2023-09-20T21:18:50.307 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-20T21:18:50.308 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} --archive /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182 --name vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi --verbose --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182/orig.config.yaml 2023-09-20T21:18:50.378 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-20T21:18:50.484 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182 branch: wip-vshankar-testing-20230920.072635 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} email: vshankar@redhat.com first_in_suite: false job_id: '7399182' kernel: client: branch: testing flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-vshankar-testing-20230920.072635 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: bf8a061ff16ac1d393cf108f740d92dfa49f2e54 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: bf8a061ff16ac1d393cf108f740d92dfa49f2e54 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20230920.072635 sha1: bf8a061ff16ac1d393cf108f740d92dfa49f2e54 owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2186 sha1: bf8a061ff16ac1d393cf108f740d92dfa49f2e54 sleep_before_teardown: 0 subset: 614/1024 suite: fs suite_branch: wip-vshankar-testing-20230920.072635 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bf8a061ff16ac1d393cf108f740d92dfa49f2e54/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bf8a061ff16ac1d393cf108f740d92dfa49f2e54 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-20_10:42:39 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1394 2023-09-20T21:18:50.485 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bf8a061ff16ac1d393cf108f740d92dfa49f2e54/qa; will attempt to use it 2023-09-20T21:18:50.486 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bf8a061ff16ac1d393cf108f740d92dfa49f2e54/qa/tasks 2023-09-20T21:18:50.486 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-20T21:18:50.488 INFO:teuthology.task.internal:Checking packages... 2023-09-20T21:18:50.512 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'bf8a061ff16ac1d393cf108f740d92dfa49f2e54' 2023-09-20T21:18:50.513 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-20T21:18:50.513 INFO:teuthology.packaging:ref: None 2023-09-20T21:18:50.514 INFO:teuthology.packaging:tag: None 2023-09-20T21:18:50.515 INFO:teuthology.packaging:branch: wip-vshankar-testing-20230920.072635 2023-09-20T21:18:50.515 INFO:teuthology.packaging:sha1: bf8a061ff16ac1d393cf108f740d92dfa49f2e54 2023-09-20T21:18:50.516 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing-20230920.072635 2023-09-20T21:18:50.662 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6213.gbf8a061f 2023-09-20T21:18:50.662 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-20T21:18:50.669 INFO:teuthology.task.internal:no buildpackages task found 2023-09-20T21:18:50.670 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-20T21:18:50.683 INFO:teuthology.task.internal:Saving configuration 2023-09-20T21:18:50.707 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-20T21:18:50.714 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-20T21:18:50.740 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-20 21:13:35.765830', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-20T21:18:50.762 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-20 21:13:35.767733', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-20T21:18:50.786 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-20 21:13:35.769330', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-20T21:18:50.786 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-20T21:18:50.793 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-09-20T21:18:50.794 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-09-20T21:18:50.794 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-09-20T21:18:50.795 INFO:teuthology.run_tasks:Running task console_log... 2023-09-20T21:18:50.885 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc22eca2ca0>, signals=[15]) 2023-09-20T21:18:50.886 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-20T21:18:50.894 INFO:teuthology.task.internal:Opening connections... 2023-09-20T21:18:50.895 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2023-09-20T21:18:50.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-20T21:18:50.968 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2023-09-20T21:18:50.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-20T21:18:51.036 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2023-09-20T21:18:51.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-20T21:18:51.105 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-20T21:18:51.112 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2023-09-20T21:18:51.135 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2023-09-20T21:18:51.135 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2023-09-20T21:18:51.195 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="Red Hat Enterprise Linux" 2023-09-20T21:18:51.196 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="8.6 (Ootpa)" 2023-09-20T21:18:51.196 INFO:teuthology.orchestra.run.smithi022.stdout:ID="rhel" 2023-09-20T21:18:51.196 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="fedora" 2023-09-20T21:18:51.196 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="8.6" 2023-09-20T21:18:51.196 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el8" 2023-09-20T21:18:51.197 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-20T21:18:51.197 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2023-09-20T21:18:51.197 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-20T21:18:51.197 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.redhat.com/" 2023-09-20T21:18:51.197 INFO:teuthology.orchestra.run.smithi022.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-20T21:18:51.197 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-20T21:18:51.198 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:18:51.198 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-20T21:18:51.198 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-20T21:18:51.198 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-20T21:18:51.198 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-20T21:18:51.199 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2023-09-20T21:18:51.223 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2023-09-20T21:18:51.244 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2023-09-20T21:18:51.245 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2023-09-20T21:18:51.303 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Red Hat Enterprise Linux" 2023-09-20T21:18:51.303 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8.6 (Ootpa)" 2023-09-20T21:18:51.303 INFO:teuthology.orchestra.run.smithi089.stdout:ID="rhel" 2023-09-20T21:18:51.304 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="fedora" 2023-09-20T21:18:51.304 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8.6" 2023-09-20T21:18:51.304 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2023-09-20T21:18:51.304 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-20T21:18:51.304 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2023-09-20T21:18:51.305 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-20T21:18:51.305 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.redhat.com/" 2023-09-20T21:18:51.305 INFO:teuthology.orchestra.run.smithi089.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-20T21:18:51.305 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-20T21:18:51.305 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:18:51.306 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-20T21:18:51.306 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-20T21:18:51.306 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-20T21:18:51.306 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-20T21:18:51.306 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2023-09-20T21:18:51.340 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2023-09-20T21:18:51.361 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2023-09-20T21:18:51.361 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-09-20T21:18:51.421 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux" 2023-09-20T21:18:51.421 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.6 (Ootpa)" 2023-09-20T21:18:51.421 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel" 2023-09-20T21:18:51.422 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora" 2023-09-20T21:18:51.422 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.6" 2023-09-20T21:18:51.422 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2023-09-20T21:18:51.422 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-20T21:18:51.423 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2023-09-20T21:18:51.423 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-20T21:18:51.423 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/" 2023-09-20T21:18:51.423 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-20T21:18:51.424 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-20T21:18:51.424 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:18:51.424 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-20T21:18:51.425 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-20T21:18:51.425 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-20T21:18:51.425 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-20T21:18:51.426 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-09-20T21:18:51.460 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-20T21:18:51.470 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-20T21:18:51.476 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-20T21:18:51.476 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2023-09-20T21:18:51.479 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2023-09-20T21:18:51.481 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2023-09-20T21:18:51.501 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-20T21:18:51.509 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-20T21:18:51.509 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2023-09-20T21:18:51.539 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2023-09-20T21:18:51.542 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2023-09-20T21:18:51.583 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-20T21:18:51.660 INFO:teuthology.run_tasks:Running task kernel... 2023-09-20T21:18:51.678 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-09-20T21:18:51.679 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-09-20T21:18:51.679 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-09-20T21:18:51.680 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-20T21:18:51.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T21:18:51.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-09-20T21:18:51.887 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-20T21:18:51.887 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-09-20T21:18:51.888 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2023-09-20T21:18:51.908 INFO:teuthology.orchestra.run.smithi022.stdout:4.18.0-372.9.1.el8.x86_64 2023-09-20T21:18:51.909 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-20T21:18:51.909 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-09-20T21:18:51.910 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum upgrade -y linux-firmware 2023-09-20T21:18:53.138 INFO:teuthology.orchestra.run.smithi022.stdout:Updating Subscription Management repositories. 2023-09-20T21:18:53.138 INFO:teuthology.orchestra.run.smithi022.stdout:Unable to read consumer identity 2023-09-20T21:18:54.559 INFO:teuthology.orchestra.run.smithi022.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-09-20T21:18:55.467 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 18 MB/s | 16 MB 00:00 2023-09-20T21:18:59.284 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 573 kB/s | 24 kB 00:00 2023-09-20T21:19:00.014 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2023-09-20T21:19:00.015 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2023-09-20T21:19:00.015 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2023-09-20T21:19:00.154 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-09-20T21:19:00.155 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-20T21:19:00.382 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-09-20T21:19:00.382 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-09-20T21:19:00.384 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi022.front.sepia.ceph.com to complete... 2023-09-20T21:19:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20 21:19:30 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-09-20T21:19:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:FINISHED --2023-09-20 21:19:30-- 2023-09-20T21:19:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:Total wall clock time: 30s 2023-09-20T21:19:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:Downloaded: 1 files, 363M in 30s (12.3 MB/s) 2023-09-20T21:19:30.257 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-09-20T21:19:30.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-09-20T21:19:31.926 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying... ######################################## 2023-09-20T21:19:32.302 INFO:teuthology.orchestra.run.smithi022.stdout:Preparing... ######################################## 2023-09-20T21:19:32.657 INFO:teuthology.orchestra.run.smithi022.stdout:Updating / installing... 2023-09-20T21:20:13.122 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-09-20T21:20:40.259 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-09-20T21:20:40.259 DEBUG:teuthology.orchestra.run.smithi022:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-09-20T21:20:40.312 INFO:teuthology.orchestra.run.smithi022.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-20T21:20:40.312 INFO:teuthology.orchestra.run.smithi022.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-09-20T21:20:40.313 INFO:teuthology.orchestra.run.smithi022.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-09-20T21:20:40.313 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-09-20T21:20:40.313 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-09-20T21:20:40.313 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-09-20T21:20:40.313 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-09-20T21:20:40.314 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-09-20T21:20:40.314 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-09-20T21:20:40.314 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-09-20T21:20:40.314 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-09-20T21:20:40.314 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-09-20T21:20:40.315 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-09-20T21:20:40.315 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-09-20T21:20:40.315 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-09-20T21:20:40.315 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-09-20T21:20:40.316 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-09-20T21:20:40.316 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-09-20T21:20:40.316 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-09-20T21:20:40.316 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-09-20T21:20:40.316 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-09-20T21:20:40.316 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-09-20T21:20:40.317 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-09-20T21:20:40.317 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-09-20T21:20:40.317 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-09-20T21:20:40.317 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-09-20T21:20:40.317 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-09-20T21:20:40.318 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-09-20T21:20:40.318 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-09-20T21:20:40.318 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-09-20T21:20:40.318 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-09-20T21:20:40.318 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-09-20T21:20:40.319 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-09-20T21:20:40.319 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-09-20T21:20:40.319 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-09-20T21:20:40.319 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-09-20T21:20:40.319 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-09-20T21:20:40.320 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-09-20T21:20:40.320 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-09-20T21:20:40.320 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-09-20T21:20:40.320 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-09-20T21:20:40.320 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-09-20T21:20:40.321 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-09-20T21:20:40.321 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-09-20T21:20:40.321 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-09-20T21:20:40.321 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-09-20T21:20:40.321 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-09-20T21:20:40.322 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-09-20T21:20:40.322 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-09-20T21:20:40.322 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-09-20T21:20:40.322 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-09-20T21:20:40.322 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-09-20T21:20:40.323 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-09-20T21:20:40.323 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-09-20T21:20:40.323 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-09-20T21:20:40.323 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-09-20T21:20:40.323 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-09-20T21:20:40.324 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-09-20T21:20:40.324 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-09-20T21:20:40.324 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-09-20T21:20:40.324 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-09-20T21:20:40.324 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-09-20T21:20:40.324 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-09-20T21:20:40.325 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-09-20T21:20:40.325 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-09-20T21:20:40.325 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-09-20T21:20:40.325 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-09-20T21:20:40.325 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-09-20T21:20:40.326 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-09-20T21:20:40.326 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-09-20T21:20:40.326 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-09-20T21:20:40.326 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-09-20T21:20:40.326 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-09-20T21:20:40.327 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-09-20T21:20:40.327 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-09-20T21:20:40.327 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-09-20T21:20:40.327 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-09-20T21:20:40.327 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-09-20T21:20:40.328 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-09-20T21:20:40.328 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-09-20T21:20:40.328 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-09-20T21:20:40.328 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-09-20T21:20:40.328 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-09-20T21:20:40.329 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-09-20T21:20:40.329 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-09-20T21:20:40.329 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-09-20T21:20:40.329 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-09-20T21:20:40.329 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-09-20T21:20:40.329 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-09-20T21:20:40.330 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-09-20T21:20:40.330 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-09-20T21:20:40.330 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-09-20T21:20:40.330 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-09-20T21:20:40.330 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-09-20T21:20:40.331 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-09-20T21:20:40.331 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-09-20T21:20:40.331 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-09-20T21:20:40.331 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-09-20T21:20:40.331 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-09-20T21:20:40.332 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-09-20T21:20:40.332 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-09-20T21:20:40.332 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-09-20T21:20:40.332 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-09-20T21:20:40.332 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-09-20T21:20:40.333 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-09-20T21:20:40.333 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-09-20T21:20:40.333 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-09-20T21:20:40.333 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-09-20T21:20:40.333 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-09-20T21:20:40.333 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-09-20T21:20:40.334 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-09-20T21:20:40.334 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-09-20T21:20:40.334 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-09-20T21:20:40.334 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-09-20T21:20:40.335 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-09-20T21:20:40.335 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-09-20T21:20:40.335 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-09-20T21:20:40.335 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-09-20T21:20:40.335 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-09-20T21:20:40.335 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-09-20T21:20:40.336 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-09-20T21:20:40.336 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-09-20T21:20:40.336 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-09-20T21:20:40.336 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-09-20T21:20:40.336 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-09-20T21:20:40.337 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-09-20T21:20:40.337 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-09-20T21:20:40.337 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-09-20T21:20:40.337 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-09-20T21:20:40.337 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-09-20T21:20:40.338 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-09-20T21:20:40.338 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-09-20T21:20:40.338 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-09-20T21:20:40.338 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-09-20T21:20:40.338 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-09-20T21:20:40.339 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-09-20T21:20:40.339 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-09-20T21:20:40.339 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-09-20T21:20:40.339 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-09-20T21:20:40.339 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-09-20T21:20:40.340 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-09-20T21:20:40.340 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-09-20T21:20:40.340 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-09-20T21:20:40.340 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-09-20T21:20:40.340 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-09-20T21:20:40.341 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-09-20T21:20:40.341 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-09-20T21:20:40.341 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-09-20T21:20:40.341 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-09-20T21:20:40.341 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-09-20T21:20:40.341 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-09-20T21:20:40.342 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-09-20T21:20:40.342 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-09-20T21:20:40.342 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-09-20T21:20:40.342 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-09-20T21:20:40.343 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-09-20T21:20:40.343 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-09-20T21:20:40.343 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-09-20T21:20:40.343 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-09-20T21:20:40.343 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-09-20T21:20:40.343 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-09-20T21:20:40.344 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-09-20T21:20:40.344 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-09-20T21:20:40.344 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-09-20T21:20:40.344 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-09-20T21:20:40.344 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-09-20T21:20:40.345 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-09-20T21:20:40.345 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-09-20T21:20:40.345 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-09-20T21:20:40.345 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-09-20T21:20:40.345 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-09-20T21:20:40.346 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-09-20T21:20:40.346 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-09-20T21:20:40.346 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-09-20T21:20:40.346 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-09-20T21:20:40.346 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-09-20T21:20:40.347 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-09-20T21:20:40.347 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-09-20T21:20:40.347 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-09-20T21:20:40.347 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-09-20T21:20:40.347 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-09-20T21:20:40.348 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-09-20T21:20:40.348 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-09-20T21:20:40.348 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-09-20T21:20:40.348 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-09-20T21:20:40.348 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-09-20T21:20:40.348 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-09-20T21:20:40.349 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-09-20T21:20:40.349 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-09-20T21:20:40.349 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-09-20T21:20:40.349 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-09-20T21:20:40.349 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-09-20T21:20:40.350 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-09-20T21:20:40.350 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-09-20T21:20:40.350 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-09-20T21:20:40.350 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-09-20T21:20:40.350 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-09-20T21:20:40.351 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-09-20T21:20:40.351 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-09-20T21:20:40.351 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-09-20T21:20:40.351 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-09-20T21:20:40.351 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-09-20T21:20:40.352 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-09-20T21:20:40.352 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-09-20T21:20:40.352 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-09-20T21:20:40.352 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-09-20T21:20:40.352 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-09-20T21:20:40.352 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-09-20T21:20:40.353 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-09-20T21:20:40.353 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-09-20T21:20:40.353 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-09-20T21:20:40.353 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-09-20T21:20:40.353 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-09-20T21:20:40.354 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-09-20T21:20:40.354 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-09-20T21:20:40.354 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-09-20T21:20:40.354 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-09-20T21:20:40.354 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-09-20T21:20:40.355 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-09-20T21:20:40.355 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-09-20T21:20:40.355 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-09-20T21:20:40.355 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-09-20T21:20:40.355 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-09-20T21:20:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-09-20T21:20:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-09-20T21:20:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-09-20T21:20:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-09-20T21:20:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-09-20T21:20:40.356 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-09-20T21:20:40.357 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-09-20T21:20:40.357 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-09-20T21:20:40.357 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-09-20T21:20:40.357 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-09-20T21:20:40.357 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-09-20T21:20:40.358 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-09-20T21:20:40.358 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-09-20T21:20:40.358 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-09-20T21:20:40.358 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-09-20T21:20:40.358 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-09-20T21:20:40.359 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-09-20T21:20:40.359 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-09-20T21:20:40.359 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-09-20T21:20:40.359 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-09-20T21:20:40.359 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-09-20T21:20:40.360 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-09-20T21:20:40.360 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-09-20T21:20:40.360 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-09-20T21:20:40.360 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-09-20T21:20:40.360 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-09-20T21:20:40.361 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-09-20T21:20:40.361 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-09-20T21:20:40.361 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-09-20T21:20:40.361 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-09-20T21:20:40.361 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-09-20T21:20:40.362 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-09-20T21:20:40.362 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-09-20T21:20:40.362 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-09-20T21:20:40.362 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-09-20T21:20:40.362 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-09-20T21:20:40.363 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-09-20T21:20:40.363 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-09-20T21:20:40.363 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-09-20T21:20:40.363 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-09-20T21:20:40.363 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-09-20T21:20:40.364 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-09-20T21:20:40.364 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-09-20T21:20:40.364 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-09-20T21:20:40.364 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-09-20T21:20:40.364 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-09-20T21:20:40.364 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-09-20T21:20:40.365 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-09-20T21:20:40.365 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-09-20T21:20:40.365 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-09-20T21:20:40.365 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-09-20T21:20:40.365 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-09-20T21:20:40.366 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-09-20T21:20:40.366 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-09-20T21:20:40.366 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-09-20T21:20:40.366 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-09-20T21:20:40.366 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-09-20T21:20:40.367 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-09-20T21:20:40.367 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-09-20T21:20:40.367 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-09-20T21:20:40.367 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-09-20T21:20:40.367 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-09-20T21:20:40.368 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-09-20T21:20:40.368 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-09-20T21:20:40.368 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-09-20T21:20:40.368 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-09-20T21:20:40.368 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-09-20T21:20:40.368 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-09-20T21:20:40.369 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-09-20T21:20:40.369 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-09-20T21:20:40.369 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-09-20T21:20:40.369 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-09-20T21:20:40.369 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-09-20T21:20:40.370 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-09-20T21:20:40.370 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-09-20T21:20:40.370 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-09-20T21:20:40.370 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-09-20T21:20:40.370 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-09-20T21:20:40.371 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-09-20T21:20:40.371 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-09-20T21:20:40.371 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-09-20T21:20:40.371 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-09-20T21:20:40.371 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-09-20T21:20:40.372 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-09-20T21:20:40.372 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-09-20T21:20:40.372 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-09-20T21:20:40.372 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-09-20T21:20:40.372 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-09-20T21:20:40.373 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-09-20T21:20:40.373 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-09-20T21:20:40.373 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-09-20T21:20:40.373 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-09-20T21:20:40.373 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-09-20T21:20:40.373 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-09-20T21:20:40.374 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-09-20T21:20:40.374 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-09-20T21:20:40.374 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-09-20T21:20:40.374 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-09-20T21:20:40.374 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-09-20T21:20:40.375 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-09-20T21:20:40.375 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-09-20T21:20:40.375 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-09-20T21:20:40.375 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-09-20T21:20:40.375 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-09-20T21:20:40.376 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-09-20T21:20:40.376 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-09-20T21:20:40.376 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-09-20T21:20:40.376 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-09-20T21:20:40.376 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-09-20T21:20:40.377 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-09-20T21:20:40.377 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-09-20T21:20:40.377 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-09-20T21:20:40.377 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-09-20T21:20:40.377 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-09-20T21:20:40.377 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-09-20T21:20:40.378 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-09-20T21:20:40.378 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-09-20T21:20:40.378 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-09-20T21:20:40.378 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-09-20T21:20:40.378 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-09-20T21:20:40.379 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-09-20T21:20:40.379 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-09-20T21:20:40.379 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-09-20T21:20:40.379 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-09-20T21:20:40.379 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-09-20T21:20:40.380 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-09-20T21:20:40.380 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-09-20T21:20:40.380 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-09-20T21:20:40.380 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-09-20T21:20:40.380 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-09-20T21:20:40.380 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-09-20T21:20:40.381 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-09-20T21:20:40.381 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-09-20T21:20:40.381 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-09-20T21:20:40.381 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-09-20T21:20:40.381 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-09-20T21:20:40.382 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-09-20T21:20:40.382 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-09-20T21:20:40.382 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-09-20T21:20:40.382 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-09-20T21:20:40.382 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-09-20T21:20:40.383 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-09-20T21:20:40.383 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-09-20T21:20:40.383 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-09-20T21:20:40.383 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-09-20T21:20:40.383 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-09-20T21:20:40.384 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-09-20T21:20:40.384 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-09-20T21:20:40.384 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-09-20T21:20:40.384 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-09-20T21:20:40.384 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-09-20T21:20:40.384 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-09-20T21:20:40.385 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-09-20T21:20:40.385 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-09-20T21:20:40.385 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-09-20T21:20:40.385 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-09-20T21:20:40.385 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-09-20T21:20:40.386 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-09-20T21:20:40.386 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-09-20T21:20:40.386 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-09-20T21:20:40.386 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-09-20T21:20:40.386 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-09-20T21:20:40.387 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-09-20T21:20:40.387 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-09-20T21:20:40.387 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-09-20T21:20:40.387 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-09-20T21:20:40.387 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-09-20T21:20:40.387 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-09-20T21:20:40.388 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-09-20T21:20:40.388 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-09-20T21:20:40.388 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-09-20T21:20:40.388 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-09-20T21:20:40.388 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-09-20T21:20:40.389 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-09-20T21:20:40.389 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-09-20T21:20:40.389 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-09-20T21:20:40.389 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-09-20T21:20:40.389 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-09-20T21:20:40.390 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-09-20T21:20:40.390 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-09-20T21:20:40.390 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-09-20T21:20:40.390 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-09-20T21:20:40.390 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-09-20T21:20:40.391 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-09-20T21:20:40.391 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-09-20T21:20:40.391 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-09-20T21:20:40.391 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-09-20T21:20:40.391 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-09-20T21:20:40.392 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-09-20T21:20:40.392 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-09-20T21:20:40.392 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-09-20T21:20:40.392 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-09-20T21:20:40.392 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-09-20T21:20:40.392 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-09-20T21:20:40.393 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-09-20T21:20:40.393 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-09-20T21:20:40.393 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-09-20T21:20:40.393 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-09-20T21:20:40.393 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-09-20T21:20:40.394 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-09-20T21:20:40.394 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-09-20T21:20:40.394 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-09-20T21:20:40.394 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-09-20T21:20:40.394 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-09-20T21:20:40.395 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-09-20T21:20:40.395 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-09-20T21:20:40.395 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-09-20T21:20:40.395 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-09-20T21:20:40.395 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-09-20T21:20:40.396 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-09-20T21:20:40.396 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-09-20T21:20:40.396 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-09-20T21:20:40.396 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-09-20T21:20:40.396 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-09-20T21:20:40.397 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-09-20T21:20:40.397 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-09-20T21:20:40.397 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-09-20T21:20:40.397 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-09-20T21:20:40.397 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-09-20T21:20:40.397 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-09-20T21:20:40.398 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-09-20T21:20:40.398 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-09-20T21:20:40.398 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-09-20T21:20:40.398 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-09-20T21:20:40.398 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-09-20T21:20:40.399 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-09-20T21:20:40.399 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-09-20T21:20:40.399 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-09-20T21:20:40.399 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-09-20T21:20:40.399 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-09-20T21:20:40.400 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-09-20T21:20:40.400 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-09-20T21:20:40.400 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-09-20T21:20:40.400 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-09-20T21:20:40.400 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-09-20T21:20:40.400 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-09-20T21:20:40.401 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-09-20T21:20:40.401 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-09-20T21:20:40.401 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-09-20T21:20:40.401 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-09-20T21:20:40.401 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-09-20T21:20:40.402 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-09-20T21:20:40.402 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-09-20T21:20:40.402 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-09-20T21:20:40.402 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-09-20T21:20:40.402 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-09-20T21:20:40.403 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-09-20T21:20:40.403 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-09-20T21:20:40.403 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-09-20T21:20:40.403 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-09-20T21:20:40.403 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-09-20T21:20:40.404 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-09-20T21:20:40.404 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-09-20T21:20:40.404 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-09-20T21:20:40.404 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-09-20T21:20:40.404 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-09-20T21:20:40.405 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-09-20T21:20:40.405 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-09-20T21:20:40.405 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-09-20T21:20:40.405 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-09-20T21:20:40.405 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-09-20T21:20:40.406 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-09-20T21:20:40.406 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-09-20T21:20:40.406 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-09-20T21:20:40.406 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-09-20T21:20:40.406 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-09-20T21:20:40.407 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-09-20T21:20:40.407 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-09-20T21:20:40.407 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-09-20T21:20:40.407 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-09-20T21:20:40.407 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-09-20T21:20:40.408 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-09-20T21:20:40.408 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-09-20T21:20:40.408 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-09-20T21:20:40.408 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-09-20T21:20:40.408 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-09-20T21:20:40.409 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-09-20T21:20:40.409 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-09-20T21:20:40.409 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-09-20T21:20:40.409 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-09-20T21:20:40.409 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-09-20T21:20:40.409 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-09-20T21:20:40.410 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-09-20T21:20:40.410 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-09-20T21:20:40.410 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-09-20T21:20:40.410 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-09-20T21:20:40.410 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-09-20T21:20:40.411 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-09-20T21:20:40.411 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-09-20T21:20:40.411 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-09-20T21:20:40.411 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-09-20T21:20:40.411 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-09-20T21:20:40.412 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-09-20T21:20:40.412 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-09-20T21:20:40.412 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-09-20T21:20:40.412 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-09-20T21:20:40.412 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-09-20T21:20:40.413 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-09-20T21:20:40.413 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-09-20T21:20:40.413 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-09-20T21:20:40.413 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-09-20T21:20:40.414 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-09-20T21:20:40.414 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-09-20T21:20:40.414 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-09-20T21:20:40.414 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-09-20T21:20:40.414 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-09-20T21:20:40.415 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-09-20T21:20:40.415 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-09-20T21:20:40.415 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-09-20T21:20:40.415 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-09-20T21:20:40.415 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-09-20T21:20:40.416 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-09-20T21:20:40.416 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-09-20T21:20:40.416 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-09-20T21:20:40.416 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-09-20T21:20:40.416 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-09-20T21:20:40.416 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-09-20T21:20:40.417 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-09-20T21:20:40.417 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-09-20T21:20:40.417 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-09-20T21:20:40.417 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-09-20T21:20:40.417 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-09-20T21:20:40.417 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-09-20T21:20:40.418 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-09-20T21:20:40.418 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-09-20T21:20:40.418 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-09-20T21:20:40.418 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-09-20T21:20:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-09-20T21:20:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-09-20T21:20:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-09-20T21:20:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-09-20T21:20:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-09-20T21:20:40.420 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-09-20T21:20:40.420 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-09-20T21:20:40.420 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-09-20T21:20:40.420 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-09-20T21:20:40.420 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-09-20T21:20:40.420 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-09-20T21:20:40.421 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-09-20T21:20:40.421 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-09-20T21:20:40.421 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-09-20T21:20:40.421 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-09-20T21:20:40.421 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-09-20T21:20:40.422 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-09-20T21:20:40.422 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-09-20T21:20:40.422 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-09-20T21:20:40.422 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-09-20T21:20:40.422 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-09-20T21:20:40.423 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-09-20T21:20:40.423 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-09-20T21:20:40.423 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-09-20T21:20:40.423 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-09-20T21:20:40.423 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-09-20T21:20:40.423 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-09-20T21:20:40.424 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-09-20T21:20:40.424 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-09-20T21:20:40.424 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-09-20T21:20:40.424 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-09-20T21:20:40.424 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-09-20T21:20:40.425 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-09-20T21:20:40.425 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-09-20T21:20:40.425 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-09-20T21:20:40.425 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-09-20T21:20:40.425 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-09-20T21:20:40.426 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-09-20T21:20:40.426 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-09-20T21:20:40.426 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-09-20T21:20:40.426 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-09-20T21:20:40.426 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-09-20T21:20:40.427 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-09-20T21:20:40.427 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-09-20T21:20:40.427 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-09-20T21:20:40.427 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-09-20T21:20:40.427 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-09-20T21:20:40.428 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-09-20T21:20:40.428 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-09-20T21:20:40.428 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-09-20T21:20:40.428 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-09-20T21:20:40.428 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-09-20T21:20:40.428 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-09-20T21:20:40.429 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-09-20T21:20:40.429 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-09-20T21:20:40.429 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-09-20T21:20:40.429 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-09-20T21:20:40.429 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-09-20T21:20:40.430 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-09-20T21:20:40.430 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-09-20T21:20:40.430 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-09-20T21:20:40.430 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-09-20T21:20:40.430 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-09-20T21:20:40.431 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-09-20T21:20:40.431 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-09-20T21:20:40.431 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-09-20T21:20:40.431 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-09-20T21:20:40.431 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-09-20T21:20:40.431 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-09-20T21:20:40.432 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-09-20T21:20:40.432 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-09-20T21:20:40.432 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-09-20T21:20:40.432 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-09-20T21:20:40.432 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-09-20T21:20:40.433 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-09-20T21:20:40.433 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-09-20T21:20:40.433 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-09-20T21:20:40.433 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-09-20T21:20:40.433 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-09-20T21:20:40.433 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-09-20T21:20:40.434 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-09-20T21:20:40.434 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-09-20T21:20:40.434 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-09-20T21:20:40.434 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-09-20T21:20:40.434 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-09-20T21:20:40.435 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-09-20T21:20:40.435 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-09-20T21:20:40.435 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-09-20T21:20:40.435 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-09-20T21:20:40.435 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-09-20T21:20:40.436 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-09-20T21:20:40.436 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-09-20T21:20:40.436 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-09-20T21:20:40.436 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-09-20T21:20:40.436 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-09-20T21:20:40.436 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-09-20T21:20:40.437 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-09-20T21:20:40.437 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-09-20T21:20:40.437 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-09-20T21:20:40.437 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-09-20T21:20:40.437 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-09-20T21:20:40.438 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-09-20T21:20:40.438 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-09-20T21:20:40.438 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-09-20T21:20:40.438 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-09-20T21:20:40.438 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-09-20T21:20:40.439 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-09-20T21:20:40.439 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-09-20T21:20:40.439 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-09-20T21:20:40.439 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-09-20T21:20:40.439 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-09-20T21:20:40.440 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-09-20T21:20:40.440 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-09-20T21:20:40.440 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-09-20T21:20:40.440 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-09-20T21:20:40.440 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-09-20T21:20:40.441 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-09-20T21:20:40.441 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-09-20T21:20:40.441 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-09-20T21:20:40.441 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-09-20T21:20:40.441 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-09-20T21:20:40.441 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-09-20T21:20:40.442 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-09-20T21:20:40.442 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-09-20T21:20:40.442 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-09-20T21:20:40.442 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-09-20T21:20:40.443 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-09-20T21:20:40.443 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-09-20T21:20:40.443 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-09-20T21:20:40.443 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-09-20T21:20:40.443 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-09-20T21:20:40.443 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-09-20T21:20:40.444 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-09-20T21:20:40.444 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-09-20T21:20:40.444 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-09-20T21:20:40.444 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-09-20T21:20:40.444 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-09-20T21:20:40.445 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-09-20T21:20:40.445 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-09-20T21:20:40.445 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-09-20T21:20:40.445 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-09-20T21:20:40.445 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-09-20T21:20:40.446 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-09-20T21:20:40.446 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-09-20T21:20:40.446 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-09-20T21:20:40.446 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-09-20T21:20:40.446 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-09-20T21:20:40.447 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-09-20T21:20:40.447 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-09-20T21:20:40.447 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-09-20T21:20:40.447 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-09-20T21:20:40.447 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-09-20T21:20:40.448 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-09-20T21:20:40.448 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-09-20T21:20:40.448 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-09-20T21:20:40.448 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-09-20T21:20:40.448 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-09-20T21:20:40.448 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-09-20T21:20:40.449 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-09-20T21:20:40.449 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-09-20T21:20:40.449 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-09-20T21:20:40.449 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-09-20T21:20:40.449 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-09-20T21:20:40.450 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-09-20T21:20:40.450 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-09-20T21:20:40.450 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-09-20T21:20:40.450 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-09-20T21:20:40.450 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-09-20T21:20:40.451 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-09-20T21:20:40.451 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-09-20T21:20:40.451 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-09-20T21:20:40.451 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-09-20T21:20:40.451 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-09-20T21:20:40.451 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-09-20T21:20:40.452 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-09-20T21:20:40.452 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-09-20T21:20:40.452 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-09-20T21:20:40.452 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-09-20T21:20:40.452 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-09-20T21:20:40.453 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-09-20T21:20:40.453 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-09-20T21:20:40.453 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-09-20T21:20:40.453 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-09-20T21:20:40.453 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-09-20T21:20:40.454 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-09-20T21:20:40.454 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-09-20T21:20:40.454 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-09-20T21:20:40.454 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-09-20T21:20:40.454 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-09-20T21:20:40.455 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-09-20T21:20:40.455 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-09-20T21:20:40.455 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-09-20T21:20:40.455 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-09-20T21:20:40.455 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-09-20T21:20:40.456 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-09-20T21:20:40.456 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-09-20T21:20:40.456 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-09-20T21:20:40.456 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-09-20T21:20:40.456 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-09-20T21:20:40.457 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-09-20T21:20:40.457 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-09-20T21:20:40.457 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-09-20T21:20:40.457 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-09-20T21:20:40.457 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-09-20T21:20:40.457 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-09-20T21:20:40.458 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-09-20T21:20:40.458 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-09-20T21:20:40.458 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-09-20T21:20:40.458 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-09-20T21:20:40.458 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-09-20T21:20:40.459 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-09-20T21:20:40.459 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-09-20T21:20:40.459 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-09-20T21:20:40.459 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-09-20T21:20:40.459 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-09-20T21:20:40.460 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-09-20T21:20:40.460 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-09-20T21:20:40.460 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-09-20T21:20:40.460 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-09-20T21:20:40.460 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-09-20T21:20:40.461 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-09-20T21:20:40.461 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-09-20T21:20:40.461 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-09-20T21:20:40.461 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-09-20T21:20:40.461 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-09-20T21:20:40.461 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-09-20T21:20:40.462 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-09-20T21:20:40.462 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-09-20T21:20:40.462 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-09-20T21:20:40.462 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-09-20T21:20:40.462 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-09-20T21:20:40.463 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-09-20T21:20:40.463 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-09-20T21:20:40.463 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-09-20T21:20:40.463 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-09-20T21:20:40.463 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-09-20T21:20:40.464 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-09-20T21:20:40.464 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-09-20T21:20:40.464 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-09-20T21:20:40.464 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-09-20T21:20:40.464 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-09-20T21:20:40.464 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-09-20T21:20:40.465 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-09-20T21:20:40.465 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-09-20T21:20:40.465 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-09-20T21:20:40.465 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-09-20T21:20:40.465 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-09-20T21:20:40.466 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-09-20T21:20:40.466 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-09-20T21:20:40.466 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-09-20T21:20:40.466 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-09-20T21:20:40.466 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-09-20T21:20:40.467 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-09-20T21:20:40.467 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-09-20T21:20:40.467 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-09-20T21:20:40.467 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-09-20T21:20:40.467 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-09-20T21:20:40.468 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-09-20T21:20:40.468 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-09-20T21:20:40.468 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-09-20T21:20:40.468 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-09-20T21:20:40.468 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-09-20T21:20:40.468 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-09-20T21:20:40.469 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-09-20T21:20:40.469 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-09-20T21:20:40.469 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-09-20T21:20:40.469 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-09-20T21:20:40.469 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-09-20T21:20:40.470 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-09-20T21:20:40.470 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-09-20T21:20:40.470 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-09-20T21:20:40.470 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-09-20T21:20:40.470 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-09-20T21:20:40.471 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-09-20T21:20:40.471 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-09-20T21:20:40.471 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-09-20T21:20:40.471 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-09-20T21:20:40.471 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-09-20T21:20:40.471 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-09-20T21:20:40.472 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-09-20T21:20:40.472 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-09-20T21:20:40.472 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-09-20T21:20:40.472 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-09-20T21:20:40.472 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-09-20T21:20:40.473 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-09-20T21:20:40.473 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-09-20T21:20:40.473 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-09-20T21:20:40.473 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-09-20T21:20:40.473 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-09-20T21:20:40.474 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-09-20T21:20:40.474 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-09-20T21:20:40.474 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-09-20T21:20:40.474 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-09-20T21:20:40.474 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-09-20T21:20:40.475 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-09-20T21:20:40.475 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-09-20T21:20:40.475 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-09-20T21:20:40.477 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-09-20T21:20:40.477 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-09-20T21:20:40.477 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-09-20T21:20:40.478 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-09-20T21:20:40.478 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-09-20T21:20:40.478 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-09-20T21:20:40.478 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-09-20T21:20:40.478 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-09-20T21:20:40.479 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-09-20T21:20:40.479 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-09-20T21:20:40.479 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-09-20T21:20:40.479 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-09-20T21:20:40.479 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-09-20T21:20:40.479 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-09-20T21:20:40.480 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-09-20T21:20:40.480 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-09-20T21:20:40.480 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-09-20T21:20:40.480 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-09-20T21:20:40.480 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-09-20T21:20:40.481 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-09-20T21:20:40.481 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-09-20T21:20:40.481 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-09-20T21:20:40.481 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-09-20T21:20:40.481 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-09-20T21:20:40.482 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-09-20T21:20:40.482 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-09-20T21:20:40.482 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-09-20T21:20:40.482 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-09-20T21:20:40.482 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-09-20T21:20:40.483 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-09-20T21:20:40.483 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-09-20T21:20:40.483 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-09-20T21:20:40.483 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-09-20T21:20:40.483 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-09-20T21:20:40.484 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-09-20T21:20:40.484 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-09-20T21:20:40.484 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-09-20T21:20:40.484 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-09-20T21:20:40.484 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-09-20T21:20:40.485 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-09-20T21:20:40.485 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-09-20T21:20:40.485 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-09-20T21:20:40.485 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-09-20T21:20:40.485 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-09-20T21:20:40.485 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-09-20T21:20:40.486 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-09-20T21:20:40.486 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-09-20T21:20:40.486 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-09-20T21:20:40.486 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-09-20T21:20:40.486 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-09-20T21:20:40.487 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-09-20T21:20:40.487 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-09-20T21:20:40.487 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-09-20T21:20:40.487 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-09-20T21:20:40.487 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-09-20T21:20:40.487 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-09-20T21:20:40.488 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-09-20T21:20:40.488 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-09-20T21:20:40.488 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-09-20T21:20:40.488 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-09-20T21:20:40.488 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-09-20T21:20:40.489 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-09-20T21:20:40.489 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-09-20T21:20:40.489 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-09-20T21:20:40.489 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-09-20T21:20:40.489 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-09-20T21:20:40.490 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-09-20T21:20:40.490 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-09-20T21:20:40.490 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-09-20T21:20:40.490 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-09-20T21:20:40.490 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-09-20T21:20:40.491 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-09-20T21:20:40.491 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-09-20T21:20:40.491 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-09-20T21:20:40.491 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-09-20T21:20:40.491 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-09-20T21:20:40.491 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-09-20T21:20:40.492 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-09-20T21:20:40.492 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-09-20T21:20:40.492 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-09-20T21:20:40.492 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-09-20T21:20:40.492 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-09-20T21:20:40.493 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-09-20T21:20:40.493 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-09-20T21:20:40.493 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-09-20T21:20:40.493 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-09-20T21:20:40.493 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-09-20T21:20:40.494 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-09-20T21:20:40.494 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-09-20T21:20:40.494 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-09-20T21:20:40.494 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-09-20T21:20:40.494 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-09-20T21:20:40.494 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-09-20T21:20:40.495 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-09-20T21:20:40.495 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-09-20T21:20:40.495 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-09-20T21:20:40.495 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-09-20T21:20:40.495 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-09-20T21:20:40.496 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-09-20T21:20:40.496 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-09-20T21:20:40.496 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-09-20T21:20:40.496 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-09-20T21:20:40.496 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-09-20T21:20:40.496 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-09-20T21:20:40.497 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-09-20T21:20:40.497 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-09-20T21:20:40.497 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-09-20T21:20:40.497 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-09-20T21:20:40.497 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-09-20T21:20:40.498 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-09-20T21:20:40.498 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-09-20T21:20:40.498 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-09-20T21:20:40.498 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-09-20T21:20:40.498 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-09-20T21:20:40.499 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-09-20T21:20:40.499 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-09-20T21:20:40.499 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-09-20T21:20:40.499 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-09-20T21:20:40.499 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-09-20T21:20:40.499 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-09-20T21:20:40.500 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-09-20T21:20:40.500 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-09-20T21:20:40.500 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-09-20T21:20:40.500 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-09-20T21:20:40.500 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-09-20T21:20:40.501 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-09-20T21:20:40.501 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-09-20T21:20:40.501 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-09-20T21:20:40.501 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-09-20T21:20:40.501 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-09-20T21:20:40.501 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-09-20T21:20:40.502 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-09-20T21:20:40.502 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-09-20T21:20:40.502 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-09-20T21:20:40.502 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-09-20T21:20:40.502 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-09-20T21:20:40.503 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-09-20T21:20:40.503 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-09-20T21:20:40.503 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-09-20T21:20:40.503 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-09-20T21:20:40.503 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-09-20T21:20:40.503 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-09-20T21:20:40.504 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-09-20T21:20:40.504 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-09-20T21:20:40.504 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-09-20T21:20:40.504 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-09-20T21:20:40.504 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-09-20T21:20:40.505 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-09-20T21:20:40.505 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-09-20T21:20:40.505 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-09-20T21:20:40.505 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-09-20T21:20:40.505 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-09-20T21:20:40.505 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-09-20T21:20:40.506 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-09-20T21:20:40.506 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-09-20T21:20:40.506 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-09-20T21:20:40.506 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-09-20T21:20:40.506 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-09-20T21:20:40.507 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-09-20T21:20:40.507 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-09-20T21:20:40.507 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-09-20T21:20:40.507 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-09-20T21:20:40.507 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-09-20T21:20:40.508 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-09-20T21:20:40.508 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-09-20T21:20:40.508 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-09-20T21:20:40.508 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-09-20T21:20:40.508 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-09-20T21:20:40.508 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-09-20T21:20:40.509 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-09-20T21:20:40.509 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-09-20T21:20:40.509 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-09-20T21:20:40.509 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-09-20T21:20:40.509 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-09-20T21:20:40.510 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-09-20T21:20:40.510 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-09-20T21:20:40.510 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-09-20T21:20:40.510 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-09-20T21:20:40.510 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-09-20T21:20:40.511 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-09-20T21:20:40.511 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-09-20T21:20:40.511 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-09-20T21:20:40.511 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-09-20T21:20:40.511 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-09-20T21:20:40.511 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-09-20T21:20:40.512 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-09-20T21:20:40.512 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-09-20T21:20:40.512 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-09-20T21:20:40.512 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-09-20T21:20:40.512 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-09-20T21:20:40.513 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-09-20T21:20:40.513 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-09-20T21:20:40.513 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-09-20T21:20:40.513 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-09-20T21:20:40.513 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-09-20T21:20:40.514 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-09-20T21:20:40.514 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-09-20T21:20:40.514 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-09-20T21:20:40.514 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-09-20T21:20:40.514 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-09-20T21:20:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-09-20T21:20:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-09-20T21:20:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-09-20T21:20:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-09-20T21:20:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-09-20T21:20:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-09-20T21:20:40.516 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-09-20T21:20:40.516 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-09-20T21:20:40.516 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-09-20T21:20:40.516 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-09-20T21:20:40.516 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-09-20T21:20:40.517 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-09-20T21:20:40.517 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-09-20T21:20:40.517 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-09-20T21:20:40.517 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-09-20T21:20:40.517 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-09-20T21:20:40.517 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-09-20T21:20:40.518 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-09-20T21:20:40.518 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-09-20T21:20:40.518 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-09-20T21:20:40.518 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-09-20T21:20:40.518 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-09-20T21:20:40.519 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-09-20T21:20:40.519 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-09-20T21:20:40.519 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-09-20T21:20:40.519 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-09-20T21:20:40.519 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-09-20T21:20:40.520 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-09-20T21:20:40.520 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-09-20T21:20:40.520 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-09-20T21:20:40.520 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-09-20T21:20:40.520 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-09-20T21:20:40.521 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-09-20T21:20:40.521 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-09-20T21:20:40.521 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-09-20T21:20:40.521 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-09-20T21:20:40.521 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-09-20T21:20:40.521 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-09-20T21:20:40.522 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-09-20T21:20:40.522 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-09-20T21:20:40.522 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-09-20T21:20:40.522 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-09-20T21:20:40.522 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-09-20T21:20:40.523 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-09-20T21:20:40.523 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-09-20T21:20:40.523 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-09-20T21:20:40.523 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-09-20T21:20:40.523 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-09-20T21:20:40.524 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-09-20T21:20:40.524 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-09-20T21:20:40.524 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-09-20T21:20:40.524 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-09-20T21:20:40.524 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-09-20T21:20:40.524 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-09-20T21:20:40.525 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-09-20T21:20:40.525 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-09-20T21:20:40.525 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-09-20T21:20:40.525 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-09-20T21:20:40.525 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-09-20T21:20:40.526 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-09-20T21:20:40.526 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-09-20T21:20:40.526 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-09-20T21:20:40.526 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-09-20T21:20:40.526 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-09-20T21:20:40.527 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-09-20T21:20:40.527 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-09-20T21:20:40.527 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-09-20T21:20:40.527 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-09-20T21:20:40.527 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-09-20T21:20:40.527 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-09-20T21:20:40.528 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-09-20T21:20:40.528 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-09-20T21:20:40.528 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-09-20T21:20:40.528 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-09-20T21:20:40.528 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-09-20T21:20:40.529 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-09-20T21:20:40.529 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-09-20T21:20:40.529 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-09-20T21:20:40.529 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-09-20T21:20:40.529 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-09-20T21:20:40.530 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-09-20T21:20:40.530 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-09-20T21:20:40.530 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-09-20T21:20:40.530 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-09-20T21:20:40.530 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-09-20T21:20:40.531 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-09-20T21:20:40.531 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-09-20T21:20:40.531 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-09-20T21:20:40.531 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-09-20T21:20:40.531 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-09-20T21:20:40.532 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-09-20T21:20:40.532 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-09-20T21:20:40.532 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-09-20T21:20:40.532 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-09-20T21:20:40.532 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-09-20T21:20:40.532 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-09-20T21:20:40.533 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-09-20T21:20:40.533 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-09-20T21:20:40.533 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-09-20T21:20:40.533 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-09-20T21:20:40.533 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-09-20T21:20:40.534 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-09-20T21:20:40.534 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-09-20T21:20:40.534 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-09-20T21:20:40.534 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-09-20T21:20:40.534 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-09-20T21:20:40.535 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-09-20T21:20:40.535 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-09-20T21:20:40.535 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-09-20T21:20:40.535 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-09-20T21:20:40.535 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-09-20T21:20:40.536 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-09-20T21:20:40.536 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-09-20T21:20:40.536 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-09-20T21:20:40.536 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-09-20T21:20:40.536 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-09-20T21:20:40.536 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-09-20T21:20:40.537 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-09-20T21:20:40.537 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-09-20T21:20:40.537 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-09-20T21:20:40.537 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-09-20T21:20:40.537 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-09-20T21:20:40.538 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-09-20T21:20:40.538 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-09-20T21:20:40.538 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-09-20T21:20:40.538 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-09-20T21:20:40.538 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-09-20T21:20:40.539 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-09-20T21:20:40.539 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-09-20T21:20:40.539 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-09-20T21:20:40.539 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-09-20T21:20:40.539 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-09-20T21:20:40.540 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-09-20T21:20:40.540 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-09-20T21:20:40.540 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-09-20T21:20:40.540 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-09-20T21:20:40.540 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-09-20T21:20:40.540 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-09-20T21:20:40.541 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-09-20T21:20:40.541 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-09-20T21:20:40.541 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-09-20T21:20:40.541 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-09-20T21:20:40.541 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-09-20T21:20:40.542 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-09-20T21:20:40.542 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-09-20T21:20:40.542 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-09-20T21:20:40.542 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-09-20T21:20:40.542 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-09-20T21:20:40.542 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-09-20T21:20:40.543 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-09-20T21:20:40.543 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-09-20T21:20:40.543 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-09-20T21:20:40.543 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-09-20T21:20:40.543 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-09-20T21:20:40.544 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-09-20T21:20:40.544 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-09-20T21:20:40.544 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-09-20T21:20:40.544 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-09-20T21:20:40.544 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-09-20T21:20:40.545 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-09-20T21:20:40.545 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-09-20T21:20:40.545 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-09-20T21:20:40.545 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-09-20T21:20:40.545 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-09-20T21:20:40.545 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-09-20T21:20:40.546 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-09-20T21:20:40.546 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-09-20T21:20:40.546 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-09-20T21:20:40.546 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-09-20T21:20:40.546 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-09-20T21:20:40.547 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-09-20T21:20:40.547 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-09-20T21:20:40.547 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-09-20T21:20:40.547 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-09-20T21:20:40.547 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-09-20T21:20:40.547 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-09-20T21:20:40.548 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-09-20T21:20:40.548 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-09-20T21:20:40.548 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-09-20T21:20:40.548 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-09-20T21:20:40.548 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-09-20T21:20:40.549 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-09-20T21:20:40.549 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-09-20T21:20:40.549 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-09-20T21:20:40.549 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-09-20T21:20:40.549 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-09-20T21:20:40.550 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-09-20T21:20:40.550 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-09-20T21:20:40.550 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-09-20T21:20:40.550 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-09-20T21:20:40.550 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-09-20T21:20:40.551 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-09-20T21:20:40.551 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-09-20T21:20:40.551 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-09-20T21:20:40.551 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-09-20T21:20:40.551 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-09-20T21:20:40.552 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-09-20T21:20:40.552 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-09-20T21:20:40.552 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-09-20T21:20:40.552 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-09-20T21:20:40.552 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-09-20T21:20:40.552 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-09-20T21:20:40.553 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-09-20T21:20:40.553 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-09-20T21:20:40.553 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-09-20T21:20:40.553 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-09-20T21:20:40.553 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-09-20T21:20:40.554 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-09-20T21:20:40.554 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-09-20T21:20:40.554 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-09-20T21:20:40.554 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-09-20T21:20:40.554 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-09-20T21:20:40.555 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-09-20T21:20:40.555 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-09-20T21:20:40.555 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-09-20T21:20:40.555 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-09-20T21:20:40.555 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-09-20T21:20:40.556 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-09-20T21:20:40.556 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-09-20T21:20:40.556 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-09-20T21:20:40.556 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-09-20T21:20:40.556 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-09-20T21:20:40.556 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-09-20T21:20:40.557 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-09-20T21:20:40.557 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-09-20T21:20:40.557 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-09-20T21:20:40.557 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-09-20T21:20:40.557 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-09-20T21:20:40.558 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-09-20T21:20:40.558 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-09-20T21:20:40.558 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-09-20T21:20:40.558 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-09-20T21:20:40.558 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-09-20T21:20:40.559 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-09-20T21:20:40.559 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-09-20T21:20:40.559 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-09-20T21:20:40.559 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-09-20T21:20:40.559 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-09-20T21:20:40.559 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-09-20T21:20:40.560 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-09-20T21:20:40.560 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-09-20T21:20:40.560 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-09-20T21:20:40.560 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-09-20T21:20:40.560 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-09-20T21:20:40.561 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-09-20T21:20:40.561 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-09-20T21:20:40.561 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-09-20T21:20:40.561 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-09-20T21:20:40.561 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-09-20T21:20:40.561 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-09-20T21:20:40.562 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-09-20T21:20:40.562 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-09-20T21:20:40.562 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-09-20T21:20:40.562 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-09-20T21:20:40.562 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-09-20T21:20:40.563 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-09-20T21:20:40.563 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-09-20T21:20:40.563 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-09-20T21:20:40.563 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-09-20T21:20:40.563 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-09-20T21:20:40.563 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-09-20T21:20:40.564 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-09-20T21:20:40.564 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-09-20T21:20:40.564 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-09-20T21:20:40.564 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-09-20T21:20:40.564 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-09-20T21:20:40.565 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-09-20T21:20:40.565 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-09-20T21:20:40.565 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-09-20T21:20:40.565 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-09-20T21:20:40.565 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-09-20T21:20:40.565 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-09-20T21:20:40.566 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-09-20T21:20:40.566 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-09-20T21:20:40.566 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-09-20T21:20:40.566 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-09-20T21:20:40.566 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-09-20T21:20:40.567 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-09-20T21:20:40.567 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-09-20T21:20:40.567 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-09-20T21:20:40.567 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-09-20T21:20:40.567 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-09-20T21:20:40.568 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-09-20T21:20:40.568 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-09-20T21:20:40.568 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-09-20T21:20:40.568 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-09-20T21:20:40.568 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-09-20T21:20:40.569 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-09-20T21:20:40.569 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-09-20T21:20:40.569 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-09-20T21:20:40.569 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-09-20T21:20:40.569 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-09-20T21:20:40.569 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-09-20T21:20:40.570 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-09-20T21:20:40.570 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-09-20T21:20:40.570 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-09-20T21:20:40.570 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-09-20T21:20:40.570 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-09-20T21:20:40.571 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-09-20T21:20:40.571 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-09-20T21:20:40.571 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-09-20T21:20:40.571 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-09-20T21:20:40.571 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-09-20T21:20:40.572 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-09-20T21:20:40.572 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-09-20T21:20:40.572 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-09-20T21:20:40.572 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-09-20T21:20:40.572 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-09-20T21:20:40.572 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-09-20T21:20:40.573 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-09-20T21:20:40.573 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-09-20T21:20:40.573 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-09-20T21:20:40.573 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-09-20T21:20:40.573 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-09-20T21:20:40.574 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-09-20T21:20:40.574 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-09-20T21:20:40.574 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-09-20T21:20:40.574 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-09-20T21:20:40.574 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-09-20T21:20:40.574 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-09-20T21:20:40.575 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-09-20T21:20:40.575 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-09-20T21:20:40.575 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-09-20T21:20:40.575 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-09-20T21:20:40.575 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-09-20T21:20:40.576 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-09-20T21:20:40.576 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-09-20T21:20:40.576 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-09-20T21:20:40.576 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-09-20T21:20:40.576 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-09-20T21:20:40.576 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-09-20T21:20:40.577 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-09-20T21:20:40.577 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-09-20T21:20:40.577 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-09-20T21:20:40.577 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-09-20T21:20:40.577 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-09-20T21:20:40.578 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-09-20T21:20:40.578 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-09-20T21:20:40.578 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-09-20T21:20:40.578 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-09-20T21:20:40.578 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-09-20T21:20:40.578 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-09-20T21:20:40.579 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-09-20T21:20:40.579 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-09-20T21:20:40.579 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-09-20T21:20:40.579 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-09-20T21:20:40.579 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-09-20T21:20:40.580 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-09-20T21:20:40.580 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-09-20T21:20:40.580 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-09-20T21:20:40.580 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-09-20T21:20:40.580 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-09-20T21:20:40.580 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-09-20T21:20:40.581 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-09-20T21:20:40.581 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-09-20T21:20:40.581 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-09-20T21:20:40.581 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-09-20T21:20:40.581 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-09-20T21:20:40.582 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-09-20T21:20:40.582 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-09-20T21:20:40.582 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-09-20T21:20:40.582 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-09-20T21:20:40.582 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-09-20T21:20:40.583 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-09-20T21:20:40.583 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-09-20T21:20:40.583 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-09-20T21:20:40.583 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-09-20T21:20:40.583 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-09-20T21:20:40.583 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-09-20T21:20:40.584 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-09-20T21:20:40.584 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-09-20T21:20:40.584 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-09-20T21:20:40.584 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-09-20T21:20:40.584 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-09-20T21:20:40.585 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-09-20T21:20:40.585 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-09-20T21:20:40.585 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-09-20T21:20:40.585 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-09-20T21:20:40.585 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-09-20T21:20:40.585 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-09-20T21:20:40.586 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-09-20T21:20:40.586 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-09-20T21:20:40.586 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-09-20T21:20:40.586 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-09-20T21:20:40.586 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-09-20T21:20:40.587 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-09-20T21:20:40.587 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-09-20T21:20:40.587 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-09-20T21:20:40.587 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-09-20T21:20:40.587 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-09-20T21:20:40.587 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-09-20T21:20:40.588 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-09-20T21:20:40.588 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-09-20T21:20:40.588 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-09-20T21:20:40.588 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-09-20T21:20:40.588 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-09-20T21:20:40.589 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-09-20T21:20:40.589 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-09-20T21:20:40.589 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-09-20T21:20:40.589 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-09-20T21:20:40.589 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-09-20T21:20:40.589 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-09-20T21:20:40.590 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-09-20T21:20:40.590 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-09-20T21:20:40.590 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-09-20T21:20:40.590 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-09-20T21:20:40.590 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-09-20T21:20:40.591 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-09-20T21:20:40.591 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-09-20T21:20:40.591 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-09-20T21:20:40.591 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-09-20T21:20:40.591 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-09-20T21:20:40.591 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-09-20T21:20:40.592 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-09-20T21:20:40.592 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-09-20T21:20:40.592 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-09-20T21:20:40.592 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-09-20T21:20:40.592 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-09-20T21:20:40.593 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-09-20T21:20:40.593 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-09-20T21:20:40.593 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-09-20T21:20:40.593 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-09-20T21:20:40.593 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-09-20T21:20:40.593 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-09-20T21:20:40.594 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-09-20T21:20:40.594 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-09-20T21:20:40.594 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-09-20T21:20:40.594 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-09-20T21:20:40.594 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-09-20T21:20:40.595 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-09-20T21:20:40.595 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-09-20T21:20:40.595 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-09-20T21:20:40.595 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-09-20T21:20:40.595 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-09-20T21:20:40.595 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-09-20T21:20:40.596 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-09-20T21:20:40.596 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-09-20T21:20:40.596 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-09-20T21:20:40.596 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-09-20T21:20:40.596 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-09-20T21:20:40.597 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-09-20T21:20:40.597 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-09-20T21:20:40.597 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-09-20T21:20:40.597 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-09-20T21:20:40.597 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-09-20T21:20:40.597 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-09-20T21:20:40.598 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-09-20T21:20:40.598 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-09-20T21:20:40.598 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-09-20T21:20:40.598 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-09-20T21:20:40.598 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-09-20T21:20:40.599 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-09-20T21:20:40.599 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-09-20T21:20:40.599 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-09-20T21:20:40.599 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-09-20T21:20:40.599 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-09-20T21:20:40.600 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-09-20T21:20:40.600 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-09-20T21:20:40.600 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-09-20T21:20:40.600 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-09-20T21:20:40.600 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-09-20T21:20:40.601 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-09-20T21:20:40.601 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-09-20T21:20:40.601 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-09-20T21:20:40.601 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-09-20T21:20:40.601 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-09-20T21:20:40.601 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-09-20T21:20:40.602 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-09-20T21:20:40.602 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-09-20T21:20:40.602 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-09-20T21:20:40.602 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-09-20T21:20:40.602 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-09-20T21:20:40.603 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-09-20T21:20:40.603 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-09-20T21:20:40.603 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-09-20T21:20:40.603 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-09-20T21:20:40.603 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-09-20T21:20:40.604 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-09-20T21:20:40.604 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-09-20T21:20:40.604 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-09-20T21:20:40.604 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-09-20T21:20:40.604 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-09-20T21:20:40.605 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-09-20T21:20:40.605 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-09-20T21:20:40.605 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-09-20T21:20:40.605 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-09-20T21:20:40.605 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-09-20T21:20:40.605 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-09-20T21:20:40.606 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-09-20T21:20:40.606 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-09-20T21:20:40.606 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-09-20T21:20:40.606 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-09-20T21:20:40.606 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-09-20T21:20:40.607 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-09-20T21:20:40.607 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-09-20T21:20:40.607 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-09-20T21:20:40.607 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-09-20T21:20:40.607 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-09-20T21:20:40.608 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-09-20T21:20:40.608 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-09-20T21:20:40.608 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-09-20T21:20:40.608 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-09-20T21:20:40.608 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-09-20T21:20:40.609 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-09-20T21:20:40.609 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-09-20T21:20:40.609 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-09-20T21:20:40.609 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-09-20T21:20:40.609 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-09-20T21:20:40.609 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-09-20T21:20:40.610 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-09-20T21:20:40.610 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-09-20T21:20:40.610 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-09-20T21:20:40.610 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-09-20T21:20:40.610 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-09-20T21:20:40.611 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-09-20T21:20:40.611 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-09-20T21:20:40.611 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-09-20T21:20:40.611 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-09-20T21:20:40.611 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-09-20T21:20:40.611 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-09-20T21:20:40.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-09-20T21:20:40.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-09-20T21:20:40.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-09-20T21:20:40.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-09-20T21:20:40.612 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-09-20T21:20:40.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-09-20T21:20:40.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-09-20T21:20:40.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-09-20T21:20:40.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-09-20T21:20:40.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-09-20T21:20:40.613 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-09-20T21:20:40.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-09-20T21:20:40.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-09-20T21:20:40.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-09-20T21:20:40.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-09-20T21:20:40.614 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-09-20T21:20:40.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-09-20T21:20:40.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-09-20T21:20:40.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-09-20T21:20:40.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-09-20T21:20:40.615 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-09-20T21:20:40.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-09-20T21:20:40.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-09-20T21:20:40.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-09-20T21:20:40.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-09-20T21:20:40.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-09-20T21:20:40.616 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-09-20T21:20:40.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-09-20T21:20:40.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-09-20T21:20:40.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-09-20T21:20:40.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-09-20T21:20:40.617 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-09-20T21:20:40.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-09-20T21:20:40.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-09-20T21:20:40.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-09-20T21:20:40.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-09-20T21:20:40.618 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-09-20T21:20:40.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-09-20T21:20:40.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-09-20T21:20:40.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-09-20T21:20:40.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-09-20T21:20:40.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-09-20T21:20:40.619 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-09-20T21:20:40.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-09-20T21:20:40.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-09-20T21:20:40.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-09-20T21:20:40.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-09-20T21:20:40.620 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-09-20T21:20:40.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-09-20T21:20:40.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-09-20T21:20:40.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-09-20T21:20:40.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-09-20T21:20:40.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-09-20T21:20:40.621 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-09-20T21:20:40.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-09-20T21:20:40.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-09-20T21:20:40.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-09-20T21:20:40.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-09-20T21:20:40.622 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-09-20T21:20:40.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-09-20T21:20:40.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-09-20T21:20:40.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-09-20T21:20:40.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-09-20T21:20:40.623 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-09-20T21:20:40.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-09-20T21:20:40.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-09-20T21:20:40.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-09-20T21:20:40.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-09-20T21:20:40.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-09-20T21:20:40.624 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-09-20T21:20:40.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-09-20T21:20:40.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-09-20T21:20:40.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-09-20T21:20:40.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-09-20T21:20:40.625 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-09-20T21:20:40.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-09-20T21:20:40.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-09-20T21:20:40.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-09-20T21:20:40.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-09-20T21:20:40.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-09-20T21:20:40.626 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-09-20T21:20:40.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-09-20T21:20:40.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-09-20T21:20:40.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-09-20T21:20:40.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-09-20T21:20:40.627 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-09-20T21:20:40.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-09-20T21:20:40.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-09-20T21:20:40.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-09-20T21:20:40.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-09-20T21:20:40.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-09-20T21:20:40.628 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-09-20T21:20:40.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-09-20T21:20:40.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-09-20T21:20:40.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-09-20T21:20:40.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-09-20T21:20:40.629 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-09-20T21:20:40.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-09-20T21:20:40.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-09-20T21:20:40.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-09-20T21:20:40.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-09-20T21:20:40.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-09-20T21:20:40.630 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-09-20T21:20:40.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-09-20T21:20:40.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-09-20T21:20:40.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-09-20T21:20:40.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-09-20T21:20:40.631 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-09-20T21:20:40.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-09-20T21:20:40.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-09-20T21:20:40.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-09-20T21:20:40.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-09-20T21:20:40.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-09-20T21:20:40.632 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-09-20T21:20:40.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-09-20T21:20:40.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-09-20T21:20:40.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-09-20T21:20:40.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-09-20T21:20:40.633 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-09-20T21:20:40.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-09-20T21:20:40.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-09-20T21:20:40.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-09-20T21:20:40.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-09-20T21:20:40.634 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-09-20T21:20:40.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-09-20T21:20:40.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-09-20T21:20:40.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-09-20T21:20:40.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-09-20T21:20:40.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-09-20T21:20:40.635 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-09-20T21:20:40.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-09-20T21:20:40.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-09-20T21:20:40.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-09-20T21:20:40.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-09-20T21:20:40.636 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-09-20T21:20:40.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-09-20T21:20:40.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-09-20T21:20:40.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-09-20T21:20:40.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-09-20T21:20:40.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-09-20T21:20:40.637 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-09-20T21:20:40.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-09-20T21:20:40.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-09-20T21:20:40.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-09-20T21:20:40.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-09-20T21:20:40.638 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-09-20T21:20:40.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-09-20T21:20:40.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-09-20T21:20:40.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-09-20T21:20:40.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-09-20T21:20:40.639 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-09-20T21:20:40.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-09-20T21:20:40.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-09-20T21:20:40.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-09-20T21:20:40.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-09-20T21:20:40.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-09-20T21:20:40.640 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-09-20T21:20:40.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-09-20T21:20:40.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-09-20T21:20:40.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-09-20T21:20:40.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-09-20T21:20:40.641 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-09-20T21:20:40.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-09-20T21:20:40.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-09-20T21:20:40.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-09-20T21:20:40.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-09-20T21:20:40.642 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-09-20T21:20:40.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-09-20T21:20:40.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-09-20T21:20:40.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-09-20T21:20:40.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-09-20T21:20:40.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-09-20T21:20:40.643 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-09-20T21:20:40.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-09-20T21:20:40.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-09-20T21:20:40.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-09-20T21:20:40.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-09-20T21:20:40.644 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-09-20T21:20:40.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-09-20T21:20:40.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-09-20T21:20:40.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-09-20T21:20:40.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-09-20T21:20:40.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-09-20T21:20:40.645 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-09-20T21:20:40.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-09-20T21:20:40.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-09-20T21:20:40.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-09-20T21:20:40.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-09-20T21:20:40.646 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-09-20T21:20:40.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-09-20T21:20:40.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-09-20T21:20:40.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-09-20T21:20:40.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-09-20T21:20:40.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-09-20T21:20:40.647 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-09-20T21:20:40.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-09-20T21:20:40.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-09-20T21:20:40.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-09-20T21:20:40.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-09-20T21:20:40.648 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-09-20T21:20:40.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-09-20T21:20:40.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-09-20T21:20:40.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-09-20T21:20:40.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-09-20T21:20:40.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-09-20T21:20:40.649 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-09-20T21:20:40.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-09-20T21:20:40.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-09-20T21:20:40.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-09-20T21:20:40.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-09-20T21:20:40.650 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-09-20T21:20:40.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-09-20T21:20:40.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-09-20T21:20:40.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-09-20T21:20:40.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-09-20T21:20:40.651 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-09-20T21:20:40.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-09-20T21:20:40.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-09-20T21:20:40.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-09-20T21:20:40.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-09-20T21:20:40.652 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-09-20T21:20:40.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-09-20T21:20:40.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-09-20T21:20:40.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-09-20T21:20:40.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-09-20T21:20:40.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-09-20T21:20:40.653 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-09-20T21:20:40.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-09-20T21:20:40.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-09-20T21:20:40.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-09-20T21:20:40.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-09-20T21:20:40.654 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-09-20T21:20:40.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-09-20T21:20:40.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-09-20T21:20:40.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-09-20T21:20:40.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-09-20T21:20:40.655 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-09-20T21:20:40.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-09-20T21:20:40.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-09-20T21:20:40.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-09-20T21:20:40.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-09-20T21:20:40.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-09-20T21:20:40.656 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-09-20T21:20:40.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-09-20T21:20:40.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-09-20T21:20:40.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-09-20T21:20:40.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-09-20T21:20:40.657 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-09-20T21:20:40.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-09-20T21:20:40.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-09-20T21:20:40.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-09-20T21:20:40.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-09-20T21:20:40.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-09-20T21:20:40.658 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-09-20T21:20:40.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-09-20T21:20:40.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-09-20T21:20:40.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-09-20T21:20:40.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-09-20T21:20:40.659 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-09-20T21:20:40.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-09-20T21:20:40.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-09-20T21:20:40.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-09-20T21:20:40.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-09-20T21:20:40.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-09-20T21:20:40.660 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-09-20T21:20:40.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-09-20T21:20:40.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-09-20T21:20:40.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-09-20T21:20:40.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-09-20T21:20:40.661 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-09-20T21:20:40.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-09-20T21:20:40.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-09-20T21:20:40.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-09-20T21:20:40.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-09-20T21:20:40.662 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-09-20T21:20:40.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-09-20T21:20:40.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-09-20T21:20:40.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-09-20T21:20:40.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-09-20T21:20:40.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-09-20T21:20:40.663 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-09-20T21:20:40.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-09-20T21:20:40.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-09-20T21:20:40.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-09-20T21:20:40.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-09-20T21:20:40.664 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-09-20T21:20:40.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-09-20T21:20:40.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-09-20T21:20:40.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-09-20T21:20:40.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-09-20T21:20:40.665 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-09-20T21:20:40.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-09-20T21:20:40.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-09-20T21:20:40.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-09-20T21:20:40.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-09-20T21:20:40.666 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-09-20T21:20:40.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-09-20T21:20:40.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-09-20T21:20:40.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-09-20T21:20:40.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-09-20T21:20:40.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-09-20T21:20:40.667 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-09-20T21:20:40.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-09-20T21:20:40.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-09-20T21:20:40.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-09-20T21:20:40.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-09-20T21:20:40.668 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-09-20T21:20:40.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-09-20T21:20:40.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-09-20T21:20:40.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-09-20T21:20:40.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-09-20T21:20:40.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-09-20T21:20:40.669 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-09-20T21:20:40.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-09-20T21:20:40.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-09-20T21:20:40.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-09-20T21:20:40.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-09-20T21:20:40.670 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-09-20T21:20:40.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-09-20T21:20:40.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-09-20T21:20:40.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-09-20T21:20:40.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-09-20T21:20:40.671 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-09-20T21:20:40.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-09-20T21:20:40.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-09-20T21:20:40.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-09-20T21:20:40.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-09-20T21:20:40.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-09-20T21:20:40.672 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-09-20T21:20:40.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-09-20T21:20:40.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-09-20T21:20:40.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-09-20T21:20:40.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-09-20T21:20:40.673 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-09-20T21:20:40.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-09-20T21:20:40.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-09-20T21:20:40.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-09-20T21:20:40.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-09-20T21:20:40.674 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-09-20T21:20:40.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-09-20T21:20:40.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-09-20T21:20:40.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-09-20T21:20:40.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-09-20T21:20:40.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-09-20T21:20:40.675 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-09-20T21:20:40.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-09-20T21:20:40.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-09-20T21:20:40.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-09-20T21:20:40.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-09-20T21:20:40.676 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-09-20T21:20:40.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-09-20T21:20:40.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-09-20T21:20:40.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-09-20T21:20:40.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-09-20T21:20:40.677 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-09-20T21:20:40.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-09-20T21:20:40.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-09-20T21:20:40.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-09-20T21:20:40.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-09-20T21:20:40.678 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-09-20T21:20:40.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-09-20T21:20:40.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-09-20T21:20:40.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-09-20T21:20:40.679 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-09-20T21:20:40.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-09-20T21:20:40.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-09-20T21:20:40.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-09-20T21:20:40.681 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-09-20T21:20:40.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-09-20T21:20:40.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-09-20T21:20:40.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-09-20T21:20:40.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-09-20T21:20:40.682 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-09-20T21:20:40.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-09-20T21:20:40.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-09-20T21:20:40.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-09-20T21:20:40.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-09-20T21:20:40.683 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-09-20T21:20:40.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-09-20T21:20:40.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-09-20T21:20:40.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-09-20T21:20:40.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-09-20T21:20:40.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-09-20T21:20:40.684 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-09-20T21:20:40.685 INFO:teuthology.orchestra.run.smithi022.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-09-20T21:20:40.685 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-09-20T21:20:40.686 DEBUG:teuthology.orchestra.run.smithi022:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-09-20T21:20:40.714 INFO:teuthology.orchestra.run.smithi022.stdout:postinstall scriptlet (using /bin/sh): 2023-09-20T21:20:40.714 INFO:teuthology.orchestra.run.smithi022.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-09-20T21:20:40.714 INFO:teuthology.orchestra.run.smithi022.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-20T21:20:40.715 INFO:teuthology.orchestra.run.smithi022.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-20T21:20:40.715 INFO:teuthology.orchestra.run.smithi022.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-20T21:20:40.715 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-20T21:20:40.715 INFO:teuthology.orchestra.run.smithi022.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-20T21:20:40.715 INFO:teuthology.orchestra.run.smithi022.stdout:fi 2023-09-20T21:20:40.716 INFO:teuthology.orchestra.run.smithi022.stdout:preuninstall scriptlet (using /bin/sh): 2023-09-20T21:20:40.716 INFO:teuthology.orchestra.run.smithi022.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-09-20T21:20:40.716 INFO:teuthology.orchestra.run.smithi022.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-09-20T21:20:40.716 INFO:teuthology.orchestra.run.smithi022.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-09-20T21:20:40.716 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-09-20T21:20:40.717 INFO:teuthology.orchestra.run.smithi022.stdout:fi 2023-09-20T21:20:40.717 INFO:teuthology.orchestra.run.smithi022.stdout:postuninstall scriptlet (using /bin/sh): 2023-09-20T21:20:40.717 INFO:teuthology.orchestra.run.smithi022.stdout:if [ -x /sbin/update-bootloader ]; then 2023-09-20T21:20:40.717 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-09-20T21:20:40.717 INFO:teuthology.orchestra.run.smithi022.stdout:fi 2023-09-20T21:20:40.718 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2023-09-20T21:20:40.798 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2023-09-20T21:20:40.799 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2023-09-20T21:20:40.799 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.02 2023-09-20T21:20:40.799 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 123.el8 2023-09-20T21:20:40.799 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2023-09-20T21:20:40.799 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-09-20T21:20:40.800 INFO:teuthology.orchestra.run.smithi022.stdout:Group : System Environment/Base 2023-09-20T21:20:40.800 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 9265450 2023-09-20T21:20:40.800 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2023-09-20T21:20:40.800 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-09-20T21:20:40.800 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-09-20T21:20:40.801 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-09-20T21:20:40.801 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-09-20T21:20:40.801 INFO:teuthology.orchestra.run.smithi022.stdout:Relocations : (not relocatable) 2023-09-20T21:20:40.801 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : Red Hat, Inc. 2023-09-20T21:20:40.801 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : Red Hat, Inc. 2023-09-20T21:20:40.802 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-20T21:20:40.802 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2023-09-20T21:20:40.802 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2023-09-20T21:20:40.802 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:20:40.802 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-20T21:20:40.803 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-20T21:20:40.803 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-20T21:20:40.803 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2023-09-20T21:20:40.803 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:20:40.803 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2023-09-20T21:20:40.804 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-20T21:20:40.805 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 6.5.0-rc7-gfef64f93da2b 2023-09-20T21:20:40.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-20T21:20:42.904 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2023-09-20T21:20:45.067 INFO:teuthology.orchestra.run.smithi022.stderr:done 2023-09-20T21:20:45.069 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2023-09-20T21:20:45.086 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.4P18UJFqKW 2023-09-20T21:20:45.087 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4P18UJFqKW 2023-09-20T21:20:45.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.4P18UJFqKW 2023-09-20T21:20:45.316 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.4P18UJFqKW is 6KB 2023-09-20T21:20:45.366 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.4P18UJFqKW 2023-09-20T21:20:45.382 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2023-09-20T21:20:45.461 INFO:teuthology.orchestra.run.smithi022.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-09-20T21:20:45.461 INFO:teuthology.orchestra.run.smithi022.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-09-20T21:20:45.462 INFO:teuthology.orchestra.run.smithi022.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b.conf 2023-09-20T21:20:45.463 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc7-gfef64f93da2b 2023-09-20T21:20:45.644 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2023-09-20T21:21:15.646 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-09-20T21:21:15.647 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-09-20T21:21:15.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-20T21:22:15.651 DEBUG:teuthology.orchestra.remote:timed out 2023-09-20T21:22:24.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-09-20T21:22:24.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-20T21:22:25.047 DEBUG:teuthology.orchestra.run.smithi022:> true 2023-09-20T21:22:25.552 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-09-20T21:22:25.552 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-09-20T21:22:25.553 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2023-09-20T21:22:25.594 INFO:teuthology.orchestra.run.smithi022.stdout:6.5.0-rc7-gfef64f93da2b 2023-09-20T21:22:25.595 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-09-20T21:22:25.595 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-20T21:22:25.595 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-09-20T21:22:26.596 DEBUG:teuthology.parallel:result is None 2023-09-20T21:22:26.596 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-20T21:22:26.604 INFO:teuthology.task.internal:Creating test directory... 2023-09-20T21:22:26.604 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-20T21:22:26.609 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-20T21:22:26.612 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-20T21:22:26.647 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-20T21:22:26.654 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-20T21:22:26.663 INFO:teuthology.task.internal:Creating archive directory... 2023-09-20T21:22:26.663 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-20T21:22:26.696 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-20T21:22:26.698 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-20T21:22:26.761 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-20T21:22:26.770 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-20T21:22:26.770 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-20T21:22:26.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T21:22:26.789 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-20T21:22:26.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T21:22:26.809 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-20T21:22:26.813 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-20T21:22:26.831 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-20T21:22:26.867 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-20T21:22:26.885 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-20T21:22:26.886 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-20T21:22:26.903 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-20T21:22:26.944 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-20T21:22:26.984 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-20T21:22:26.990 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-20T21:22:26.997 INFO:teuthology.task.internal:Configuring sudo... 2023-09-20T21:22:26.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-20T21:22:27.001 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-20T21:22:27.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-20T21:22:27.070 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-20T21:22:27.080 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-20T21:22:27.080 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-20T21:22:27.086 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-20T21:22:27.088 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-20T21:22:27.126 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-20T21:22:27.243 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-20T21:22:27.332 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-20T21:22:27.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-20T21:22:27.496 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:22:27.496 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-20T21:22:27.575 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-20T21:22:27.621 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-20T21:22:27.679 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-20T21:22:27.763 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-20T21:22:27.807 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:22:27.808 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-20T21:22:27.894 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-20T21:22:27.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-20T21:22:27.996 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-20T21:22:28.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-20T21:22:28.128 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:22:28.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-20T21:22:28.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2023-09-20T21:22:28.215 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2023-09-20T21:22:28.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2023-09-20T21:22:28.290 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-20T21:22:28.320 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-20T21:22:28.336 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-20T21:22:28.848 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-20T21:22:28.858 INFO:teuthology.task.internal:Starting timer... 2023-09-20T21:22:28.858 INFO:teuthology.run_tasks:Running task pcp... 2023-09-20T21:22:28.896 INFO:teuthology.run_tasks:Running task selinux... 2023-09-20T21:22:28.910 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-09-20T21:22:28.911 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2023-09-20T21:22:29.033 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2023-09-20T21:22:29.037 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2023-09-20T21:22:29.093 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2023-09-20T21:22:29.095 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2023-09-20T21:22:29.156 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2023-09-20T21:22:29.157 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-20T21:22:29.158 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2023-09-20T21:22:29.211 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2023-09-20T21:22:29.211 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2023-09-20T21:22:29.234 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2023-09-20T21:22:29.235 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2023-09-20T21:22:29.258 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2023-09-20T21:22:29.258 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2023-09-20T21:22:29.259 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-20T21:22:29.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T21:22:29.329 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-20T21:22:29.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T21:22:29.356 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-20T21:22:29.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T21:22:29.384 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-20T21:22:29.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2023-09-20T21:22:29.453 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2023-09-20T21:22:29.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2023-09-20T21:22:29.508 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-20T21:22:29.518 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-09-20T21:22:29.525 INFO:teuthology.repo_utils:Fetching main from origin 2023-09-20T21:22:29.649 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-20T21:22:29.666 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-20T21:22:29.667 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi089.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-20T21:34:50.158 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2023-09-20T21:34:50.179 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-09-20T21:34:50.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-20T21:34:50.269 DEBUG:teuthology.orchestra.run.smithi022:> true 2023-09-20T21:34:50.380 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-09-20T21:34:50.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-09-20T21:34:50.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-20T21:34:50.449 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-09-20T21:34:50.517 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-09-20T21:34:50.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-20T21:34:50.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-20T21:34:50.586 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-09-20T21:34:50.612 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-09-20T21:34:50.612 INFO:teuthology.run_tasks:Running task clock... 2023-09-20T21:34:50.625 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-20T21:34:50.626 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-20T21:34:50.627 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-20T21:34:50.631 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-20T21:34:50.632 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-20T21:34:50.634 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-20T21:34:50.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-20T21:34:50.667 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-20T21:34:50.683 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-20T21:34:50.684 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-20T21:34:50.699 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-20T21:34:50.703 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-20T21:34:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-20T21:34:50.746 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2023-09-20T21:34:50.759 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2023-09-20T21:34:50.774 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-20T21:34:50.775 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2023-09-20T21:34:50.787 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2023-09-20T21:34:50.788 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-20T21:34:50.801 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-20T21:34:50.814 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-20T21:34:50.832 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2023-09-20T21:34:50.862 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2023-09-20T21:34:50.863 INFO:teuthology.orchestra.run.smithi089.stderr:bash: ntpq: command not found 2023-09-20T21:34:50.865 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-20T21:34:50.866 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2023-09-20T21:34:50.866 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:50.866 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:50.866 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:50.867 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:50.895 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-09-20T21:34:50.897 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-20T21:34:50.898 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-20T21:34:50.898 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-09-20T21:34:50.898 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:50.898 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:50.899 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:50.899 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:50.932 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-20T21:34:51.040 INFO:teuthology.orchestra.run.smithi022.stderr:bash: ntpq: command not found 2023-09-20T21:34:51.048 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-20T21:34:51.048 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2023-09-20T21:34:51.048 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:51.049 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:51.049 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:51.049 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T21:34:51.051 INFO:teuthology.run_tasks:Running task pexec... 2023-09-20T21:34:51.061 INFO:teuthology.task.pexec:Executing custom commands... 2023-09-20T21:34:51.062 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi022.front.sepia.ceph.com 2023-09-20T21:34:51.062 DEBUG:teuthology.orchestra.run.smithi022:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-20T21:34:51.063 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi089.front.sepia.ceph.com 2023-09-20T21:34:51.063 DEBUG:teuthology.orchestra.run.smithi089:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-20T21:34:51.064 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi123.front.sepia.ceph.com 2023-09-20T21:34:51.064 DEBUG:teuthology.orchestra.run.smithi123:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-09-20T21:34:51.434 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-09-20T21:34:51.435 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-09-20T21:34:51.583 INFO:teuthology.orchestra.run.smithi022.stdout:Updating Subscription Management repositories. 2023-09-20T21:34:52.260 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:57 ago on Wed 20 Sep 2023 09:29:54 PM UTC. 2023-09-20T21:34:52.269 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:04:02 ago on Wed 20 Sep 2023 09:30:49 PM UTC. 2023-09-20T21:34:52.541 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:48 ago on Wed 20 Sep 2023 09:33:04 PM UTC. 2023-09-20T21:34:53.977 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-09-20T21:34:53.978 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-09-20T21:34:53.978 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2023-09-20T21:34:53.978 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-09-20T21:34:53.979 INFO:teuthology.orchestra.run.smithi123.stdout:Resetting modules: 2023-09-20T21:34:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: container-tools 2023-09-20T21:34:53.979 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:34:53.979 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-09-20T21:34:53.979 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-09-20T21:34:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:34:54.140 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-09-20T21:34:54.141 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-09-20T21:34:54.141 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2023-09-20T21:34:54.141 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-09-20T21:34:54.142 INFO:teuthology.orchestra.run.smithi089.stdout:Resetting modules: 2023-09-20T21:34:54.142 INFO:teuthology.orchestra.run.smithi089.stdout: container-tools 2023-09-20T21:34:54.142 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:34:54.142 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-09-20T21:34:54.142 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2023-09-20T21:34:54.143 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:34:54.532 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2023-09-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2023-09-20T21:34:54.533 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2023-09-20T21:34:54.534 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2023-09-20T21:34:54.534 INFO:teuthology.orchestra.run.smithi022.stdout:Resetting modules: 2023-09-20T21:34:54.534 INFO:teuthology.orchestra.run.smithi022.stdout: container-tools 2023-09-20T21:34:54.534 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:34:54.535 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2023-09-20T21:34:54.535 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2023-09-20T21:34:54.535 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:34:54.942 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:05:00 ago on Wed 20 Sep 2023 09:29:54 PM UTC. 2023-09-20T21:34:55.100 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:04:05 ago on Wed 20 Sep 2023 09:30:49 PM UTC. 2023-09-20T21:34:55.823 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:51 ago on Wed 20 Sep 2023 09:33:04 PM UTC. 2023-09-20T21:34:57.785 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-09-20T21:34:58.076 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-09-20T21:34:58.244 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-09-20T21:34:58.455 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2023-09-20T21:34:58.785 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-09-20T21:34:59.047 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:05:04 ago on Wed 20 Sep 2023 09:29:54 PM UTC. 2023-09-20T21:34:59.094 INFO:teuthology.orchestra.run.smithi022.stdout:Updating Subscription Management repositories. 2023-09-20T21:34:59.661 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:04:10 ago on Wed 20 Sep 2023 09:30:49 PM UTC. 2023-09-20T21:35:00.064 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:55 ago on Wed 20 Sep 2023 09:33:04 PM UTC. 2023-09-20T21:35:00.913 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-09-20T21:35:01.004 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================================================ 2023-09-20T21:35:01.005 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2023-09-20T21:35:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================================================ 2023-09-20T21:35:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:Installing group/module packages: 2023-09-20T21:35:01.005 INFO:teuthology.orchestra.run.smithi123.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-20T21:35:01.006 INFO:teuthology.orchestra.run.smithi123.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-20T21:35:01.006 INFO:teuthology.orchestra.run.smithi123.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-20T21:35:01.006 INFO:teuthology.orchestra.run.smithi123.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-20T21:35:01.006 INFO:teuthology.orchestra.run.smithi123.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-20T21:35:01.006 INFO:teuthology.orchestra.run.smithi123.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-20T21:35:01.007 INFO:teuthology.orchestra.run.smithi123.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-20T21:35:01.007 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2023-09-20T21:35:01.007 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-20T21:35:01.007 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-20T21:35:01.007 INFO:teuthology.orchestra.run.smithi123.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-20T21:35:01.008 INFO:teuthology.orchestra.run.smithi123.stdout:Installing module profiles: 2023-09-20T21:35:01.008 INFO:teuthology.orchestra.run.smithi123.stdout: container-tools/common 2023-09-20T21:35:01.008 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:35:01.008 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling module streams: 2023-09-20T21:35:01.008 INFO:teuthology.orchestra.run.smithi123.stdout: container-tools rhel8 2023-09-20T21:35:01.009 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:35:01.009 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-09-20T21:35:01.009 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================================================ 2023-09-20T21:35:01.009 INFO:teuthology.orchestra.run.smithi123.stdout:Install 10 Packages 2023-09-20T21:35:01.009 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:35:01.010 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 18 M 2023-09-20T21:35:01.010 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 65 M 2023-09-20T21:35:01.011 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2023-09-20T21:35:01.090 INFO:teuthology.orchestra.run.smithi123.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.1 MB/s | 94 kB 00:00 2023-09-20T21:35:01.108 INFO:teuthology.orchestra.run.smithi123.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 246 kB/s | 25 kB 00:00 2023-09-20T21:35:01.124 INFO:teuthology.orchestra.run.smithi123.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 347 kB/s | 41 kB 00:00 2023-09-20T21:35:01.142 INFO:teuthology.orchestra.run.smithi123.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 973 kB/s | 49 kB 00:00 2023-09-20T21:35:01.175 INFO:teuthology.orchestra.run.smithi123.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 7.6 MB/s | 524 kB 00:00 2023-09-20T21:35:01.209 INFO:teuthology.orchestra.run.smithi123.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-09-20T21:35:01.234 INFO:teuthology.orchestra.run.smithi123.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-09-20T21:35:01.342 INFO:teuthology.orchestra.run.smithi123.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-09-20T21:35:01.370 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-09-20T21:35:01.435 INFO:teuthology.orchestra.run.smithi123.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 26 MB/s | 8.0 MB 00:00 2023-09-20T21:35:01.460 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================================================ 2023-09-20T21:35:01.461 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-09-20T21:35:01.461 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================================================ 2023-09-20T21:35:01.461 INFO:teuthology.orchestra.run.smithi089.stdout:Installing group/module packages: 2023-09-20T21:35:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-20T21:35:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-20T21:35:01.462 INFO:teuthology.orchestra.run.smithi089.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-20T21:35:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-20T21:35:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-20T21:35:01.463 INFO:teuthology.orchestra.run.smithi089.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-20T21:35:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-20T21:35:01.464 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-09-20T21:35:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-20T21:35:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-20T21:35:01.464 INFO:teuthology.orchestra.run.smithi089.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-20T21:35:01.465 INFO:teuthology.orchestra.run.smithi089.stdout:Installing module profiles: 2023-09-20T21:35:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: container-tools/common 2023-09-20T21:35:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:35:01.465 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling module streams: 2023-09-20T21:35:01.465 INFO:teuthology.orchestra.run.smithi089.stdout: container-tools rhel8 2023-09-20T21:35:01.466 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:35:01.466 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-09-20T21:35:01.466 INFO:teuthology.orchestra.run.smithi089.stdout:============================================================================================================ 2023-09-20T21:35:01.466 INFO:teuthology.orchestra.run.smithi089.stdout:Install 10 Packages 2023-09-20T21:35:01.467 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:35:01.467 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 18 M 2023-09-20T21:35:01.468 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 65 M 2023-09-20T21:35:01.468 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-09-20T21:35:01.527 INFO:teuthology.orchestra.run.smithi123.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 22 MB/s | 7.0 MB 00:00 2023-09-20T21:35:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2023-09-20T21:35:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:Total 35 MB/s | 18 MB 00:00 2023-09-20T21:35:01.536 INFO:teuthology.orchestra.run.smithi089.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-09-20T21:35:01.561 INFO:teuthology.orchestra.run.smithi089.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 252 kB/s | 25 kB 00:00 2023-09-20T21:35:01.578 INFO:teuthology.orchestra.run.smithi089.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 353 kB/s | 41 kB 00:00 2023-09-20T21:35:01.595 INFO:teuthology.orchestra.run.smithi089.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 834 kB/s | 49 kB 00:00 2023-09-20T21:35:01.629 INFO:teuthology.orchestra.run.smithi089.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 7.6 MB/s | 524 kB 00:00 2023-09-20T21:35:01.645 INFO:teuthology.orchestra.run.smithi089.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.9 MB/s | 149 kB 00:00 2023-09-20T21:35:01.679 INFO:teuthology.orchestra.run.smithi089.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-09-20T21:35:01.726 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2023-09-20T21:35:01.745 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2023-09-20T21:35:01.746 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2023-09-20T21:35:01.780 INFO:teuthology.orchestra.run.smithi089.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-09-20T21:35:01.896 INFO:teuthology.orchestra.run.smithi089.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-09-20T21:35:01.927 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2023-09-20T21:35:01.931 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2023-09-20T21:35:02.013 INFO:teuthology.orchestra.run.smithi089.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-09-20T21:35:02.014 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-09-20T21:35:02.015 INFO:teuthology.orchestra.run.smithi089.stdout:Total 33 MB/s | 18 MB 00:00 2023-09-20T21:35:02.143 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2023-09-20T21:35:02.212 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-09-20T21:35:02.231 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-09-20T21:35:02.231 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-09-20T21:35:02.250 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================================ 2023-09-20T21:35:02.250 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2023-09-20T21:35:02.250 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================================ 2023-09-20T21:35:02.250 INFO:teuthology.orchestra.run.smithi022.stdout:Installing group/module packages: 2023-09-20T21:35:02.250 INFO:teuthology.orchestra.run.smithi022.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-09-20T21:35:02.251 INFO:teuthology.orchestra.run.smithi022.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-09-20T21:35:02.251 INFO:teuthology.orchestra.run.smithi022.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-09-20T21:35:02.251 INFO:teuthology.orchestra.run.smithi022.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-09-20T21:35:02.251 INFO:teuthology.orchestra.run.smithi022.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-09-20T21:35:02.251 INFO:teuthology.orchestra.run.smithi022.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-09-20T21:35:02.252 INFO:teuthology.orchestra.run.smithi022.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-09-20T21:35:02.252 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2023-09-20T21:35:02.252 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-09-20T21:35:02.252 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-09-20T21:35:02.252 INFO:teuthology.orchestra.run.smithi022.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-09-20T21:35:02.252 INFO:teuthology.orchestra.run.smithi022.stdout:Installing module profiles: 2023-09-20T21:35:02.253 INFO:teuthology.orchestra.run.smithi022.stdout: container-tools/common 2023-09-20T21:35:02.253 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:35:02.253 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling module streams: 2023-09-20T21:35:02.253 INFO:teuthology.orchestra.run.smithi022.stdout: container-tools rhel8 2023-09-20T21:35:02.253 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:35:02.254 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2023-09-20T21:35:02.254 INFO:teuthology.orchestra.run.smithi022.stdout:============================================================================================================ 2023-09-20T21:35:02.254 INFO:teuthology.orchestra.run.smithi022.stdout:Install 10 Packages 2023-09-20T21:35:02.254 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:35:02.255 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 18 M 2023-09-20T21:35:02.255 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 65 M 2023-09-20T21:35:02.255 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2023-09-20T21:35:02.326 INFO:teuthology.orchestra.run.smithi022.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-09-20T21:35:02.351 INFO:teuthology.orchestra.run.smithi022.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 255 kB/s | 25 kB 00:00 2023-09-20T21:35:02.376 INFO:teuthology.orchestra.run.smithi022.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 332 kB/s | 41 kB 00:00 2023-09-20T21:35:02.402 INFO:teuthology.orchestra.run.smithi022.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-09-20T21:35:02.412 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-09-20T21:35:02.416 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-09-20T21:35:02.435 INFO:teuthology.orchestra.run.smithi022.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-09-20T21:35:02.451 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2023-09-20T21:35:02.460 INFO:teuthology.orchestra.run.smithi022.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-09-20T21:35:02.494 INFO:teuthology.orchestra.run.smithi022.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.5 MB/s | 213 kB 00:00 2023-09-20T21:35:02.620 INFO:teuthology.orchestra.run.smithi022.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 18 MB/s | 2.2 MB 00:00 2023-09-20T21:35:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-20T21:35:02.787 INFO:teuthology.orchestra.run.smithi022.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 22 MB/s | 7.0 MB 00:00 2023-09-20T21:35:02.823 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-20T21:35:02.887 INFO:teuthology.orchestra.run.smithi022.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 16 MB/s | 8.0 MB 00:00 2023-09-20T21:35:02.887 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2023-09-20T21:35:02.888 INFO:teuthology.orchestra.run.smithi022.stdout:Total 29 MB/s | 18 MB 00:00 2023-09-20T21:35:02.915 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-09-20T21:35:03.017 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-20T21:35:03.082 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-20T21:35:03.203 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-20T21:35:03.230 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2023-09-20T21:35:03.254 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2023-09-20T21:35:03.254 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2023-09-20T21:35:03.255 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-20T21:35:03.431 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-20T21:35:03.593 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-20T21:35:03.626 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-20T21:35:03.758 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2023-09-20T21:35:03.763 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2023-09-20T21:35:03.974 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-20T21:35:04.327 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2023-09-20T21:35:04.363 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-20T21:35:04.508 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-09-20T21:35:04.686 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-20T21:35:04.719 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-20T21:35:04.949 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-09-20T21:35:05.168 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-09-20T21:35:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-20T21:35:05.484 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-20T21:35:05.574 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-09-20T21:35:05.576 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-20T21:35:05.647 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-20T21:35:05.816 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-20T21:35:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-20T21:35:05.978 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-20T21:35:06.110 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-20T21:35:06.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-20T21:35:06.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-20T21:35:06.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-20T21:35:06.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-20T21:35:06.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-20T21:35:06.111 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-20T21:35:06.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-20T21:35:06.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-20T21:35:06.112 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-20T21:35:06.155 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-20T21:35:06.420 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-20T21:35:06.420 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-20T21:35:06.421 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-20T21:35:06.421 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-20T21:35:06.421 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-20T21:35:06.421 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-20T21:35:06.421 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-20T21:35:06.422 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-20T21:35:06.422 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-20T21:35:06.422 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-20T21:35:06.432 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-09-20T21:35:07.286 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-20T21:35:07.286 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:05:12 ago on Wed 20 Sep 2023 09:29:54 PM UTC. 2023-09-20T21:35:07.437 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-09-20T21:35:07.576 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-20T21:35:07.577 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:04:18 ago on Wed 20 Sep 2023 09:30:49 PM UTC. 2023-09-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-09-20T21:35:07.889 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-09-20T21:35:08.081 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-20T21:35:08.485 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-09-20T21:35:08.485 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-09-20T21:35:08.486 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-09-20T21:35:08.486 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-09-20T21:35:08.486 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-09-20T21:35:08.487 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-09-20T21:35:08.487 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-09-20T21:35:08.487 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-09-20T21:35:08.488 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-09-20T21:35:08.488 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-09-20T21:35:09.895 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-09-20T21:35:09.895 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:02:05 ago on Wed 20 Sep 2023 09:33:04 PM UTC. 2023-09-20T21:35:10.362 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2023-09-20T21:35:10.593 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:35:10.593 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2023-09-20T21:35:10.593 INFO:teuthology.orchestra.run.smithi123.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:10.594 INFO:teuthology.orchestra.run.smithi123.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-20T21:35:10.594 INFO:teuthology.orchestra.run.smithi123.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:10.594 INFO:teuthology.orchestra.run.smithi123.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-20T21:35:10.594 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-20T21:35:10.594 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-20T21:35:10.595 INFO:teuthology.orchestra.run.smithi123.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:10.595 INFO:teuthology.orchestra.run.smithi123.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:10.595 INFO:teuthology.orchestra.run.smithi123.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-20T21:35:10.595 INFO:teuthology.orchestra.run.smithi123.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-20T21:35:10.595 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:35:10.596 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-09-20T21:35:10.613 INFO:teuthology.orchestra.run.smithi089.stdout:Installed products updated. 2023-09-20T21:35:10.810 DEBUG:teuthology.parallel:result is None 2023-09-20T21:35:10.830 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:35:10.831 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-09-20T21:35:10.831 INFO:teuthology.orchestra.run.smithi089.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:10.831 INFO:teuthology.orchestra.run.smithi089.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-20T21:35:10.831 INFO:teuthology.orchestra.run.smithi089.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:10.831 INFO:teuthology.orchestra.run.smithi089.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-20T21:35:10.832 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-20T21:35:10.832 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-20T21:35:10.832 INFO:teuthology.orchestra.run.smithi089.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:10.832 INFO:teuthology.orchestra.run.smithi089.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:10.832 INFO:teuthology.orchestra.run.smithi089.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-20T21:35:10.833 INFO:teuthology.orchestra.run.smithi089.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-20T21:35:10.833 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:35:10.833 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-09-20T21:35:11.006 DEBUG:teuthology.parallel:result is None 2023-09-20T21:35:12.984 INFO:teuthology.orchestra.run.smithi022.stdout:Installed products updated. 2023-09-20T21:35:13.205 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:35:13.205 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2023-09-20T21:35:13.206 INFO:teuthology.orchestra.run.smithi022.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:13.206 INFO:teuthology.orchestra.run.smithi022.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-09-20T21:35:13.206 INFO:teuthology.orchestra.run.smithi022.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:13.206 INFO:teuthology.orchestra.run.smithi022.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-09-20T21:35:13.206 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-09-20T21:35:13.207 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-09-20T21:35:13.207 INFO:teuthology.orchestra.run.smithi022.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:13.207 INFO:teuthology.orchestra.run.smithi022.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-09-20T21:35:13.207 INFO:teuthology.orchestra.run.smithi022.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-09-20T21:35:13.207 INFO:teuthology.orchestra.run.smithi022.stdout: yajl-2.1.0-10.el8.x86_64 2023-09-20T21:35:13.208 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:35:13.208 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2023-09-20T21:35:13.420 DEBUG:teuthology.parallel:result is None 2023-09-20T21:35:13.421 INFO:teuthology.run_tasks:Running task install... 2023-09-20T21:35:13.431 DEBUG:teuthology.task.install:project ceph 2023-09-20T21:35:13.432 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bf8a061ff16ac1d393cf108f740d92dfa49f2e54'}} 2023-09-20T21:35:13.432 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bf8a061ff16ac1d393cf108f740d92dfa49f2e54'} 2023-09-20T21:35:13.433 INFO:teuthology.task.install:Using flavor: default 2023-09-20T21:35:13.440 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-20T21:35:13.440 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-20T21:35:13.441 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bf8a061ff16ac1d393cf108f740d92dfa49f2e54', 'tag': None, 'wait_for_package': False} 2023-09-20T21:35:13.441 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bf8a061ff16ac1d393cf108f740d92dfa49f2e54 2023-09-20T21:35:13.443 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bf8a061ff16ac1d393cf108f740d92dfa49f2e54', 'tag': None, 'wait_for_package': False} 2023-09-20T21:35:13.444 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bf8a061ff16ac1d393cf108f740d92dfa49f2e54 2023-09-20T21:35:13.445 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bf8a061ff16ac1d393cf108f740d92dfa49f2e54', 'tag': None, 'wait_for_package': False} 2023-09-20T21:35:13.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bf8a061ff16ac1d393cf108f740d92dfa49f2e54 2023-09-20T21:35:13.583 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/ 2023-09-20T21:35:13.583 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6213.gbf8a061f 2023-09-20T21:35:13.594 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/ 2023-09-20T21:35:13.595 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6213.gbf8a061f 2023-09-20T21:35:13.611 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/ 2023-09-20T21:35:13.611 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6213.gbf8a061f 2023-09-20T21:35:13.747 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-20T21:35:13.748 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:35:13.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-20T21:35:13.757 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-20T21:35:13.757 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:35:13.757 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-20T21:35:13.772 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-20T21:35:13.772 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:35:13.773 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-20T21:35:13.784 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-20T21:35:13.785 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-20T21:35:13.799 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-20T21:35:13.800 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-20T21:35:13.818 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-20T21:35:13.819 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-20T21:35:13.852 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-20T21:35:13.866 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-20T21:35:13.929 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-20T21:35:13.931 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-20T21:35:13.943 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-20T21:35:13.997 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2023-09-20T21:35:13.998 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2023-09-20T21:35:14.009 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2023-09-20T21:35:14.010 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2023-09-20T21:35:14.029 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-20T21:35:14.097 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2023-09-20T21:35:14.101 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2023-09-20T21:35:14.390 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-09-20T21:35:14.447 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-09-20T21:35:14.581 INFO:teuthology.orchestra.run.smithi022.stdout:Updating Subscription Management repositories. 2023-09-20T21:35:14.886 INFO:teuthology.orchestra.run.smithi123.stdout:44 files removed 2023-09-20T21:35:14.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-20T21:35:14.987 INFO:teuthology.orchestra.run.smithi089.stdout:44 files removed 2023-09-20T21:35:15.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-20T21:35:15.088 INFO:teuthology.orchestra.run.smithi022.stdout:44 files removed 2023-09-20T21:35:15.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-20T21:35:15.346 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-09-20T21:35:15.424 INFO:teuthology.orchestra.run.smithi089.stdout:Updating Subscription Management repositories. 2023-09-20T21:35:15.628 INFO:teuthology.orchestra.run.smithi022.stdout:Updating Subscription Management repositories. 2023-09-20T21:35:16.952 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 122 kB/s | 81 kB 00:00 2023-09-20T21:35:16.998 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 118 kB/s | 81 kB 00:00 2023-09-20T21:35:17.450 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 79 kB/s | 81 kB 00:01 2023-09-20T21:35:17.553 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 17 kB/s | 9.4 kB 00:00 2023-09-20T21:35:17.625 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 16 kB/s | 9.4 kB 00:00 2023-09-20T21:35:18.118 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 15 kB/s | 9.4 kB 00:00 2023-09-20T21:35:18.226 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2023-09-20T21:35:18.271 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 2.1 kB/s | 1.5 kB 00:00 2023-09-20T21:35:18.435 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2023-09-20T21:35:18.476 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-09-20T21:35:18.784 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 2.3 kB/s | 1.5 kB 00:00 2023-09-20T21:35:18.935 INFO:teuthology.orchestra.run.smithi022.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-09-20T21:35:19.680 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-09-20T21:35:19.774 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-09-20T21:35:21.055 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 16 MB 00:02 2023-09-20T21:35:23.272 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 397 kB/s | 24 kB 00:00 2023-09-20T21:35:23.410 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 383 kB/s | 24 kB 00:00 2023-09-20T21:35:24.755 INFO:teuthology.orchestra.run.smithi089.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 MB/s | 47 MB 00:01 2023-09-20T21:35:24.782 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 MB/s | 47 MB 00:01 2023-09-20T21:35:25.336 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 381 kB/s | 24 kB 00:00 2023-09-20T21:35:26.696 INFO:teuthology.orchestra.run.smithi022.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 MB/s | 53 MB 00:01 2023-09-20T21:35:32.996 INFO:teuthology.orchestra.run.smithi089.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 53 MB 00:01 2023-09-20T21:35:33.032 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 MB/s | 53 MB 00:01 2023-09-20T21:35:35.939 INFO:teuthology.orchestra.run.smithi022.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 47 MB 00:01 2023-09-20T21:35:44.078 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-20T21:35:44.080 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-20T21:35:44.097 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-20T21:35:44.097 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-20T21:35:44.097 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-20T21:35:44.098 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-20T21:35:44.100 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-20T21:35:44.104 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-20T21:35:44.105 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-20T21:35:44.189 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-20T21:35:44.190 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-20T21:35:44.205 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-20T21:35:44.206 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-20T21:35:44.208 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-20T21:35:44.209 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-20T21:35:44.211 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-20T21:35:44.214 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-20T21:35:44.216 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-20T21:35:44.237 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2023-09-20T21:35:44.249 INFO:teuthology.orchestra.run.smithi089.stdout:=========================================================================================================================== 2023-09-20T21:35:44.249 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2023-09-20T21:35:44.250 INFO:teuthology.orchestra.run.smithi089.stdout:=========================================================================================================================== 2023-09-20T21:35:44.250 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2023-09-20T21:35:44.250 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-20T21:35:44.250 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-20T21:35:44.250 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 6.4 k 2023-09-20T21:35:44.251 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 5.4 M 2023-09-20T21:35:44.251 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 907 k 2023-09-20T21:35:44.251 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 167 k 2023-09-20T21:35:44.251 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 1.6 M 2023-09-20T21:35:44.251 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 132 k 2023-09-20T21:35:44.252 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 1.7 M 2023-09-20T21:35:44.252 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 7.4 M 2023-09-20T21:35:44.252 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 47 k 2023-09-20T21:35:44.252 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 14 M 2023-09-20T21:35:44.252 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 49 M 2023-09-20T21:35:44.252 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 261 k 2023-09-20T21:35:44.253 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 179 k 2023-09-20T21:35:44.253 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 261 k 2023-09-20T21:35:44.253 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 18 k 2023-09-20T21:35:44.253 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-20T21:35:44.253 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-20T21:35:44.254 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-20T21:35:44.254 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 31 k 2023-09-20T21:35:44.254 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 827 k 2023-09-20T21:35:44.254 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 139 k 2023-09-20T21:35:44.254 INFO:teuthology.orchestra.run.smithi089.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-20T21:35:44.255 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-20T21:35:44.255 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-20T21:35:44.255 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-20T21:35:44.255 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-20T21:35:44.255 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-20T21:35:44.256 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 197 k 2023-09-20T21:35:44.256 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 371 k 2023-09-20T21:35:44.256 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 364 k 2023-09-20T21:35:44.256 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 106 k 2023-09-20T21:35:44.256 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 94 k 2023-09-20T21:35:44.256 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 3.6 M 2023-09-20T21:35:44.257 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 179 k 2023-09-20T21:35:44.257 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-20T21:35:44.257 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2023-09-20T21:35:44.257 INFO:teuthology.orchestra.run.smithi089.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-20T21:35:44.257 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 3.7 M 2023-09-20T21:35:44.258 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 3.7 M 2023-09-20T21:35:44.258 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2023-09-20T21:35:44.258 INFO:teuthology.orchestra.run.smithi089.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-20T21:35:44.258 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 23 M 2023-09-20T21:35:44.258 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 23 k 2023-09-20T21:35:44.259 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 2.3 M 2023-09-20T21:35:44.259 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 242 k 2023-09-20T21:35:44.259 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 4.6 M 2023-09-20T21:35:44.259 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 18 M 2023-09-20T21:35:44.259 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 15 k 2023-09-20T21:35:44.260 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 24 k 2023-09-20T21:35:44.260 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-20T21:35:44.260 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-20T21:35:44.260 INFO:teuthology.orchestra.run.smithi089.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-20T21:35:44.260 INFO:teuthology.orchestra.run.smithi089.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-20T21:35:44.260 INFO:teuthology.orchestra.run.smithi089.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-20T21:35:44.261 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 184 k 2023-09-20T21:35:44.261 INFO:teuthology.orchestra.run.smithi089.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-20T21:35:44.261 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-20T21:35:44.261 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-20T21:35:44.261 INFO:teuthology.orchestra.run.smithi089.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-20T21:35:44.262 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-20T21:35:44.262 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-20T21:35:44.262 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 519 k 2023-09-20T21:35:44.262 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-20T21:35:44.262 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 6.9 M 2023-09-20T21:35:44.263 INFO:teuthology.orchestra.run.smithi089.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-20T21:35:44.263 INFO:teuthology.orchestra.run.smithi089.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-20T21:35:44.263 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-20T21:35:44.263 INFO:teuthology.orchestra.run.smithi089.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-20T21:35:44.263 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-20T21:35:44.263 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-20T21:35:44.264 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-20T21:35:44.264 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-20T21:35:44.264 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-20T21:35:44.264 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-20T21:35:44.264 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-20T21:35:44.265 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-20T21:35:44.265 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-20T21:35:44.265 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-20T21:35:44.265 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-20T21:35:44.265 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-20T21:35:44.266 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 45 k 2023-09-20T21:35:44.266 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 125 k 2023-09-20T21:35:44.266 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-20T21:35:44.266 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-20T21:35:44.266 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-20T21:35:44.267 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-20T21:35:44.267 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-20T21:35:44.267 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-20T21:35:44.267 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-20T21:35:44.267 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-20T21:35:44.268 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-20T21:35:44.268 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-20T21:35:44.268 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-20T21:35:44.268 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-20T21:35:44.268 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-20T21:35:44.268 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-20T21:35:44.269 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-20T21:35:44.269 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-20T21:35:44.269 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-20T21:35:44.269 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-20T21:35:44.269 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-20T21:35:44.269 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-20T21:35:44.270 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-20T21:35:44.270 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-20T21:35:44.270 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-20T21:35:44.270 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-20T21:35:44.270 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-20T21:35:44.271 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-20T21:35:44.271 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-20T21:35:44.271 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-20T21:35:44.271 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-20T21:35:44.271 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-20T21:35:44.271 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-20T21:35:44.272 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-20T21:35:44.272 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-20T21:35:44.272 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-20T21:35:44.272 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-20T21:35:44.272 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-20T21:35:44.273 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-20T21:35:44.273 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-20T21:35:44.273 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-20T21:35:44.273 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-20T21:35:44.273 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-20T21:35:44.273 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-20T21:35:44.274 INFO:teuthology.orchestra.run.smithi089.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-20T21:35:44.275 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-20T21:35:44.275 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-20T21:35:44.275 INFO:teuthology.orchestra.run.smithi089.stdout:Enabling module streams: 2023-09-20T21:35:44.275 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql 10 2023-09-20T21:35:44.275 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:35:44.276 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2023-09-20T21:35:44.276 INFO:teuthology.orchestra.run.smithi089.stdout:=========================================================================================================================== 2023-09-20T21:35:44.276 INFO:teuthology.orchestra.run.smithi089.stdout:Install 128 Packages 2023-09-20T21:35:44.276 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 3 Packages 2023-09-20T21:35:44.276 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:35:44.277 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 208 M 2023-09-20T21:35:44.277 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2023-09-20T21:35:44.348 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-09-20T21:35:44.360 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-09-20T21:35:44.360 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2023-09-20T21:35:44.361 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-09-20T21:35:44.361 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2023-09-20T21:35:44.361 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-20T21:35:44.362 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-20T21:35:44.362 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 6.4 k 2023-09-20T21:35:44.362 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 5.4 M 2023-09-20T21:35:44.362 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 907 k 2023-09-20T21:35:44.362 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 167 k 2023-09-20T21:35:44.363 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 1.6 M 2023-09-20T21:35:44.363 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 132 k 2023-09-20T21:35:44.363 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 1.7 M 2023-09-20T21:35:44.364 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 7.4 M 2023-09-20T21:35:44.364 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 47 k 2023-09-20T21:35:44.364 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 14 M 2023-09-20T21:35:44.365 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 49 M 2023-09-20T21:35:44.365 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 261 k 2023-09-20T21:35:44.366 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 179 k 2023-09-20T21:35:44.366 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 261 k 2023-09-20T21:35:44.366 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 18 k 2023-09-20T21:35:44.366 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-20T21:35:44.367 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-20T21:35:44.367 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-20T21:35:44.367 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 31 k 2023-09-20T21:35:44.368 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 827 k 2023-09-20T21:35:44.368 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 139 k 2023-09-20T21:35:44.368 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-20T21:35:44.368 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 197 k 2023-09-20T21:35:44.370 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 371 k 2023-09-20T21:35:44.370 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 364 k 2023-09-20T21:35:44.370 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 106 k 2023-09-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 94 k 2023-09-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 3.6 M 2023-09-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 179 k 2023-09-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-20T21:35:44.372 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2023-09-20T21:35:44.372 INFO:teuthology.orchestra.run.smithi123.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-20T21:35:44.372 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 3.7 M 2023-09-20T21:35:44.373 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 3.7 M 2023-09-20T21:35:44.373 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2023-09-20T21:35:44.373 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-20T21:35:44.374 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 23 M 2023-09-20T21:35:44.374 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 23 k 2023-09-20T21:35:44.374 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 2.3 M 2023-09-20T21:35:44.374 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 242 k 2023-09-20T21:35:44.375 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 4.6 M 2023-09-20T21:35:44.375 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 18 M 2023-09-20T21:35:44.375 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 15 k 2023-09-20T21:35:44.376 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 24 k 2023-09-20T21:35:44.376 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-20T21:35:44.376 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-20T21:35:44.377 INFO:teuthology.orchestra.run.smithi123.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-20T21:35:44.377 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-20T21:35:44.377 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-20T21:35:44.378 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 184 k 2023-09-20T21:35:44.378 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-20T21:35:44.378 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-20T21:35:44.379 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-20T21:35:44.379 INFO:teuthology.orchestra.run.smithi123.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-20T21:35:44.379 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-20T21:35:44.379 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-20T21:35:44.380 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 519 k 2023-09-20T21:35:44.380 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-20T21:35:44.380 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 6.9 M 2023-09-20T21:35:44.381 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-20T21:35:44.381 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-20T21:35:44.381 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-20T21:35:44.381 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-20T21:35:44.382 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-20T21:35:44.382 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-20T21:35:44.382 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-20T21:35:44.382 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-20T21:35:44.382 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-20T21:35:44.383 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-20T21:35:44.383 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-20T21:35:44.383 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-20T21:35:44.383 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-20T21:35:44.383 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-20T21:35:44.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-20T21:35:44.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-20T21:35:44.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 45 k 2023-09-20T21:35:44.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 125 k 2023-09-20T21:35:44.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-20T21:35:44.384 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-20T21:35:44.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-20T21:35:44.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-20T21:35:44.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-20T21:35:44.385 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-20T21:35:44.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-20T21:35:44.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-20T21:35:44.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-20T21:35:44.386 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-20T21:35:44.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-20T21:35:44.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-20T21:35:44.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-20T21:35:44.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-20T21:35:44.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-20T21:35:44.387 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-20T21:35:44.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-20T21:35:44.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-20T21:35:44.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-20T21:35:44.388 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-20T21:35:44.389 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-20T21:35:44.389 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-20T21:35:44.389 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-20T21:35:44.389 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-20T21:35:44.389 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-20T21:35:44.390 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-20T21:35:44.390 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-20T21:35:44.390 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-20T21:35:44.391 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-20T21:35:44.391 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-20T21:35:44.391 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-20T21:35:44.392 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-20T21:35:44.392 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-20T21:35:44.392 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-20T21:35:44.392 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-20T21:35:44.392 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-20T21:35:44.393 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-20T21:35:44.393 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-20T21:35:44.393 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-20T21:35:44.393 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-20T21:35:44.393 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-20T21:35:44.394 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-20T21:35:44.394 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-20T21:35:44.394 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-20T21:35:44.394 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-20T21:35:44.395 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-20T21:35:44.395 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-20T21:35:44.395 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-20T21:35:44.395 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-20T21:35:44.395 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling module streams: 2023-09-20T21:35:44.396 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql 10 2023-09-20T21:35:44.396 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:35:44.396 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-09-20T21:35:44.396 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-09-20T21:35:44.397 INFO:teuthology.orchestra.run.smithi123.stdout:Install 128 Packages 2023-09-20T21:35:44.397 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 3 Packages 2023-09-20T21:35:44.397 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:35:44.398 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 208 M 2023-09-20T21:35:44.398 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2023-09-20T21:35:44.965 INFO:teuthology.orchestra.run.smithi123.stdout:(1/131): ceph-18.0.0-6213.gbf8a061f.el8.x86_64. 15 kB/s | 6.4 kB 00:00 2023-09-20T21:35:45.014 INFO:teuthology.orchestra.run.smithi089.stdout:(1/131): ceph-18.0.0-6213.gbf8a061f.el8.x86_64. 18 kB/s | 6.4 kB 00:00 2023-09-20T21:35:45.666 INFO:teuthology.orchestra.run.smithi123.stdout:(2/131): ceph-base-18.0.0-6213.gbf8a061f.el8.x8 4.8 MB/s | 5.4 MB 00:01 2023-09-20T21:35:45.792 INFO:teuthology.orchestra.run.smithi123.stdout:(3/131): ceph-immutable-object-cache-18.0.0-621 1.3 MB/s | 167 kB 00:00 2023-09-20T21:35:46.022 INFO:teuthology.orchestra.run.smithi089.stdout:(2/131): ceph-base-18.0.0-6213.gbf8a061f.el8.x8 3.9 MB/s | 5.4 MB 00:01 2023-09-20T21:35:46.090 INFO:teuthology.orchestra.run.smithi089.stdout:(3/131): ceph-fuse-18.0.0-6213.gbf8a061f.el8.x8 843 kB/s | 907 kB 00:01 2023-09-20T21:35:46.092 INFO:teuthology.orchestra.run.smithi123.stdout:(4/131): ceph-fuse-18.0.0-6213.gbf8a061f.el8.x8 805 kB/s | 907 kB 00:01 2023-09-20T21:35:46.115 INFO:teuthology.orchestra.run.smithi089.stdout:(4/131): ceph-immutable-object-cache-18.0.0-621 1.8 MB/s | 167 kB 00:00 2023-09-20T21:35:46.334 INFO:teuthology.orchestra.run.smithi123.stdout:(5/131): ceph-mds-18.0.0-6213.gbf8a061f.el8.x86 4.3 MB/s | 2.3 MB 00:00 2023-09-20T21:35:46.557 INFO:teuthology.orchestra.run.smithi089.stdout:(5/131): ceph-mgr-18.0.0-6213.gbf8a061f.el8.x86 3.7 MB/s | 1.6 MB 00:00 2023-09-20T21:35:46.617 INFO:teuthology.orchestra.run.smithi123.stdout:(6/131): ceph-mgr-18.0.0-6213.gbf8a061f.el8.x86 3.1 MB/s | 1.6 MB 00:00 2023-09-20T21:35:46.841 INFO:teuthology.orchestra.run.smithi089.stdout:(6/131): ceph-mds-18.0.0-6213.gbf8a061f.el8.x86 3.1 MB/s | 2.3 MB 00:00 2023-09-20T21:35:47.249 INFO:teuthology.orchestra.run.smithi089.stdout:(7/131): ceph-mon-18.0.0-6213.gbf8a061f.el8.x86 6.7 MB/s | 4.6 MB 00:00 2023-09-20T21:35:48.250 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-20T21:35:48.251 INFO:teuthology.orchestra.run.smithi123.stdout:(7/131): ceph-mon-18.0.0-6213.gbf8a061f.el8.x86 2.4 MB/s | 4.6 MB 00:01 2023-09-20T21:35:48.251 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-20T21:35:48.266 INFO:teuthology.orchestra.run.smithi022.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-09-20T21:35:48.267 INFO:teuthology.orchestra.run.smithi022.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-09-20T21:35:48.269 INFO:teuthology.orchestra.run.smithi022.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-09-20T21:35:48.270 INFO:teuthology.orchestra.run.smithi022.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-09-20T21:35:48.272 INFO:teuthology.orchestra.run.smithi022.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-09-20T21:35:48.276 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-09-20T21:35:48.277 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-09-20T21:35:48.426 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2023-09-20T21:35:48.440 INFO:teuthology.orchestra.run.smithi022.stdout:=========================================================================================================================== 2023-09-20T21:35:48.440 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2023-09-20T21:35:48.441 INFO:teuthology.orchestra.run.smithi022.stdout:=========================================================================================================================== 2023-09-20T21:35:48.441 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2023-09-20T21:35:48.441 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-09-20T21:35:48.441 INFO:teuthology.orchestra.run.smithi022.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-09-20T21:35:48.441 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 6.4 k 2023-09-20T21:35:48.442 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 5.4 M 2023-09-20T21:35:48.442 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 907 k 2023-09-20T21:35:48.442 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 167 k 2023-09-20T21:35:48.442 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 1.6 M 2023-09-20T21:35:48.442 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 132 k 2023-09-20T21:35:48.443 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 1.7 M 2023-09-20T21:35:48.443 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 7.4 M 2023-09-20T21:35:48.443 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 47 k 2023-09-20T21:35:48.443 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 14 M 2023-09-20T21:35:48.443 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 49 M 2023-09-20T21:35:48.444 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 261 k 2023-09-20T21:35:48.444 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 179 k 2023-09-20T21:35:48.444 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 261 k 2023-09-20T21:35:48.444 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 18 k 2023-09-20T21:35:48.444 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-09-20T21:35:48.444 INFO:teuthology.orchestra.run.smithi022.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-09-20T21:35:48.445 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-09-20T21:35:48.445 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 31 k 2023-09-20T21:35:48.445 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 827 k 2023-09-20T21:35:48.445 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 139 k 2023-09-20T21:35:48.445 INFO:teuthology.orchestra.run.smithi022.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-09-20T21:35:48.446 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-09-20T21:35:48.446 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-09-20T21:35:48.446 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-09-20T21:35:48.446 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-09-20T21:35:48.446 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-09-20T21:35:48.447 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 197 k 2023-09-20T21:35:48.447 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 371 k 2023-09-20T21:35:48.447 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 364 k 2023-09-20T21:35:48.447 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 106 k 2023-09-20T21:35:48.447 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 94 k 2023-09-20T21:35:48.448 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 3.6 M 2023-09-20T21:35:48.448 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 179 k 2023-09-20T21:35:48.448 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-09-20T21:35:48.448 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2023-09-20T21:35:48.448 INFO:teuthology.orchestra.run.smithi022.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-09-20T21:35:48.448 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 3.7 M 2023-09-20T21:35:48.449 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 3.7 M 2023-09-20T21:35:48.449 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2023-09-20T21:35:48.449 INFO:teuthology.orchestra.run.smithi022.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-09-20T21:35:48.449 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 23 M 2023-09-20T21:35:48.449 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 23 k 2023-09-20T21:35:48.450 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 2.3 M 2023-09-20T21:35:48.450 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 242 k 2023-09-20T21:35:48.450 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 4.6 M 2023-09-20T21:35:48.450 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 18 M 2023-09-20T21:35:48.450 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6213.gbf8a061f.el8 ceph-noarch 15 k 2023-09-20T21:35:48.451 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 24 k 2023-09-20T21:35:48.451 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-09-20T21:35:48.451 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-20T21:35:48.451 INFO:teuthology.orchestra.run.smithi022.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-09-20T21:35:48.451 INFO:teuthology.orchestra.run.smithi022.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-09-20T21:35:48.452 INFO:teuthology.orchestra.run.smithi022.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-09-20T21:35:48.452 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 184 k 2023-09-20T21:35:48.452 INFO:teuthology.orchestra.run.smithi022.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-09-20T21:35:48.452 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-09-20T21:35:48.452 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-20T21:35:48.452 INFO:teuthology.orchestra.run.smithi022.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-09-20T21:35:48.453 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-09-20T21:35:48.453 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-09-20T21:35:48.453 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 519 k 2023-09-20T21:35:48.453 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-09-20T21:35:48.453 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 6.9 M 2023-09-20T21:35:48.454 INFO:teuthology.orchestra.run.smithi022.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-09-20T21:35:48.454 INFO:teuthology.orchestra.run.smithi022.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-09-20T21:35:48.454 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-20T21:35:48.454 INFO:teuthology.orchestra.run.smithi022.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-09-20T21:35:48.454 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-09-20T21:35:48.455 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-09-20T21:35:48.455 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-09-20T21:35:48.455 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-09-20T21:35:48.455 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-09-20T21:35:48.455 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-09-20T21:35:48.456 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-09-20T21:35:48.456 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-09-20T21:35:48.456 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-09-20T21:35:48.456 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-20T21:35:48.456 INFO:teuthology.orchestra.run.smithi022.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-20T21:35:48.456 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-20T21:35:48.457 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 45 k 2023-09-20T21:35:48.457 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:18.0.0-6213.gbf8a061f.el8 ceph 125 k 2023-09-20T21:35:48.457 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-20T21:35:48.457 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-09-20T21:35:48.457 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-20T21:35:48.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-20T21:35:48.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-09-20T21:35:48.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-20T21:35:48.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-20T21:35:48.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-20T21:35:48.458 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-09-20T21:35:48.459 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-09-20T21:35:48.459 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-09-20T21:35:48.459 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-20T21:35:48.459 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-20T21:35:48.459 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-20T21:35:48.460 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-09-20T21:35:48.460 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-09-20T21:35:48.460 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-20T21:35:48.460 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-09-20T21:35:48.460 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-09-20T21:35:48.461 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-09-20T21:35:48.461 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-09-20T21:35:48.461 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-20T21:35:48.461 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-20T21:35:48.461 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-09-20T21:35:48.461 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-09-20T21:35:48.462 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-09-20T21:35:48.462 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-09-20T21:35:48.462 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-09-20T21:35:48.462 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-09-20T21:35:48.462 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-20T21:35:48.463 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-09-20T21:35:48.463 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-20T21:35:48.463 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-20T21:35:48.463 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-09-20T21:35:48.463 INFO:teuthology.orchestra.run.smithi022.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-20T21:35:48.463 INFO:teuthology.orchestra.run.smithi022.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-20T21:35:48.464 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-20T21:35:48.464 INFO:teuthology.orchestra.run.smithi022.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-20T21:35:48.464 INFO:teuthology.orchestra.run.smithi022.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-20T21:35:48.464 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-20T21:35:48.464 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-20T21:35:48.465 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-20T21:35:48.465 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-09-20T21:35:48.465 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-20T21:35:48.465 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-09-20T21:35:48.465 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-20T21:35:48.465 INFO:teuthology.orchestra.run.smithi022.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-09-20T21:35:48.466 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-09-20T21:35:48.466 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-09-20T21:35:48.466 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling module streams: 2023-09-20T21:35:48.466 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql 10 2023-09-20T21:35:48.466 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:35:48.467 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2023-09-20T21:35:48.467 INFO:teuthology.orchestra.run.smithi022.stdout:=========================================================================================================================== 2023-09-20T21:35:48.467 INFO:teuthology.orchestra.run.smithi022.stdout:Install 128 Packages 2023-09-20T21:35:48.467 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 3 Packages 2023-09-20T21:35:48.467 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:35:48.468 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 208 M 2023-09-20T21:35:48.468 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2023-09-20T21:35:48.643 INFO:teuthology.orchestra.run.smithi123.stdout:(8/131): ceph-common-18.0.0-6213.gbf8a061f.el8. 5.5 MB/s | 23 MB 00:04 2023-09-20T21:35:48.693 INFO:teuthology.orchestra.run.smithi123.stdout:(9/131): ceph-selinux-18.0.0-6213.gbf8a061f.el8 489 kB/s | 24 kB 00:00 2023-09-20T21:35:48.807 INFO:teuthology.orchestra.run.smithi089.stdout:(8/131): ceph-common-18.0.0-6213.gbf8a061f.el8. 5.4 MB/s | 23 MB 00:04 2023-09-20T21:35:49.048 INFO:teuthology.orchestra.run.smithi022.stdout:(1/131): ceph-18.0.0-6213.gbf8a061f.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2023-09-20T21:35:49.099 INFO:teuthology.orchestra.run.smithi089.stdout:(9/131): ceph-radosgw-18.0.0-6213.gbf8a061f.el8 7.5 MB/s | 14 MB 00:01 2023-09-20T21:35:49.125 INFO:teuthology.orchestra.run.smithi089.stdout:(10/131): ceph-selinux-18.0.0-6213.gbf8a061f.el 77 kB/s | 24 kB 00:00 2023-09-20T21:35:49.308 INFO:teuthology.orchestra.run.smithi089.stdout:(11/131): cephfs-mirror-18.0.0-6213.gbf8a061f.e 1.4 MB/s | 261 kB 00:00 2023-09-20T21:35:49.417 INFO:teuthology.orchestra.run.smithi089.stdout:(12/131): libcephfs-devel-18.0.0-6213.gbf8a061f 286 kB/s | 31 kB 00:00 2023-09-20T21:35:49.851 INFO:teuthology.orchestra.run.smithi089.stdout:(13/131): libcephfs2-18.0.0-6213.gbf8a061f.el8. 1.9 MB/s | 827 kB 00:00 2023-09-20T21:35:49.898 INFO:teuthology.orchestra.run.smithi022.stdout:(2/131): ceph-fuse-18.0.0-6213.gbf8a061f.el8.x8 1.0 MB/s | 907 kB 00:00 2023-09-20T21:35:49.976 INFO:teuthology.orchestra.run.smithi089.stdout:(14/131): libcephsqlite-18.0.0-6213.gbf8a061f.e 1.4 MB/s | 184 kB 00:00 2023-09-20T21:35:50.059 INFO:teuthology.orchestra.run.smithi089.stdout:(15/131): librados-devel-18.0.0-6213.gbf8a061f. 1.6 MB/s | 139 kB 00:00 2023-09-20T21:35:50.090 INFO:teuthology.orchestra.run.smithi022.stdout:(3/131): ceph-immutable-object-cache-18.0.0-621 875 kB/s | 167 kB 00:00 2023-09-20T21:35:50.310 INFO:teuthology.orchestra.run.smithi089.stdout:(16/131): libradosstriper1-18.0.0-6213.gbf8a061 2.0 MB/s | 519 kB 00:00 2023-09-20T21:35:51.144 INFO:teuthology.orchestra.run.smithi123.stdout:(10/131): ceph-radosgw-18.0.0-6213.gbf8a061f.el 4.8 MB/s | 14 MB 00:02 2023-09-20T21:35:51.224 INFO:teuthology.orchestra.run.smithi022.stdout:(4/131): ceph-mds-18.0.0-6213.gbf8a061f.el8.x86 2.1 MB/s | 2.3 MB 00:01 2023-09-20T21:35:51.269 INFO:teuthology.orchestra.run.smithi123.stdout:(11/131): cephfs-mirror-18.0.0-6213.gbf8a061f.e 2.0 MB/s | 261 kB 00:00 2023-09-20T21:35:51.370 INFO:teuthology.orchestra.run.smithi123.stdout:(12/131): libcephfs-devel-18.0.0-6213.gbf8a061f 308 kB/s | 31 kB 00:00 2023-09-20T21:35:51.594 INFO:teuthology.orchestra.run.smithi123.stdout:(13/131): libcephfs2-18.0.0-6213.gbf8a061f.el8. 3.6 MB/s | 827 kB 00:00 2023-09-20T21:35:51.686 INFO:teuthology.orchestra.run.smithi123.stdout:(14/131): libcephsqlite-18.0.0-6213.gbf8a061f.e 2.0 MB/s | 184 kB 00:00 2023-09-20T21:35:51.749 INFO:teuthology.orchestra.run.smithi022.stdout:(5/131): ceph-base-18.0.0-6213.gbf8a061f.el8.x8 1.8 MB/s | 5.4 MB 00:02 2023-09-20T21:35:51.761 INFO:teuthology.orchestra.run.smithi123.stdout:(15/131): librados-devel-18.0.0-6213.gbf8a061f. 1.8 MB/s | 139 kB 00:00 2023-09-20T21:35:51.937 INFO:teuthology.orchestra.run.smithi123.stdout:(16/131): libradosstriper1-18.0.0-6213.gbf8a061 2.9 MB/s | 519 kB 00:00 2023-09-20T21:35:52.024 INFO:teuthology.orchestra.run.smithi022.stdout:(6/131): ceph-mgr-18.0.0-6213.gbf8a061f.el8.x86 2.0 MB/s | 1.6 MB 00:00 2023-09-20T21:35:52.354 INFO:teuthology.orchestra.run.smithi123.stdout:(17/131): ceph-osd-18.0.0-6213.gbf8a061f.el8.x8 3.1 MB/s | 18 MB 00:05 2023-09-20T21:35:52.496 INFO:teuthology.orchestra.run.smithi123.stdout:(18/131): python3-ceph-argparse-18.0.0-6213.gbf 316 kB/s | 45 kB 00:00 2023-09-20T21:35:52.654 INFO:teuthology.orchestra.run.smithi123.stdout:(19/131): python3-ceph-common-18.0.0-6213.gbf8a 795 kB/s | 125 kB 00:00 2023-09-20T21:35:52.754 INFO:teuthology.orchestra.run.smithi123.stdout:(20/131): python3-cephfs-18.0.0-6213.gbf8a061f. 1.9 MB/s | 197 kB 00:00 2023-09-20T21:35:52.835 INFO:teuthology.orchestra.run.smithi089.stdout:(17/131): librgw2-18.0.0-6213.gbf8a061f.el8.x86 2.7 MB/s | 6.9 MB 00:02 2023-09-20T21:35:52.938 INFO:teuthology.orchestra.run.smithi123.stdout:(21/131): python3-rados-18.0.0-6213.gbf8a061f.e 2.0 MB/s | 371 kB 00:00 2023-09-20T21:35:53.008 INFO:teuthology.orchestra.run.smithi022.stdout:(7/131): ceph-mon-18.0.0-6213.gbf8a061f.el8.x86 3.7 MB/s | 4.6 MB 00:01 2023-09-20T21:35:53.010 INFO:teuthology.orchestra.run.smithi089.stdout:(18/131): ceph-osd-18.0.0-6213.gbf8a061f.el8.x8 2.9 MB/s | 18 MB 00:06 2023-09-20T21:35:53.027 INFO:teuthology.orchestra.run.smithi089.stdout:(19/131): python3-ceph-argparse-18.0.0-6213.gbf 233 kB/s | 45 kB 00:00 2023-09-20T21:35:53.180 INFO:teuthology.orchestra.run.smithi123.stdout:(22/131): python3-rbd-18.0.0-6213.gbf8a061f.el8 1.5 MB/s | 364 kB 00:00 2023-09-20T21:35:53.211 INFO:teuthology.orchestra.run.smithi089.stdout:(20/131): python3-cephfs-18.0.0-6213.gbf8a061f. 1.0 MB/s | 197 kB 00:00 2023-09-20T21:35:53.227 INFO:teuthology.orchestra.run.smithi089.stdout:(21/131): python3-ceph-common-18.0.0-6213.gbf8a 579 kB/s | 125 kB 00:00 2023-09-20T21:35:53.297 INFO:teuthology.orchestra.run.smithi123.stdout:(23/131): python3-rgw-18.0.0-6213.gbf8a061f.el8 913 kB/s | 106 kB 00:00 2023-09-20T21:35:53.381 INFO:teuthology.orchestra.run.smithi123.stdout:(24/131): librgw2-18.0.0-6213.gbf8a061f.el8.x86 4.8 MB/s | 6.9 MB 00:01 2023-09-20T21:35:53.423 INFO:teuthology.orchestra.run.smithi123.stdout:(25/131): rbd-fuse-18.0.0-6213.gbf8a061f.el8.x8 748 kB/s | 94 kB 00:00 2023-09-20T21:35:53.444 INFO:teuthology.orchestra.run.smithi089.stdout:(22/131): python3-rados-18.0.0-6213.gbf8a061f.e 1.6 MB/s | 371 kB 00:00 2023-09-20T21:35:53.470 INFO:teuthology.orchestra.run.smithi089.stdout:(23/131): python3-rbd-18.0.0-6213.gbf8a061f.el8 1.5 MB/s | 364 kB 00:00 2023-09-20T21:35:53.539 INFO:teuthology.orchestra.run.smithi123.stdout:(26/131): rbd-nbd-18.0.0-6213.gbf8a061f.el8.x86 1.5 MB/s | 179 kB 00:00 2023-09-20T21:35:53.545 INFO:teuthology.orchestra.run.smithi089.stdout:(24/131): python3-rgw-18.0.0-6213.gbf8a061f.el8 1.0 MB/s | 106 kB 00:00 2023-09-20T21:35:53.579 INFO:teuthology.orchestra.run.smithi089.stdout:(25/131): rbd-fuse-18.0.0-6213.gbf8a061f.el8.x8 864 kB/s | 94 kB 00:00 2023-09-20T21:35:53.640 INFO:teuthology.orchestra.run.smithi123.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6213.g 231 kB/s | 23 kB 00:00 2023-09-20T21:35:53.696 INFO:teuthology.orchestra.run.smithi089.stdout:(26/131): rbd-nbd-18.0.0-6213.gbf8a061f.el8.x86 1.5 MB/s | 179 kB 00:00 2023-09-20T21:35:53.757 INFO:teuthology.orchestra.run.smithi123.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6213.gbf8a061 1.1 MB/s | 132 kB 00:00 2023-09-20T21:35:53.796 INFO:teuthology.orchestra.run.smithi089.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6213.g 231 kB/s | 23 kB 00:00 2023-09-20T21:35:53.905 INFO:teuthology.orchestra.run.smithi089.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6213.gbf8a061 1.2 MB/s | 132 kB 00:00 2023-09-20T21:35:54.240 INFO:teuthology.orchestra.run.smithi123.stdout:(29/131): rbd-mirror-18.0.0-6213.gbf8a061f.el8. 4.2 MB/s | 3.6 MB 00:00 2023-09-20T21:35:54.407 INFO:teuthology.orchestra.run.smithi123.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6213.gbf8a0 2.7 MB/s | 1.7 MB 00:00 2023-09-20T21:35:54.551 INFO:teuthology.orchestra.run.smithi123.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6213.gbf 1.7 MB/s | 242 kB 00:00 2023-09-20T21:35:54.666 INFO:teuthology.orchestra.run.smithi123.stdout:(32/131): ceph-mgr-rook-18.0.0-6213.gbf8a061f.e 412 kB/s | 47 kB 00:00 2023-09-20T21:35:54.697 INFO:teuthology.orchestra.run.smithi089.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6213.gbf8a0 2.2 MB/s | 1.7 MB 00:00 2023-09-20T21:35:54.758 INFO:teuthology.orchestra.run.smithi123.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6213.gb 159 kB/s | 15 kB 00:00 2023-09-20T21:35:54.867 INFO:teuthology.orchestra.run.smithi123.stdout:(34/131): ceph-volume-18.0.0-6213.gbf8a061f.el8 2.4 MB/s | 261 kB 00:00 2023-09-20T21:35:54.970 INFO:teuthology.orchestra.run.smithi123.stdout:(35/131): cephadm-18.0.0-6213.gbf8a061f.el8.noa 1.8 MB/s | 179 kB 00:00 2023-09-20T21:35:55.067 INFO:teuthology.orchestra.run.smithi123.stdout:(36/131): cephfs-top-18.0.0-6213.gbf8a061f.el8. 181 kB/s | 18 kB 00:00 2023-09-20T21:35:55.264 INFO:teuthology.orchestra.run.smithi089.stdout:(30/131): rbd-mirror-18.0.0-6213.gbf8a061f.el8. 2.1 MB/s | 3.6 MB 00:01 2023-09-20T21:35:55.381 INFO:teuthology.orchestra.run.smithi089.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6213.gbf 2.0 MB/s | 242 kB 00:00 2023-09-20T21:35:55.441 INFO:teuthology.orchestra.run.smithi022.stdout:(8/131): ceph-radosgw-18.0.0-6213.gbf8a061f.el8 5.7 MB/s | 14 MB 00:02 2023-09-20T21:35:55.448 INFO:teuthology.orchestra.run.smithi089.stdout:(32/131): ceph-mgr-rook-18.0.0-6213.gbf8a061f.e 711 kB/s | 47 kB 00:00 2023-09-20T21:35:55.516 INFO:teuthology.orchestra.run.smithi022.stdout:(9/131): ceph-selinux-18.0.0-6213.gbf8a061f.el8 326 kB/s | 24 kB 00:00 2023-09-20T21:35:55.523 INFO:teuthology.orchestra.run.smithi089.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6213.gb 194 kB/s | 15 kB 00:00 2023-09-20T21:35:55.657 INFO:teuthology.orchestra.run.smithi089.stdout:(34/131): ceph-volume-18.0.0-6213.gbf8a061f.el8 1.9 MB/s | 261 kB 00:00 2023-09-20T21:35:55.687 INFO:teuthology.orchestra.run.smithi123.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 494 kB/s | 306 kB 00:00 2023-09-20T21:35:55.807 INFO:teuthology.orchestra.run.smithi089.stdout:(35/131): cephadm-18.0.0-6213.gbf8a061f.el8.noa 1.2 MB/s | 179 kB 00:00 2023-09-20T21:35:55.810 INFO:teuthology.orchestra.run.smithi123.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 502 kB/s | 59 kB 00:00 2023-09-20T21:35:55.890 INFO:teuthology.orchestra.run.smithi089.stdout:(36/131): cephfs-top-18.0.0-6213.gbf8a061f.el8. 217 kB/s | 18 kB 00:00 2023-09-20T21:35:55.910 INFO:teuthology.orchestra.run.smithi123.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 752 kB/s | 75 kB 00:00 2023-09-20T21:35:56.127 INFO:teuthology.orchestra.run.smithi123.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 3.9 MB/s | 7.4 MB 00:01 2023-09-20T21:35:56.152 INFO:teuthology.orchestra.run.smithi123.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2023-09-20T21:35:56.253 INFO:teuthology.orchestra.run.smithi123.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-09-20T21:35:56.353 INFO:teuthology.orchestra.run.smithi123.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-09-20T21:35:56.354 INFO:teuthology.orchestra.run.smithi089.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 661 kB/s | 306 kB 00:00 2023-09-20T21:35:56.437 INFO:teuthology.orchestra.run.smithi123.stdout:(44/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 141 kB/s | 44 kB 00:00 2023-09-20T21:35:56.481 INFO:teuthology.orchestra.run.smithi123.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 127 kB/s | 16 kB 00:00 2023-09-20T21:35:56.587 INFO:teuthology.orchestra.run.smithi123.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.6 MB/s | 384 kB 00:00 2023-09-20T21:35:56.599 INFO:teuthology.orchestra.run.smithi089.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 241 kB/s | 59 kB 00:00 2023-09-20T21:35:56.671 INFO:teuthology.orchestra.run.smithi123.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 738 kB/s | 173 kB 00:00 2023-09-20T21:35:56.675 INFO:teuthology.orchestra.run.smithi089.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2023-09-20T21:35:56.696 INFO:teuthology.orchestra.run.smithi123.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 749 kB/s | 81 kB 00:00 2023-09-20T21:35:56.763 INFO:teuthology.orchestra.run.smithi123.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-09-20T21:35:56.788 INFO:teuthology.orchestra.run.smithi123.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2023-09-20T21:35:56.792 INFO:teuthology.orchestra.run.smithi089.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.0 MB/s | 476 kB 00:00 2023-09-20T21:35:56.851 INFO:teuthology.orchestra.run.smithi089.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 750 kB/s | 44 kB 00:00 2023-09-20T21:35:56.880 INFO:teuthology.orchestra.run.smithi123.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 532 kB/s | 49 kB 00:00 2023-09-20T21:35:56.926 INFO:teuthology.orchestra.run.smithi089.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2023-09-20T21:35:56.972 INFO:teuthology.orchestra.run.smithi123.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 642 kB/s | 59 kB 00:00 2023-09-20T21:35:56.985 INFO:teuthology.orchestra.run.smithi089.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 566 kB/s | 33 kB 00:00 2023-09-20T21:35:57.014 INFO:teuthology.orchestra.run.smithi123.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 4.6 MB/s | 1.2 MB 00:00 2023-09-20T21:35:57.043 INFO:teuthology.orchestra.run.smithi089.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 280 kB/s | 16 kB 00:00 2023-09-20T21:35:57.064 INFO:teuthology.orchestra.run.smithi123.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 657 kB/s | 60 kB 00:00 2023-09-20T21:35:57.118 INFO:teuthology.orchestra.run.smithi089.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-09-20T21:35:57.140 INFO:teuthology.orchestra.run.smithi123.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-09-20T21:35:57.165 INFO:teuthology.orchestra.run.smithi123.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-09-20T21:35:57.227 INFO:teuthology.orchestra.run.smithi089.stdout:(46/131): ceph-mgr-diskprediction-local-18.0.0- 2.9 MB/s | 7.4 MB 00:02 2023-09-20T21:35:57.232 INFO:teuthology.orchestra.run.smithi123.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 367 kB/s | 34 kB 00:00 2023-09-20T21:35:57.252 INFO:teuthology.orchestra.run.smithi089.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2023-09-20T21:35:57.265 INFO:teuthology.orchestra.run.smithi123.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-09-20T21:35:57.319 INFO:teuthology.orchestra.run.smithi089.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2023-09-20T21:35:57.324 INFO:teuthology.orchestra.run.smithi123.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 699 kB/s | 64 kB 00:00 2023-09-20T21:35:57.349 INFO:teuthology.orchestra.run.smithi123.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 223 kB/s | 19 kB 00:00 2023-09-20T21:35:57.386 INFO:teuthology.orchestra.run.smithi089.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 267 kB/s | 18 kB 00:00 2023-09-20T21:35:57.408 INFO:teuthology.orchestra.run.smithi123.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 286 kB/s | 24 kB 00:00 2023-09-20T21:35:57.433 INFO:teuthology.orchestra.run.smithi123.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 350 kB/s | 29 kB 00:00 2023-09-20T21:35:57.492 INFO:teuthology.orchestra.run.smithi123.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 329 kB/s | 27 kB 00:00 2023-09-20T21:35:57.495 INFO:teuthology.orchestra.run.smithi089.stdout:(50/131): python3-google-auth-1.1.1-10.el8.noar 304 kB/s | 81 kB 00:00 2023-09-20T21:35:57.529 INFO:teuthology.orchestra.run.smithi089.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2023-09-20T21:35:57.534 INFO:teuthology.orchestra.run.smithi123.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-09-20T21:35:57.567 INFO:teuthology.orchestra.run.smithi022.stdout:(10/131): ceph-common-18.0.0-6213.gbf8a061f.el8 2.6 MB/s | 23 MB 00:08 2023-09-20T21:35:57.579 INFO:teuthology.orchestra.run.smithi089.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 583 kB/s | 49 kB 00:00 2023-09-20T21:35:57.593 INFO:teuthology.orchestra.run.smithi123.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-09-20T21:35:57.604 INFO:teuthology.orchestra.run.smithi089.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 780 kB/s | 59 kB 00:00 2023-09-20T21:35:57.626 INFO:teuthology.orchestra.run.smithi123.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 664 kB/s | 61 kB 00:00 2023-09-20T21:35:57.685 INFO:teuthology.orchestra.run.smithi123.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 925 kB/s | 85 kB 00:00 2023-09-20T21:35:57.687 INFO:teuthology.orchestra.run.smithi089.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 555 kB/s | 60 kB 00:00 2023-09-20T21:35:57.708 INFO:teuthology.orchestra.run.smithi022.stdout:(11/131): cephfs-mirror-18.0.0-6213.gbf8a061f.e 1.8 MB/s | 261 kB 00:00 2023-09-20T21:35:57.713 INFO:teuthology.orchestra.run.smithi089.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-09-20T21:35:57.719 INFO:teuthology.orchestra.run.smithi123.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 249 kB/s | 23 kB 00:00 2023-09-20T21:35:57.770 INFO:teuthology.orchestra.run.smithi123.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.3 MB/s | 70 kB 00:00 2023-09-20T21:35:57.775 INFO:teuthology.orchestra.run.smithi022.stdout:(12/131): libcephfs-devel-18.0.0-6213.gbf8a061f 466 kB/s | 31 kB 00:00 2023-09-20T21:35:57.894 INFO:teuthology.orchestra.run.smithi123.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 8.3 MB/s | 1.7 MB 00:00 2023-09-20T21:35:57.953 INFO:teuthology.orchestra.run.smithi123.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-09-20T21:35:57.995 INFO:teuthology.orchestra.run.smithi123.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-20T21:35:58.026 INFO:teuthology.orchestra.run.smithi022.stdout:(13/131): libcephfs2-18.0.0-6213.gbf8a061f.el8. 3.2 MB/s | 827 kB 00:00 2023-09-20T21:35:58.036 INFO:teuthology.orchestra.run.smithi123.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-09-20T21:35:58.079 INFO:teuthology.orchestra.run.smithi123.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-09-20T21:35:58.120 INFO:teuthology.orchestra.run.smithi123.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 934 kB/s | 39 kB 00:00 2023-09-20T21:35:58.126 INFO:teuthology.orchestra.run.smithi022.stdout:(14/131): libcephsqlite-18.0.0-6213.gbf8a061f.e 1.8 MB/s | 184 kB 00:00 2023-09-20T21:35:58.164 INFO:teuthology.orchestra.run.smithi089.stdout:(56/131): ceph-test-18.0.0-6213.gbf8a061f.el8.x 5.4 MB/s | 49 MB 00:09 2023-09-20T21:35:58.197 INFO:teuthology.orchestra.run.smithi089.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 31 kB/s | 16 kB 00:00 2023-09-20T21:35:58.218 INFO:teuthology.orchestra.run.smithi022.stdout:(15/131): librados-devel-18.0.0-6213.gbf8a061f. 1.5 MB/s | 139 kB 00:00 2023-09-20T21:35:58.222 INFO:teuthology.orchestra.run.smithi089.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 66 kB/s | 34 kB 00:00 2023-09-20T21:35:58.263 INFO:teuthology.orchestra.run.smithi123.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 28 MB/s | 14 MB 00:00 2023-09-20T21:35:58.289 INFO:teuthology.orchestra.run.smithi089.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 281 kB/s | 19 kB 00:00 2023-09-20T21:35:58.296 INFO:teuthology.orchestra.run.smithi123.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2023-09-20T21:35:58.306 INFO:teuthology.orchestra.run.smithi089.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-09-20T21:35:58.330 INFO:teuthology.orchestra.run.smithi123.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-09-20T21:35:58.356 INFO:teuthology.orchestra.run.smithi089.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 362 kB/s | 24 kB 00:00 2023-09-20T21:35:58.382 INFO:teuthology.orchestra.run.smithi089.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2023-09-20T21:35:58.424 INFO:teuthology.orchestra.run.smithi089.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 411 kB/s | 27 kB 00:00 2023-09-20T21:35:58.441 INFO:teuthology.orchestra.run.smithi089.stdout:(64/131): python3-rsa-4.9-2.el8.noarch.rpm 264 kB/s | 64 kB 00:00 2023-09-20T21:35:58.483 INFO:teuthology.orchestra.run.smithi089.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-09-20T21:35:58.508 INFO:teuthology.orchestra.run.smithi089.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2023-09-20T21:35:58.533 INFO:teuthology.orchestra.run.smithi089.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 666 kB/s | 61 kB 00:00 2023-09-20T21:35:58.550 INFO:teuthology.orchestra.run.smithi089.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-09-20T21:35:58.576 INFO:teuthology.orchestra.run.smithi089.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 341 kB/s | 23 kB 00:00 2023-09-20T21:35:58.585 INFO:teuthology.orchestra.run.smithi022.stdout:(16/131): ceph-osd-18.0.0-6213.gbf8a061f.el8.x8 2.7 MB/s | 18 MB 00:06 2023-09-20T21:35:58.592 INFO:teuthology.orchestra.run.smithi089.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-09-20T21:35:58.611 INFO:teuthology.orchestra.run.smithi022.stdout:(17/131): libradosstriper1-18.0.0-6213.gbf8a061 1.3 MB/s | 519 kB 00:00 2023-09-20T21:35:58.651 INFO:teuthology.orchestra.run.smithi089.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.0 MB/s | 110 kB 00:00 2023-09-20T21:35:58.693 INFO:teuthology.orchestra.run.smithi089.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-20T21:35:58.727 INFO:teuthology.orchestra.run.smithi022.stdout:(18/131): python3-ceph-argparse-18.0.0-6213.gbf 384 kB/s | 45 kB 00:00 2023-09-20T21:35:58.735 INFO:teuthology.orchestra.run.smithi089.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-09-20T21:35:58.747 INFO:teuthology.orchestra.run.smithi123.stdout:(79/131): ceph-test-18.0.0-6213.gbf8a061f.el8.x 4.9 MB/s | 49 MB 00:10 2023-09-20T21:35:58.772 INFO:teuthology.orchestra.run.smithi123.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 330 kB/s | 157 kB 00:00 2023-09-20T21:35:58.777 INFO:teuthology.orchestra.run.smithi089.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-09-20T21:35:58.797 INFO:teuthology.orchestra.run.smithi123.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 401 kB/s | 187 kB 00:00 2023-09-20T21:35:58.819 INFO:teuthology.orchestra.run.smithi089.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 934 kB/s | 39 kB 00:00 2023-09-20T21:35:58.848 INFO:teuthology.orchestra.run.smithi123.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 7.0 MB/s | 713 kB 00:00 2023-09-20T21:35:58.852 INFO:teuthology.orchestra.run.smithi089.stdout:(76/131): thrift-0.13.0-2.el8.x86_64.rpm 5.4 MB/s | 1.7 MB 00:00 2023-09-20T21:35:58.861 INFO:teuthology.orchestra.run.smithi022.stdout:(19/131): python3-ceph-common-18.0.0-6213.gbf8a 941 kB/s | 125 kB 00:00 2023-09-20T21:35:58.873 INFO:teuthology.orchestra.run.smithi123.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 353 kB/s | 27 kB 00:00 2023-09-20T21:35:58.906 INFO:teuthology.orchestra.run.smithi123.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 5.4 MB/s | 320 kB 00:00 2023-09-20T21:35:58.911 INFO:teuthology.orchestra.run.smithi089.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 7.3 MB/s | 688 kB 00:00 2023-09-20T21:35:58.931 INFO:teuthology.orchestra.run.smithi123.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 753 kB/s | 44 kB 00:00 2023-09-20T21:35:58.945 INFO:teuthology.orchestra.run.smithi089.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 5.7 MB/s | 538 kB 00:00 2023-09-20T21:35:58.961 INFO:teuthology.orchestra.run.smithi022.stdout:(20/131): python3-cephfs-18.0.0-6213.gbf8a061f. 1.9 MB/s | 197 kB 00:00 2023-09-20T21:35:58.970 INFO:teuthology.orchestra.run.smithi089.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-20T21:35:58.982 INFO:teuthology.orchestra.run.smithi123.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-09-20T21:35:58.995 INFO:teuthology.orchestra.run.smithi089.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-09-20T21:35:59.037 INFO:teuthology.orchestra.run.smithi089.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-09-20T21:35:59.050 INFO:teuthology.orchestra.run.smithi123.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.2 MB/s | 225 kB 00:00 2023-09-20T21:35:59.073 INFO:teuthology.orchestra.run.smithi089.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 796 kB/s | 27 kB 00:00 2023-09-20T21:35:59.091 INFO:teuthology.orchestra.run.smithi123.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.6 MB/s | 64 kB 00:00 2023-09-20T21:35:59.113 INFO:teuthology.orchestra.run.smithi089.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-09-20T21:35:59.138 INFO:teuthology.orchestra.run.smithi089.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.7 MB/s | 44 kB 00:00 2023-09-20T21:35:59.145 INFO:teuthology.orchestra.run.smithi022.stdout:(21/131): python3-rados-18.0.0-6213.gbf8a061f.e 2.0 MB/s | 371 kB 00:00 2023-09-20T21:35:59.158 INFO:teuthology.orchestra.run.smithi123.stdout:(89/131): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2023-09-20T21:35:59.225 INFO:teuthology.orchestra.run.smithi123.stdout:(90/131): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-09-20T21:35:59.250 INFO:teuthology.orchestra.run.smithi123.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 218 kB/s | 20 kB 00:00 2023-09-20T21:35:59.275 INFO:teuthology.orchestra.run.smithi123.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 686 kB/s | 126 kB 00:00 2023-09-20T21:35:59.281 INFO:teuthology.orchestra.run.smithi089.stdout:(85/131): python3-scipy-1.0.0-21.module+el8.5.0 19 MB/s | 14 MB 00:00 2023-09-20T21:35:59.300 INFO:teuthology.orchestra.run.smithi123.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.3 MB/s | 103 kB 00:00 2023-09-20T21:35:59.320 INFO:teuthology.orchestra.run.smithi022.stdout:(22/131): python3-rbd-18.0.0-6213.gbf8a061f.el8 2.0 MB/s | 364 kB 00:00 2023-09-20T21:35:59.342 INFO:teuthology.orchestra.run.smithi123.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2023-09-20T21:35:59.355 INFO:teuthology.orchestra.run.smithi089.stdout:(86/131): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2023-09-20T21:35:59.368 INFO:teuthology.orchestra.run.smithi123.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-09-20T21:35:59.389 INFO:teuthology.orchestra.run.smithi089.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 4.1 MB/s | 457 kB 00:00 2023-09-20T21:35:59.410 INFO:teuthology.orchestra.run.smithi123.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-09-20T21:35:59.414 INFO:teuthology.orchestra.run.smithi089.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.8 MB/s | 225 kB 00:00 2023-09-20T21:35:59.429 INFO:teuthology.orchestra.run.smithi022.stdout:(23/131): python3-rgw-18.0.0-6213.gbf8a061f.el8 983 kB/s | 106 kB 00:00 2023-09-20T21:35:59.440 INFO:teuthology.orchestra.run.smithi089.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-09-20T21:35:59.452 INFO:teuthology.orchestra.run.smithi123.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-20T21:35:59.465 INFO:teuthology.orchestra.run.smithi089.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-09-20T21:35:59.490 INFO:teuthology.orchestra.run.smithi089.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-09-20T21:35:59.516 INFO:teuthology.orchestra.run.smithi089.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-09-20T21:35:59.529 INFO:teuthology.orchestra.run.smithi022.stdout:(24/131): rbd-fuse-18.0.0-6213.gbf8a061f.el8.x8 937 kB/s | 94 kB 00:00 2023-09-20T21:35:59.535 INFO:teuthology.orchestra.run.smithi123.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 19 MB/s | 4.9 MB 00:00 2023-09-20T21:35:59.549 INFO:teuthology.orchestra.run.smithi089.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-09-20T21:35:59.591 INFO:teuthology.orchestra.run.smithi089.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-09-20T21:35:59.611 INFO:teuthology.orchestra.run.smithi123.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-09-20T21:35:59.658 INFO:teuthology.orchestra.run.smithi089.stdout:(95/131): python3-numpy-1.14.3-10.el8.x86_64.rp 7.2 MB/s | 3.7 MB 00:00 2023-09-20T21:35:59.686 INFO:teuthology.orchestra.run.smithi123.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.2 MB/s | 808 kB 00:00 2023-09-20T21:35:59.700 INFO:teuthology.orchestra.run.smithi089.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-09-20T21:35:59.719 INFO:teuthology.orchestra.run.smithi123.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-09-20T21:35:59.742 INFO:teuthology.orchestra.run.smithi089.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-09-20T21:35:59.761 INFO:teuthology.orchestra.run.smithi123.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 10 MB/s | 1.5 MB 00:00 2023-09-20T21:35:59.787 INFO:teuthology.orchestra.run.smithi123.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-09-20T21:35:59.812 INFO:teuthology.orchestra.run.smithi123.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-09-20T21:35:59.834 INFO:teuthology.orchestra.run.smithi089.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 15 MB/s | 4.9 MB 00:00 2023-09-20T21:35:59.838 INFO:teuthology.orchestra.run.smithi123.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-09-20T21:35:59.863 INFO:teuthology.orchestra.run.smithi123.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 954 kB/s | 48 kB 00:00 2023-09-20T21:35:59.888 INFO:teuthology.orchestra.run.smithi123.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-20T21:35:59.913 INFO:teuthology.orchestra.run.smithi123.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 853 kB/s | 43 kB 00:00 2023-09-20T21:35:59.917 INFO:teuthology.orchestra.run.smithi089.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-09-20T21:35:59.939 INFO:teuthology.orchestra.run.smithi123.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-09-20T21:35:59.951 INFO:teuthology.orchestra.run.smithi089.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-09-20T21:35:59.964 INFO:teuthology.orchestra.run.smithi123.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-09-20T21:35:59.985 INFO:teuthology.orchestra.run.smithi089.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-09-20T21:36:00.027 INFO:teuthology.orchestra.run.smithi089.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-09-20T21:36:00.030 INFO:teuthology.orchestra.run.smithi123.stdout:(111/131): postgresql-server-10.21-2.module+el8 8.8 MB/s | 5.1 MB 00:00 2023-09-20T21:36:00.052 INFO:teuthology.orchestra.run.smithi089.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-09-20T21:36:00.056 INFO:teuthology.orchestra.run.smithi123.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 204 kB/s | 19 kB 00:00 2023-09-20T21:36:00.081 INFO:teuthology.orchestra.run.smithi123.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-09-20T21:36:00.102 INFO:teuthology.orchestra.run.smithi089.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 567 kB/s | 43 kB 00:00 2023-09-20T21:36:00.107 INFO:teuthology.orchestra.run.smithi123.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.3 MB/s | 560 kB 00:00 2023-09-20T21:36:00.132 INFO:teuthology.orchestra.run.smithi123.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 355 kB/s | 18 kB 00:00 2023-09-20T21:36:00.136 INFO:teuthology.orchestra.run.smithi089.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2023-09-20T21:36:00.162 INFO:teuthology.orchestra.run.smithi089.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 817 kB/s | 48 kB 00:00 2023-09-20T21:36:00.165 INFO:teuthology.orchestra.run.smithi123.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 5.0 MB/s | 559 kB 00:00 2023-09-20T21:36:00.186 INFO:teuthology.orchestra.run.smithi089.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-09-20T21:36:00.212 INFO:teuthology.orchestra.run.smithi089.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 846 kB/s | 43 kB 00:00 2023-09-20T21:36:00.237 INFO:teuthology.orchestra.run.smithi089.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-09-20T21:36:00.240 INFO:teuthology.orchestra.run.smithi123.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-09-20T21:36:00.262 INFO:teuthology.orchestra.run.smithi089.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-09-20T21:36:00.266 INFO:teuthology.orchestra.run.smithi123.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.7 MB/s | 238 kB 00:00 2023-09-20T21:36:00.304 INFO:teuthology.orchestra.run.smithi089.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-09-20T21:36:00.322 INFO:teuthology.orchestra.run.smithi123.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 886 kB/s | 72 kB 00:00 2023-09-20T21:36:00.329 INFO:teuthology.orchestra.run.smithi089.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 279 kB/s | 19 kB 00:00 2023-09-20T21:36:00.358 INFO:teuthology.orchestra.run.smithi123.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 902 kB/s | 83 kB 00:00 2023-09-20T21:36:00.383 INFO:teuthology.orchestra.run.smithi123.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 989 kB/s | 60 kB 00:00 2023-09-20T21:36:00.408 INFO:teuthology.orchestra.run.smithi123.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-09-20T21:36:00.413 INFO:teuthology.orchestra.run.smithi089.stdout:(113/131): postgresql-server-10.21-2.module+el8 7.5 MB/s | 5.1 MB 00:00 2023-09-20T21:36:00.434 INFO:teuthology.orchestra.run.smithi123.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-09-20T21:36:00.439 INFO:teuthology.orchestra.run.smithi089.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-09-20T21:36:00.456 INFO:teuthology.orchestra.run.smithi089.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 423 kB/s | 18 kB 00:00 2023-09-20T21:36:00.459 INFO:teuthology.orchestra.run.smithi123.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-09-20T21:36:00.489 INFO:teuthology.orchestra.run.smithi089.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.4 MB/s | 559 kB 00:00 2023-09-20T21:36:00.493 INFO:teuthology.orchestra.run.smithi123.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-09-20T21:36:00.514 INFO:teuthology.orchestra.run.smithi089.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.8 MB/s | 605 kB 00:00 2023-09-20T21:36:00.518 INFO:teuthology.orchestra.run.smithi123.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.8 MB/s | 171 kB 00:00 2023-09-20T21:36:00.540 INFO:teuthology.orchestra.run.smithi089.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-09-20T21:36:00.543 INFO:teuthology.orchestra.run.smithi123.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-20T21:36:00.565 INFO:teuthology.orchestra.run.smithi089.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-09-20T21:36:00.590 INFO:teuthology.orchestra.run.smithi089.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-20T21:36:00.594 INFO:teuthology.orchestra.run.smithi123.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 5.4 MB/s | 2.3 MB 00:00 2023-09-20T21:36:00.616 INFO:teuthology.orchestra.run.smithi089.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-09-20T21:36:00.632 INFO:teuthology.orchestra.run.smithi089.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-09-20T21:36:00.644 INFO:teuthology.orchestra.run.smithi123.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-09-20T21:36:00.658 INFO:teuthology.orchestra.run.smithi089.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-09-20T21:36:00.683 INFO:teuthology.orchestra.run.smithi089.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 766 kB/s | 39 kB 00:00 2023-09-20T21:36:00.717 INFO:teuthology.orchestra.run.smithi089.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-09-20T21:36:00.742 INFO:teuthology.orchestra.run.smithi089.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-09-20T21:36:00.767 INFO:teuthology.orchestra.run.smithi089.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-09-20T21:36:00.788 INFO:teuthology.orchestra.run.smithi022.stdout:(25/131): rbd-mirror-18.0.0-6213.gbf8a061f.el8. 2.8 MB/s | 3.6 MB 00:01 2023-09-20T21:36:00.817 INFO:teuthology.orchestra.run.smithi089.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 7.1 MB/s | 2.3 MB 00:00 2023-09-20T21:36:00.860 INFO:teuthology.orchestra.run.smithi089.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2023-09-20T21:36:00.896 INFO:teuthology.orchestra.run.smithi022.stdout:(26/131): rbd-nbd-18.0.0-6213.gbf8a061f.el8.x86 1.6 MB/s | 179 kB 00:00 2023-09-20T21:36:01.006 INFO:teuthology.orchestra.run.smithi022.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6213.g 213 kB/s | 23 kB 00:00 2023-09-20T21:36:01.113 INFO:teuthology.orchestra.run.smithi022.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6213.gbf8a061 1.2 MB/s | 132 kB 00:00 2023-09-20T21:36:01.197 INFO:teuthology.orchestra.run.smithi022.stdout:(29/131): librgw2-18.0.0-6213.gbf8a061f.el8.x86 2.6 MB/s | 6.9 MB 00:02 2023-09-20T21:36:01.781 INFO:teuthology.orchestra.run.smithi022.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6213.gbf8a0 2.6 MB/s | 1.7 MB 00:00 2023-09-20T21:36:01.794 INFO:teuthology.orchestra.run.smithi123.stdout:(130/131): librbd1-18.0.0-6213.gbf8a061f.el8.x8 3.0 MB/s | 3.7 MB 00:01 2023-09-20T21:36:01.914 INFO:teuthology.orchestra.run.smithi022.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6213.gbf 1.8 MB/s | 242 kB 00:00 2023-09-20T21:36:02.040 INFO:teuthology.orchestra.run.smithi022.stdout:(32/131): ceph-mgr-rook-18.0.0-6213.gbf8a061f.e 379 kB/s | 47 kB 00:00 2023-09-20T21:36:02.085 INFO:teuthology.orchestra.run.smithi089.stdout:(130/131): librbd1-18.0.0-6213.gbf8a061f.el8.x8 2.8 MB/s | 3.7 MB 00:01 2023-09-20T21:36:02.086 INFO:teuthology.orchestra.run.smithi123.stdout:(131/131): librados2-18.0.0-6213.gbf8a061f.el8. 2.4 MB/s | 3.7 MB 00:01 2023-09-20T21:36:02.089 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2023-09-20T21:36:02.089 INFO:teuthology.orchestra.run.smithi123.stdout:Total 12 MB/s | 208 MB 00:17 2023-09-20T21:36:02.157 INFO:teuthology.orchestra.run.smithi022.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6213.gb 124 kB/s | 15 kB 00:00 2023-09-20T21:36:02.282 INFO:teuthology.orchestra.run.smithi022.stdout:(34/131): ceph-volume-18.0.0-6213.gbf8a061f.el8 2.0 MB/s | 261 kB 00:00 2023-09-20T21:36:02.408 INFO:teuthology.orchestra.run.smithi022.stdout:(35/131): cephadm-18.0.0-6213.gbf8a061f.el8.noa 1.4 MB/s | 179 kB 00:00 2023-09-20T21:36:02.525 INFO:teuthology.orchestra.run.smithi022.stdout:(36/131): cephfs-top-18.0.0-6213.gbf8a061f.el8. 155 kB/s | 18 kB 00:00 2023-09-20T21:36:02.618 INFO:teuthology.orchestra.run.smithi089.stdout:(131/131): librados2-18.0.0-6213.gbf8a061f.el8. 2.0 MB/s | 3.7 MB 00:01 2023-09-20T21:36:02.623 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2023-09-20T21:36:02.624 INFO:teuthology.orchestra.run.smithi089.stdout:Total 11 MB/s | 208 MB 00:18 2023-09-20T21:36:02.775 INFO:teuthology.orchestra.run.smithi022.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-09-20T21:36:02.825 INFO:teuthology.orchestra.run.smithi022.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.2 MB/s | 59 kB 00:00 2023-09-20T21:36:02.892 INFO:teuthology.orchestra.run.smithi022.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-09-20T21:36:02.942 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2023-09-20T21:36:02.992 INFO:teuthology.orchestra.run.smithi022.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.6 MB/s | 476 kB 00:00 2023-09-20T21:36:03.059 INFO:teuthology.orchestra.run.smithi022.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2023-09-20T21:36:03.061 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2023-09-20T21:36:03.061 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2023-09-20T21:36:03.126 INFO:teuthology.orchestra.run.smithi022.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-09-20T21:36:03.185 INFO:teuthology.orchestra.run.smithi022.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 566 kB/s | 33 kB 00:00 2023-09-20T21:36:03.244 INFO:teuthology.orchestra.run.smithi022.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 280 kB/s | 16 kB 00:00 2023-09-20T21:36:03.294 INFO:teuthology.orchestra.run.smithi022.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2023-09-20T21:36:03.361 INFO:teuthology.orchestra.run.smithi022.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.6 MB/s | 384 kB 00:00 2023-09-20T21:36:03.420 INFO:teuthology.orchestra.run.smithi022.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-09-20T21:36:03.465 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2023-09-20T21:36:03.478 INFO:teuthology.orchestra.run.smithi022.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-09-20T21:36:03.538 INFO:teuthology.orchestra.run.smithi022.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 303 kB/s | 18 kB 00:00 2023-09-20T21:36:03.588 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2023-09-20T21:36:03.589 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2023-09-20T21:36:03.646 INFO:teuthology.orchestra.run.smithi022.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 11 MB/s | 1.2 MB 00:00 2023-09-20T21:36:03.705 INFO:teuthology.orchestra.run.smithi022.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 839 kB/s | 49 kB 00:00 2023-09-20T21:36:03.764 INFO:teuthology.orchestra.run.smithi022.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-09-20T21:36:04.204 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2023-09-20T21:36:04.208 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2023-09-20T21:36:04.272 INFO:teuthology.orchestra.run.smithi022.stdout:(53/131): ceph-test-18.0.0-6213.gbf8a061f.el8.x 5.6 MB/s | 49 MB 00:08 2023-09-20T21:36:04.297 INFO:teuthology.orchestra.run.smithi022.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 113 kB/s | 60 kB 00:00 2023-09-20T21:36:04.398 INFO:teuthology.orchestra.run.smithi022.stdout:(55/131): ceph-mgr-diskprediction-local-18.0.0- 2.3 MB/s | 7.4 MB 00:03 2023-09-20T21:36:04.423 INFO:teuthology.orchestra.run.smithi022.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 124 kB/s | 16 kB 00:00 2023-09-20T21:36:04.456 INFO:teuthology.orchestra.run.smithi022.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 577 kB/s | 34 kB 00:00 2023-09-20T21:36:04.515 INFO:teuthology.orchestra.run.smithi022.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-09-20T21:36:04.574 INFO:teuthology.orchestra.run.smithi022.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 322 kB/s | 19 kB 00:00 2023-09-20T21:36:04.641 INFO:teuthology.orchestra.run.smithi022.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 360 kB/s | 24 kB 00:00 2023-09-20T21:36:04.666 INFO:teuthology.orchestra.run.smithi022.stdout:(61/131): python3-pecan-1.3.2-9.el8.noarch.rpm 719 kB/s | 283 kB 00:00 2023-09-20T21:36:04.691 INFO:teuthology.orchestra.run.smithi022.stdout:(62/131): python3-routes-2.4.1-12.el8.noarch.rp 732 kB/s | 196 kB 00:00 2023-09-20T21:36:04.717 INFO:teuthology.orchestra.run.smithi022.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 388 kB/s | 29 kB 00:00 2023-09-20T21:36:04.717 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2023-09-20T21:36:04.721 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2023-09-20T21:36:04.774 INFO:teuthology.orchestra.run.smithi022.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 256 kB/s | 27 kB 00:00 2023-09-20T21:36:04.809 INFO:teuthology.orchestra.run.smithi022.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.1 MB/s | 255 kB 00:00 2023-09-20T21:36:04.834 INFO:teuthology.orchestra.run.smithi022.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-09-20T21:36:04.885 INFO:teuthology.orchestra.run.smithi022.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 556 kB/s | 61 kB 00:00 2023-09-20T21:36:04.910 INFO:teuthology.orchestra.run.smithi022.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 306 kB/s | 23 kB 00:00 2023-09-20T21:36:04.935 INFO:teuthology.orchestra.run.smithi022.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 852 kB/s | 85 kB 00:00 2023-09-20T21:36:04.960 INFO:teuthology.orchestra.run.smithi022.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-09-20T21:36:05.011 INFO:teuthology.orchestra.run.smithi022.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 624 kB/s | 47 kB 00:00 2023-09-20T21:36:05.044 INFO:teuthology.orchestra.run.smithi022.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.6 MB/s | 200 kB 00:00 2023-09-20T21:36:05.070 INFO:teuthology.orchestra.run.smithi022.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 729 kB/s | 43 kB 00:00 2023-09-20T21:36:05.112 INFO:teuthology.orchestra.run.smithi022.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 7.7 MB/s | 1.7 MB 00:00 2023-09-20T21:36:05.137 INFO:teuthology.orchestra.run.smithi022.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 720 kB/s | 48 kB 00:00 2023-09-20T21:36:05.171 INFO:teuthology.orchestra.run.smithi022.stdout:(76/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-09-20T21:36:05.196 INFO:teuthology.orchestra.run.smithi022.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 852 kB/s | 43 kB 00:00 2023-09-20T21:36:05.221 INFO:teuthology.orchestra.run.smithi022.stdout:(78/131): libsepol-devel-2.9-3.el8.x86_64.rpm 796 kB/s | 87 kB 00:00 2023-09-20T21:36:05.247 INFO:teuthology.orchestra.run.smithi022.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-09-20T21:36:05.280 INFO:teuthology.orchestra.run.smithi022.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-09-20T21:36:05.305 INFO:teuthology.orchestra.run.smithi022.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2023-09-20T21:36:05.364 INFO:teuthology.orchestra.run.smithi022.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.8 MB/s | 155 kB 00:00 2023-09-20T21:36:05.406 INFO:teuthology.orchestra.run.smithi022.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.0 MB/s | 560 kB 00:00 2023-09-20T21:36:05.431 INFO:teuthology.orchestra.run.smithi022.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 268 kB/s | 18 kB 00:00 2023-09-20T21:36:05.465 INFO:teuthology.orchestra.run.smithi022.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 3.4 MB/s | 559 kB 00:00 2023-09-20T21:36:05.499 INFO:teuthology.orchestra.run.smithi022.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-09-20T21:36:05.533 INFO:teuthology.orchestra.run.smithi022.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-09-20T21:36:05.558 INFO:teuthology.orchestra.run.smithi022.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-09-20T21:36:05.583 INFO:teuthology.orchestra.run.smithi022.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-09-20T21:36:05.609 INFO:teuthology.orchestra.run.smithi022.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-09-20T21:36:05.634 INFO:teuthology.orchestra.run.smithi022.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-09-20T21:36:05.659 INFO:teuthology.orchestra.run.smithi022.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-09-20T21:36:05.684 INFO:teuthology.orchestra.run.smithi022.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-09-20T21:36:05.726 INFO:teuthology.orchestra.run.smithi022.stdout:(94/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 4.0 MB/s | 171 kB 00:00 2023-09-20T21:36:05.760 INFO:teuthology.orchestra.run.smithi022.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.3 MB/s | 644 kB 00:00 2023-09-20T21:36:05.785 INFO:teuthology.orchestra.run.smithi022.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-09-20T21:36:05.827 INFO:teuthology.orchestra.run.smithi022.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-09-20T21:36:05.869 INFO:teuthology.orchestra.run.smithi022.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-09-20T21:36:05.936 INFO:teuthology.orchestra.run.smithi022.stdout:(99/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 4.9 MB/s | 2.3 MB 00:00 2023-09-20T21:36:05.970 INFO:teuthology.orchestra.run.smithi022.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.5 MB/s | 259 kB 00:00 2023-09-20T21:36:06.003 INFO:teuthology.orchestra.run.smithi022.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-09-20T21:36:06.029 INFO:teuthology.orchestra.run.smithi022.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 665 kB/s | 39 kB 00:00 2023-09-20T21:36:06.079 INFO:teuthology.orchestra.run.smithi022.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 8.9 MB/s | 688 kB 00:00 2023-09-20T21:36:06.113 INFO:teuthology.orchestra.run.smithi022.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-09-20T21:36:06.138 INFO:teuthology.orchestra.run.smithi022.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-09-20T21:36:06.172 INFO:teuthology.orchestra.run.smithi022.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-09-20T21:36:06.222 INFO:teuthology.orchestra.run.smithi022.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 8.3 MB/s | 713 kB 00:00 2023-09-20T21:36:06.256 INFO:teuthology.orchestra.run.smithi022.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 796 kB/s | 27 kB 00:00 2023-09-20T21:36:06.306 INFO:teuthology.orchestra.run.smithi022.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-09-20T21:36:06.339 INFO:teuthology.orchestra.run.smithi022.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-09-20T21:36:06.532 INFO:teuthology.orchestra.run.smithi022.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2023-09-20T21:36:06.582 INFO:teuthology.orchestra.run.smithi022.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-09-20T21:36:06.658 INFO:teuthology.orchestra.run.smithi022.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 12 MB/s | 3.7 MB 00:00 2023-09-20T21:36:06.691 INFO:teuthology.orchestra.run.smithi022.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.0 MB/s | 225 kB 00:00 2023-09-20T21:36:06.716 INFO:teuthology.orchestra.run.smithi022.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-09-20T21:36:06.741 INFO:teuthology.orchestra.run.smithi022.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-09-20T21:36:06.767 INFO:teuthology.orchestra.run.smithi022.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-09-20T21:36:06.792 INFO:teuthology.orchestra.run.smithi022.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-09-20T21:36:06.843 INFO:teuthology.orchestra.run.smithi022.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-09-20T21:36:06.885 INFO:teuthology.orchestra.run.smithi022.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-09-20T21:36:07.268 INFO:teuthology.orchestra.run.smithi022.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 9.1 MB/s | 14 MB 00:01 2023-09-20T21:36:07.344 INFO:teuthology.orchestra.run.smithi022.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 2.6 MB/s | 202 kB 00:00 2023-09-20T21:36:07.419 INFO:teuthology.orchestra.run.smithi022.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 4.6 MB/s | 354 kB 00:00 2023-09-20T21:36:07.536 INFO:teuthology.orchestra.run.smithi022.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 6.5 MB/s | 4.9 MB 00:00 2023-09-20T21:36:07.670 INFO:teuthology.orchestra.run.smithi022.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 6.0 MB/s | 4.7 MB 00:00 2023-09-20T21:36:07.712 INFO:teuthology.orchestra.run.smithi022.stdout:(126/131): postgresql-contrib-10.21-2.module+el 4.5 MB/s | 808 kB 00:00 2023-09-20T21:36:07.787 INFO:teuthology.orchestra.run.smithi022.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-09-20T21:36:07.896 INFO:teuthology.orchestra.run.smithi022.stdout:(128/131): postgresql-server-10.21-2.module+el8 11 MB/s | 5.1 MB 00:00 2023-09-20T21:36:07.963 INFO:teuthology.orchestra.run.smithi022.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-09-20T21:36:08.846 INFO:teuthology.orchestra.run.smithi022.stdout:(130/131): librbd1-18.0.0-6213.gbf8a061f.el8.x8 3.5 MB/s | 3.7 MB 00:01 2023-09-20T21:36:09.121 INFO:teuthology.orchestra.run.smithi022.stdout:(131/131): librados2-18.0.0-6213.gbf8a061f.el8. 2.6 MB/s | 3.7 MB 00:01 2023-09-20T21:36:09.125 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2023-09-20T21:36:09.125 INFO:teuthology.orchestra.run.smithi022.stdout:Total 10 MB/s | 208 MB 00:20 2023-09-20T21:36:09.126 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2023-09-20T21:36:09.331 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-20T21:36:09.486 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-20T21:36:09.565 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2023-09-20T21:36:09.699 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-20T21:36:09.738 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-20T21:36:09.875 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-20T21:36:09.876 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-20T21:36:10.001 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-20T21:36:10.113 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-09-20T21:36:10.216 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-20T21:36:10.306 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-20T21:36:10.414 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-09-20T21:36:10.618 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-20T21:36:10.640 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-20T21:36:10.813 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2023-09-20T21:36:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 6/134 2023-09-20T21:36:10.959 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2023-09-20T21:36:10.960 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2023-09-20T21:36:11.049 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-09-20T21:36:11.216 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 6/134 2023-09-20T21:36:11.223 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 6/134 2023-09-20T21:36:11.433 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 7/134 2023-09-20T21:36:11.548 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 7/134 2023-09-20T21:36:11.616 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 6/134 2023-09-20T21:36:11.758 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 8/134 2023-09-20T21:36:11.790 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 7/134 2023-09-20T21:36:11.897 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 8/134 2023-09-20T21:36:11.906 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 7/134 2023-09-20T21:36:12.081 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 8/134 2023-09-20T21:36:12.174 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 8/134 2023-09-20T21:36:12.201 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:18.0.0-6213.gbf8a061f.el8.x86_64 9/134 2023-09-20T21:36:12.348 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:18.0.0-6213.gbf8a061f.el8.x86_64 9/134 2023-09-20T21:36:12.438 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-20T21:36:12.578 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2023-09-20T21:36:12.578 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-20T21:36:12.583 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2023-09-20T21:36:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-20T21:36:12.969 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-20T21:36:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-20T21:36:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-20T21:36:13.349 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-20T21:36:13.466 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 14/134 2023-09-20T21:36:13.558 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 14/134 2023-09-20T21:36:13.597 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-20T21:36:13.708 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86 15/134 2023-09-20T21:36:13.722 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 14/134 2023-09-20T21:36:13.816 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 14/134 2023-09-20T21:36:13.819 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86 15/134 2023-09-20T21:36:13.949 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-20T21:36:13.997 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86 15/134 2023-09-20T21:36:14.071 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86 15/134 2023-09-20T21:36:14.088 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-20T21:36:14.197 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-20T21:36:14.241 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-20T21:36:14.326 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-09-20T21:36:14.489 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-20T21:36:14.852 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-20T21:36:15.057 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-20T21:36:15.101 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-09-20T21:36:15.314 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-20T21:36:15.651 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-20T21:36:15.840 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-20T21:36:15.908 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-09-20T21:36:16.080 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-09-20T21:36:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-20T21:36:16.774 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-09-20T21:36:16.805 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-20T21:36:17.029 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-09-20T21:36:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-20T21:36:17.231 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-09-20T21:36:17.277 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-20T21:36:17.434 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-09-20T21:36:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-20T21:36:17.611 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-09-20T21:36:17.745 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2023-09-20T21:36:17.747 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-20T21:36:17.827 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-09-20T21:36:17.945 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-09-20T21:36:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-20T21:36:18.065 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-20T21:36:18.155 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:18.0.0-6213.gbf8a061f.el8. 26/134 2023-09-20T21:36:18.191 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-09-20T21:36:18.245 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:18.0.0-6213.gbf8a061f.el8. 26/134 2023-09-20T21:36:18.310 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:18.0.0-6213.gbf8a061f.el 27/134 2023-09-20T21:36:18.354 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-20T21:36:18.358 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:18.0.0-6213.gbf8a061f.el 27/134 2023-09-20T21:36:18.499 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-09-20T21:36:18.729 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-20T21:36:19.202 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:18.0.0-6213.gbf8a061f.el8.x86_6 28/134 2023-09-20T21:36:19.229 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-09-20T21:36:19.248 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:18.0.0-6213.gbf8a061f.el8.x86_6 28/134 2023-09-20T21:36:19.445 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 5/134 2023-09-20T21:36:19.506 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-20T21:36:19.536 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-09-20T21:36:19.777 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-20T21:36:19.777 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-09-20T21:36:19.937 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 5/134 2023-09-20T21:36:20.006 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-20T21:36:20.016 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-09-20T21:36:20.120 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 6/134 2023-09-20T21:36:20.196 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-20T21:36:20.248 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-20T21:36:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 6/134 2023-09-20T21:36:20.495 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 7/134 2023-09-20T21:36:20.600 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 7/134 2023-09-20T21:36:20.751 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:18.0.0-6213.gbf8a061f.el8.x86_64 8/134 2023-09-20T21:36:20.930 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-09-20T21:36:21.191 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-09-20T21:36:21.402 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-09-20T21:36:21.852 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-09-20T21:36:21.957 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-20T21:36:22.001 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-09-20T21:36:22.088 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-09-20T21:36:22.261 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 14/134 2023-09-20T21:36:22.264 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-20T21:36:22.366 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-09-20T21:36:22.483 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-20T21:36:22.576 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-20T21:36:22.612 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 14/134 2023-09-20T21:36:22.673 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-20T21:36:22.799 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-20T21:36:22.804 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86 15/134 2023-09-20T21:36:22.894 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 36/134 2023-09-20T21:36:22.986 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86 15/134 2023-09-20T21:36:22.995 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 36/134 2023-09-20T21:36:23.041 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:18.0.0-6213.gbf8a061f.el8.x86_6 37/134 2023-09-20T21:36:23.185 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:18.0.0-6213.gbf8a061f.el8.x86_6 37/134 2023-09-20T21:36:23.203 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-09-20T21:36:23.211 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-20T21:36:23.380 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-09-20T21:36:23.397 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-09-20T21:36:23.406 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-20T21:36:23.533 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-09-20T21:36:23.590 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-20T21:36:23.605 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-09-20T21:36:23.734 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-09-20T21:36:23.741 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-20T21:36:23.744 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-20T21:36:23.905 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-09-20T21:36:23.926 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-09-20T21:36:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-20T21:36:24.061 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-20T21:36:24.168 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-20T21:36:24.173 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-09-20T21:36:24.371 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-09-20T21:36:24.541 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-20T21:36:24.735 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-09-20T21:36:24.803 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-09-20T21:36:24.824 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-20T21:36:25.014 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-20T21:36:25.020 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-20T21:36:25.026 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-09-20T21:36:25.224 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-20T21:36:25.232 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-09-20T21:36:25.475 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-20T21:36:25.476 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-09-20T21:36:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-09-20T21:36:25.741 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-09-20T21:36:25.761 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-20T21:36:25.903 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-09-20T21:36:25.953 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-20T21:36:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-09-20T21:36:26.145 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-20T21:36:26.238 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-09-20T21:36:26.350 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-20T21:36:26.413 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-09-20T21:36:26.516 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-20T21:36:26.651 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-09-20T21:36:26.715 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-20T21:36:26.803 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-09-20T21:36:26.817 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-09-20T21:36:26.992 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-09-20T21:36:27.004 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-20T21:36:27.027 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-09-20T21:36:27.250 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-20T21:36:27.251 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-09-20T21:36:27.328 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-09-20T21:36:27.475 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-20T21:36:27.570 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-09-20T21:36:27.579 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-09-20T21:36:27.702 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-20T21:36:27.752 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:18.0.0-6213.gbf8a061f.el8. 26/134 2023-09-20T21:36:27.844 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-20T21:36:27.934 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:18.0.0-6213.gbf8a061f.el 27/134 2023-09-20T21:36:28.005 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-20T21:36:28.161 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-09-20T21:36:28.162 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-20T21:36:28.188 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:18.0.0-6213.gbf8a061f.el8.x86_6 28/134 2023-09-20T21:36:28.220 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-20T21:36:28.379 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-09-20T21:36:28.386 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-20T21:36:28.465 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-20T21:36:28.571 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-09-20T21:36:28.706 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-09-20T21:36:28.880 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-09-20T21:36:28.924 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-20T21:36:28.950 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-09-20T21:36:29.054 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-20T21:36:29.133 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-20T21:36:29.217 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-09-20T21:36:29.393 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-20T21:36:29.399 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-09-20T21:36:29.555 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-20T21:36:29.563 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-20T21:36:29.637 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-09-20T21:36:29.704 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-20T21:36:29.875 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-20T21:36:29.892 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-09-20T21:36:30.056 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-09-20T21:36:30.120 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-20T21:36:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-20T21:36:30.319 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-20T21:36:30.410 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-20T21:36:30.492 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-20T21:36:30.608 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-09-20T21:36:30.633 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-20T21:36:30.647 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-20T21:36:30.814 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-20T21:36:30.869 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-20T21:36:31.000 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-20T21:36:31.025 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-20T21:36:31.173 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-20T21:36:31.190 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-20T21:36:31.340 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-20T21:36:31.378 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-20T21:36:31.468 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-20T21:36:31.574 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-20T21:36:31.693 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-20T21:36:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-20T21:36:31.833 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-09-20T21:36:31.940 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-20T21:36:31.966 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-20T21:36:32.102 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-20T21:36:32.126 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-20T21:36:32.219 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-09-20T21:36:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-20T21:36:32.359 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-20T21:36:32.482 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-09-20T21:36:32.561 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-20T21:36:32.566 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-20T21:36:32.719 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-20T21:36:32.720 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-09-20T21:36:32.744 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-20T21:36:32.920 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-20T21:36:32.952 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-09-20T21:36:32.960 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-20T21:36:33.179 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-20T21:36:33.193 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-09-20T21:36:33.335 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-20T21:36:33.368 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 36/134 2023-09-20T21:36:33.560 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:18.0.0-6213.gbf8a061f.el8.x86_6 37/134 2023-09-20T21:36:33.588 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-20T21:36:33.745 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-09-20T21:36:33.933 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-09-20T21:36:34.016 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-20T21:36:34.134 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-09-20T21:36:34.257 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-09-20T21:36:34.374 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-20T21:36:34.506 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-09-20T21:36:34.628 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-20T21:36:34.708 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-20T21:36:34.734 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-09-20T21:36:34.913 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-20T21:36:34.926 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-20T21:36:34.999 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-20T21:36:35.009 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-09-20T21:36:35.164 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-20T21:36:35.252 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-09-20T21:36:35.262 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-20T21:36:35.377 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-20T21:36:35.449 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-09-20T21:36:35.572 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-20T21:36:35.596 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-09-20T21:36:35.757 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-09-20T21:36:35.849 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-20T21:36:35.971 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-09-20T21:36:36.044 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-20T21:36:36.142 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-09-20T21:36:36.183 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-20T21:36:36.344 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 92/134 2023-09-20T21:36:36.369 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-09-20T21:36:36.418 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 92/134 2023-09-20T21:36:36.595 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 93/134 2023-09-20T21:36:36.613 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-09-20T21:36:36.859 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-20T21:36:37.029 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 92/134 2023-09-20T21:36:37.040 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-09-20T21:36:37.099 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 92/134 2023-09-20T21:36:37.262 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 93/134 2023-09-20T21:36:37.360 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-09-20T21:36:37.594 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-09-20T21:36:37.881 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-09-20T21:36:38.114 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-09-20T21:36:38.483 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-09-20T21:36:38.728 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-09-20T21:36:38.962 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-09-20T21:36:39.181 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-09-20T21:36:39.368 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-09-20T21:36:39.558 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-09-20T21:36:39.936 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-09-20T21:36:40.232 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-09-20T21:36:40.302 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 94/134 2023-09-20T21:36:40.487 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-09-20T21:36:40.527 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 94/134 2023-09-20T21:36:40.746 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-09-20T21:36:40.919 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-09-20T21:36:40.953 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 94/134 2023-09-20T21:36:41.111 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-09-20T21:36:41.137 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 94/134 2023-09-20T21:36:41.212 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 94/134 2023-09-20T21:36:41.399 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 95/134 2023-09-20T21:36:41.410 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-09-20T21:36:41.470 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 95/134 2023-09-20T21:36:41.585 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 96/134 2023-09-20T21:36:41.643 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-09-20T21:36:41.821 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 94/134 2023-09-20T21:36:41.831 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-09-20T21:36:41.990 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-09-20T21:36:42.108 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 95/134 2023-09-20T21:36:42.187 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-09-20T21:36:42.192 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 95/134 2023-09-20T21:36:42.378 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 96/134 2023-09-20T21:36:42.412 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-09-20T21:36:42.629 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-09-20T21:36:42.851 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-09-20T21:36:43.008 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-09-20T21:36:43.257 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-09-20T21:36:43.500 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-09-20T21:36:43.700 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-09-20T21:36:43.985 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-09-20T21:36:44.219 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-09-20T21:36:44.404 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-09-20T21:36:44.684 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-09-20T21:36:46.095 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-09-20T21:36:46.407 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-09-20T21:36:46.683 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-09-20T21:36:47.086 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-09-20T21:36:47.356 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-09-20T21:36:47.590 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-09-20T21:36:48.489 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-09-20T21:36:48.699 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 92/134 2023-09-20T21:36:48.782 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 92/134 2023-09-20T21:36:48.966 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 93/134 2023-09-20T21:36:54.398 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 94/134 2023-09-20T21:36:54.599 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 94/134 2023-09-20T21:36:55.355 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 94/134 2023-09-20T21:36:55.648 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 95/134 2023-09-20T21:36:55.784 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 95/134 2023-09-20T21:36:55.931 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 96/134 2023-09-20T21:37:00.428 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 96/134 2023-09-20T21:37:00.428 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2023-09-20T21:37:00.428 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2023-09-20T21:37:00.428 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-09-20T21:37:00.429 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /run 2023-09-20T21:37:00.429 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2023-09-20T21:37:00.429 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2023-09-20T21:37:00.429 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2023-09-20T21:37:00.429 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2023-09-20T21:37:00.430 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /dev 2023-09-20T21:37:00.430 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:37:00.882 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 97/134 2023-09-20T21:37:01.159 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 96/134 2023-09-20T21:37:01.160 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2023-09-20T21:37:01.160 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2023-09-20T21:37:01.160 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-09-20T21:37:01.160 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /run 2023-09-20T21:37:01.160 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2023-09-20T21:37:01.161 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2023-09-20T21:37:01.161 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2023-09-20T21:37:01.161 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2023-09-20T21:37:01.161 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-09-20T21:37:01.161 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:37:01.517 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 97/134 2023-09-20T21:37:02.464 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 97/134 2023-09-20T21:37:02.646 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 98/134 2023-09-20T21:37:02.868 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 98/134 2023-09-20T21:37:03.041 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 99/134 2023-09-20T21:37:03.099 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 97/134 2023-09-20T21:37:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 98/134 2023-09-20T21:37:03.485 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 98/134 2023-09-20T21:37:03.497 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 99/134 2023-09-20T21:37:03.624 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 100/134 2023-09-20T21:37:03.640 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 100/134 2023-09-20T21:37:03.667 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 99/134 2023-09-20T21:37:04.122 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 99/134 2023-09-20T21:37:04.293 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 100/134 2023-09-20T21:37:04.308 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 100/134 2023-09-20T21:37:05.425 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 101/134 2023-09-20T21:37:05.640 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 101/134 2023-09-20T21:37:05.763 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6213.gbf8a061f.e 102/134 2023-09-20T21:37:05.956 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6213.gbf8a061f. 103/134 2023-09-20T21:37:06.131 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noa 104/134 2023-09-20T21:37:06.145 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 101/134 2023-09-20T21:37:06.349 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noa 104/134 2023-09-20T21:37:06.357 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 101/134 2023-09-20T21:37:06.480 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6213.gbf8a061f.e 102/134 2023-09-20T21:37:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.n 105/134 2023-09-20T21:37:06.675 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6213.gbf8a061f. 103/134 2023-09-20T21:37:06.874 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noa 104/134 2023-09-20T21:37:07.092 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noa 104/134 2023-09-20T21:37:07.379 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.n 105/134 2023-09-20T21:37:07.600 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.n 105/134 2023-09-20T21:37:07.764 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8 106/134 2023-09-20T21:37:07.855 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8 106/134 2023-09-20T21:37:08.186 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6213.gbf8a061f.el 107/134 2023-09-20T21:37:08.337 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.n 105/134 2023-09-20T21:37:08.381 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 108/134 2023-09-20T21:37:08.416 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 108/134 2023-09-20T21:37:08.507 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8 106/134 2023-09-20T21:37:08.573 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 109/134 2023-09-20T21:37:08.584 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 109/134 2023-09-20T21:37:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8 106/134 2023-09-20T21:37:08.920 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6213.gbf8a061f.el 107/134 2023-09-20T21:37:09.143 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 108/134 2023-09-20T21:37:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 108/134 2023-09-20T21:37:09.366 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 109/134 2023-09-20T21:37:09.377 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 109/134 2023-09-20T21:37:09.890 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:18.0.0-6213.gbf8a061f.el8.x86_64 110/134 2023-09-20T21:37:10.084 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 111/134 2023-09-20T21:37:10.128 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 111/134 2023-09-20T21:37:10.267 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6213.gbf8a0 112/134 2023-09-20T21:37:10.306 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6213.gbf8a0 112/134 2023-09-20T21:37:10.456 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 113/134 2023-09-20T21:37:10.711 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:18.0.0-6213.gbf8a061f.el8.x86_64 110/134 2023-09-20T21:37:10.814 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 113/134 2023-09-20T21:37:10.910 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 111/134 2023-09-20T21:37:10.953 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 111/134 2023-09-20T21:37:10.997 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 114/134 2023-09-20T21:37:11.099 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6213.gbf8a0 112/134 2023-09-20T21:37:11.141 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6213.gbf8a0 112/134 2023-09-20T21:37:11.307 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 113/134 2023-09-20T21:37:11.665 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 113/134 2023-09-20T21:37:11.882 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 114/134 2023-09-20T21:37:15.650 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 114/134 2023-09-20T21:37:15.926 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:18.0.0-6213.gbf8a061f.el8.x86_64 115/134 2023-09-20T21:37:16.063 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-20T21:37:16.261 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-20T21:37:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 114/134 2023-09-20T21:37:16.845 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:18.0.0-6213.gbf8a061f.el8.x86_64 115/134 2023-09-20T21:37:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-09-20T21:37:17.137 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-20T21:37:17.243 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-09-20T21:37:17.441 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-20T21:37:17.520 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 96/134 2023-09-20T21:37:17.520 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2023-09-20T21:37:17.520 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2023-09-20T21:37:17.521 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /dev 2023-09-20T21:37:17.521 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /run 2023-09-20T21:37:17.521 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2023-09-20T21:37:17.521 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2023-09-20T21:37:17.522 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2023-09-20T21:37:17.522 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2023-09-20T21:37:17.522 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /dev 2023-09-20T21:37:17.522 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:37:17.607 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 119/134 2023-09-20T21:37:17.800 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-20T21:37:17.945 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 97/134 2023-09-20T21:37:17.992 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:18.0.0-6213.gbf8a061f.el8.x86_ 121/134 2023-09-20T21:37:18.104 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-09-20T21:37:18.131 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-20T21:37:18.433 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-09-20T21:37:18.600 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 119/134 2023-09-20T21:37:18.793 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-20T21:37:18.985 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:18.0.0-6213.gbf8a061f.el8.x86_ 121/134 2023-09-20T21:37:19.132 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-20T21:37:19.805 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 97/134 2023-09-20T21:37:20.069 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 98/134 2023-09-20T21:37:20.341 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-20T21:37:20.358 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 98/134 2023-09-20T21:37:20.544 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 99/134 2023-09-20T21:37:20.604 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-20T21:37:20.627 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-20T21:37:20.775 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-top-2:18.0.0-6213.gbf8a061f.el8.noarch 124/134 2023-09-20T21:37:21.045 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 125/134 2023-09-20T21:37:21.106 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 99/134 2023-09-20T21:37:21.276 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-20T21:37:21.286 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 100/134 2023-09-20T21:37:21.329 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 100/134 2023-09-20T21:37:21.337 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 126/134 2023-09-20T21:37:21.563 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-20T21:37:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-20T21:37:21.631 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-20T21:37:21.747 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:18.0.0-6213.gbf8a061f.el8.noarch 124/134 2023-09-20T21:37:21.795 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-20T21:37:21.805 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-20T21:37:21.985 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-20T21:37:22.021 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 125/134 2023-09-20T21:37:22.162 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-20T21:37:22.246 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-20T21:37:22.347 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 126/134 2023-09-20T21:37:22.437 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-20T21:37:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-20T21:37:22.450 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-20T21:37:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-20T21:37:22.632 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-20T21:37:22.633 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-20T21:37:22.805 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-20T21:37:22.813 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-20T21:37:22.860 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-20T21:37:23.003 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-09-20T21:37:23.094 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-20T21:37:23.180 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-09-20T21:37:23.266 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-09-20T21:37:23.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-09-20T21:37:23.484 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-09-20T21:37:23.484 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-20T21:37:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-20T21:37:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:18.0.0-6213.gbf8a061f.el8.x86_64 1/134 2023-09-20T21:37:23.614 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 2/134 2023-09-20T21:37:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 3/134 2023-09-20T21:37:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 4/134 2023-09-20T21:37:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6213.gbf8a0 5/134 2023-09-20T21:37:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 6/134 2023-09-20T21:37:23.615 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 7/134 2023-09-20T21:37:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 8/134 2023-09-20T21:37:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 9/134 2023-09-20T21:37:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 10/134 2023-09-20T21:37:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 11/134 2023-09-20T21:37:23.616 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:18.0.0-6213.gbf8a061f.el8.x86_64 12/134 2023-09-20T21:37:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 13/134 2023-09-20T21:37:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:18.0.0-6213.gbf8a061f.el8.x86_ 14/134 2023-09-20T21:37:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 15/134 2023-09-20T21:37:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 16/134 2023-09-20T21:37:23.617 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:18.0.0-6213.gbf8a061f.el8.x86_6 17/134 2023-09-20T21:37:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86 18/134 2023-09-20T21:37:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 19/134 2023-09-20T21:37:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6213.gbf8a061f.el 20/134 2023-09-20T21:37:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:18.0.0-6213.gbf8a061f.el8. 21/134 2023-09-20T21:37:23.618 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:18.0.0-6213.gbf8a061f.el8.x86_6 22/134 2023-09-20T21:37:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:18.0.0-6213.gbf8a061f.el8.x86_64 23/134 2023-09-20T21:37:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 24/134 2023-09-20T21:37:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 25/134 2023-09-20T21:37:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 26/134 2023-09-20T21:37:23.619 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 27/134 2023-09-20T21:37:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 28/134 2023-09-20T21:37:23.620 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6213.gbf8a061f. 29/134 2023-09-20T21:37:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noa 30/134 2023-09-20T21:37:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.n 31/134 2023-09-20T21:37:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8 32/134 2023-09-20T21:37:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6213.gbf8a061f.el 33/134 2023-09-20T21:37:23.629 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 34/134 2023-09-20T21:37:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6213.gbf8a061f.e 35/134 2023-09-20T21:37:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 36/134 2023-09-20T21:37:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 37/134 2023-09-20T21:37:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-top-2:18.0.0-6213.gbf8a061f.el8.noarch 38/134 2023-09-20T21:37:23.630 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-20T21:37:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-20T21:37:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-20T21:37:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-20T21:37:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-20T21:37:23.631 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-20T21:37:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-20T21:37:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-20T21:37:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-20T21:37:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-20T21:37:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-20T21:37:23.632 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-20T21:37:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-20T21:37:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-20T21:37:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-20T21:37:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-20T21:37:23.633 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-20T21:37:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-20T21:37:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-20T21:37:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-20T21:37:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-20T21:37:23.634 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-20T21:37:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-20T21:37:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-20T21:37:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-20T21:37:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-20T21:37:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-20T21:37:23.635 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-20T21:37:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-20T21:37:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-20T21:37:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-20T21:37:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-20T21:37:23.636 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-20T21:37:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-20T21:37:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-20T21:37:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-20T21:37:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-20T21:37:23.637 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-20T21:37:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-20T21:37:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-20T21:37:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-20T21:37:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-20T21:37:23.638 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-20T21:37:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-20T21:37:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-20T21:37:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-20T21:37:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-20T21:37:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-20T21:37:23.639 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-20T21:37:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-20T21:37:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-20T21:37:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-20T21:37:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-20T21:37:23.640 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-20T21:37:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-20T21:37:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-20T21:37:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-20T21:37:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-20T21:37:23.641 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-20T21:37:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-20T21:37:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-20T21:37:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-20T21:37:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-20T21:37:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-20T21:37:23.642 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-20T21:37:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-20T21:37:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-20T21:37:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-20T21:37:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-20T21:37:23.643 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-20T21:37:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-20T21:37:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-20T21:37:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-20T21:37:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-20T21:37:23.644 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-20T21:37:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-20T21:37:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-20T21:37:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-20T21:37:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-20T21:37:23.645 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-20T21:37:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-20T21:37:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-20T21:37:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-20T21:37:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-20T21:37:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-20T21:37:23.646 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-20T21:37:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-20T21:37:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-20T21:37:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-20T21:37:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-20T21:37:23.647 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 129/134 2023-09-20T21:37:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-20T21:37:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 131/134 2023-09-20T21:37:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-20T21:37:23.648 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-20T21:37:23.683 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-20T21:37:23.683 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-20T21:37:23.910 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-20T21:37:24.145 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-20T21:37:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 101/134 2023-09-20T21:37:24.388 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 101/134 2023-09-20T21:37:24.493 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6213.gbf8a061f.e 102/134 2023-09-20T21:37:24.665 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-20T21:37:24.665 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:18.0.0-6213.gbf8a061f.el8.x86_64 1/134 2023-09-20T21:37:24.666 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 2/134 2023-09-20T21:37:24.666 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 3/134 2023-09-20T21:37:24.666 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 4/134 2023-09-20T21:37:24.666 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6213.gbf8a0 5/134 2023-09-20T21:37:24.666 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 6/134 2023-09-20T21:37:24.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 7/134 2023-09-20T21:37:24.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 8/134 2023-09-20T21:37:24.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 9/134 2023-09-20T21:37:24.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 10/134 2023-09-20T21:37:24.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 11/134 2023-09-20T21:37:24.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:18.0.0-6213.gbf8a061f.el8.x86_64 12/134 2023-09-20T21:37:24.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 13/134 2023-09-20T21:37:24.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:18.0.0-6213.gbf8a061f.el8.x86_ 14/134 2023-09-20T21:37:24.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 15/134 2023-09-20T21:37:24.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 16/134 2023-09-20T21:37:24.669 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:18.0.0-6213.gbf8a061f.el8.x86_6 17/134 2023-09-20T21:37:24.669 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86 18/134 2023-09-20T21:37:24.669 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 19/134 2023-09-20T21:37:24.669 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6213.gbf8a061f.el 20/134 2023-09-20T21:37:24.669 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:18.0.0-6213.gbf8a061f.el8. 21/134 2023-09-20T21:37:24.669 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:18.0.0-6213.gbf8a061f.el8.x86_6 22/134 2023-09-20T21:37:24.670 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:18.0.0-6213.gbf8a061f.el8.x86_64 23/134 2023-09-20T21:37:24.670 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 24/134 2023-09-20T21:37:24.670 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 25/134 2023-09-20T21:37:24.670 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 26/134 2023-09-20T21:37:24.670 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 27/134 2023-09-20T21:37:24.671 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 28/134 2023-09-20T21:37:24.671 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6213.gbf8a061f. 29/134 2023-09-20T21:37:24.679 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noa 30/134 2023-09-20T21:37:24.680 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.n 31/134 2023-09-20T21:37:24.680 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8 32/134 2023-09-20T21:37:24.680 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6213.gbf8a061f.el 33/134 2023-09-20T21:37:24.680 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 34/134 2023-09-20T21:37:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6213.gbf8a061f.e 35/134 2023-09-20T21:37:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 36/134 2023-09-20T21:37:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 37/134 2023-09-20T21:37:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:18.0.0-6213.gbf8a061f.el8.noarch 38/134 2023-09-20T21:37:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-20T21:37:24.681 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-20T21:37:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-20T21:37:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-20T21:37:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-20T21:37:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-20T21:37:24.682 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-20T21:37:24.683 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-20T21:37:24.683 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-20T21:37:24.683 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-20T21:37:24.683 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-20T21:37:24.683 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-20T21:37:24.684 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-20T21:37:24.684 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-20T21:37:24.684 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-20T21:37:24.684 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-20T21:37:24.684 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-20T21:37:24.684 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-20T21:37:24.685 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-20T21:37:24.685 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-20T21:37:24.685 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-20T21:37:24.685 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-20T21:37:24.685 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-20T21:37:24.686 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-20T21:37:24.686 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-20T21:37:24.686 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-20T21:37:24.686 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-20T21:37:24.686 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-20T21:37:24.687 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-20T21:37:24.687 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-20T21:37:24.687 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-20T21:37:24.687 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-20T21:37:24.687 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-20T21:37:24.687 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-09-20T21:37:24.688 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-09-20T21:37:24.688 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-09-20T21:37:24.688 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-09-20T21:37:24.688 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-09-20T21:37:24.688 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-09-20T21:37:24.689 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-09-20T21:37:24.689 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-09-20T21:37:24.689 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-09-20T21:37:24.689 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-09-20T21:37:24.689 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-09-20T21:37:24.689 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-09-20T21:37:24.690 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-09-20T21:37:24.690 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-09-20T21:37:24.690 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-09-20T21:37:24.690 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-09-20T21:37:24.690 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-09-20T21:37:24.691 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-09-20T21:37:24.691 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-09-20T21:37:24.691 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-09-20T21:37:24.691 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-09-20T21:37:24.691 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-09-20T21:37:24.691 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-09-20T21:37:24.692 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-09-20T21:37:24.692 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-09-20T21:37:24.692 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-09-20T21:37:24.692 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-09-20T21:37:24.692 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-09-20T21:37:24.693 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-09-20T21:37:24.693 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-09-20T21:37:24.693 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-09-20T21:37:24.693 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-09-20T21:37:24.693 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-09-20T21:37:24.694 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-09-20T21:37:24.694 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-09-20T21:37:24.694 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-09-20T21:37:24.694 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-09-20T21:37:24.694 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-09-20T21:37:24.695 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-09-20T21:37:24.695 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-09-20T21:37:24.695 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-09-20T21:37:24.695 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-09-20T21:37:24.695 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-09-20T21:37:24.695 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-09-20T21:37:24.696 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-09-20T21:37:24.696 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-09-20T21:37:24.696 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-09-20T21:37:24.696 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-09-20T21:37:24.696 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-09-20T21:37:24.697 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-09-20T21:37:24.697 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-20T21:37:24.697 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-09-20T21:37:24.697 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-09-20T21:37:24.697 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-09-20T21:37:24.697 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-09-20T21:37:24.698 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-09-20T21:37:24.698 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-09-20T21:37:24.698 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 129/134 2023-09-20T21:37:24.698 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-20T21:37:24.698 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 131/134 2023-09-20T21:37:24.699 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-20T21:37:24.699 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-20T21:37:24.699 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6213.gbf8a061f. 103/134 2023-09-20T21:37:24.943 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noa 104/134 2023-09-20T21:37:25.225 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noa 104/134 2023-09-20T21:37:25.426 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.n 105/134 2023-09-20T21:37:26.074 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-20T21:37:26.074 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:54 ago on Wed 20 Sep 2023 09:35:32 PM UTC. 2023-09-20T21:37:26.619 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.n 105/134 2023-09-20T21:37:26.793 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8 106/134 2023-09-20T21:37:26.931 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8 106/134 2023-09-20T21:37:27.207 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-20T21:37:27.208 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:55 ago on Wed 20 Sep 2023 09:35:32 PM UTC. 2023-09-20T21:37:27.326 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6213.gbf8a061f.el 107/134 2023-09-20T21:37:27.551 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 108/134 2023-09-20T21:37:27.619 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 108/134 2023-09-20T21:37:27.851 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 109/134 2023-09-20T21:37:27.881 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 109/134 2023-09-20T21:37:28.527 INFO:teuthology.orchestra.run.smithi089.stdout:Installed products updated. 2023-09-20T21:37:28.741 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:37:28.741 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2023-09-20T21:37:28.741 INFO:teuthology.orchestra.run.smithi089.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-20T21:37:28.742 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.742 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.742 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2023-09-20T21:37:28.742 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-20T21:37:28.742 INFO:teuthology.orchestra.run.smithi089.stdout: automake-1.16.1-7.el8.noarch 2023-09-20T21:37:28.743 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.0.4-10.el8.x86_64 2023-09-20T21:37:28.743 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.743 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.743 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.744 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.744 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:28.744 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.744 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.744 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.745 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:28.745 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:28.745 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:28.745 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:28.745 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:28.746 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.746 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.746 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:28.746 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.746 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.747 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.747 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:28.747 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:28.747 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.747 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:28.748 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-20T21:37:28.748 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.1-9.el8.x86_64 2023-09-20T21:37:28.748 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-20T21:37:28.748 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-20T21:37:28.748 INFO:teuthology.orchestra.run.smithi089.stdout: jq-1.6-3.el8.x86_64 2023-09-20T21:37:28.749 INFO:teuthology.orchestra.run.smithi089.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-20T21:37:28.749 INFO:teuthology.orchestra.run.smithi089.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-20T21:37:28.749 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-20T21:37:28.749 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.750 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.750 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.750 INFO:teuthology.orchestra.run.smithi089.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-20T21:37:28.750 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-20T21:37:28.750 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-20T21:37:28.751 INFO:teuthology.orchestra.run.smithi089.stdout: libpq-13.5-1.el8.x86_64 2023-09-20T21:37:28.751 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-20T21:37:28.751 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-20T21:37:28.751 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.751 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.752 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-20T21:37:28.752 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.752 INFO:teuthology.orchestra.run.smithi089.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-20T21:37:28.752 INFO:teuthology.orchestra.run.smithi089.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-20T21:37:28.752 INFO:teuthology.orchestra.run.smithi089.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-20T21:37:28.753 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-20T21:37:28.753 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-20T21:37:28.753 INFO:teuthology.orchestra.run.smithi089.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-20T21:37:28.753 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-20T21:37:28.753 INFO:teuthology.orchestra.run.smithi089.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-20T21:37:28.754 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-20T21:37:28.754 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-20T21:37:28.754 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-20T21:37:28.754 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-20T21:37:28.754 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-20T21:37:28.755 INFO:teuthology.orchestra.run.smithi089.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-20T21:37:28.755 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-20T21:37:28.755 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-20T21:37:28.755 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-20T21:37:28.755 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-20T21:37:28.756 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-20T21:37:28.756 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-20T21:37:28.756 INFO:teuthology.orchestra.run.smithi089.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-20T21:37:28.756 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-20T21:37:28.756 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.757 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.757 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.757 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-20T21:37:28.757 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-20T21:37:28.757 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-20T21:37:28.757 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-20T21:37:28.758 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-20T21:37:28.758 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-20T21:37:28.758 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-20T21:37:28.758 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-20T21:37:28.758 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-20T21:37:28.759 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-20T21:37:28.759 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-20T21:37:28.759 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-20T21:37:28.759 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-20T21:37:28.759 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-20T21:37:28.760 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-20T21:37:28.760 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-20T21:37:28.760 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-20T21:37:28.760 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-20T21:37:28.760 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-20T21:37:28.761 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-20T21:37:28.761 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-20T21:37:28.761 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-20T21:37:28.761 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-2.6-1.el8.noarch 2023-09-20T21:37:28.762 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-20T21:37:28.762 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-20T21:37:28.762 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-20T21:37:28.762 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-20T21:37:28.762 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-20T21:37:28.763 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-20T21:37:28.763 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.763 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.763 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-20T21:37:28.763 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-20T21:37:28.763 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.764 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-20T21:37:28.764 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-20T21:37:28.764 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-20T21:37:28.764 INFO:teuthology.orchestra.run.smithi089.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-20T21:37:28.764 INFO:teuthology.orchestra.run.smithi089.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-20T21:37:28.765 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-20T21:37:28.765 INFO:teuthology.orchestra.run.smithi089.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-20T21:37:28.765 INFO:teuthology.orchestra.run.smithi089.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-20T21:37:28.765 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-20T21:37:28.765 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-20T21:37:28.766 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-20T21:37:28.766 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-20T21:37:28.766 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-20T21:37:28.766 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.766 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.767 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:28.767 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-20T21:37:28.767 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-20T21:37:28.767 INFO:teuthology.orchestra.run.smithi089.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-20T21:37:28.767 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-20T21:37:28.768 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-20T21:37:28.768 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-20T21:37:28.768 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-09-20T21:37:28.768 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2023-09-20T21:37:28.926 DEBUG:teuthology.parallel:result is None 2023-09-20T21:37:29.401 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:18.0.0-6213.gbf8a061f.el8.x86_64 110/134 2023-09-20T21:37:29.701 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 111/134 2023-09-20T21:37:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2023-09-20T21:37:29.778 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 111/134 2023-09-20T21:37:29.918 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6213.gbf8a0 112/134 2023-09-20T21:37:29.957 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:37:29.958 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2023-09-20T21:37:29.958 INFO:teuthology.orchestra.run.smithi123.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-20T21:37:29.958 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.958 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.959 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2023-09-20T21:37:29.959 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-20T21:37:29.959 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.1-7.el8.noarch 2023-09-20T21:37:29.959 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.0.4-10.el8.x86_64 2023-09-20T21:37:29.959 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.960 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.960 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.960 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.960 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:29.960 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.960 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.961 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.961 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:29.961 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:29.961 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:29.961 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:29.962 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:29.962 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.962 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.964 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:29.965 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.965 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.966 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.966 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:29.966 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:29.966 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.966 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:29.967 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-20T21:37:29.967 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.1-9.el8.x86_64 2023-09-20T21:37:29.967 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-20T21:37:29.967 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-20T21:37:29.967 INFO:teuthology.orchestra.run.smithi123.stdout: jq-1.6-3.el8.x86_64 2023-09-20T21:37:29.968 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-20T21:37:29.968 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-20T21:37:29.968 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-20T21:37:29.968 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.968 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-20T21:37:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-20T21:37:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-20T21:37:29.969 INFO:teuthology.orchestra.run.smithi123.stdout: libpq-13.5-1.el8.x86_64 2023-09-20T21:37:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-20T21:37:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-20T21:37:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-20T21:37:29.970 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.971 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-20T21:37:29.971 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-20T21:37:29.971 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-20T21:37:29.971 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-20T21:37:29.971 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-20T21:37:29.972 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-20T21:37:29.972 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-20T21:37:29.972 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-20T21:37:29.972 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-20T21:37:29.972 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-20T21:37:29.973 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-20T21:37:29.973 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-20T21:37:29.973 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-20T21:37:29.973 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-20T21:37:29.973 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-20T21:37:29.973 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-20T21:37:29.974 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-20T21:37:29.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-20T21:37:29.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-20T21:37:29.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-20T21:37:29.974 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-20T21:37:29.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-20T21:37:29.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.975 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-20T21:37:29.976 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-20T21:37:29.976 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-20T21:37:29.976 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-20T21:37:29.976 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-20T21:37:29.976 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-20T21:37:29.977 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-20T21:37:29.977 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-20T21:37:29.977 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-20T21:37:29.977 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-20T21:37:29.977 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-20T21:37:29.978 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-20T21:37:29.978 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-20T21:37:29.978 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-20T21:37:29.978 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-20T21:37:29.978 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-20T21:37:29.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-20T21:37:29.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-20T21:37:29.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-20T21:37:29.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-20T21:37:29.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-20T21:37:29.979 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-20T21:37:29.980 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-2.6-1.el8.noarch 2023-09-20T21:37:29.980 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-20T21:37:29.980 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-20T21:37:29.980 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-20T21:37:29.980 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-20T21:37:29.980 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-20T21:37:29.981 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-20T21:37:29.981 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.981 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.981 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-20T21:37:29.981 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-20T21:37:29.981 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.982 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-20T21:37:29.982 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-20T21:37:29.982 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-20T21:37:29.982 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-20T21:37:29.982 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-20T21:37:29.982 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-20T21:37:29.983 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-20T21:37:29.983 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-20T21:37:29.983 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-20T21:37:29.983 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-20T21:37:29.983 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-20T21:37:29.984 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-20T21:37:29.984 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-20T21:37:29.984 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.984 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.984 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:29.985 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-20T21:37:29.985 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-20T21:37:29.985 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-20T21:37:29.985 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-20T21:37:29.985 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-20T21:37:29.986 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-20T21:37:29.986 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:37:29.986 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-09-20T21:37:29.991 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6213.gbf8a0 112/134 2023-09-20T21:37:30.134 DEBUG:teuthology.parallel:result is None 2023-09-20T21:37:30.185 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 113/134 2023-09-20T21:37:30.646 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 113/134 2023-09-20T21:37:30.834 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 114/134 2023-09-20T21:37:36.327 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 114/134 2023-09-20T21:37:37.671 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:18.0.0-6213.gbf8a061f.el8.x86_64 115/134 2023-09-20T21:37:38.014 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-09-20T21:37:38.232 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 117/134 2023-09-20T21:37:38.492 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-09-20T21:37:38.682 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-09-20T21:37:38.956 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-09-20T21:37:39.180 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-09-20T21:37:39.388 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:18.0.0-6213.gbf8a061f.el8.x86_ 121/134 2023-09-20T21:37:39.526 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-09-20T21:37:43.267 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-20T21:37:43.732 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-20T21:37:43.771 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-09-20T21:37:43.967 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-top-2:18.0.0-6213.gbf8a061f.el8.noarch 124/134 2023-09-20T21:37:44.184 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 125/134 2023-09-20T21:37:44.505 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 126/134 2023-09-20T21:37:44.897 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-09-20T21:37:45.106 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-09-20T21:37:45.225 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-09-20T21:37:45.473 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-09-20T21:37:45.502 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-09-20T21:37:45.740 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-20T21:37:45.763 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-09-20T21:37:45.919 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-09-20T21:37:45.919 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-20T21:37:46.153 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-20T21:37:46.153 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-20T21:37:46.423 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-09-20T21:37:46.607 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-20T21:37:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-09-20T21:37:46.877 INFO:teuthology.orchestra.run.smithi022.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-09-20T21:37:46.877 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:37:47.531 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:18.0.0-6213.gbf8a061f.el8.x86_64 1/134 2023-09-20T21:37:47.532 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 2/134 2023-09-20T21:37:47.532 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 3/134 2023-09-20T21:37:47.532 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 4/134 2023-09-20T21:37:47.532 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6213.gbf8a0 5/134 2023-09-20T21:37:47.532 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 6/134 2023-09-20T21:37:47.533 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 7/134 2023-09-20T21:37:47.533 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 8/134 2023-09-20T21:37:47.533 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 9/134 2023-09-20T21:37:47.533 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 10/134 2023-09-20T21:37:47.533 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 11/134 2023-09-20T21:37:47.534 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:18.0.0-6213.gbf8a061f.el8.x86_64 12/134 2023-09-20T21:37:47.534 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 13/134 2023-09-20T21:37:47.534 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:18.0.0-6213.gbf8a061f.el8.x86_ 14/134 2023-09-20T21:37:47.534 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 15/134 2023-09-20T21:37:47.534 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 16/134 2023-09-20T21:37:47.535 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:18.0.0-6213.gbf8a061f.el8.x86_6 17/134 2023-09-20T21:37:47.535 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86 18/134 2023-09-20T21:37:47.535 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 19/134 2023-09-20T21:37:47.535 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6213.gbf8a061f.el 20/134 2023-09-20T21:37:47.535 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:18.0.0-6213.gbf8a061f.el8. 21/134 2023-09-20T21:37:47.536 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:18.0.0-6213.gbf8a061f.el8.x86_6 22/134 2023-09-20T21:37:47.536 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:18.0.0-6213.gbf8a061f.el8.x86_64 23/134 2023-09-20T21:37:47.536 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 24/134 2023-09-20T21:37:47.536 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 25/134 2023-09-20T21:37:47.536 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 26/134 2023-09-20T21:37:47.537 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 27/134 2023-09-20T21:37:47.546 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 28/134 2023-09-20T21:37:47.546 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6213.gbf8a061f. 29/134 2023-09-20T21:37:47.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noa 30/134 2023-09-20T21:37:47.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.n 31/134 2023-09-20T21:37:47.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8 32/134 2023-09-20T21:37:47.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6213.gbf8a061f.el 33/134 2023-09-20T21:37:47.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 34/134 2023-09-20T21:37:47.547 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6213.gbf8a061f.e 35/134 2023-09-20T21:37:47.548 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 36/134 2023-09-20T21:37:47.548 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 37/134 2023-09-20T21:37:47.548 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-top-2:18.0.0-6213.gbf8a061f.el8.noarch 38/134 2023-09-20T21:37:47.548 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-09-20T21:37:47.548 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-09-20T21:37:47.549 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-09-20T21:37:47.549 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-09-20T21:37:47.549 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-09-20T21:37:47.549 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-09-20T21:37:47.549 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-09-20T21:37:47.550 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-09-20T21:37:47.550 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-09-20T21:37:47.550 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-09-20T21:37:47.550 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-09-20T21:37:47.550 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-09-20T21:37:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-09-20T21:37:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-09-20T21:37:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-09-20T21:37:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-09-20T21:37:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-09-20T21:37:47.552 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-09-20T21:37:47.552 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-09-20T21:37:47.552 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-09-20T21:37:47.552 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-09-20T21:37:47.552 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-09-20T21:37:47.553 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-09-20T21:37:47.553 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-09-20T21:37:47.553 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-09-20T21:37:47.553 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-09-20T21:37:47.553 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-09-20T21:37:47.554 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-09-20T21:37:47.554 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-09-20T21:37:47.554 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-09-20T21:37:47.554 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-09-20T21:37:47.554 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-09-20T21:37:47.555 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-09-20T21:37:47.555 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-09-20T21:37:47.555 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-09-20T21:37:47.555 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-09-20T21:37:47.555 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-09-20T21:37:47.556 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-09-20T21:37:47.556 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-09-20T21:37:47.556 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-09-20T21:37:47.556 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-09-20T21:37:47.556 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-09-20T21:37:47.557 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-09-20T21:37:47.557 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-09-20T21:37:47.557 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-09-20T21:37:47.557 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-09-20T21:37:47.557 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-09-20T21:37:47.557 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-09-20T21:37:47.558 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-09-20T21:37:47.558 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-09-20T21:37:47.558 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-09-20T21:37:47.558 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-09-20T21:37:47.558 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-09-20T21:37:47.559 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-09-20T21:37:47.559 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-09-20T21:37:47.559 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-09-20T21:37:47.559 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-09-20T21:37:47.559 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-09-20T21:37:47.560 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-09-20T21:37:47.560 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-09-20T21:37:47.560 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-09-20T21:37:47.560 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-09-20T21:37:47.560 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-09-20T21:37:47.560 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-09-20T21:37:47.561 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-09-20T21:37:47.561 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-09-20T21:37:47.561 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-09-20T21:37:47.561 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-09-20T21:37:47.561 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-09-20T21:37:47.562 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-09-20T21:37:47.562 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-09-20T21:37:47.562 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-09-20T21:37:47.562 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-09-20T21:37:47.562 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-09-20T21:37:47.562 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-09-20T21:37:47.563 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-09-20T21:37:47.563 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-09-20T21:37:47.563 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-09-20T21:37:47.563 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-09-20T21:37:47.563 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-09-20T21:37:47.564 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-09-20T21:37:47.564 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-09-20T21:37:47.564 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-09-20T21:37:47.564 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-09-20T21:37:47.564 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-09-20T21:37:47.564 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-09-20T21:37:47.565 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-09-20T21:37:47.565 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-09-20T21:37:47.565 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-09-20T21:37:47.565 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-09-20T21:37:47.565 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 129/134 2023-09-20T21:37:47.566 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-09-20T21:37:47.566 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 131/134 2023-09-20T21:37:47.569 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-09-20T21:37:47.569 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-09-20T21:37:50.203 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-09-20T21:37:50.204 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:02:15 ago on Wed 20 Sep 2023 09:35:35 PM UTC. 2023-09-20T21:37:52.888 INFO:teuthology.orchestra.run.smithi022.stdout:Installed products updated. 2023-09-20T21:37:53.129 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:37:53.130 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2023-09-20T21:37:53.130 INFO:teuthology.orchestra.run.smithi022.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-09-20T21:37:53.130 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.130 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.131 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2023-09-20T21:37:53.131 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-09-20T21:37:53.131 INFO:teuthology.orchestra.run.smithi022.stdout: automake-1.16.1-7.el8.noarch 2023-09-20T21:37:53.131 INFO:teuthology.orchestra.run.smithi022.stdout: bison-3.0.4-10.el8.x86_64 2023-09-20T21:37:53.131 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.132 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.132 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.132 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.132 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:53.133 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.133 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.133 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.133 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:53.133 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:53.134 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:53.134 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:53.134 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:53.134 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.134 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.135 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:53.135 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.135 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.135 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.135 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:53.136 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:53.136 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.136 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top-2:18.0.0-6213.gbf8a061f.el8.noarch 2023-09-20T21:37:53.136 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-09-20T21:37:53.136 INFO:teuthology.orchestra.run.smithi022.stdout: flex-2.6.1-9.el8.x86_64 2023-09-20T21:37:53.137 INFO:teuthology.orchestra.run.smithi022.stdout: fuse-2.9.7-15.el8.x86_64 2023-09-20T21:37:53.137 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-20T21:37:53.137 INFO:teuthology.orchestra.run.smithi022.stdout: jq-1.6-3.el8.x86_64 2023-09-20T21:37:53.137 INFO:teuthology.orchestra.run.smithi022.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-20T21:37:53.137 INFO:teuthology.orchestra.run.smithi022.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-09-20T21:37:53.138 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-09-20T21:37:53.138 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.138 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.138 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.138 INFO:teuthology.orchestra.run.smithi022.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-09-20T21:37:53.139 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-09-20T21:37:53.139 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-20T21:37:53.139 INFO:teuthology.orchestra.run.smithi022.stdout: libpq-13.5-1.el8.x86_64 2023-09-20T21:37:53.139 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-09-20T21:37:53.139 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-09-20T21:37:53.140 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.140 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.140 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-20T21:37:53.140 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.140 INFO:teuthology.orchestra.run.smithi022.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-09-20T21:37:53.140 INFO:teuthology.orchestra.run.smithi022.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-20T21:37:53.141 INFO:teuthology.orchestra.run.smithi022.stdout: libtool-2.4.6-25.el8.x86_64 2023-09-20T21:37:53.141 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-20T21:37:53.141 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-09-20T21:37:53.141 INFO:teuthology.orchestra.run.smithi022.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-09-20T21:37:53.141 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-20T21:37:53.142 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-20T21:37:53.142 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.15-3.el8.x86_64 2023-09-20T21:37:53.142 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-09-20T21:37:53.142 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-09-20T21:37:53.142 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-09-20T21:37:53.143 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-09-20T21:37:53.143 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-09-20T21:37:53.143 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-20T21:37:53.143 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-20T21:37:53.143 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-09-20T21:37:53.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-09-20T21:37:53.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.5.1-7.el8.noarch 2023-09-20T21:37:53.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-20T21:37:53.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-20T21:37:53.144 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-20T21:37:53.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-20T21:37:53.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-09-20T21:37:53.145 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-20T21:37:53.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-20T21:37:53.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-09-20T21:37:53.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-20T21:37:53.146 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-20T21:37:53.147 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-20T21:37:53.147 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-09-20T21:37:53.147 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-20T21:37:53.147 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-20T21:37:53.147 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-20T21:37:53.148 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-20T21:37:53.148 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-20T21:37:53.148 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.0.6-13.el8.noarch 2023-09-20T21:37:53.148 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-20T21:37:53.148 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-20T21:37:53.149 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-09-20T21:37:53.149 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-20T21:37:53.149 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-20T21:37:53.149 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-20T21:37:53.149 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-20T21:37:53.150 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-2.6-1.el8.noarch 2023-09-20T21:37:53.150 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-20T21:37:53.150 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-20T21:37:53.150 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-20T21:37:53.150 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-20T21:37:53.151 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-20T21:37:53.151 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2017.2-9.el8.noarch 2023-09-20T21:37:53.151 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.151 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.151 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-20T21:37:53.152 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-20T21:37:53.152 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.152 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-20T21:37:53.152 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-20T21:37:53.152 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-09-20T21:37:53.153 INFO:teuthology.orchestra.run.smithi022.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-20T21:37:53.153 INFO:teuthology.orchestra.run.smithi022.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-20T21:37:53.153 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-20T21:37:53.153 INFO:teuthology.orchestra.run.smithi022.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-20T21:37:53.153 INFO:teuthology.orchestra.run.smithi022.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-20T21:37:53.154 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-20T21:37:53.154 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-20T21:37:53.154 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-20T21:37:53.154 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-20T21:37:53.155 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-20T21:37:53.155 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.155 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.155 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:18.0.0-6213.gbf8a061f.el8.x86_64 2023-09-20T21:37:53.155 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-20T21:37:53.156 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-20T21:37:53.156 INFO:teuthology.orchestra.run.smithi022.stdout: uuid-1.6.2-43.el8.x86_64 2023-09-20T21:37:53.156 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-09-20T21:37:53.156 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-20T21:37:53.157 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-09-20T21:37:53.157 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:37:53.157 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2023-09-20T21:37:53.455 DEBUG:teuthology.parallel:result is None 2023-09-20T21:37:53.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bf8a061ff16ac1d393cf108f740d92dfa49f2e54 2023-09-20T21:37:53.595 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-20T21:37:53.650 INFO:teuthology.orchestra.run.smithi022.stdout:18.0.0-6213.gbf8a061f.el8 2023-09-20T21:37:53.651 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6213.gbf8a061f.el8 2023-09-20T21:37:53.651 INFO:teuthology.task.install:The correct ceph version 18.0.0-6213.gbf8a061f is installed. 2023-09-20T21:37:53.652 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bf8a061ff16ac1d393cf108f740d92dfa49f2e54 2023-09-20T21:37:53.789 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-20T21:37:53.818 INFO:teuthology.orchestra.run.smithi089.stdout:18.0.0-6213.gbf8a061f.el8 2023-09-20T21:37:53.818 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6213.gbf8a061f.el8 2023-09-20T21:37:53.818 INFO:teuthology.task.install:The correct ceph version 18.0.0-6213.gbf8a061f is installed. 2023-09-20T21:37:53.819 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bf8a061ff16ac1d393cf108f740d92dfa49f2e54 2023-09-20T21:37:53.959 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-20T21:37:53.987 INFO:teuthology.orchestra.run.smithi123.stdout:18.0.0-6213.gbf8a061f.el8 2023-09-20T21:37:53.988 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6213.gbf8a061f.el8 2023-09-20T21:37:53.988 INFO:teuthology.task.install:The correct ceph version 18.0.0-6213.gbf8a061f is installed. 2023-09-20T21:37:53.988 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-20T21:37:53.989 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:37:53.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-20T21:37:54.080 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:37:54.080 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-20T21:37:54.128 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:37:54.128 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-20T21:37:54.177 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-20T21:37:54.177 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:37:54.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2023-09-20T21:37:54.248 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-20T21:37:54.359 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:37:54.359 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2023-09-20T21:37:54.391 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-20T21:37:54.463 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:37:54.463 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2023-09-20T21:37:54.497 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-20T21:37:54.569 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-20T21:37:54.570 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:37:54.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-20T21:37:54.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-20T21:37:54.751 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:37:54.752 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-20T21:37:54.783 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-20T21:37:54.853 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:37:54.853 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-20T21:37:54.886 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-20T21:37:54.958 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-20T21:37:54.958 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:37:54.959 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2023-09-20T21:37:55.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-20T21:37:55.141 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:37:55.141 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2023-09-20T21:37:55.174 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-20T21:37:55.245 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:37:55.245 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2023-09-20T21:37:55.278 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-20T21:37:55.350 INFO:teuthology.run_tasks:Running task cephadm... 2023-09-20T21:37:55.427 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bf8a061ff16ac1d393cf108f740d92dfa49f2e54'} 2023-09-20T21:37:55.428 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 2023-09-20T21:37:55.429 INFO:tasks.cephadm:Cluster fsid is f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:37:55.429 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-09-20T21:37:55.429 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.22', 'mon.b': '172.21.15.89', 'mon.c': '172.21.15.123'} 2023-09-20T21:37:55.430 INFO:tasks.cephadm:First mon is mon.a on smithi022 2023-09-20T21:37:55.430 INFO:tasks.cephadm:First mgr is x 2023-09-20T21:37:55.430 INFO:tasks.cephadm:Normalizing hostnames... 2023-09-20T21:37:55.431 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname $(hostname -s) 2023-09-20T21:37:55.504 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname $(hostname -s) 2023-09-20T21:37:55.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2023-09-20T21:37:55.570 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-09-20T21:37:55.571 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bf8a061ff16ac1d393cf108f740d92dfa49f2e54 2023-09-20T21:37:55.709 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bf8a061ff16ac1d393cf108f740d92dfa49f2e54', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/72688/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6213-gbf8a061f', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6213.gbf8a061f'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-09-20 09:17:22.841997', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20230920.072635', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-09-20T21:37:55.931 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-vshankar-testing-20230920.072635, sha1 bf8a061ff16ac1d393cf108f740d92dfa49f2e54 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=bf8a061ff16ac1d393cf108f740d92dfa49f2e54 2023-09-20T21:37:55.931 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/x86_64/flavors/default/cephadm 2023-09-20T21:37:55.932 DEBUG:teuthology.orchestra.run.smithi022:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-20T21:37:56.873 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-rw-r--. 1 ubuntu ubuntu 733450 Sep 20 21:37 /home/ubuntu/cephtest/cephadm 2023-09-20T21:37:56.874 DEBUG:teuthology.orchestra.run.smithi089:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-20T21:37:57.546 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-rw-r--. 1 ubuntu ubuntu 733450 Sep 20 21:37 /home/ubuntu/cephtest/cephadm 2023-09-20T21:37:57.546 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20230920.072635/bf8a061ff16ac1d393cf108f740d92dfa49f2e54/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-09-20T21:37:58.224 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-rw-r--. 1 ubuntu ubuntu 733450 Sep 20 21:37 /home/ubuntu/cephtest/cephadm 2023-09-20T21:37:58.224 DEBUG:teuthology.orchestra.run.smithi022:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-20T21:37:58.276 DEBUG:teuthology.orchestra.run.smithi089:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-20T21:37:58.298 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-09-20T21:37:58.355 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 on all hosts... 2023-09-20T21:37:58.355 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 pull 2023-09-20T21:37:58.360 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 pull 2023-09-20T21:37:58.363 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 pull 2023-09-20T21:37:59.687 INFO:teuthology.orchestra.run.smithi089.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54... 2023-09-20T21:37:59.688 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54... 2023-09-20T21:38:02.659 INFO:teuthology.orchestra.run.smithi022.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54... 2023-09-20T21:38:30.227 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:38:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 18.0.0-6213-gbf8a061f (bf8a061ff16ac1d393cf108f740d92dfa49f2e54) reef (dev)", 2023-09-20T21:38:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "image_id": "6308322be975aa4495e6fffb6e31dbd07c38647cee234909b177f890e8cdc58e", 2023-09-20T21:38:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "repo_digests": [ 2023-09-20T21:38:30.228 INFO:teuthology.orchestra.run.smithi022.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b0ec4686b9c685bcc4ffaf8e80e1d2d088d15c1baac2421c646b69bff2d8bb68" 2023-09-20T21:38:30.229 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-09-20T21:38:30.229 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:38:50.193 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-20T21:38:50.194 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 18.0.0-6213-gbf8a061f (bf8a061ff16ac1d393cf108f740d92dfa49f2e54) reef (dev)", 2023-09-20T21:38:50.194 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "6308322be975aa4495e6fffb6e31dbd07c38647cee234909b177f890e8cdc58e", 2023-09-20T21:38:50.194 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2023-09-20T21:38:50.194 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b0ec4686b9c685bcc4ffaf8e80e1d2d088d15c1baac2421c646b69bff2d8bb68" 2023-09-20T21:38:50.195 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2023-09-20T21:38:50.195 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-20T21:38:51.190 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-09-20T21:38:51.191 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 18.0.0-6213-gbf8a061f (bf8a061ff16ac1d393cf108f740d92dfa49f2e54) reef (dev)", 2023-09-20T21:38:51.191 INFO:teuthology.orchestra.run.smithi089.stdout: "image_id": "6308322be975aa4495e6fffb6e31dbd07c38647cee234909b177f890e8cdc58e", 2023-09-20T21:38:51.191 INFO:teuthology.orchestra.run.smithi089.stdout: "repo_digests": [ 2023-09-20T21:38:51.191 INFO:teuthology.orchestra.run.smithi089.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b0ec4686b9c685bcc4ffaf8e80e1d2d088d15c1baac2421c646b69bff2d8bb68" 2023-09-20T21:38:51.191 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2023-09-20T21:38:51.192 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-09-20T21:38:51.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph 2023-09-20T21:38:51.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph 2023-09-20T21:38:51.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2023-09-20T21:38:51.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /etc/ceph 2023-09-20T21:38:51.431 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /etc/ceph 2023-09-20T21:38:51.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2023-09-20T21:38:51.489 INFO:tasks.cephadm:Writing seed config... 2023-09-20T21:38:51.490 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-09-20T21:38:51.491 INFO:tasks.cephadm: override: [client] debug client = 20 2023-09-20T21:38:51.491 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-09-20T21:38:51.491 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-09-20T21:38:51.491 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-09-20T21:38:51.492 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-09-20T21:38:51.492 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-09-20T21:38:51.492 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-09-20T21:38:51.493 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-09-20T21:38:51.493 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-09-20T21:38:51.493 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-09-20T21:38:51.493 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-09-20T21:38:51.494 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-09-20T21:38:51.494 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-09-20T21:38:51.494 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-09-20T21:38:51.495 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-09-20T21:38:51.495 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-09-20T21:38:51.495 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-09-20T21:38:51.495 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-09-20T21:38:51.496 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-09-20T21:38:51.496 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-09-20T21:38:51.496 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-09-20T21:38:51.496 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-09-20T21:38:51.497 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-09-20T21:38:51.497 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-09-20T21:38:51.497 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-09-20T21:38:51.498 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-09-20T21:38:51.498 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-09-20T21:38:51.498 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-09-20T21:38:51.498 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-09-20T21:38:51.499 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-09-20T21:38:51.499 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-09-20T21:38:51.499 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-09-20T21:38:51.499 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-09-20T21:38:51.500 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-09-20T21:38:51.500 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-09-20T21:38:51.500 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-09-20T21:38:51.500 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-09-20T21:38:51.501 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-09-20T21:38:51.501 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-09-20T21:38:51.501 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-09-20T21:38:51.501 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-09-20T21:38:51.502 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-09-20T21:38:51.503 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:38:51.503 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-09-20T21:38:51.547 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-09-20T21:38:51.548 DEBUG:teuthology.orchestra.run.smithi022:mon.a> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.a.service 2023-09-20T21:38:51.592 DEBUG:teuthology.orchestra.run.smithi022:mgr.x> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.x.service 2023-09-20T21:38:51.637 INFO:tasks.cephadm:Bootstrapping... 2023-09-20T21:38:51.638 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 -v bootstrap --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.22 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-09-20T21:38:51.660 INFO:journalctl@ceph.mon.a.smithi022.stdout:-- Logs begin at Wed 2023-09-20 21:22:06 UTC. -- 2023-09-20T21:38:51.706 INFO:journalctl@ceph.mgr.x.smithi022.stdout:-- Logs begin at Wed 2023-09-20 21:22:06 UTC. -- 2023-09-20T21:38:51.871 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2023-09-20T21:38:51.871 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54', '-v', 'bootstrap', '--fsid', 'f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.22', '--skip-admin-label'] 2023-09-20T21:38:52.045 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.1.1 2023-09-20T21:38:52.046 INFO:teuthology.orchestra.run.smithi022.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-09-20T21:38:52.046 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying podman|docker is present... 2023-09-20T21:38:52.216 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.1.1 2023-09-20T21:38:52.217 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying lvm2 is present... 2023-09-20T21:38:52.217 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying time synchronization is in place... 2023-09-20T21:38:52.232 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-20T21:38:52.232 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-20T21:38:52.245 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-20T21:38:52.245 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2023-09-20T21:38:52.259 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2023-09-20T21:38:52.273 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2023-09-20T21:38:52.273 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2023-09-20T21:38:52.273 INFO:teuthology.orchestra.run.smithi022.stdout:Repeating the final host check... 2023-09-20T21:38:52.524 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.1.1 2023-09-20T21:38:52.705 INFO:teuthology.orchestra.run.smithi022.stdout:podman (/bin/podman) version 4.1.1 is present 2023-09-20T21:38:52.706 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl is present 2023-09-20T21:38:52.706 INFO:teuthology.orchestra.run.smithi022.stdout:lvcreate is present 2023-09-20T21:38:52.724 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-09-20T21:38:52.724 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-09-20T21:38:52.741 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-09-20T21:38:52.741 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2023-09-20T21:38:52.758 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2023-09-20T21:38:52.772 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2023-09-20T21:38:52.773 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2023-09-20T21:38:52.773 INFO:teuthology.orchestra.run.smithi022.stdout:Host looks OK 2023-09-20T21:38:52.773 INFO:teuthology.orchestra.run.smithi022.stdout:Cluster fsid: f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:38:52.774 INFO:teuthology.orchestra.run.smithi022.stdout:Acquiring lock 140206400895464 on /run/cephadm/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da.lock 2023-09-20T21:38:52.774 INFO:teuthology.orchestra.run.smithi022.stdout:Lock 140206400895464 acquired on /run/cephadm/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da.lock 2023-09-20T21:38:52.774 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 3300 ... 2023-09-20T21:38:52.775 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 6789 ... 2023-09-20T21:38:52.775 INFO:teuthology.orchestra.run.smithi022.stdout:Base mon IP(s) is [172.21.15.22:3300, 172.21.15.22:6789], mon addrv is [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2023-09-20T21:38:52.782 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-09-20T21:38:52.783 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.22 metric 100 2023-09-20T21:38:52.791 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-09-20T21:38:52.791 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-09-20T21:38:52.791 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1759sec hoplimit 64 pref medium 2023-09-20T21:38:52.798 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-09-20T21:38:52.798 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-09-20T21:38:52.798 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-20T21:38:52.798 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-09-20T21:38:52.799 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-09-20T21:38:52.799 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-09-20T21:38:52.800 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2023-09-20T21:38:52.800 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2023-09-20T21:38:52.800 INFO:teuthology.orchestra.run.smithi022.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-09-20T21:38:52.800 INFO:teuthology.orchestra.run.smithi022.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-09-20T21:38:52.801 INFO:teuthology.orchestra.run.smithi022.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54... 2023-09-20T21:38:59.100 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 6308322be975aa4495e6fffb6e31dbd07c38647cee234909b177f890e8cdc58e 2023-09-20T21:38:59.100 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54... 2023-09-20T21:38:59.100 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Getting image source signatures 2023-09-20T21:38:59.100 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:e58c58ff9ddbe36aa81383722955dea81127ecf69578e402a788fa7993bed9e7 2023-09-20T21:38:59.100 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:3f4fd5c3c800c6da9ef94691df203a05a5bbbf32c483d09d620ec06eef42ea2b 2023-09-20T21:38:59.101 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying config sha256:6308322be975aa4495e6fffb6e31dbd07c38647cee234909b177f890e8cdc58e 2023-09-20T21:38:59.101 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Writing manifest to image destination 2023-09-20T21:38:59.101 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Storing signatures 2023-09-20T21:39:00.081 INFO:teuthology.orchestra.run.smithi022.stdout:ceph: stdout ceph version 18.0.0-6213-gbf8a061f (bf8a061ff16ac1d393cf108f740d92dfa49f2e54) reef (dev) 2023-09-20T21:39:00.081 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph version: ceph version 18.0.0-6213-gbf8a061f (bf8a061ff16ac1d393cf108f740d92dfa49f2e54) reef (dev) 2023-09-20T21:39:00.082 INFO:teuthology.orchestra.run.smithi022.stdout:Extracting ceph user uid/gid from container image... 2023-09-20T21:39:00.856 INFO:teuthology.orchestra.run.smithi022.stdout:stat: stdout 167 167 2023-09-20T21:39:00.856 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial keys... 2023-09-20T21:39:01.730 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQB1Zgtlh1+sDhAAbDD47DcyVlSqy+immDiTZQ== 2023-09-20T21:39:02.613 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQB2ZgtlZq03ChAAR4MZo0d5nfaSJIIKlOHkWg== 2023-09-20T21:39:03.388 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQB2ZgtlkbBYOxAAtZC4TfZX04hr5oREp+qE+A== 2023-09-20T21:39:03.389 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial monmap... 2023-09-20T21:39:04.165 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-20T21:39:04.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-09-20T21:39:04.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:39:04.166 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-20T21:39:04.166 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool for a [v2:172.21.15.22:3300,v1:172.21.15.22:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-09-20T21:39:04.167 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = pacific 2023-09-20T21:39:04.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: set fsid to f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:39:04.167 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-09-20T21:39:04.167 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:39:04.167 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mon... 2023-09-20T21:39:05.046 INFO:teuthology.orchestra.run.smithi022.stdout:create mon.a on 2023-09-20T21:39:05.305 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-09-20T21:39:05.516 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-09-20T21:39:05.782 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da.target → /etc/systemd/system/ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da.target. 2023-09-20T21:39:05.782 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da.target → /etc/systemd/system/ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da.target. 2023-09-20T21:39:06.172 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.a 2023-09-20T21:39:06.173 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.a.service: Unit ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.a.service not loaded. 2023-09-20T21:39:06.389 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da.target.wants/ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.a.service → /etc/systemd/system/ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@.service. 2023-09-20T21:39:06.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:06 smithi022 systemd[1]: Starting Ceph mon.a for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:39:07.131 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:06 smithi022 podman[97106]: 2023-09-20T21:39:07.132 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:07 smithi022 bash[97106]: e9c14c8629d49c46903b9fbb6ce1ee87e4885d30a217135e5fc93a89fc23fe8c 2023-09-20T21:39:07.202 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-20T21:39:07.202 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2023-09-20T21:39:07.218 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-20T21:39:07.218 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2023-09-20T21:39:07.218 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2023-09-20T21:39:07.219 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2023-09-20T21:39:07.219 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon to start... 2023-09-20T21:39:07.219 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon... 2023-09-20T21:39:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:07 smithi022 ceph-mon[97146]: mkfs f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:39:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:07 smithi022 ceph-mon[97146]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-20T21:39:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:07 smithi022 systemd[1]: Started Ceph mon.a for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:39:08.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:08 smithi022 ceph-mon[97146]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-20T21:39:08.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:08 smithi022 ceph-mon[97146]: monmap e1: 1 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0]} removed_ranks: {} 2023-09-20T21:39:08.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:08 smithi022 ceph-mon[97146]: fsmap 2023-09-20T21:39:08.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:08 smithi022 ceph-mon[97146]: osdmap e1: 0 total, 0 up, 0 in 2023-09-20T21:39:08.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:08 smithi022 ceph-mon[97146]: mgrmap e1: no daemons active 2023-09-20T21:39:08.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:08 smithi022 ceph-mon[97146]: from='client.? 172.21.15.22:0/2754805512' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-09-20T21:39:08.304 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout cluster: 2023-09-20T21:39:08.314 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout id: f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:39:08.314 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-09-20T21:39:08.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:08.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout services: 2023-09-20T21:39:08.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.741386s) 2023-09-20T21:39:08.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-09-20T21:39:08.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-09-20T21:39:08.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:08.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout data: 2023-09-20T21:39:08.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-09-20T21:39:08.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-09-20T21:39:08.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-09-20T21:39:08.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pgs: 2023-09-20T21:39:08.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:08.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.779+0000 7f25244cc700 1 Processor -- start 2023-09-20T21:39:08.404 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.779+0000 7f25244cc700 1 -- start start 2023-09-20T21:39:08.404 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.780+0000 7f25244cc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c07a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:08.405 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.780+0000 7f25244cc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c07a870 con 0x7f251c07be40 2023-09-20T21:39:08.405 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.781+0000 7f2522268700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c07a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:08.405 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.781+0000 7f2522268700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c07a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48606/0 (socket says 172.21.15.22:48606) 2023-09-20T21:39:08.405 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.781+0000 7f2522268700 1 -- 172.21.15.22:0/168923838 learned_addr learned my addr 172.21.15.22:0/168923838 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:08.405 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.781+0000 7f2522268700 1 -- 172.21.15.22:0/168923838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c07a9b0 con 0x7f251c07be40 2023-09-20T21:39:08.406 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.781+0000 7f2522268700 1 --2- 172.21.15.22:0/168923838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c07a2a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25180098a0 tx=0x7f2518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e9ce279c1736c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:08.406 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.782+0000 7f2521266700 1 -- 172.21.15.22:0/168923838 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2518004160 con 0x7f251c07be40 2023-09-20T21:39:08.406 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.782+0000 7f2521266700 1 -- 172.21.15.22:0/168923838 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25180042c0 con 0x7f251c07be40 2023-09-20T21:39:08.406 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.782+0000 7f2521266700 1 -- 172.21.15.22:0/168923838 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25180045a0 con 0x7f251c07be40 2023-09-20T21:39:08.406 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.782+0000 7f25244cc700 1 -- 172.21.15.22:0/168923838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 msgr2=0x7f251c07a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:08.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.782+0000 7f25244cc700 1 --2- 172.21.15.22:0/168923838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c07a2a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25180098a0 tx=0x7f2518009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:08.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.782+0000 7f25244cc700 1 -- 172.21.15.22:0/168923838 shutdown_connections 2023-09-20T21:39:08.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.782+0000 7f25244cc700 1 --2- 172.21.15.22:0/168923838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c07a2a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:08.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.782+0000 7f25244cc700 1 -- 172.21.15.22:0/168923838 >> 172.21.15.22:0/168923838 conn(0x7f251c101b30 msgr2=0x7f251c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:08.407 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.782+0000 7f25244cc700 1 -- 172.21.15.22:0/168923838 shutdown_connections 2023-09-20T21:39:08.408 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.783+0000 7f25244cc700 1 -- 172.21.15.22:0/168923838 wait complete. 2023-09-20T21:39:08.408 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.784+0000 7f25244cc700 1 Processor -- start 2023-09-20T21:39:08.408 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.784+0000 7f25244cc700 1 -- start start 2023-09-20T21:39:08.408 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.785+0000 7f25244cc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:08.408 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.785+0000 7f25244cc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c19c1c0 con 0x7f251c07be40 2023-09-20T21:39:08.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.785+0000 7f2522268700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:08.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.785+0000 7f2522268700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c19bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48618/0 (socket says 172.21.15.22:48618) 2023-09-20T21:39:08.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.785+0000 7f2522268700 1 -- 172.21.15.22:0/2754805512 learned_addr learned my addr 172.21.15.22:0/2754805512 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:08.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.785+0000 7f2522268700 1 -- 172.21.15.22:0/2754805512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2518009550 con 0x7f251c07be40 2023-09-20T21:39:08.409 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.786+0000 7f2522268700 1 --2- 172.21.15.22:0/2754805512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c19bc80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f251c07b310 tx=0x7f2518004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:08.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.786+0000 7f25137fe700 1 -- 172.21.15.22:0/2754805512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2518011630 con 0x7f251c07be40 2023-09-20T21:39:08.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.786+0000 7f25137fe700 1 -- 172.21.15.22:0/2754805512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2518011790 con 0x7f251c07be40 2023-09-20T21:39:08.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.786+0000 7f25137fe700 1 -- 172.21.15.22:0/2754805512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2518011a30 con 0x7f251c07be40 2023-09-20T21:39:08.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.786+0000 7f25244cc700 1 -- 172.21.15.22:0/2754805512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c19c3c0 con 0x7f251c07be40 2023-09-20T21:39:08.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.786+0000 7f25244cc700 1 -- 172.21.15.22:0/2754805512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c19c7e0 con 0x7f251c07be40 2023-09-20T21:39:08.410 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.786+0000 7f25137fe700 1 -- 172.21.15.22:0/2754805512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2518022070 con 0x7f251c07be40 2023-09-20T21:39:08.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.786+0000 7f25137fe700 1 -- 172.21.15.22:0/2754805512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f251801c450 con 0x7f251c07be40 2023-09-20T21:39:08.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.787+0000 7f25244cc700 1 -- 172.21.15.22:0/2754805512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c195b40 con 0x7f251c07be40 2023-09-20T21:39:08.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.790+0000 7f25137fe700 1 -- 172.21.15.22:0/2754805512 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f251801b450 con 0x7f251c07be40 2023-09-20T21:39:08.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.868+0000 7f25244cc700 1 -- 172.21.15.22:0/2754805512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f251c02cef0 con 0x7f251c07be40 2023-09-20T21:39:08.411 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.868+0000 7f25137fe700 1 -- 172.21.15.22:0/2754805512 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f251801b5f0 con 0x7f251c07be40 2023-09-20T21:39:08.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.869+0000 7f25244cc700 1 -- 172.21.15.22:0/2754805512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 msgr2=0x7f251c19bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:08.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.869+0000 7f25244cc700 1 --2- 172.21.15.22:0/2754805512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c19bc80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f251c07b310 tx=0x7f2518004750 comp rx=0 tx=0).stop 2023-09-20T21:39:08.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.869+0000 7f25244cc700 1 -- 172.21.15.22:0/2754805512 shutdown_connections 2023-09-20T21:39:08.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.869+0000 7f25244cc700 1 --2- 172.21.15.22:0/2754805512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f251c07be40 0x7f251c19bc80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:08.412 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.869+0000 7f25244cc700 1 -- 172.21.15.22:0/2754805512 >> 172.21.15.22:0/2754805512 conn(0x7f251c101b30 msgr2=0x7f251c103570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:08.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.869+0000 7f25244cc700 1 -- 172.21.15.22:0/2754805512 shutdown_connections 2023-09-20T21:39:08.413 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:07.869+0000 7f25244cc700 1 -- 172.21.15.22:0/2754805512 wait complete. 2023-09-20T21:39:08.413 INFO:teuthology.orchestra.run.smithi022.stdout:mon is available 2023-09-20T21:39:08.413 INFO:teuthology.orchestra.run.smithi022.stdout:Assimilating anything we can from ceph.conf... 2023-09-20T21:39:09.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:09 smithi022 ceph-mon[97146]: from='client.? 172.21.15.22:0/949496586' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-20T21:39:09.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:09 smithi022 ceph-mon[97146]: from='client.? 172.21.15.22:0/949496586' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-09-20T21:39:09.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:09.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2023-09-20T21:39:09.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:39:09.390 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2023-09-20T21:39:09.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-20T21:39:09.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-20T21:39:09.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-20T21:39:09.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-20T21:39:09.391 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:09.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-20T21:39:09.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-20T21:39:09.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:09.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2023-09-20T21:39:09.392 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-20T21:39:09.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-20T21:39:09.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-20T21:39:09.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-20T21:39:09.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-20T21:39:09.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.845+0000 7fe669afa700 1 Processor -- start 2023-09-20T21:39:09.393 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.846+0000 7fe669afa700 1 -- start start 2023-09-20T21:39:09.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.846+0000 7fe669afa700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:09.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.846+0000 7fe669afa700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66407a890 con 0x7fe66407be60 2023-09-20T21:39:09.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.846+0000 7fe6637fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:09.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.846+0000 7fe6637fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48630/0 (socket says 172.21.15.22:48630) 2023-09-20T21:39:09.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.846+0000 7fe6637fe700 1 -- 172.21.15.22:0/503661320 learned_addr learned my addr 172.21.15.22:0/503661320 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:09.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.847+0000 7fe6637fe700 1 -- 172.21.15.22:0/503661320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66407a9d0 con 0x7fe66407be60 2023-09-20T21:39:09.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.847+0000 7fe6637fe700 1 --2- 172.21.15.22:0/503661320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66407a2c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe6540098a0 tx=0x7fe654009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15902951013792 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:09.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.847+0000 7fe6627fc700 1 -- 172.21.15.22:0/503661320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe654004160 con 0x7fe66407be60 2023-09-20T21:39:09.395 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.847+0000 7fe6627fc700 1 -- 172.21.15.22:0/503661320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6540042c0 con 0x7fe66407be60 2023-09-20T21:39:09.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.847+0000 7fe6627fc700 1 -- 172.21.15.22:0/503661320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6540045a0 con 0x7fe66407be60 2023-09-20T21:39:09.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.848+0000 7fe669afa700 1 -- 172.21.15.22:0/503661320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 msgr2=0x7fe66407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:09.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.848+0000 7fe669afa700 1 --2- 172.21.15.22:0/503661320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66407a2c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe6540098a0 tx=0x7fe654009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:09.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.848+0000 7fe669afa700 1 -- 172.21.15.22:0/503661320 shutdown_connections 2023-09-20T21:39:09.396 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.848+0000 7fe669afa700 1 --2- 172.21.15.22:0/503661320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66407a2c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:09.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.848+0000 7fe669afa700 1 -- 172.21.15.22:0/503661320 >> 172.21.15.22:0/503661320 conn(0x7fe664101b30 msgr2=0x7fe664103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:09.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.848+0000 7fe669afa700 1 -- 172.21.15.22:0/503661320 shutdown_connections 2023-09-20T21:39:09.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.848+0000 7fe669afa700 1 -- 172.21.15.22:0/503661320 wait complete. 2023-09-20T21:39:09.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.849+0000 7fe669afa700 1 Processor -- start 2023-09-20T21:39:09.397 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.849+0000 7fe669afa700 1 -- start start 2023-09-20T21:39:09.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.849+0000 7fe669afa700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:09.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.849+0000 7fe669afa700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66419c1d0 con 0x7fe66407be60 2023-09-20T21:39:09.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.850+0000 7fe6637fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:09.398 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.850+0000 7fe6637fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48646/0 (socket says 172.21.15.22:48646) 2023-09-20T21:39:09.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.850+0000 7fe6637fe700 1 -- 172.21.15.22:0/949496586 learned_addr learned my addr 172.21.15.22:0/949496586 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:09.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.850+0000 7fe6637fe700 1 -- 172.21.15.22:0/949496586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe654009550 con 0x7fe66407be60 2023-09-20T21:39:09.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.851+0000 7fe6637fe700 1 --2- 172.21.15.22:0/949496586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66419bc90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe66407b330 tx=0x7fe654004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:09.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.851+0000 7fe660ff9700 1 -- 172.21.15.22:0/949496586 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe654011630 con 0x7fe66407be60 2023-09-20T21:39:09.399 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.851+0000 7fe660ff9700 1 -- 172.21.15.22:0/949496586 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe654011790 con 0x7fe66407be60 2023-09-20T21:39:09.400 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.851+0000 7fe660ff9700 1 -- 172.21.15.22:0/949496586 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe654011a30 con 0x7fe66407be60 2023-09-20T21:39:09.400 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.851+0000 7fe669afa700 1 -- 172.21.15.22:0/949496586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66419c3d0 con 0x7fe66407be60 2023-09-20T21:39:09.400 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.851+0000 7fe669afa700 1 -- 172.21.15.22:0/949496586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66419c7f0 con 0x7fe66407be60 2023-09-20T21:39:09.400 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.851+0000 7fe660ff9700 1 -- 172.21.15.22:0/949496586 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe654022070 con 0x7fe66407be60 2023-09-20T21:39:09.400 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.851+0000 7fe660ff9700 1 -- 172.21.15.22:0/949496586 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe65401c450 con 0x7fe66407be60 2023-09-20T21:39:09.400 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.852+0000 7fe669afa700 1 -- 172.21.15.22:0/949496586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe664195b90 con 0x7fe66407be60 2023-09-20T21:39:09.401 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.855+0000 7fe660ff9700 1 -- 172.21.15.22:0/949496586 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fe65401b450 con 0x7fe66407be60 2023-09-20T21:39:09.401 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.929+0000 7fe669afa700 1 -- 172.21.15.22:0/949496586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe66407ad40 con 0x7fe66407be60 2023-09-20T21:39:09.401 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.933+0000 7fe660ff9700 1 -- 172.21.15.22:0/949496586 <== mon.0 v2:172.21.15.22:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe65401b5f0 con 0x7fe66407be60 2023-09-20T21:39:09.401 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.933+0000 7fe660ff9700 1 -- 172.21.15.22:0/949496586 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fe654036590 con 0x7fe66407be60 2023-09-20T21:39:09.401 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.934+0000 7fe669afa700 1 -- 172.21.15.22:0/949496586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 msgr2=0x7fe66419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:09.402 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.934+0000 7fe669afa700 1 --2- 172.21.15.22:0/949496586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66419bc90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe66407b330 tx=0x7fe654004750 comp rx=0 tx=0).stop 2023-09-20T21:39:09.402 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.934+0000 7fe669afa700 1 -- 172.21.15.22:0/949496586 shutdown_connections 2023-09-20T21:39:09.402 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.934+0000 7fe669afa700 1 --2- 172.21.15.22:0/949496586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe66407be60 0x7fe66419bc90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:09.402 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.934+0000 7fe669afa700 1 -- 172.21.15.22:0/949496586 >> 172.21.15.22:0/949496586 conn(0x7fe664101b30 msgr2=0x7fe664103520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:09.403 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.934+0000 7fe669afa700 1 -- 172.21.15.22:0/949496586 shutdown_connections 2023-09-20T21:39:09.403 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:08.934+0000 7fe669afa700 1 -- 172.21.15.22:0/949496586 wait complete. 2023-09-20T21:39:09.403 INFO:teuthology.orchestra.run.smithi022.stdout:Generating new minimal ceph.conf... 2023-09-20T21:39:10.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:10 smithi022 ceph-mon[97146]: from='client.? 172.21.15.22:0/4127534430' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:39:10.473 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.936+0000 7f38d6c22700 1 Processor -- start 2023-09-20T21:39:10.473 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.936+0000 7f38d6c22700 1 -- start start 2023-09-20T21:39:10.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.936+0000 7f38d6c22700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:10.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.936+0000 7f38d6c22700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d007a8d0 con 0x7f38d007bea0 2023-09-20T21:39:10.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.937+0000 7f38d49be700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:10.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.937+0000 7f38d49be700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48656/0 (socket says 172.21.15.22:48656) 2023-09-20T21:39:10.474 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.937+0000 7f38d49be700 1 -- 172.21.15.22:0/225115784 learned_addr learned my addr 172.21.15.22:0/225115784 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:10.475 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.937+0000 7f38d49be700 1 -- 172.21.15.22:0/225115784 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d007aa10 con 0x7f38d007bea0 2023-09-20T21:39:10.475 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.937+0000 7f38d49be700 1 --2- 172.21.15.22:0/225115784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d007a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f38c40098a0 tx=0x7f38c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4e1f8b52bad6b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:10.476 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.938+0000 7f38cf7fe700 1 -- 172.21.15.22:0/225115784 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38c4004160 con 0x7f38d007bea0 2023-09-20T21:39:10.476 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.938+0000 7f38cf7fe700 1 -- 172.21.15.22:0/225115784 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f38c40042c0 con 0x7f38d007bea0 2023-09-20T21:39:10.476 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.938+0000 7f38cf7fe700 1 -- 172.21.15.22:0/225115784 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38c4011a00 con 0x7f38d007bea0 2023-09-20T21:39:10.476 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.938+0000 7f38d6c22700 1 -- 172.21.15.22:0/225115784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 msgr2=0x7f38d007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:10.476 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.938+0000 7f38d6c22700 1 --2- 172.21.15.22:0/225115784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d007a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f38c40098a0 tx=0x7f38c4009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:10.476 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.938+0000 7f38d6c22700 1 -- 172.21.15.22:0/225115784 shutdown_connections 2023-09-20T21:39:10.477 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.938+0000 7f38d6c22700 1 --2- 172.21.15.22:0/225115784 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d007a300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:10.477 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.938+0000 7f38d6c22700 1 -- 172.21.15.22:0/225115784 >> 172.21.15.22:0/225115784 conn(0x7f38d0101b30 msgr2=0x7f38d0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:10.477 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.938+0000 7f38d6c22700 1 -- 172.21.15.22:0/225115784 shutdown_connections 2023-09-20T21:39:10.478 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.938+0000 7f38d6c22700 1 -- 172.21.15.22:0/225115784 wait complete. 2023-09-20T21:39:10.478 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.939+0000 7f38d6c22700 1 Processor -- start 2023-09-20T21:39:10.478 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.939+0000 7f38d6c22700 1 -- start start 2023-09-20T21:39:10.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.940+0000 7f38d6c22700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:10.479 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.940+0000 7f38d6c22700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d019ca80 con 0x7f38d007bea0 2023-09-20T21:39:10.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.940+0000 7f38d49be700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:10.480 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.940+0000 7f38d49be700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48658/0 (socket says 172.21.15.22:48658) 2023-09-20T21:39:10.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.940+0000 7f38d49be700 1 -- 172.21.15.22:0/4127534430 learned_addr learned my addr 172.21.15.22:0/4127534430 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:10.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.940+0000 7f38d49be700 1 -- 172.21.15.22:0/4127534430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c4009550 con 0x7f38d007bea0 2023-09-20T21:39:10.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.941+0000 7f38d49be700 1 --2- 172.21.15.22:0/4127534430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d019c540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009130 tx=0x7f38c4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:10.481 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.941+0000 7f38cdffb700 1 -- 172.21.15.22:0/4127534430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38c4003ab0 con 0x7f38d007bea0 2023-09-20T21:39:10.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.941+0000 7f38cdffb700 1 -- 172.21.15.22:0/4127534430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f38c4003c10 con 0x7f38d007bea0 2023-09-20T21:39:10.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.941+0000 7f38cdffb700 1 -- 172.21.15.22:0/4127534430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38c40247d0 con 0x7f38d007bea0 2023-09-20T21:39:10.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.941+0000 7f38d6c22700 1 -- 172.21.15.22:0/4127534430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d019cc80 con 0x7f38d007bea0 2023-09-20T21:39:10.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.941+0000 7f38d6c22700 1 -- 172.21.15.22:0/4127534430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d019d0a0 con 0x7f38d007bea0 2023-09-20T21:39:10.482 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.941+0000 7f38cdffb700 1 -- 172.21.15.22:0/4127534430 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f38c4024d00 con 0x7f38d007bea0 2023-09-20T21:39:10.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.941+0000 7f38cdffb700 1 -- 172.21.15.22:0/4127534430 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f38c40193c0 con 0x7f38d007bea0 2023-09-20T21:39:10.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.942+0000 7f38d6c22700 1 -- 172.21.15.22:0/4127534430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d0196590 con 0x7f38d007bea0 2023-09-20T21:39:10.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:09.945+0000 7f38cdffb700 1 -- 172.21.15.22:0/4127534430 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f38c401abb0 con 0x7f38d007bea0 2023-09-20T21:39:10.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:10.018+0000 7f38d6c22700 1 -- 172.21.15.22:0/4127534430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f38d0062410 con 0x7f38d007bea0 2023-09-20T21:39:10.483 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:10.018+0000 7f38cdffb700 1 -- 172.21.15.22:0/4127534430 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f38c401ad50 con 0x7f38d007bea0 2023-09-20T21:39:10.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:10.019+0000 7f38d6c22700 1 -- 172.21.15.22:0/4127534430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 msgr2=0x7f38d019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:10.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:10.019+0000 7f38d6c22700 1 --2- 172.21.15.22:0/4127534430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d019c540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009130 tx=0x7f38c4011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:10.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:10.019+0000 7f38d6c22700 1 -- 172.21.15.22:0/4127534430 shutdown_connections 2023-09-20T21:39:10.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:10.019+0000 7f38d6c22700 1 --2- 172.21.15.22:0/4127534430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38d007bea0 0x7f38d019c540 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:10.484 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:10.019+0000 7f38d6c22700 1 -- 172.21.15.22:0/4127534430 >> 172.21.15.22:0/4127534430 conn(0x7f38d0101b30 msgr2=0x7f38d01032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:10.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:10.019+0000 7f38d6c22700 1 -- 172.21.15.22:0/4127534430 shutdown_connections 2023-09-20T21:39:10.485 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:10.019+0000 7f38d6c22700 1 -- 172.21.15.22:0/4127534430 wait complete. 2023-09-20T21:39:10.485 INFO:teuthology.orchestra.run.smithi022.stdout:Restarting the monitor... 2023-09-20T21:39:10.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:10 smithi022 systemd[1]: Stopping Ceph mon.a for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:39:11.198 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:10 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-a[97122]: 2023-09-20T21:39:10.936+0000 7fe7f368a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T21:39:11.198 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:10 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-a[97122]: 2023-09-20T21:39:10.936+0000 7fe7f368a700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-09-20T21:39:11.478 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:11 smithi022 bash[97484]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-a 2023-09-20T21:39:11.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:11 smithi022 systemd[1]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.a.service: Succeeded. 2023-09-20T21:39:11.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:11 smithi022 systemd[1]: Stopped Ceph mon.a for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:39:11.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:11 smithi022 systemd[1]: Starting Ceph mon.a for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:39:12.409 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 podman[97611]: 2023-09-20T21:39:12.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 bash[97611]: 0ee46ea65b53f579748a7ad790c528a49475b9ef39d0a9a3aed494e98660009f 2023-09-20T21:39:12.410 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: set uid:gid to 167:167 (ceph:ceph) 2023-09-20T21:39:12.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: ceph version 18.0.0-6213-gbf8a061f (bf8a061ff16ac1d393cf108f740d92dfa49f2e54) reef (dev), process ceph-mon, pid 7 2023-09-20T21:39:12.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: pidfile_write: ignore empty --pid-file 2023-09-20T21:39:12.429 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: load: jerasure load: lrc 2023-09-20T21:39:12.430 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: RocksDB version: 7.9.2 2023-09-20T21:39:12.430 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Git sha 0 2023-09-20T21:39:12.430 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Compile date 2023-09-20 08:00:43 2023-09-20T21:39:12.430 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: DB SUMMARY 2023-09-20T21:39:12.430 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: DB Session ID: R1QBZXDCE9Z0X6Z8PSKI 2023-09-20T21:39:12.431 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: CURRENT file: CURRENT 2023-09-20T21:39:12.431 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: IDENTITY file: IDENTITY 2023-09-20T21:39:12.431 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-09-20T21:39:12.431 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-09-20T21:39:12.431 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98343 ; 2023-09-20T21:39:12.432 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.error_if_exists: 0 2023-09-20T21:39:12.432 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.create_if_missing: 0 2023-09-20T21:39:12.432 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.paranoid_checks: 1 2023-09-20T21:39:12.432 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-20T21:39:12.432 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-20T21:39:12.433 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-20T21:39:12.433 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.env: 0x563a4c03cda0 2023-09-20T21:39:12.433 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.fs: PosixFileSystem 2023-09-20T21:39:12.433 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.info_log: 0x563a4e2f5580 2023-09-20T21:39:12.433 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_file_opening_threads: 16 2023-09-20T21:39:12.433 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.statistics: (nil) 2023-09-20T21:39:12.434 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.use_fsync: 0 2023-09-20T21:39:12.434 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_log_file_size: 0 2023-09-20T21:39:12.434 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-20T21:39:12.434 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-20T21:39:12.434 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.keep_log_file_num: 1000 2023-09-20T21:39:12.435 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.recycle_log_file_num: 0 2023-09-20T21:39:12.435 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.allow_fallocate: 1 2023-09-20T21:39:12.435 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.allow_mmap_reads: 0 2023-09-20T21:39:12.435 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.allow_mmap_writes: 0 2023-09-20T21:39:12.435 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.use_direct_reads: 0 2023-09-20T21:39:12.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-20T21:39:12.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.create_missing_column_families: 0 2023-09-20T21:39:12.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.db_log_dir: 2023-09-20T21:39:12.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.wal_dir: 2023-09-20T21:39:12.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-20T21:39:12.437 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-20T21:39:12.437 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-20T21:39:12.437 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-20T21:39:12.437 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-20T21:39:12.438 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-20T21:39:12.438 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.advise_random_on_open: 1 2023-09-20T21:39:12.438 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.db_write_buffer_size: 0 2023-09-20T21:39:12.438 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.write_buffer_manager: 0x563a4e2ad4a0 2023-09-20T21:39:12.438 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-20T21:39:12.439 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-20T21:39:12.439 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-20T21:39:12.439 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.rate_limiter: (nil) 2023-09-20T21:39:12.439 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-20T21:39:12.439 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.wal_recovery_mode: 2 2023-09-20T21:39:12.440 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.enable_thread_tracking: 0 2023-09-20T21:39:12.440 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.enable_pipelined_write: 0 2023-09-20T21:39:12.440 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.unordered_write: 0 2023-09-20T21:39:12.440 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-20T21:39:12.440 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-20T21:39:12.441 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-20T21:39:12.441 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-20T21:39:12.476 INFO:teuthology.orchestra.run.smithi022.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-09-20T21:39:12.742 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.row_cache: None 2023-09-20T21:39:12.742 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.wal_filter: None 2023-09-20T21:39:12.742 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-20T21:39:12.742 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.allow_ingest_behind: 0 2023-09-20T21:39:12.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.two_write_queues: 0 2023-09-20T21:39:12.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.manual_wal_flush: 0 2023-09-20T21:39:12.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.wal_compression: 0 2023-09-20T21:39:12.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.atomic_flush: 0 2023-09-20T21:39:12.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-20T21:39:12.743 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-20T21:39:12.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-20T21:39:12.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.log_readahead_size: 0 2023-09-20T21:39:12.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-20T21:39:12.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.best_efforts_recovery: 0 2023-09-20T21:39:12.744 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-20T21:39:12.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-20T21:39:12.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.allow_data_in_errors: 0 2023-09-20T21:39:12.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.db_host_id: __hostname__ 2023-09-20T21:39:12.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-20T21:39:12.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_background_jobs: 2 2023-09-20T21:39:12.745 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_background_compactions: -1 2023-09-20T21:39:12.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_subcompactions: 1 2023-09-20T21:39:12.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-20T21:39:12.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-20T21:39:12.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-20T21:39:12.746 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_total_wal_size: 0 2023-09-20T21:39:12.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-20T21:39:12.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-20T21:39:12.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-20T21:39:12.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-20T21:39:12.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_open_files: -1 2023-09-20T21:39:12.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bytes_per_sync: 0 2023-09-20T21:39:12.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-20T21:39:12.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-20T21:39:12.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_readahead_size: 0 2023-09-20T21:39:12.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_background_flushes: -1 2023-09-20T21:39:12.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Compression algorithms supported: 2023-09-20T21:39:12.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-20T21:39:12.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: kZSTD supported: 0 2023-09-20T21:39:12.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: kXpressCompression supported: 0 2023-09-20T21:39:12.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: kLZ4HCCompression supported: 1 2023-09-20T21:39:12.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: kZlibCompression supported: 1 2023-09-20T21:39:12.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: kSnappyCompression supported: 1 2023-09-20T21:39:12.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: kLZ4Compression supported: 1 2023-09-20T21:39:12.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: kBZip2Compression supported: 0 2023-09-20T21:39:12.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-20T21:39:12.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-20T21:39:12.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-09-20T21:39:12.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-20T21:39:12.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-20T21:39:12.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.merge_operator: 2023-09-20T21:39:12.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_filter: None 2023-09-20T21:39:12.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_filter_factory: None 2023-09-20T21:39:12.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.sst_partitioner_factory: None 2023-09-20T21:39:12.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-20T21:39:12.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-20T21:39:12.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563a4e2f5680) 2023-09-20T21:39:12.754 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks: 1 2023-09-20T21:39:12.754 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-20T21:39:12.755 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-20T21:39:12.755 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_top_level_index_and_filter: 1 2023-09-20T21:39:12.755 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_type: 0 2023-09-20T21:39:12.755 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_index_type: 0 2023-09-20T21:39:12.755 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_shortening: 1 2023-09-20T21:39:12.756 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-20T21:39:12.756 INFO:journalctl@ceph.mon.a.smithi022.stdout: checksum: 4 2023-09-20T21:39:12.756 INFO:journalctl@ceph.mon.a.smithi022.stdout: no_block_cache: 0 2023-09-20T21:39:12.756 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache: 0x563a4e34b1f0 2023-09-20T21:39:12.756 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_name: BinnedLRUCache 2023-09-20T21:39:12.756 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_options: 2023-09-20T21:39:12.757 INFO:journalctl@ceph.mon.a.smithi022.stdout: capacity : 536870912 2023-09-20T21:39:12.757 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_shard_bits : 4 2023-09-20T21:39:12.757 INFO:journalctl@ceph.mon.a.smithi022.stdout: strict_capacity_limit : 0 2023-09-20T21:39:12.757 INFO:journalctl@ceph.mon.a.smithi022.stdout: high_pri_pool_ratio: 0.000 2023-09-20T21:39:12.757 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_compressed: (nil) 2023-09-20T21:39:12.757 INFO:journalctl@ceph.mon.a.smithi022.stdout: persistent_cache: (nil) 2023-09-20T21:39:12.758 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size: 4096 2023-09-20T21:39:12.758 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size_deviation: 10 2023-09-20T21:39:12.758 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_restart_interval: 16 2023-09-20T21:39:12.758 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_block_restart_interval: 1 2023-09-20T21:39:12.758 INFO:journalctl@ceph.mon.a.smithi022.stdout: metadata_block_size: 4096 2023-09-20T21:39:12.758 INFO:journalctl@ceph.mon.a.smithi022.stdout: partition_filters: 0 2023-09-20T21:39:12.759 INFO:journalctl@ceph.mon.a.smithi022.stdout: use_delta_encoding: 1 2023-09-20T21:39:12.759 INFO:journalctl@ceph.mon.a.smithi022.stdout: filter_policy: bloomfilter 2023-09-20T21:39:12.759 INFO:journalctl@ceph.mon.a.smithi022.stdout: whole_key_filtering: 1 2023-09-20T21:39:12.759 INFO:journalctl@ceph.mon.a.smithi022.stdout: verify_compression: 0 2023-09-20T21:39:12.759 INFO:journalctl@ceph.mon.a.smithi022.stdout: read_amp_bytes_per_bit: 0 2023-09-20T21:39:12.759 INFO:journalctl@ceph.mon.a.smithi022.stdout: format_version: 5 2023-09-20T21:39:12.760 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_index_compression: 1 2023-09-20T21:39:12.760 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_align: 0 2023-09-20T21:39:12.760 INFO:journalctl@ceph.mon.a.smithi022.stdout: max_auto_readahead_size: 262144 2023-09-20T21:39:12.760 INFO:journalctl@ceph.mon.a.smithi022.stdout: prepopulate_block_cache: 0 2023-09-20T21:39:12.760 INFO:journalctl@ceph.mon.a.smithi022.stdout: initial_auto_readahead_size: 8192 2023-09-20T21:39:12.760 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_file_reads_for_auto_readahead: 2 2023-09-20T21:39:12.761 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.write_buffer_size: 33554432 2023-09-20T21:39:12.761 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_write_buffer_number: 2 2023-09-20T21:39:12.761 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compression: NoCompression 2023-09-20T21:39:12.761 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bottommost_compression: Disabled 2023-09-20T21:39:12.761 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.prefix_extractor: nullptr 2023-09-20T21:39:12.761 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-20T21:39:12.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.num_levels: 7 2023-09-20T21:39:12.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-20T21:39:12.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-20T21:39:12.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-20T21:39:12.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-20T21:39:12.763 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-20T21:39:12.763 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-20T21:39:12.763 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-20T21:39:12.763 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-20T21:39:12.763 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-20T21:39:12.763 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-20T21:39:12.763 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-20T21:39:12.764 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-20T21:39:12.764 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-20T21:39:12.764 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compression_opts.level: 32767 2023-09-20T21:39:12.764 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compression_opts.strategy: 0 2023-09-20T21:39:12.764 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-20T21:39:12.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-20T21:39:12.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-20T21:39:12.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-20T21:39:12.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compression_opts.enabled: false 2023-09-20T21:39:12.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-20T21:39:12.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-20T21:39:12.766 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-20T21:39:12.766 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-20T21:39:12.766 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.target_file_size_base: 67108864 2023-09-20T21:39:12.766 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-20T21:39:12.766 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-20T21:39:12.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-20T21:39:12.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-20T21:39:12.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-20T21:39:12.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-20T21:39:12.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-20T21:39:12.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-20T21:39:12.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-20T21:39:12.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-20T21:39:12.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-20T21:39:12.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-20T21:39:12.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-20T21:39:12.768 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-20T21:39:12.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.arena_block_size: 1048576 2023-09-20T21:39:12.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-20T21:39:12.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-20T21:39:12.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.disable_auto_compactions: 0 2023-09-20T21:39:12.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-20T21:39:12.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-20T21:39:12.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-20T21:39:12.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-20T21:39:12.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-20T21:39:12.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-20T21:39:12.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-20T21:39:12.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-20T21:39:12.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-20T21:39:12.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-20T21:39:12.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.table_properties_collectors: 2023-09-20T21:39:12.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.inplace_update_support: 0 2023-09-20T21:39:12.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-20T21:39:12.771 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-20T21:39:12.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-20T21:39:12.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-20T21:39:12.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.bloom_locality: 0 2023-09-20T21:39:12.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.max_successive_merges: 0 2023-09-20T21:39:12.772 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-20T21:39:12.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.paranoid_file_checks: 0 2023-09-20T21:39:12.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.force_consistency_checks: 1 2023-09-20T21:39:12.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.report_bg_io_stats: 0 2023-09-20T21:39:12.773 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.ttl: 2592000 2023-09-20T21:39:12.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-20T21:39:12.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-20T21:39:12.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-20T21:39:12.774 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.enable_blob_files: false 2023-09-20T21:39:12.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.min_blob_size: 0 2023-09-20T21:39:12.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.blob_file_size: 268435456 2023-09-20T21:39:12.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-20T21:39:12.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-20T21:39:12.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-20T21:39:12.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-20T21:39:12.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-20T21:39:12.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.blob_file_starting_level: 0 2023-09-20T21:39:12.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-20T21:39:12.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-09-20T21:39:12.776 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-09-20T21:39:12.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cac97cb1-2ee8-4c82-a99d-7933f5a26b4e 2023-09-20T21:39:12.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695245952423161, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-09-20T21:39:12.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-09-20T21:39:12.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695245952425944, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93639, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 312, "table_properties": {"data_size": 91639, "index_size": 282, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13531, "raw_average_key_size": 49, "raw_value_size": 84253, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1695245952, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cac97cb1-2ee8-4c82-a99d-7933f5a26b4e", "db_session_id": "R1QBZXDCE9Z0X6Z8PSKI", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-09-20T21:39:12.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695245952426390, "job": 1, "event": "recovery_finished"} 2023-09-20T21:39:12.777 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-09-20T21:39:12.778 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-20T21:39:12.778 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563a4e3e8000 2023-09-20T21:39:12.778 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: DB pointer 0x563a4e3d4000 2023-09-20T21:39:12.778 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-20T21:39:12.778 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-20T21:39:12.778 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** DB Stats ** 2023-09-20T21:39:12.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-20T21:39:12.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-20T21:39:12.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-20T21:39:12.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-20T21:39:12.779 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-20T21:39:12.780 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-20T21:39:12.780 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-20T21:39:12.780 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2023-09-20T21:39:12.780 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2023-09-20T21:39:12.780 INFO:journalctl@ceph.mon.a.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-20T21:39:12.780 INFO:journalctl@ceph.mon.a.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-20T21:39:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout: L0 2/0 93.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 51.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-20T21:39:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout: Sum 2/0 93.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 51.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-20T21:39:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 51.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-20T21:39:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2023-09-20T21:39:12.781 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2023-09-20T21:39:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-20T21:39:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-20T21:39:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 51.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2023-09-20T21:39:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2023-09-20T21:39:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-20T21:39:12.782 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2023-09-20T21:39:12.783 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-20T21:39:12.783 INFO:journalctl@ceph.mon.a.smithi022.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-20T21:39:12.783 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-20T21:39:12.783 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-20T21:39:12.783 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-20T21:39:12.784 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-20T21:39:12.784 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative compaction: 0.00 GB write, 10.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-20T21:39:12.784 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval compaction: 0.00 GB write, 10.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-20T21:39:12.784 INFO:journalctl@ceph.mon.a.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-20T21:39:12.784 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache BinnedLRUCache@0x563a4e34b1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-09-20T21:39:12.784 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-09-20T21:39:12.785 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2023-09-20T21:39:12.785 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-20T21:39:12.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: starting mon.a rank 0 at public addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] at bind addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:39:12.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: mon.a@-1(???) e1 preinit fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:39:12.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: mon.a@-1(???).mds e1 new map 2023-09-20T21:39:12.785 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: mon.a@-1(???).mds e1 print_map 2023-09-20T21:39:12.786 INFO:journalctl@ceph.mon.a.smithi022.stdout: e1 2023-09-20T21:39:12.786 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-20T21:39:12.786 INFO:journalctl@ceph.mon.a.smithi022.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-20T21:39:12.786 INFO:journalctl@ceph.mon.a.smithi022.stdout: legacy client fscid: -1 2023-09-20T21:39:12.786 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2023-09-20T21:39:12.787 INFO:journalctl@ceph.mon.a.smithi022.stdout: No filesystems configured 2023-09-20T21:39:12.787 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-20T21:39:12.787 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-20T21:39:12.787 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-20T21:39:12.787 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-09-20T21:39:12.787 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-09-20T21:39:12.788 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 systemd[1]: Started Ceph mon.a for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:39:12.788 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: mon.a is new leader, mons a in quorum (ranks 0) 2023-09-20T21:39:12.788 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: monmap e1: 1 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0]} removed_ranks: {} 2023-09-20T21:39:12.788 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: fsmap 2023-09-20T21:39:12.788 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: osdmap e1: 0 total, 0 up, 0 in 2023-09-20T21:39:12.788 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:12 smithi022 ceph-mon[97651]: mgrmap e1: no daemons active 2023-09-20T21:39:13.458 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.008+0000 7f9b009ad700 1 Processor -- start 2023-09-20T21:39:13.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.008+0000 7f9b009ad700 1 -- start start 2023-09-20T21:39:13.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.009+0000 7f9b009ad700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:13.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.009+0000 7f9b009ad700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc07a8d0 con 0x7f9afc07bea0 2023-09-20T21:39:13.459 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.010+0000 7f9afa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:13.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.010+0000 7f9afa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48674/0 (socket says 172.21.15.22:48674) 2023-09-20T21:39:13.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.010+0000 7f9afa59c700 1 -- 172.21.15.22:0/3700664545 learned_addr learned my addr 172.21.15.22:0/3700664545 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:13.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.010+0000 7f9afa59c700 1 -- 172.21.15.22:0/3700664545 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc07aa10 con 0x7f9afc07bea0 2023-09-20T21:39:13.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.010+0000 7f9afa59c700 1 --2- 172.21.15.22:0/3700664545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc07a300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0098a0 tx=0x7f9aec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5285bcc7ca44450f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:13.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.010+0000 7f9af959a700 1 -- 172.21.15.22:0/3700664545 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9aec004160 con 0x7f9afc07bea0 2023-09-20T21:39:13.460 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.011+0000 7f9af959a700 1 -- 172.21.15.22:0/3700664545 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9aec0042c0 con 0x7f9afc07bea0 2023-09-20T21:39:13.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.011+0000 7f9af959a700 1 -- 172.21.15.22:0/3700664545 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9aec011a00 con 0x7f9afc07bea0 2023-09-20T21:39:13.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.011+0000 7f9b009ad700 1 -- 172.21.15.22:0/3700664545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 msgr2=0x7f9afc07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:13.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.011+0000 7f9b009ad700 1 --2- 172.21.15.22:0/3700664545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc07a300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0098a0 tx=0x7f9aec009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:13.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.011+0000 7f9b009ad700 1 -- 172.21.15.22:0/3700664545 shutdown_connections 2023-09-20T21:39:13.461 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.011+0000 7f9b009ad700 1 --2- 172.21.15.22:0/3700664545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc07a300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:13.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.011+0000 7f9b009ad700 1 -- 172.21.15.22:0/3700664545 >> 172.21.15.22:0/3700664545 conn(0x7f9afc101b30 msgr2=0x7f9afc103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:13.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.011+0000 7f9b009ad700 1 -- 172.21.15.22:0/3700664545 shutdown_connections 2023-09-20T21:39:13.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.011+0000 7f9b009ad700 1 -- 172.21.15.22:0/3700664545 wait complete. 2023-09-20T21:39:13.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.012+0000 7f9b009ad700 1 Processor -- start 2023-09-20T21:39:13.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.013+0000 7f9b009ad700 1 -- start start 2023-09-20T21:39:13.462 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.013+0000 7f9b009ad700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:13.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.013+0000 7f9b009ad700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc19ca80 con 0x7f9afc07bea0 2023-09-20T21:39:13.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.013+0000 7f9afa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:13.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.013+0000 7f9afa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48680/0 (socket says 172.21.15.22:48680) 2023-09-20T21:39:13.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.013+0000 7f9afa59c700 1 -- 172.21.15.22:0/4223400878 learned_addr learned my addr 172.21.15.22:0/4223400878 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:13.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.014+0000 7f9afa59c700 1 -- 172.21.15.22:0/4223400878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec009550 con 0x7f9afc07bea0 2023-09-20T21:39:13.463 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.014+0000 7f9afa59c700 1 --2- 172.21.15.22:0/4223400878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc19c540 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009130 tx=0x7f9aec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:13.464 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.014+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/4223400878 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9aec003ab0 con 0x7f9afc07bea0 2023-09-20T21:39:13.464 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.014+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/4223400878 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9aec003c10 con 0x7f9afc07bea0 2023-09-20T21:39:13.464 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.014+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/4223400878 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9aec0247d0 con 0x7f9afc07bea0 2023-09-20T21:39:13.464 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.014+0000 7f9b009ad700 1 -- 172.21.15.22:0/4223400878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc19cc80 con 0x7f9afc07bea0 2023-09-20T21:39:13.464 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.014+0000 7f9b009ad700 1 -- 172.21.15.22:0/4223400878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc19d0a0 con 0x7f9afc07bea0 2023-09-20T21:39:13.464 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.015+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/4223400878 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9aec024d00 con 0x7f9afc07bea0 2023-09-20T21:39:13.465 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.015+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/4223400878 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9aec01a430 con 0x7f9afc07bea0 2023-09-20T21:39:13.465 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.015+0000 7f9b009ad700 1 -- 172.21.15.22:0/4223400878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc196590 con 0x7f9afc07bea0 2023-09-20T21:39:13.465 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.018+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/4223400878 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f9aec01abb0 con 0x7f9afc07bea0 2023-09-20T21:39:13.465 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.092+0000 7f9b009ad700 1 -- 172.21.15.22:0/4223400878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f9afc10cb60 con 0x7f9afc07bea0 2023-09-20T21:39:13.465 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.094+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/4223400878 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9aec01ad50 con 0x7f9afc07bea0 2023-09-20T21:39:13.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.095+0000 7f9b009ad700 1 -- 172.21.15.22:0/4223400878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 msgr2=0x7f9afc19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:13.466 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.095+0000 7f9b009ad700 1 --2- 172.21.15.22:0/4223400878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc19c540 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009130 tx=0x7f9aec011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:13.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.095+0000 7f9b009ad700 1 -- 172.21.15.22:0/4223400878 shutdown_connections 2023-09-20T21:39:13.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.095+0000 7f9b009ad700 1 --2- 172.21.15.22:0/4223400878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9afc07bea0 0x7f9afc19c540 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:13.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.095+0000 7f9b009ad700 1 -- 172.21.15.22:0/4223400878 >> 172.21.15.22:0/4223400878 conn(0x7f9afc101b30 msgr2=0x7f9afc1032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:13.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.095+0000 7f9b009ad700 1 -- 172.21.15.22:0/4223400878 shutdown_connections 2023-09-20T21:39:13.467 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:13.095+0000 7f9b009ad700 1 -- 172.21.15.22:0/4223400878 wait complete. 2023-09-20T21:39:13.468 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote config to /etc/ceph/ceph.conf 2023-09-20T21:39:13.482 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-09-20T21:39:13.482 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mgr... 2023-09-20T21:39:13.483 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:9283 ... 2023-09-20T21:39:13.483 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:8765 ... 2023-09-20T21:39:13.883 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.x 2023-09-20T21:39:13.883 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.x.service: Unit ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.x.service not loaded. 2023-09-20T21:39:14.090 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da.target.wants/ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.x.service → /etc/systemd/system/ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@.service. 2023-09-20T21:39:14.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:14 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4223400878' entity='client.admin' 2023-09-20T21:39:14.248 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:14 smithi022 systemd[1]: Starting Ceph mgr.x for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:39:14.881 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:14 smithi022 podman[97884]: 2023-09-20T21:39:14.881 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:14 smithi022 bash[97884]: 0f9ad732a2192c38b9f770996d1ffbb00d27f13b972a469ce5131f3a2622be49 2023-09-20T21:39:14.906 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-20T21:39:14.907 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2023-09-20T21:39:14.923 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-20T21:39:14.923 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2023-09-20T21:39:14.923 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2023-09-20T21:39:14.924 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2023-09-20T21:39:14.942 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-20T21:39:14.942 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2023-09-20T21:39:14.959 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-20T21:39:14.960 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2023-09-20T21:39:14.960 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2023-09-20T21:39:14.961 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-09-20T21:39:14.961 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr to start... 2023-09-20T21:39:14.961 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr... 2023-09-20T21:39:15.143 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:14 smithi022 systemd[1]: Started Ceph mgr.x for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:39:15.144 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:15.034+0000 7fb1191c9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-20T21:39:15.144 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:15.142+0000 7fb1191c9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-20T21:39:15.401 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:15.266+0000 7fb1191c9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-20T21:39:15.402 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:15.400+0000 7fb1191c9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-20T21:39:15.651 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:15.640+0000 7fb1191c9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-20T21:39:15.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:15 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1858667576' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-20T21:39:16.045 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:16.046 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2023-09-20T21:39:16.046 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:39:16.046 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2023-09-20T21:39:16.046 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-20T21:39:16.047 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-20T21:39:16.047 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-20T21:39:16.047 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:16.047 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-20T21:39:16.047 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-20T21:39:16.048 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2023-09-20T21:39:16.048 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:16.048 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-20T21:39:16.048 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2023-09-20T21:39:16.048 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:16.049 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-09-20T21:39:16.049 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-20T21:39:16.049 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:16.049 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-20T21:39:16.049 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-20T21:39:16.050 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:16.050 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-20T21:39:16.050 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:16.050 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-20T21:39:16.050 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-20T21:39:16.058 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-20T21:39:16.058 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-20T21:39:16.058 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-20T21:39:16.058 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-20T21:39:16.059 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:16.059 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-20T21:39:16.059 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-20T21:39:16.059 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-20T21:39:16.060 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-20T21:39:16.060 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-20T21:39:16.060 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-20T21:39:16.060 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-20T21:39:16.060 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-20T21:39:16.061 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-20T21:39:16.061 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:16.061 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-20T21:39:16.061 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:16.061 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-20T21:39:16.062 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-20T21:39:16.062 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:16.062 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-20T21:39:16.062 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-20T21:39:16.062 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-20T21:39:16.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-20T21:39:16.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-20T21:39:16.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-20T21:39:16.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2023-09-20T21:39:16.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:16.063 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-20T21:39:16.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:16.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-20T21:39:16.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:16.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2023-09-20T21:39:07.129574+0000", 2023-09-20T21:39:16.064 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-20T21:39:16.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:16.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-20T21:39:16.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2023-09-20T21:39:16.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.538+0000 7f697d542700 1 Processor -- start 2023-09-20T21:39:16.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.539+0000 7f697d542700 1 -- start start 2023-09-20T21:39:16.065 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.539+0000 7f697d542700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f697807a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:16.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.539+0000 7f697d542700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697807a890 con 0x7f697807be60 2023-09-20T21:39:16.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.540+0000 7f6976ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f697807a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:16.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.540+0000 7f6976ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f697807a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48690/0 (socket says 172.21.15.22:48690) 2023-09-20T21:39:16.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.540+0000 7f6976ffd700 1 -- 172.21.15.22:0/1282155190 learned_addr learned my addr 172.21.15.22:0/1282155190 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:16.066 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.540+0000 7f6976ffd700 1 -- 172.21.15.22:0/1282155190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697807a9d0 con 0x7f697807be60 2023-09-20T21:39:16.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.540+0000 7f6976ffd700 1 --2- 172.21.15.22:0/1282155190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f697807a2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f69680098a0 tx=0x7f6968009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505ab037a264af90 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:16.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.541+0000 7f6975ffb700 1 -- 172.21.15.22:0/1282155190 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6968004160 con 0x7f697807be60 2023-09-20T21:39:16.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.541+0000 7f6975ffb700 1 -- 172.21.15.22:0/1282155190 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69680042c0 con 0x7f697807be60 2023-09-20T21:39:16.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.541+0000 7f6975ffb700 1 -- 172.21.15.22:0/1282155190 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6968011a00 con 0x7f697807be60 2023-09-20T21:39:16.067 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.541+0000 7f697d542700 1 -- 172.21.15.22:0/1282155190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 msgr2=0x7f697807a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:16.068 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.541+0000 7f697d542700 1 --2- 172.21.15.22:0/1282155190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f697807a2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f69680098a0 tx=0x7f6968009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:16.068 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.541+0000 7f697d542700 1 -- 172.21.15.22:0/1282155190 shutdown_connections 2023-09-20T21:39:16.068 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.541+0000 7f697d542700 1 --2- 172.21.15.22:0/1282155190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f697807a2c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:16.068 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.541+0000 7f697d542700 1 -- 172.21.15.22:0/1282155190 >> 172.21.15.22:0/1282155190 conn(0x7f6978101b70 msgr2=0x7f6978103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:16.068 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.541+0000 7f697d542700 1 -- 172.21.15.22:0/1282155190 shutdown_connections 2023-09-20T21:39:16.069 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.542+0000 7f697d542700 1 -- 172.21.15.22:0/1282155190 wait complete. 2023-09-20T21:39:16.069 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.542+0000 7f697d542700 1 Processor -- start 2023-09-20T21:39:16.069 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.543+0000 7f697d542700 1 -- start start 2023-09-20T21:39:16.069 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.543+0000 7f697d542700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f6978109940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:16.069 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.543+0000 7f697d542700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6978109e80 con 0x7f697807be60 2023-09-20T21:39:16.070 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.543+0000 7f6976ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f6978109940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:16.070 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.543+0000 7f6976ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f6978109940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48704/0 (socket says 172.21.15.22:48704) 2023-09-20T21:39:16.070 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.543+0000 7f6976ffd700 1 -- 172.21.15.22:0/1858667576 learned_addr learned my addr 172.21.15.22:0/1858667576 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:16.070 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.544+0000 7f6976ffd700 1 -- 172.21.15.22:0/1858667576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6968009550 con 0x7f697807be60 2023-09-20T21:39:16.070 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.544+0000 7f6976ffd700 1 --2- 172.21.15.22:0/1858667576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f6978109940 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6968006af0 tx=0x7f6968011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:16.071 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.544+0000 7f695ffff700 1 -- 172.21.15.22:0/1858667576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6968003a40 con 0x7f697807be60 2023-09-20T21:39:16.071 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.544+0000 7f695ffff700 1 -- 172.21.15.22:0/1858667576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6968003ba0 con 0x7f697807be60 2023-09-20T21:39:16.071 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.544+0000 7f695ffff700 1 -- 172.21.15.22:0/1858667576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f696801a6e0 con 0x7f697807be60 2023-09-20T21:39:16.071 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.544+0000 7f697d542700 1 -- 172.21.15.22:0/1858667576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697810a080 con 0x7f697807be60 2023-09-20T21:39:16.071 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.544+0000 7f697d542700 1 -- 172.21.15.22:0/1858667576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6978106570 con 0x7f697807be60 2023-09-20T21:39:16.072 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.545+0000 7f695ffff700 1 -- 172.21.15.22:0/1858667576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f696801ab60 con 0x7f697807be60 2023-09-20T21:39:16.072 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.545+0000 7f695ffff700 1 -- 172.21.15.22:0/1858667576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6968019700 con 0x7f697807be60 2023-09-20T21:39:16.072 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.545+0000 7f697d542700 1 -- 172.21.15.22:0/1858667576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697810a210 con 0x7f697807be60 2023-09-20T21:39:16.072 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.548+0000 7f695ffff700 1 -- 172.21.15.22:0/1858667576 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f696802dde0 con 0x7f697807be60 2023-09-20T21:39:16.072 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.626+0000 7f697d542700 1 -- 172.21.15.22:0/1858667576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6978062410 con 0x7f697807be60 2023-09-20T21:39:16.072 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.627+0000 7f695ffff700 1 -- 172.21.15.22:0/1858667576 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f6968024a60 con 0x7f697807be60 2023-09-20T21:39:16.073 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.628+0000 7f697d542700 1 -- 172.21.15.22:0/1858667576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 msgr2=0x7f6978109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:16.073 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.628+0000 7f697d542700 1 --2- 172.21.15.22:0/1858667576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f6978109940 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6968006af0 tx=0x7f6968011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:16.073 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.628+0000 7f697d542700 1 -- 172.21.15.22:0/1858667576 shutdown_connections 2023-09-20T21:39:16.073 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.628+0000 7f697d542700 1 --2- 172.21.15.22:0/1858667576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f697807be60 0x7f6978109940 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:16.073 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.628+0000 7f697d542700 1 -- 172.21.15.22:0/1858667576 >> 172.21.15.22:0/1858667576 conn(0x7f6978101b70 msgr2=0x7f69781032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:16.073 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.628+0000 7f697d542700 1 -- 172.21.15.22:0/1858667576 shutdown_connections 2023-09-20T21:39:16.074 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:15.628+0000 7f697d542700 1 -- 172.21.15.22:0/1858667576 wait complete. 2023-09-20T21:39:16.074 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (1/15)... 2023-09-20T21:39:16.316 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:16 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:16.148+0000 7fb1191c9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-20T21:39:16.566 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:16 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:16.315+0000 7fb1191c9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-20T21:39:16.567 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:16 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:16.435+0000 7fb1191c9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-20T21:39:16.567 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:16 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:16.565+0000 7fb1191c9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-20T21:39:17.497 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:17 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:17.126+0000 7fb1191c9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-20T21:39:18.335 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:18 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:18.055+0000 7fb1191c9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-20T21:39:18.335 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:18 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:18.171+0000 7fb1191c9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-20T21:39:18.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:18 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/238685709' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-20T21:39:19.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:19.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2023-09-20T21:39:19.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:39:19.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2023-09-20T21:39:19.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-20T21:39:19.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-20T21:39:19.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-20T21:39:19.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:19.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-20T21:39:19.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-20T21:39:19.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2023-09-20T21:39:19.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:19.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-20T21:39:19.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2023-09-20T21:39:19.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:19.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-09-20T21:39:19.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-20T21:39:19.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:19.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-20T21:39:19.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-20T21:39:19.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:19.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-20T21:39:19.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:19.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-20T21:39:19.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-20T21:39:19.147 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-20T21:39:19.147 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-20T21:39:19.147 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-20T21:39:19.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-20T21:39:19.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:19.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-20T21:39:19.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-20T21:39:19.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-20T21:39:19.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-20T21:39:19.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-20T21:39:19.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-20T21:39:19.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-20T21:39:19.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-20T21:39:19.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-20T21:39:19.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:19.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-20T21:39:19.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:19.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-20T21:39:19.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-20T21:39:19.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:19.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-20T21:39:19.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-20T21:39:19.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-20T21:39:19.151 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-20T21:39:19.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-20T21:39:19.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-20T21:39:19.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2023-09-20T21:39:19.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:19.152 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-20T21:39:19.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:19.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-20T21:39:19.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:19.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2023-09-20T21:39:07.129574+0000", 2023-09-20T21:39:19.153 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-20T21:39:19.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:19.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-20T21:39:19.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2023-09-20T21:39:19.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.612+0000 7fc4a2194700 1 Processor -- start 2023-09-20T21:39:19.154 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.612+0000 7fc4a2194700 1 -- start start 2023-09-20T21:39:19.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.612+0000 7fc4a2194700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c1069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:19.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.612+0000 7fc4a2194700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c106fa0 con 0x7fc49c1065b0 2023-09-20T21:39:19.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.613+0000 7fc49b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c1069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:19.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.613+0000 7fc49b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c1069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59204/0 (socket says 172.21.15.22:59204) 2023-09-20T21:39:19.155 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.613+0000 7fc49b7fe700 1 -- 172.21.15.22:0/54843089 learned_addr learned my addr 172.21.15.22:0/54843089 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:19.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.613+0000 7fc49b7fe700 1 -- 172.21.15.22:0/54843089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c1077b0 con 0x7fc49c1065b0 2023-09-20T21:39:19.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.613+0000 7fc49b7fe700 1 --2- 172.21.15.22:0/54843089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c1069d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0098a0 tx=0x7fc48c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e726f3993fed7477 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:19.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.614+0000 7fc49a7fc700 1 -- 172.21.15.22:0/54843089 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc48c004160 con 0x7fc49c1065b0 2023-09-20T21:39:19.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.614+0000 7fc49a7fc700 1 -- 172.21.15.22:0/54843089 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc48c0042c0 con 0x7fc49c1065b0 2023-09-20T21:39:19.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.614+0000 7fc49a7fc700 1 -- 172.21.15.22:0/54843089 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc48c011a00 con 0x7fc49c1065b0 2023-09-20T21:39:19.156 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.614+0000 7fc4a2194700 1 -- 172.21.15.22:0/54843089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 msgr2=0x7fc49c1069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:19.157 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.614+0000 7fc4a2194700 1 --2- 172.21.15.22:0/54843089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c1069d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0098a0 tx=0x7fc48c009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:19.157 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.615+0000 7fc4a2194700 1 -- 172.21.15.22:0/54843089 shutdown_connections 2023-09-20T21:39:19.157 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.615+0000 7fc4a2194700 1 --2- 172.21.15.22:0/54843089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c1069d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:19.157 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.615+0000 7fc4a2194700 1 -- 172.21.15.22:0/54843089 >> 172.21.15.22:0/54843089 conn(0x7fc49c101b30 msgr2=0x7fc49c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:19.157 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.615+0000 7fc4a2194700 1 -- 172.21.15.22:0/54843089 shutdown_connections 2023-09-20T21:39:19.158 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.615+0000 7fc4a2194700 1 -- 172.21.15.22:0/54843089 wait complete. 2023-09-20T21:39:19.158 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.616+0000 7fc4a2194700 1 Processor -- start 2023-09-20T21:39:19.158 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.616+0000 7fc4a2194700 1 -- start start 2023-09-20T21:39:19.158 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.616+0000 7fc4a2194700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c07bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:19.158 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.616+0000 7fc4a2194700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c07c170 con 0x7fc49c1065b0 2023-09-20T21:39:19.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.616+0000 7fc49b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c07bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:19.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.616+0000 7fc49b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c07bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59220/0 (socket says 172.21.15.22:59220) 2023-09-20T21:39:19.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.616+0000 7fc49b7fe700 1 -- 172.21.15.22:0/238685709 learned_addr learned my addr 172.21.15.22:0/238685709 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:19.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.617+0000 7fc49b7fe700 1 -- 172.21.15.22:0/238685709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c009550 con 0x7fc49c1065b0 2023-09-20T21:39:19.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.617+0000 7fc49b7fe700 1 --2- 172.21.15.22:0/238685709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c07bc30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009fb0 tx=0x7fc48c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:19.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.617+0000 7fc498ff9700 1 -- 172.21.15.22:0/238685709 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc48c003a40 con 0x7fc49c1065b0 2023-09-20T21:39:19.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.617+0000 7fc498ff9700 1 -- 172.21.15.22:0/238685709 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc48c003ba0 con 0x7fc49c1065b0 2023-09-20T21:39:19.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.617+0000 7fc4a2194700 1 -- 172.21.15.22:0/238685709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c07a280 con 0x7fc49c1065b0 2023-09-20T21:39:19.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.617+0000 7fc498ff9700 1 -- 172.21.15.22:0/238685709 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc48c01a750 con 0x7fc49c1065b0 2023-09-20T21:39:19.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.618+0000 7fc4a2194700 1 -- 172.21.15.22:0/238685709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c07a6a0 con 0x7fc49c1065b0 2023-09-20T21:39:19.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.618+0000 7fc498ff9700 1 -- 172.21.15.22:0/238685709 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc48c01acd0 con 0x7fc49c1065b0 2023-09-20T21:39:19.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.618+0000 7fc498ff9700 1 -- 172.21.15.22:0/238685709 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc48c0193c0 con 0x7fc49c1065b0 2023-09-20T21:39:19.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.619+0000 7fc4a2194700 1 -- 172.21.15.22:0/238685709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c062410 con 0x7fc49c1065b0 2023-09-20T21:39:19.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.621+0000 7fc498ff9700 1 -- 172.21.15.22:0/238685709 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fc48c024b60 con 0x7fc49c1065b0 2023-09-20T21:39:19.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.700+0000 7fc4a2194700 1 -- 172.21.15.22:0/238685709 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc49c10a9b0 con 0x7fc49c1065b0 2023-09-20T21:39:19.161 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.700+0000 7fc498ff9700 1 -- 172.21.15.22:0/238685709 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc48c024d00 con 0x7fc49c1065b0 2023-09-20T21:39:19.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.701+0000 7fc4a2194700 1 -- 172.21.15.22:0/238685709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 msgr2=0x7fc49c07bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:19.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.701+0000 7fc4a2194700 1 --2- 172.21.15.22:0/238685709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c07bc30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009fb0 tx=0x7fc48c011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:19.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.701+0000 7fc4a2194700 1 -- 172.21.15.22:0/238685709 shutdown_connections 2023-09-20T21:39:19.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.701+0000 7fc4a2194700 1 --2- 172.21.15.22:0/238685709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc49c1065b0 0x7fc49c07bc30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:19.162 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.701+0000 7fc4a2194700 1 -- 172.21.15.22:0/238685709 >> 172.21.15.22:0/238685709 conn(0x7fc49c101b30 msgr2=0x7fc49c103300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:19.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.701+0000 7fc4a2194700 1 -- 172.21.15.22:0/238685709 shutdown_connections 2023-09-20T21:39:19.163 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:18.701+0000 7fc4a2194700 1 -- 172.21.15.22:0/238685709 wait complete. 2023-09-20T21:39:19.163 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (2/15)... 2023-09-20T21:39:19.343 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:19 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:19.220+0000 7fb1191c9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-20T21:39:19.747 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:19 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:19.343+0000 7fb1191c9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-20T21:39:19.748 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:19 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:19.574+0000 7fb1191c9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-20T21:39:20.235 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:19 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:19.952+0000 7fb1191c9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-20T21:39:20.497 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:20 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:20.234+0000 7fb1191c9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-20T21:39:20.498 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:20 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:20.436+0000 7fb1191c9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-20T21:39:21.248 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:20 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:20.791+0000 7fb1191c9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-20T21:39:22.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:22.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2023-09-20T21:39:22.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:39:22.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2023-09-20T21:39:22.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-20T21:39:22.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-20T21:39:22.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-20T21:39:22.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:22.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-20T21:39:22.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-20T21:39:22.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2023-09-20T21:39:22.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:22.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-20T21:39:22.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2023-09-20T21:39:22.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:22.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-09-20T21:39:22.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-20T21:39:22.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:22.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-20T21:39:22.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-20T21:39:22.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:22.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-20T21:39:22.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:22.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-20T21:39:22.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-20T21:39:22.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-20T21:39:22.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-20T21:39:22.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-20T21:39:22.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-20T21:39:22.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:22.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-20T21:39:22.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-20T21:39:22.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-20T21:39:22.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-20T21:39:22.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-20T21:39:22.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-20T21:39:22.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-20T21:39:22.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-20T21:39:22.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-20T21:39:22.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:22.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-20T21:39:22.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:22.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-20T21:39:22.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-20T21:39:22.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:22.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-20T21:39:22.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-20T21:39:22.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-20T21:39:22.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-20T21:39:22.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-20T21:39:22.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-20T21:39:22.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2023-09-20T21:39:22.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:22.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-20T21:39:22.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:22.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-20T21:39:22.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:22.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2023-09-20T21:39:07.129574+0000", 2023-09-20T21:39:22.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-20T21:39:22.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:22.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-20T21:39:22.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2023-09-20T21:39:22.141 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.667+0000 7f1312b05700 1 Processor -- start 2023-09-20T21:39:22.142 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.667+0000 7f1312b05700 1 -- start start 2023-09-20T21:39:22.142 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.668+0000 7f1312b05700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:22.142 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.668+0000 7f1312b05700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c07a890 con 0x7f130c07be60 2023-09-20T21:39:22.142 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.668+0000 7f13108a1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:22.142 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.668+0000 7f13108a1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59234/0 (socket says 172.21.15.22:59234) 2023-09-20T21:39:22.143 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.668+0000 7f13108a1700 1 -- 172.21.15.22:0/1738261792 learned_addr learned my addr 172.21.15.22:0/1738261792 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:22.143 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.669+0000 7f13108a1700 1 -- 172.21.15.22:0/1738261792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c07a9d0 con 0x7f130c07be60 2023-09-20T21:39:22.143 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.669+0000 7f13108a1700 1 --2- 172.21.15.22:0/1738261792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c07a2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f13000098a0 tx=0x7f1300009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e862fc35d3b2990 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:22.143 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.669+0000 7f130b7fe700 1 -- 172.21.15.22:0/1738261792 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1300004160 con 0x7f130c07be60 2023-09-20T21:39:22.143 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.669+0000 7f130b7fe700 1 -- 172.21.15.22:0/1738261792 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13000042c0 con 0x7f130c07be60 2023-09-20T21:39:22.144 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.669+0000 7f130b7fe700 1 -- 172.21.15.22:0/1738261792 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1300011a00 con 0x7f130c07be60 2023-09-20T21:39:22.144 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.670+0000 7f1312b05700 1 -- 172.21.15.22:0/1738261792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 msgr2=0x7f130c07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:22.144 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.670+0000 7f1312b05700 1 --2- 172.21.15.22:0/1738261792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c07a2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f13000098a0 tx=0x7f1300009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:22.144 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.670+0000 7f1312b05700 1 -- 172.21.15.22:0/1738261792 shutdown_connections 2023-09-20T21:39:22.144 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.670+0000 7f1312b05700 1 --2- 172.21.15.22:0/1738261792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c07a2c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:22.145 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.670+0000 7f1312b05700 1 -- 172.21.15.22:0/1738261792 >> 172.21.15.22:0/1738261792 conn(0x7f130c101b30 msgr2=0x7f130c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:22.145 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.670+0000 7f1312b05700 1 -- 172.21.15.22:0/1738261792 shutdown_connections 2023-09-20T21:39:22.145 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.670+0000 7f1312b05700 1 -- 172.21.15.22:0/1738261792 wait complete. 2023-09-20T21:39:22.145 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.671+0000 7f1312b05700 1 Processor -- start 2023-09-20T21:39:22.145 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.671+0000 7f1312b05700 1 -- start start 2023-09-20T21:39:22.145 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.672+0000 7f1312b05700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:22.146 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.672+0000 7f1312b05700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c19ca30 con 0x7f130c07be60 2023-09-20T21:39:22.146 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.672+0000 7f13108a1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:22.146 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.672+0000 7f13108a1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59248/0 (socket says 172.21.15.22:59248) 2023-09-20T21:39:22.146 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.672+0000 7f13108a1700 1 -- 172.21.15.22:0/2344659780 learned_addr learned my addr 172.21.15.22:0/2344659780 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:22.146 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.672+0000 7f13108a1700 1 -- 172.21.15.22:0/2344659780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1300009550 con 0x7f130c07be60 2023-09-20T21:39:22.147 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.672+0000 7f13108a1700 1 --2- 172.21.15.22:0/2344659780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c19c4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1300009130 tx=0x7f1300011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:22.147 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.673+0000 7f1309ffb700 1 -- 172.21.15.22:0/2344659780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1300003b70 con 0x7f130c07be60 2023-09-20T21:39:22.147 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.673+0000 7f1312b05700 1 -- 172.21.15.22:0/2344659780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c19cc30 con 0x7f130c07be60 2023-09-20T21:39:22.147 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.673+0000 7f1309ffb700 1 -- 172.21.15.22:0/2344659780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1300003cd0 con 0x7f130c07be60 2023-09-20T21:39:22.147 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.673+0000 7f1309ffb700 1 -- 172.21.15.22:0/2344659780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13000197d0 con 0x7f130c07be60 2023-09-20T21:39:22.147 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.673+0000 7f1312b05700 1 -- 172.21.15.22:0/2344659780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c19d050 con 0x7f130c07be60 2023-09-20T21:39:22.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.674+0000 7f1309ffb700 1 -- 172.21.15.22:0/2344659780 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f130001a430 con 0x7f130c07be60 2023-09-20T21:39:22.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.674+0000 7f1309ffb700 1 -- 172.21.15.22:0/2344659780 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1300025bf0 con 0x7f130c07be60 2023-09-20T21:39:22.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.674+0000 7f1312b05700 1 -- 172.21.15.22:0/2344659780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c196580 con 0x7f130c07be60 2023-09-20T21:39:22.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.677+0000 7f1309ffb700 1 -- 172.21.15.22:0/2344659780 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f130002a060 con 0x7f130c07be60 2023-09-20T21:39:22.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.755+0000 7f1312b05700 1 -- 172.21.15.22:0/2344659780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f130c10c9a0 con 0x7f130c07be60 2023-09-20T21:39:22.148 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.755+0000 7f1309ffb700 1 -- 172.21.15.22:0/2344659780 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f1300025db0 con 0x7f130c07be60 2023-09-20T21:39:22.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.756+0000 7f1312b05700 1 -- 172.21.15.22:0/2344659780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 msgr2=0x7f130c19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:22.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.756+0000 7f1312b05700 1 --2- 172.21.15.22:0/2344659780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c19c4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1300009130 tx=0x7f1300011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:22.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.756+0000 7f1312b05700 1 -- 172.21.15.22:0/2344659780 shutdown_connections 2023-09-20T21:39:22.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.756+0000 7f1312b05700 1 --2- 172.21.15.22:0/2344659780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f130c07be60 0x7f130c19c4f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:22.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.756+0000 7f1312b05700 1 -- 172.21.15.22:0/2344659780 >> 172.21.15.22:0/2344659780 conn(0x7f130c101b30 msgr2=0x7f130c103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:22.149 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.756+0000 7f1312b05700 1 -- 172.21.15.22:0/2344659780 shutdown_connections 2023-09-20T21:39:22.150 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:21.756+0000 7f1312b05700 1 -- 172.21.15.22:0/2344659780 wait complete. 2023-09-20T21:39:22.150 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (3/15)... 2023-09-20T21:39:22.173 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:21 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2344659780' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-20T21:39:22.173 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:21 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:21.797+0000 7fb1191c9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-20T21:39:22.173 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:21 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:21.988+0000 7fb1191c9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-20T21:39:22.497 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:22 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:22.245+0000 7fb1191c9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-20T21:39:22.498 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:22 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:22.397+0000 7fb1191c9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-20T21:39:22.998 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:22 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:22.684+0000 7fb1191c9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-20T21:39:24.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: Activating manager daemon x 2023-09-20T21:39:24.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: mgrmap e2: x(active, starting, since 0.00435385s) 2023-09-20T21:39:24.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: from='mgr.14100 172.21.15.22:0/3738102541' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-20T21:39:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: from='mgr.14100 172.21.15.22:0/3738102541' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-20T21:39:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: from='mgr.14100 172.21.15.22:0/3738102541' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-20T21:39:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: from='mgr.14100 172.21.15.22:0/3738102541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:39:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: from='mgr.14100 172.21.15.22:0/3738102541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-20T21:39:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: Manager daemon x is now available 2023-09-20T21:39:24.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: from='mgr.14100 172.21.15.22:0/3738102541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-20T21:39:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: from='mgr.14100 172.21.15.22:0/3738102541' entity='mgr.x' 2023-09-20T21:39:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: from='mgr.14100 172.21.15.22:0/3738102541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-20T21:39:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: from='mgr.14100 172.21.15.22:0/3738102541' entity='mgr.x' 2023-09-20T21:39:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-mon[97651]: from='mgr.14100 172.21.15.22:0/3738102541' entity='mgr.x' 2023-09-20T21:39:24.251 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:23 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:23.809+0000 7fb1191c9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-20T21:39:25.116 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:25.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2023-09-20T21:39:25.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:39:25.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2023-09-20T21:39:25.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-20T21:39:25.117 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-20T21:39:25.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-20T21:39:25.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:25.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-20T21:39:25.118 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-20T21:39:25.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2023-09-20T21:39:25.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:25.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-20T21:39:25.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2023-09-20T21:39:25.119 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:25.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-09-20T21:39:25.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-20T21:39:25.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:25.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-20T21:39:25.120 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-20T21:39:25.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:25.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-20T21:39:25.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:25.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-20T21:39:25.121 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-20T21:39:25.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-20T21:39:25.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-20T21:39:25.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-20T21:39:25.122 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-20T21:39:25.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:25.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-20T21:39:25.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-20T21:39:25.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-20T21:39:25.123 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-20T21:39:25.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-20T21:39:25.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-20T21:39:25.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-20T21:39:25.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-20T21:39:25.124 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-20T21:39:25.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:25.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-20T21:39:25.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:25.125 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-20T21:39:25.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-20T21:39:25.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:25.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-20T21:39:25.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2023-09-20T21:39:25.126 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-20T21:39:25.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-20T21:39:25.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-20T21:39:25.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-20T21:39:25.127 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2023-09-20T21:39:25.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:25.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-20T21:39:25.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:25.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-20T21:39:25.128 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:25.129 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2023-09-20T21:39:07.129574+0000", 2023-09-20T21:39:25.129 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-20T21:39:25.129 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:25.129 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-20T21:39:25.129 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2023-09-20T21:39:25.130 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.649+0000 7feff2d32700 1 Processor -- start 2023-09-20T21:39:25.130 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.649+0000 7feff2d32700 1 -- start start 2023-09-20T21:39:25.130 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.650+0000 7feff2d32700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec07a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:25.130 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.650+0000 7feff2d32700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec07a870 con 0x7fefec07be40 2023-09-20T21:39:25.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.650+0000 7feff0ace700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec07a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:25.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.650+0000 7feff0ace700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec07a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59306/0 (socket says 172.21.15.22:59306) 2023-09-20T21:39:25.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.650+0000 7feff0ace700 1 -- 172.21.15.22:0/1879513558 learned_addr learned my addr 172.21.15.22:0/1879513558 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:25.131 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.650+0000 7feff0ace700 1 -- 172.21.15.22:0/1879513558 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec07a9b0 con 0x7fefec07be40 2023-09-20T21:39:25.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.651+0000 7feff0ace700 1 --2- 172.21.15.22:0/1879513558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec07a2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fefe00098a0 tx=0x7fefe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bc2efc8c74842c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:25.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.651+0000 7fefeb7fe700 1 -- 172.21.15.22:0/1879513558 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fefe0004160 con 0x7fefec07be40 2023-09-20T21:39:25.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.651+0000 7fefeb7fe700 1 -- 172.21.15.22:0/1879513558 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fefe00042c0 con 0x7fefec07be40 2023-09-20T21:39:25.132 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.651+0000 7fefeb7fe700 1 -- 172.21.15.22:0/1879513558 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fefe0011a00 con 0x7fefec07be40 2023-09-20T21:39:25.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.651+0000 7feff2d32700 1 -- 172.21.15.22:0/1879513558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 msgr2=0x7fefec07a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:25.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.651+0000 7feff2d32700 1 --2- 172.21.15.22:0/1879513558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec07a2a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fefe00098a0 tx=0x7fefe0009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:25.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.652+0000 7feff2d32700 1 -- 172.21.15.22:0/1879513558 shutdown_connections 2023-09-20T21:39:25.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.652+0000 7feff2d32700 1 --2- 172.21.15.22:0/1879513558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec07a2a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:25.133 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.652+0000 7feff2d32700 1 -- 172.21.15.22:0/1879513558 >> 172.21.15.22:0/1879513558 conn(0x7fefec101b30 msgr2=0x7fefec103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:25.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.652+0000 7feff2d32700 1 -- 172.21.15.22:0/1879513558 shutdown_connections 2023-09-20T21:39:25.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.652+0000 7feff2d32700 1 -- 172.21.15.22:0/1879513558 wait complete. 2023-09-20T21:39:25.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.653+0000 7feff2d32700 1 Processor -- start 2023-09-20T21:39:25.134 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.653+0000 7feff2d32700 1 -- start start 2023-09-20T21:39:25.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.653+0000 7feff2d32700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:25.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.653+0000 7feff2d32700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec19ca40 con 0x7fefec07be40 2023-09-20T21:39:25.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.653+0000 7feff0ace700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:25.135 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.653+0000 7feff0ace700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59316/0 (socket says 172.21.15.22:59316) 2023-09-20T21:39:25.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.653+0000 7feff0ace700 1 -- 172.21.15.22:0/110193032 learned_addr learned my addr 172.21.15.22:0/110193032 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:25.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.654+0000 7feff0ace700 1 -- 172.21.15.22:0/110193032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0009550 con 0x7fefec07be40 2023-09-20T21:39:25.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.654+0000 7feff0ace700 1 --2- 172.21.15.22:0/110193032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec19c500 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009130 tx=0x7fefe0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:25.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.654+0000 7fefe9ffb700 1 -- 172.21.15.22:0/110193032 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fefe0003b70 con 0x7fefec07be40 2023-09-20T21:39:25.136 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.654+0000 7fefe9ffb700 1 -- 172.21.15.22:0/110193032 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fefe0003cd0 con 0x7fefec07be40 2023-09-20T21:39:25.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.654+0000 7fefe9ffb700 1 -- 172.21.15.22:0/110193032 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fefe00197d0 con 0x7fefec07be40 2023-09-20T21:39:25.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.654+0000 7feff2d32700 1 -- 172.21.15.22:0/110193032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefec19cc40 con 0x7fefec07be40 2023-09-20T21:39:25.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.654+0000 7feff2d32700 1 -- 172.21.15.22:0/110193032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec19d060 con 0x7fefec07be40 2023-09-20T21:39:25.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.655+0000 7fefe9ffb700 1 -- 172.21.15.22:0/110193032 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 2) v1 ==== 45374+0+0 (secure 0 0 0) 0x7fefe001abb0 con 0x7fefec07be40 2023-09-20T21:39:25.137 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.656+0000 7feff2d32700 1 -- 172.21.15.22:0/110193032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefec062410 con 0x7fefec07be40 2023-09-20T21:39:25.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.656+0000 7fefe9ffb700 1 -- 172.21.15.22:0/110193032 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fefe004c610 con 0x7fefec07be40 2023-09-20T21:39:25.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.658+0000 7fefe9ffb700 1 -- 172.21.15.22:0/110193032 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fefe001a430 con 0x7fefec07be40 2023-09-20T21:39:25.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.733+0000 7feff2d32700 1 -- 172.21.15.22:0/110193032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fefec10c960 con 0x7fefec07be40 2023-09-20T21:39:25.138 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.734+0000 7fefe9ffb700 1 -- 172.21.15.22:0/110193032 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fefe0023370 con 0x7fefec07be40 2023-09-20T21:39:25.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.735+0000 7feff2d32700 1 -- 172.21.15.22:0/110193032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 msgr2=0x7fefec19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:25.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.735+0000 7feff2d32700 1 --2- 172.21.15.22:0/110193032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec19c500 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009130 tx=0x7fefe0011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:25.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.735+0000 7feff2d32700 1 -- 172.21.15.22:0/110193032 shutdown_connections 2023-09-20T21:39:25.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.735+0000 7feff2d32700 1 --2- 172.21.15.22:0/110193032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefec07be40 0x7fefec19c500 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:25.139 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.735+0000 7feff2d32700 1 -- 172.21.15.22:0/110193032 >> 172.21.15.22:0/110193032 conn(0x7fefec101b30 msgr2=0x7fefec1032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:25.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.735+0000 7feff2d32700 1 -- 172.21.15.22:0/110193032 shutdown_connections 2023-09-20T21:39:25.140 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:24.735+0000 7feff2d32700 1 -- 172.21.15.22:0/110193032 wait complete. 2023-09-20T21:39:25.140 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (4/15)... 2023-09-20T21:39:25.151 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:24 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/110193032' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-20T21:39:25.151 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:24 smithi022 ceph-mon[97651]: mgrmap e3: x(active, since 1.00897s) 2023-09-20T21:39:27.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:26 smithi022 ceph-mon[97651]: mgrmap e4: x(active, since 2s) 2023-09-20T21:39:28.307 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:28.308 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2023-09-20T21:39:28.308 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:39:28.308 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2023-09-20T21:39:28.308 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-09-20T21:39:28.308 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-09-20T21:39:28.309 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-09-20T21:39:28.309 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:28.309 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-09-20T21:39:28.309 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-09-20T21:39:28.309 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2023-09-20T21:39:28.310 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:28.310 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-09-20T21:39:28.310 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2023-09-20T21:39:28.310 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:28.310 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2023-09-20T21:39:28.311 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2023-09-20T21:39:28.311 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:28.311 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-09-20T21:39:28.311 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-09-20T21:39:28.312 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:28.312 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-09-20T21:39:28.312 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:28.312 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-09-20T21:39:28.312 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-09-20T21:39:28.313 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-09-20T21:39:28.313 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-09-20T21:39:28.313 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-09-20T21:39:28.313 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-09-20T21:39:28.313 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:28.313 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-09-20T21:39:28.314 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-09-20T21:39:28.314 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-09-20T21:39:28.314 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-09-20T21:39:28.314 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-09-20T21:39:28.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-09-20T21:39:28.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-09-20T21:39:28.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-09-20T21:39:28.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-09-20T21:39:28.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:28.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-09-20T21:39:28.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:28.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-09-20T21:39:28.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-09-20T21:39:28.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:28.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-09-20T21:39:28.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-20T21:39:28.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-09-20T21:39:28.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2023-09-20T21:39:28.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2023-09-20T21:39:28.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2023-09-20T21:39:28.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2023-09-20T21:39:28.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2023-09-20T21:39:28.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-20T21:39:28.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:28.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-09-20T21:39:28.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-09-20T21:39:28.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2023-09-20T21:39:07.129574+0000", 2023-09-20T21:39:28.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2023-09-20T21:39:28.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2023-09-20T21:39:28.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-09-20T21:39:28.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2023-09-20T21:39:28.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.632+0000 7fa4a450b700 1 Processor -- start 2023-09-20T21:39:28.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.633+0000 7fa4a450b700 1 -- start start 2023-09-20T21:39:28.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.633+0000 7fa4a450b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:28.321 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.633+0000 7fa4a450b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c07a890 con 0x7fa49c07be60 2023-09-20T21:39:28.321 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.633+0000 7fa4a22a7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:28.321 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.633+0000 7fa4a22a7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60622/0 (socket says 172.21.15.22:60622) 2023-09-20T21:39:28.321 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.633+0000 7fa4a22a7700 1 -- 172.21.15.22:0/3660090095 learned_addr learned my addr 172.21.15.22:0/3660090095 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:28.321 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.634+0000 7fa4a22a7700 1 -- 172.21.15.22:0/3660090095 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c07a9d0 con 0x7fa49c07be60 2023-09-20T21:39:28.322 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.634+0000 7fa4a22a7700 1 --2- 172.21.15.22:0/3660090095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c07a2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4980098a0 tx=0x7fa498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a337b435af542c93 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:28.322 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.634+0000 7fa4a12a5700 1 -- 172.21.15.22:0/3660090095 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa498004160 con 0x7fa49c07be60 2023-09-20T21:39:28.322 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.634+0000 7fa4a12a5700 1 -- 172.21.15.22:0/3660090095 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa4980042c0 con 0x7fa49c07be60 2023-09-20T21:39:28.322 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.634+0000 7fa4a12a5700 1 -- 172.21.15.22:0/3660090095 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa498011a00 con 0x7fa49c07be60 2023-09-20T21:39:28.322 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.635+0000 7fa4a450b700 1 -- 172.21.15.22:0/3660090095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 msgr2=0x7fa49c07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:28.323 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.635+0000 7fa4a450b700 1 --2- 172.21.15.22:0/3660090095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c07a2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4980098a0 tx=0x7fa498009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:28.323 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.635+0000 7fa4a450b700 1 -- 172.21.15.22:0/3660090095 shutdown_connections 2023-09-20T21:39:28.323 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.635+0000 7fa4a450b700 1 --2- 172.21.15.22:0/3660090095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c07a2c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:28.323 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.635+0000 7fa4a450b700 1 -- 172.21.15.22:0/3660090095 >> 172.21.15.22:0/3660090095 conn(0x7fa49c101b30 msgr2=0x7fa49c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:28.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.635+0000 7fa4a450b700 1 -- 172.21.15.22:0/3660090095 shutdown_connections 2023-09-20T21:39:28.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.635+0000 7fa4a450b700 1 -- 172.21.15.22:0/3660090095 wait complete. 2023-09-20T21:39:28.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.636+0000 7fa4a450b700 1 Processor -- start 2023-09-20T21:39:28.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.636+0000 7fa4a450b700 1 -- start start 2023-09-20T21:39:28.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.636+0000 7fa4a450b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:28.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.636+0000 7fa4a450b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c19ca30 con 0x7fa49c07be60 2023-09-20T21:39:28.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.637+0000 7fa4a22a7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:28.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.637+0000 7fa4a22a7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60638/0 (socket says 172.21.15.22:60638) 2023-09-20T21:39:28.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.637+0000 7fa4a22a7700 1 -- 172.21.15.22:0/2541903885 learned_addr learned my addr 172.21.15.22:0/2541903885 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:28.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.637+0000 7fa4a22a7700 1 -- 172.21.15.22:0/2541903885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa498009550 con 0x7fa49c07be60 2023-09-20T21:39:28.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.637+0000 7fa4a22a7700 1 --2- 172.21.15.22:0/2541903885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c19c4f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa498009130 tx=0x7fa498011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:28.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.637+0000 7fa4937fe700 1 -- 172.21.15.22:0/2541903885 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa498003b70 con 0x7fa49c07be60 2023-09-20T21:39:28.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.637+0000 7fa4937fe700 1 -- 172.21.15.22:0/2541903885 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa498003cd0 con 0x7fa49c07be60 2023-09-20T21:39:28.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.637+0000 7fa4937fe700 1 -- 172.21.15.22:0/2541903885 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4980197d0 con 0x7fa49c07be60 2023-09-20T21:39:28.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.637+0000 7fa4a450b700 1 -- 172.21.15.22:0/2541903885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c19cc30 con 0x7fa49c07be60 2023-09-20T21:39:28.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.638+0000 7fa4a450b700 1 -- 172.21.15.22:0/2541903885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c19d050 con 0x7fa49c07be60 2023-09-20T21:39:28.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.639+0000 7fa4a450b700 1 -- 172.21.15.22:0/2541903885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c196580 con 0x7fa49c07be60 2023-09-20T21:39:28.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.639+0000 7fa4937fe700 1 -- 172.21.15.22:0/2541903885 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 45607+0+0 (secure 0 0 0) 0x7fa49801abb0 con 0x7fa49c07be60 2023-09-20T21:39:28.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.640+0000 7fa4937fe700 1 --2- 172.21.15.22:0/2541903885 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fa4880387a0 0x7fa48803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:28.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.640+0000 7fa4937fe700 1 -- 172.21.15.22:0/2541903885 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa49804cf20 con 0x7fa49c07be60 2023-09-20T21:39:28.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.640+0000 7fa4a1aa6700 1 --2- 172.21.15.22:0/2541903885 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fa4880387a0 0x7fa48803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:28.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.640+0000 7fa4a1aa6700 1 --2- 172.21.15.22:0/2541903885 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fa4880387a0 0x7fa48803ac60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa48c006fd0 tx=0x7fa48c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:28.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.645+0000 7fa4937fe700 1 -- 172.21.15.22:0/2541903885 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa498023370 con 0x7fa49c07be60 2023-09-20T21:39:28.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.940+0000 7fa4a450b700 1 -- 172.21.15.22:0/2541903885 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa49c10ca20 con 0x7fa49c07be60 2023-09-20T21:39:28.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.941+0000 7fa4937fe700 1 -- 172.21.15.22:0/2541903885 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fa49802a030 con 0x7fa49c07be60 2023-09-20T21:39:28.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.943+0000 7fa4a450b700 1 -- 172.21.15.22:0/2541903885 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fa4880387a0 msgr2=0x7fa48803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:28.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.943+0000 7fa4a450b700 1 --2- 172.21.15.22:0/2541903885 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fa4880387a0 0x7fa48803ac60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa48c006fd0 tx=0x7fa48c006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:28.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.943+0000 7fa4a450b700 1 -- 172.21.15.22:0/2541903885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 msgr2=0x7fa49c19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:28.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.943+0000 7fa4a450b700 1 --2- 172.21.15.22:0/2541903885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c19c4f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa498009130 tx=0x7fa498011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:28.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.943+0000 7fa4a450b700 1 -- 172.21.15.22:0/2541903885 shutdown_connections 2023-09-20T21:39:28.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.943+0000 7fa4a450b700 1 --2- 172.21.15.22:0/2541903885 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fa4880387a0 0x7fa48803ac60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:28.330 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.943+0000 7fa4a450b700 1 --2- 172.21.15.22:0/2541903885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa49c07be60 0x7fa49c19c4f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:28.330 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.943+0000 7fa4a450b700 1 -- 172.21.15.22:0/2541903885 >> 172.21.15.22:0/2541903885 conn(0x7fa49c101b30 msgr2=0x7fa49c103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:28.330 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.943+0000 7fa4a450b700 1 -- 172.21.15.22:0/2541903885 shutdown_connections 2023-09-20T21:39:28.330 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:27.943+0000 7fa4a450b700 1 -- 172.21.15.22:0/2541903885 wait complete. 2023-09-20T21:39:28.330 INFO:teuthology.orchestra.run.smithi022.stdout:mgr is available 2023-09-20T21:39:28.635 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:28 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2541903885' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-09-20T21:39:29.490 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:29.490 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2023-09-20T21:39:29.491 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:39:29.491 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-09-20T21:39:29.491 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-09-20T21:39:29.491 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-09-20T21:39:29.492 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-09-20T21:39:29.492 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:29.492 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2023-09-20T21:39:29.492 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-09-20T21:39:29.492 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2023-09-20T21:39:29.493 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2023-09-20T21:39:29.493 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-09-20T21:39:29.493 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-09-20T21:39:29.493 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-09-20T21:39:29.493 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-09-20T21:39:29.494 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-09-20T21:39:29.494 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.833+0000 7f2a45577700 1 Processor -- start 2023-09-20T21:39:29.494 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.833+0000 7f2a45577700 1 -- start start 2023-09-20T21:39:29.494 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.834+0000 7f2a45577700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:29.494 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.834+0000 7f2a45577700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4007a890 con 0x7f2a4007be60 2023-09-20T21:39:29.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.834+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:29.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.834+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60640/0 (socket says 172.21.15.22:60640) 2023-09-20T21:39:29.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.834+0000 7f2a3effd700 1 -- 172.21.15.22:0/4043604495 learned_addr learned my addr 172.21.15.22:0/4043604495 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:29.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.835+0000 7f2a3effd700 1 -- 172.21.15.22:0/4043604495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4007a9d0 con 0x7f2a4007be60 2023-09-20T21:39:29.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.835+0000 7f2a3effd700 1 --2- 172.21.15.22:0/4043604495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4007a2c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a300098a0 tx=0x7f2a30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cf2c2a3d7de05a server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:29.495 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.835+0000 7f2a3dffb700 1 -- 172.21.15.22:0/4043604495 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a30004160 con 0x7f2a4007be60 2023-09-20T21:39:29.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.835+0000 7f2a3dffb700 1 -- 172.21.15.22:0/4043604495 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a300042c0 con 0x7f2a4007be60 2023-09-20T21:39:29.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.835+0000 7f2a3dffb700 1 -- 172.21.15.22:0/4043604495 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a30011a00 con 0x7f2a4007be60 2023-09-20T21:39:29.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.835+0000 7f2a45577700 1 -- 172.21.15.22:0/4043604495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 msgr2=0x7f2a4007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:29.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.835+0000 7f2a45577700 1 --2- 172.21.15.22:0/4043604495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4007a2c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a300098a0 tx=0x7f2a30009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:29.496 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.836+0000 7f2a45577700 1 -- 172.21.15.22:0/4043604495 shutdown_connections 2023-09-20T21:39:29.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.836+0000 7f2a45577700 1 --2- 172.21.15.22:0/4043604495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4007a2c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:29.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.836+0000 7f2a45577700 1 -- 172.21.15.22:0/4043604495 >> 172.21.15.22:0/4043604495 conn(0x7f2a40101b30 msgr2=0x7f2a40103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:29.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.836+0000 7f2a45577700 1 -- 172.21.15.22:0/4043604495 shutdown_connections 2023-09-20T21:39:29.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.836+0000 7f2a45577700 1 -- 172.21.15.22:0/4043604495 wait complete. 2023-09-20T21:39:29.497 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.837+0000 7f2a45577700 1 Processor -- start 2023-09-20T21:39:29.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.837+0000 7f2a45577700 1 -- start start 2023-09-20T21:39:29.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.837+0000 7f2a45577700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:29.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.837+0000 7f2a45577700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4019ca30 con 0x7f2a4007be60 2023-09-20T21:39:29.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.837+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:29.498 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.838+0000 7f2a3effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60652/0 (socket says 172.21.15.22:60652) 2023-09-20T21:39:29.499 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.838+0000 7f2a3effd700 1 -- 172.21.15.22:0/539744968 learned_addr learned my addr 172.21.15.22:0/539744968 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:29.499 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.838+0000 7f2a3effd700 1 -- 172.21.15.22:0/539744968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a30009550 con 0x7f2a4007be60 2023-09-20T21:39:29.499 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.838+0000 7f2a3effd700 1 --2- 172.21.15.22:0/539744968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4019c4f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009fb0 tx=0x7f2a30011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:29.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.838+0000 7f2a27fff700 1 -- 172.21.15.22:0/539744968 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a30003a40 con 0x7f2a4007be60 2023-09-20T21:39:29.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.838+0000 7f2a27fff700 1 -- 172.21.15.22:0/539744968 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a30003ba0 con 0x7f2a4007be60 2023-09-20T21:39:29.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.838+0000 7f2a45577700 1 -- 172.21.15.22:0/539744968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4019cc30 con 0x7f2a4007be60 2023-09-20T21:39:29.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.838+0000 7f2a27fff700 1 -- 172.21.15.22:0/539744968 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a3001a750 con 0x7f2a4007be60 2023-09-20T21:39:29.501 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.839+0000 7f2a45577700 1 -- 172.21.15.22:0/539744968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4019d050 con 0x7f2a4007be60 2023-09-20T21:39:29.502 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.839+0000 7f2a45577700 1 -- 172.21.15.22:0/539744968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a40196580 con 0x7f2a4007be60 2023-09-20T21:39:29.502 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.840+0000 7f2a27fff700 1 -- 172.21.15.22:0/539744968 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 45607+0+0 (secure 0 0 0) 0x7f2a30024b60 con 0x7f2a4007be60 2023-09-20T21:39:29.502 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.841+0000 7f2a27fff700 1 --2- 172.21.15.22:0/539744968 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f2a280387a0 0x7f2a2803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:29.502 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.841+0000 7f2a27fff700 1 -- 172.21.15.22:0/539744968 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2a3004c420 con 0x7f2a4007be60 2023-09-20T21:39:29.502 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.841+0000 7f2a3e7fc700 1 --2- 172.21.15.22:0/539744968 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f2a280387a0 0x7f2a2803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:29.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.841+0000 7f2a3e7fc700 1 --2- 172.21.15.22:0/539744968 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f2a280387a0 0x7f2a2803ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a34006fd0 tx=0x7f2a34006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:29.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:28.845+0000 7f2a27fff700 1 -- 172.21.15.22:0/539744968 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2a30024430 con 0x7f2a4007be60 2023-09-20T21:39:29.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.054+0000 7f2a45577700 1 -- 172.21.15.22:0/539744968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2a40062410 con 0x7f2a4007be60 2023-09-20T21:39:29.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.055+0000 7f2a27fff700 1 -- 172.21.15.22:0/539744968 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f2a30018350 con 0x7f2a4007be60 2023-09-20T21:39:29.503 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.057+0000 7f2a45577700 1 -- 172.21.15.22:0/539744968 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f2a280387a0 msgr2=0x7f2a2803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:29.504 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.057+0000 7f2a45577700 1 --2- 172.21.15.22:0/539744968 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f2a280387a0 0x7f2a2803ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a34006fd0 tx=0x7f2a34006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:29.504 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.057+0000 7f2a45577700 1 -- 172.21.15.22:0/539744968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 msgr2=0x7f2a4019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:29.504 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.057+0000 7f2a45577700 1 --2- 172.21.15.22:0/539744968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4019c4f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009fb0 tx=0x7f2a30011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:29.504 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.057+0000 7f2a45577700 1 -- 172.21.15.22:0/539744968 shutdown_connections 2023-09-20T21:39:29.505 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.057+0000 7f2a45577700 1 --2- 172.21.15.22:0/539744968 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f2a280387a0 0x7f2a2803ac60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:29.505 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.057+0000 7f2a45577700 1 --2- 172.21.15.22:0/539744968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a4007be60 0x7f2a4019c4f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:29.505 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.057+0000 7f2a45577700 1 -- 172.21.15.22:0/539744968 >> 172.21.15.22:0/539744968 conn(0x7f2a40101b30 msgr2=0x7f2a40103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:29.505 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.057+0000 7f2a45577700 1 -- 172.21.15.22:0/539744968 shutdown_connections 2023-09-20T21:39:29.505 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:29.057+0000 7f2a45577700 1 -- 172.21.15.22:0/539744968 wait complete. 2023-09-20T21:39:29.506 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling cephadm module... 2023-09-20T21:39:29.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:29 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/539744968' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-09-20T21:39:30.712 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:30 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/373891549' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-09-20T21:39:30.712 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: ignoring --setuser ceph since I am not root 2023-09-20T21:39:30.712 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: ignoring --setgroup ceph since I am not root 2023-09-20T21:39:30.713 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:30.711+0000 7f1ebf702200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-20T21:39:30.976 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.019+0000 7fb678167700 1 Processor -- start 2023-09-20T21:39:30.976 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.019+0000 7fb678167700 1 -- start start 2023-09-20T21:39:30.977 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.019+0000 7fb678167700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:30.977 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.019+0000 7fb678167700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67007a8b0 con 0x7fb67007be80 2023-09-20T21:39:30.977 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.019+0000 7fb675f03700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:30.977 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.020+0000 7fb675f03700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60660/0 (socket says 172.21.15.22:60660) 2023-09-20T21:39:30.978 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.020+0000 7fb675f03700 1 -- 172.21.15.22:0/2973358552 learned_addr learned my addr 172.21.15.22:0/2973358552 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:30.978 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.020+0000 7fb675f03700 1 -- 172.21.15.22:0/2973358552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67007a9f0 con 0x7fb67007be80 2023-09-20T21:39:30.978 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.020+0000 7fb675f03700 1 --2- 172.21.15.22:0/2973358552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67007a2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0098a0 tx=0x7fb66c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea3e3e779022828 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:30.978 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.020+0000 7fb674f01700 1 -- 172.21.15.22:0/2973358552 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb66c004160 con 0x7fb67007be80 2023-09-20T21:39:30.978 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.020+0000 7fb674f01700 1 -- 172.21.15.22:0/2973358552 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb66c0042c0 con 0x7fb67007be80 2023-09-20T21:39:30.979 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.020+0000 7fb674f01700 1 -- 172.21.15.22:0/2973358552 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb66c011a00 con 0x7fb67007be80 2023-09-20T21:39:30.979 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.021+0000 7fb678167700 1 -- 172.21.15.22:0/2973358552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 msgr2=0x7fb67007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:30.979 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.021+0000 7fb678167700 1 --2- 172.21.15.22:0/2973358552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67007a2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0098a0 tx=0x7fb66c009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:30.979 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.021+0000 7fb678167700 1 -- 172.21.15.22:0/2973358552 shutdown_connections 2023-09-20T21:39:30.980 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.021+0000 7fb678167700 1 --2- 172.21.15.22:0/2973358552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67007a2e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:30.980 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.021+0000 7fb678167700 1 -- 172.21.15.22:0/2973358552 >> 172.21.15.22:0/2973358552 conn(0x7fb670101b30 msgr2=0x7fb670103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:30.980 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.021+0000 7fb678167700 1 -- 172.21.15.22:0/2973358552 shutdown_connections 2023-09-20T21:39:30.980 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.021+0000 7fb678167700 1 -- 172.21.15.22:0/2973358552 wait complete. 2023-09-20T21:39:30.980 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.022+0000 7fb678167700 1 Processor -- start 2023-09-20T21:39:30.981 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.022+0000 7fb678167700 1 -- start start 2023-09-20T21:39:30.981 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.023+0000 7fb678167700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:30.981 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.023+0000 7fb678167700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67019ca40 con 0x7fb67007be80 2023-09-20T21:39:30.981 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.023+0000 7fb675f03700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:30.981 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.023+0000 7fb675f03700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60672/0 (socket says 172.21.15.22:60672) 2023-09-20T21:39:30.982 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.023+0000 7fb675f03700 1 -- 172.21.15.22:0/373891549 learned_addr learned my addr 172.21.15.22:0/373891549 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:30.982 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.023+0000 7fb675f03700 1 -- 172.21.15.22:0/373891549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c009550 con 0x7fb67007be80 2023-09-20T21:39:30.982 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.024+0000 7fb675f03700 1 --2- 172.21.15.22:0/373891549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67019c500 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009130 tx=0x7fb66c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:30.982 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.024+0000 7fb666ffd700 1 -- 172.21.15.22:0/373891549 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb66c003ab0 con 0x7fb67007be80 2023-09-20T21:39:30.982 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.024+0000 7fb666ffd700 1 -- 172.21.15.22:0/373891549 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb66c003c10 con 0x7fb67007be80 2023-09-20T21:39:30.983 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.024+0000 7fb666ffd700 1 -- 172.21.15.22:0/373891549 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb66c0247d0 con 0x7fb67007be80 2023-09-20T21:39:30.983 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.024+0000 7fb678167700 1 -- 172.21.15.22:0/373891549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67019cc40 con 0x7fb67007be80 2023-09-20T21:39:30.983 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.024+0000 7fb678167700 1 -- 172.21.15.22:0/373891549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67019d060 con 0x7fb67007be80 2023-09-20T21:39:30.983 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.025+0000 7fb666ffd700 1 -- 172.21.15.22:0/373891549 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 45607+0+0 (secure 0 0 0) 0x7fb66c01abb0 con 0x7fb67007be80 2023-09-20T21:39:30.983 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.025+0000 7fb678167700 1 -- 172.21.15.22:0/373891549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb670196590 con 0x7fb67007be80 2023-09-20T21:39:30.984 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.025+0000 7fb666ffd700 1 --2- 172.21.15.22:0/373891549 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fb65c0387a0 0x7fb65c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:30.984 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.026+0000 7fb666ffd700 1 -- 172.21.15.22:0/373891549 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb66c04c4f0 con 0x7fb67007be80 2023-09-20T21:39:30.984 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.026+0000 7fb675702700 1 --2- 172.21.15.22:0/373891549 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fb65c0387a0 0x7fb65c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:30.984 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.026+0000 7fb675702700 1 --2- 172.21.15.22:0/373891549 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fb65c0387a0 0x7fb65c03ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb660006fd0 tx=0x7fb660006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:30.985 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.030+0000 7fb666ffd700 1 -- 172.21.15.22:0/373891549 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb66c018710 con 0x7fb67007be80 2023-09-20T21:39:30.985 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.294+0000 7fb678167700 1 -- 172.21.15.22:0/373891549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb670062410 con 0x7fb67007be80 2023-09-20T21:39:30.985 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.442+0000 7fb666ffd700 1 -- 172.21.15.22:0/373891549 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 5) v1 ==== 45618+0+0 (secure 0 0 0) 0x7fb66c02d770 con 0x7fb67007be80 2023-09-20T21:39:30.985 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.444+0000 7fb666ffd700 1 -- 172.21.15.22:0/373891549 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb66c02ad60 con 0x7fb67007be80 2023-09-20T21:39:30.985 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.446+0000 7fb678167700 1 -- 172.21.15.22:0/373891549 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fb65c0387a0 msgr2=0x7fb65c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:30.986 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.446+0000 7fb678167700 1 --2- 172.21.15.22:0/373891549 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fb65c0387a0 0x7fb65c03ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb660006fd0 tx=0x7fb660006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:30.986 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.446+0000 7fb678167700 1 -- 172.21.15.22:0/373891549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 msgr2=0x7fb67019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:30.986 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.446+0000 7fb678167700 1 --2- 172.21.15.22:0/373891549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67019c500 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009130 tx=0x7fb66c011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:30.986 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.447+0000 7fb678167700 1 -- 172.21.15.22:0/373891549 shutdown_connections 2023-09-20T21:39:30.986 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.447+0000 7fb678167700 1 --2- 172.21.15.22:0/373891549 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7fb65c0387a0 0x7fb65c03ac60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:30.987 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.447+0000 7fb678167700 1 --2- 172.21.15.22:0/373891549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb67007be80 0x7fb67019c500 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:30.987 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.447+0000 7fb678167700 1 -- 172.21.15.22:0/373891549 >> 172.21.15.22:0/373891549 conn(0x7fb670101b30 msgr2=0x7fb670103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:30.987 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.447+0000 7fb678167700 1 -- 172.21.15.22:0/373891549 shutdown_connections 2023-09-20T21:39:30.987 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:30.447+0000 7fb678167700 1 -- 172.21.15.22:0/373891549 wait complete. 2023-09-20T21:39:30.997 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:30.821+0000 7f1ebf702200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-20T21:39:30.997 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:30.942+0000 7f1ebf702200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-20T21:39:31.442 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:31 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:31.077+0000 7f1ebf702200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-20T21:39:31.443 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:31 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:31.322+0000 7f1ebf702200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-20T21:39:31.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:31 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/373891549' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-09-20T21:39:31.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:31 smithi022 ceph-mon[97651]: mgrmap e5: x(active, since 6s) 2023-09-20T21:39:32.102 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:31 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:31.805+0000 7f1ebf702200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-20T21:39:32.103 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:31 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:31.980+0000 7f1ebf702200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-20T21:39:32.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2023-09-20T21:39:32.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-09-20T21:39:32.256 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-20T21:39:32.257 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-20T21:39:32.257 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-20T21:39:32.257 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2023-09-20T21:39:32.257 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.502+0000 7f1ad3695700 1 Processor -- start 2023-09-20T21:39:32.257 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.503+0000 7f1ad3695700 1 -- start start 2023-09-20T21:39:32.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.503+0000 7f1ad3695700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:32.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.503+0000 7f1ad3695700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc07a8d0 con 0x7f1acc07bea0 2023-09-20T21:39:32.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.503+0000 7f1ad1431700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:32.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.503+0000 7f1ad1431700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60708/0 (socket says 172.21.15.22:60708) 2023-09-20T21:39:32.258 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.503+0000 7f1ad1431700 1 -- 172.21.15.22:0/3763359592 learned_addr learned my addr 172.21.15.22:0/3763359592 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.504+0000 7f1ad1431700 1 -- 172.21.15.22:0/3763359592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc07aa10 con 0x7f1acc07bea0 2023-09-20T21:39:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.504+0000 7f1ad1431700 1 --2- 172.21.15.22:0/3763359592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc07a300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80098a0 tx=0x7f1ac8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed19a464a895f34 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.504+0000 7f1ac3fff700 1 -- 172.21.15.22:0/3763359592 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ac8004160 con 0x7f1acc07bea0 2023-09-20T21:39:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.504+0000 7f1ac3fff700 1 -- 172.21.15.22:0/3763359592 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ac80042c0 con 0x7f1acc07bea0 2023-09-20T21:39:32.259 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.504+0000 7f1ac3fff700 1 -- 172.21.15.22:0/3763359592 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ac8011a00 con 0x7f1acc07bea0 2023-09-20T21:39:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.505+0000 7f1ad3695700 1 -- 172.21.15.22:0/3763359592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 msgr2=0x7f1acc07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.505+0000 7f1ad3695700 1 --2- 172.21.15.22:0/3763359592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc07a300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80098a0 tx=0x7f1ac8009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.505+0000 7f1ad3695700 1 -- 172.21.15.22:0/3763359592 shutdown_connections 2023-09-20T21:39:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.505+0000 7f1ad3695700 1 --2- 172.21.15.22:0/3763359592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc07a300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.505+0000 7f1ad3695700 1 -- 172.21.15.22:0/3763359592 >> 172.21.15.22:0/3763359592 conn(0x7f1acc101b30 msgr2=0x7f1acc103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:32.260 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.505+0000 7f1ad3695700 1 -- 172.21.15.22:0/3763359592 shutdown_connections 2023-09-20T21:39:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.505+0000 7f1ad3695700 1 -- 172.21.15.22:0/3763359592 wait complete. 2023-09-20T21:39:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.506+0000 7f1ad3695700 1 Processor -- start 2023-09-20T21:39:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.506+0000 7f1ad3695700 1 -- start start 2023-09-20T21:39:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.507+0000 7f1ad3695700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:32.261 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.507+0000 7f1ad3695700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc19ca80 con 0x7f1acc07bea0 2023-09-20T21:39:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.507+0000 7f1ad1431700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.507+0000 7f1ad1431700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60710/0 (socket says 172.21.15.22:60710) 2023-09-20T21:39:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.507+0000 7f1ad1431700 1 -- 172.21.15.22:0/1535126449 learned_addr learned my addr 172.21.15.22:0/1535126449 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.507+0000 7f1ad1431700 1 -- 172.21.15.22:0/1535126449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac8009550 con 0x7f1acc07bea0 2023-09-20T21:39:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.508+0000 7f1ad1431700 1 --2- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc19c540 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009130 tx=0x7f1ac8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:32.262 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.508+0000 7f1ac27fc700 1 -- 172.21.15.22:0/1535126449 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ac8003ab0 con 0x7f1acc07bea0 2023-09-20T21:39:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.508+0000 7f1ac27fc700 1 -- 172.21.15.22:0/1535126449 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ac8003c10 con 0x7f1acc07bea0 2023-09-20T21:39:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.508+0000 7f1ac27fc700 1 -- 172.21.15.22:0/1535126449 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1ac80247d0 con 0x7f1acc07bea0 2023-09-20T21:39:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.508+0000 7f1ad3695700 1 -- 172.21.15.22:0/1535126449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc19cc80 con 0x7f1acc07bea0 2023-09-20T21:39:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.508+0000 7f1ad3695700 1 -- 172.21.15.22:0/1535126449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc19d0a0 con 0x7f1acc07bea0 2023-09-20T21:39:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.509+0000 7f1ac27fc700 1 -- 172.21.15.22:0/1535126449 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f1ac801a430 con 0x7f1acc07bea0 2023-09-20T21:39:32.263 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.510+0000 7f1ad3695700 1 -- 172.21.15.22:0/1535126449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc196590 con 0x7f1acc07bea0 2023-09-20T21:39:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.510+0000 7f1ac27fc700 1 --2- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f1ab8038840 0x7f1ab803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.510+0000 7f1ac27fc700 1 -- 172.21.15.22:0/1535126449 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1ac804c590 con 0x7f1acc07bea0 2023-09-20T21:39:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.510+0000 7f1ad0c30700 1 -- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f1ab8038840 msgr2=0x7f1ab803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1128310681 2023-09-20T21:39:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.510+0000 7f1ad0c30700 1 --2- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f1ab8038840 0x7f1ab803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-20T21:39:32.264 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.515+0000 7f1ac27fc700 1 -- 172.21.15.22:0/1535126449 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1ac8018b40 con 0x7f1acc07bea0 2023-09-20T21:39:32.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.711+0000 7f1ad0c30700 1 -- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f1ab8038840 msgr2=0x7f1ab803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1128310681 2023-09-20T21:39:32.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.711+0000 7f1ad0c30700 1 --2- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f1ab8038840 0x7f1ab803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-20T21:39:32.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.782+0000 7f1ad3695700 1 -- 172.21.15.22:0/1535126449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1acc062460 con 0x7f1acc07bea0 2023-09-20T21:39:32.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.782+0000 7f1ac27fc700 1 -- 172.21.15.22:0/1535126449 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f1ac8018b40 con 0x7f1acc07bea0 2023-09-20T21:39:32.265 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.784+0000 7f1ad3695700 1 -- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f1ab8038840 msgr2=0x7f1ab803ad00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:39:32.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.784+0000 7f1ad3695700 1 --2- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f1ab8038840 0x7f1ab803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:32.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.784+0000 7f1ad3695700 1 -- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 msgr2=0x7f1acc19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:32.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.784+0000 7f1ad3695700 1 --2- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc19c540 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009130 tx=0x7f1ac8011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:32.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.784+0000 7f1ad3695700 1 -- 172.21.15.22:0/1535126449 shutdown_connections 2023-09-20T21:39:32.266 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.784+0000 7f1ad3695700 1 --2- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7f1ab8038840 0x7f1ab803ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:32.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.784+0000 7f1ad3695700 1 --2- 172.21.15.22:0/1535126449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1acc07bea0 0x7f1acc19c540 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:32.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.785+0000 7f1ad3695700 1 -- 172.21.15.22:0/1535126449 >> 172.21.15.22:0/1535126449 conn(0x7f1acc101b30 msgr2=0x7f1acc1032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:32.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.785+0000 7f1ad3695700 1 -- 172.21.15.22:0/1535126449 shutdown_connections 2023-09-20T21:39:32.267 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:31.785+0000 7f1ad3695700 1 -- 172.21.15.22:0/1535126449 wait complete. 2023-09-20T21:39:32.267 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2023-09-20T21:39:32.268 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 5... 2023-09-20T21:39:32.443 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:32 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:32.102+0000 7f1ebf702200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-20T21:39:32.443 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:32 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:32.231+0000 7f1ebf702200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-20T21:39:32.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:32 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1535126449' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-20T21:39:33.247 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:32 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:32.823+0000 7f1ebf702200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-20T21:39:33.997 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:33 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:33.708+0000 7f1ebf702200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-20T21:39:33.998 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:33 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:33.820+0000 7f1ebf702200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-20T21:39:35.171 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:34 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:34.815+0000 7f1ebf702200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-20T21:39:35.171 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:34 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:34.939+0000 7f1ebf702200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-20T21:39:35.171 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:35 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:35.170+0000 7f1ebf702200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-20T21:39:35.835 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:35 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:35.549+0000 7f1ebf702200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-20T21:39:36.248 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:35 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:35.834+0000 7f1ebf702200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-20T21:39:36.248 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:36 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:36.037+0000 7f1ebf702200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-20T21:39:36.747 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:36 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:36.391+0000 7f1ebf702200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-20T21:39:37.747 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:37 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:37.360+0000 7f1ebf702200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-20T21:39:37.748 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:37 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:37.544+0000 7f1ebf702200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-20T21:39:38.244 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:37 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:37.802+0000 7f1ebf702200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-20T21:39:38.245 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:37 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:37.956+0000 7f1ebf702200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-20T21:39:38.497 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:38 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:38.244+0000 7f1ebf702200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-20T21:39:39.748 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:39.360+0000 7f1ebf702200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-20T21:39:39.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: Active manager daemon x restarted 2023-09-20T21:39:39.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: Activating manager daemon x 2023-09-20T21:39:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: osdmap e2: 0 total, 0 up, 0 in 2023-09-20T21:39:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: mgrmap e6: x(active, starting, since 0.00416472s) 2023-09-20T21:39:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:39:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-20T21:39:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-20T21:39:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-20T21:39:39.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-20T21:39:39.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: Manager daemon x is now available 2023-09-20T21:39:39.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:39.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:39.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:39:39.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:39 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:39:40.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:40 smithi022 ceph-mon[97651]: Found migration_current of "None". Setting to last migration. 2023-09-20T21:39:40.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:40 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:39:40.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:40 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-20T21:39:40.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:40 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-20T21:39:40.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:40 smithi022 ceph-mon[97651]: mgrmap e7: x(active, since 1.0072s) 2023-09-20T21:39:40.860 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2023-09-20T21:39:40.860 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-09-20T21:39:40.860 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-20T21:39:40.861 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2023-09-20T21:39:40.861 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.761+0000 7efcc61f3700 1 Processor -- start 2023-09-20T21:39:40.861 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.761+0000 7efcc61f3700 1 -- start start 2023-09-20T21:39:40.861 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.761+0000 7efcc61f3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc01069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:40.861 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.762+0000 7efcc61f3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0106fa0 con 0x7efcc01065b0 2023-09-20T21:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.762+0000 7efcbf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc01069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.762+0000 7efcbf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc01069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60720/0 (socket says 172.21.15.22:60720) 2023-09-20T21:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.762+0000 7efcbf7fe700 1 -- 172.21.15.22:0/2621157902 learned_addr learned my addr 172.21.15.22:0/2621157902 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.762+0000 7efcbf7fe700 1 -- 172.21.15.22:0/2621157902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc01077b0 con 0x7efcc01065b0 2023-09-20T21:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.763+0000 7efcbf7fe700 1 --2- 172.21.15.22:0/2621157902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc01069d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efcb00098a0 tx=0x7efcb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d0e8da298e3a77 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.763+0000 7efcbe7fc700 1 -- 172.21.15.22:0/2621157902 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcb0004160 con 0x7efcc01065b0 2023-09-20T21:39:40.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.763+0000 7efcbe7fc700 1 -- 172.21.15.22:0/2621157902 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcb00042c0 con 0x7efcc01065b0 2023-09-20T21:39:40.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.763+0000 7efcbe7fc700 1 -- 172.21.15.22:0/2621157902 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcb0011a00 con 0x7efcc01065b0 2023-09-20T21:39:40.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.764+0000 7efcc61f3700 1 -- 172.21.15.22:0/2621157902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 msgr2=0x7efcc01069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:40.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.764+0000 7efcc61f3700 1 --2- 172.21.15.22:0/2621157902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc01069d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efcb00098a0 tx=0x7efcb0009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:40.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.764+0000 7efcc61f3700 1 -- 172.21.15.22:0/2621157902 shutdown_connections 2023-09-20T21:39:40.863 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.764+0000 7efcc61f3700 1 --2- 172.21.15.22:0/2621157902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc01069d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:40.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.764+0000 7efcc61f3700 1 -- 172.21.15.22:0/2621157902 >> 172.21.15.22:0/2621157902 conn(0x7efcc0101b30 msgr2=0x7efcc0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:40.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.764+0000 7efcc61f3700 1 -- 172.21.15.22:0/2621157902 shutdown_connections 2023-09-20T21:39:40.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.764+0000 7efcc61f3700 1 -- 172.21.15.22:0/2621157902 wait complete. 2023-09-20T21:39:40.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.765+0000 7efcc61f3700 1 Processor -- start 2023-09-20T21:39:40.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.765+0000 7efcc61f3700 1 -- start start 2023-09-20T21:39:40.864 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.765+0000 7efcc61f3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc007bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:40.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.765+0000 7efcc61f3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc007c170 con 0x7efcc01065b0 2023-09-20T21:39:40.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.766+0000 7efcbf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc007bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:40.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.766+0000 7efcbf7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc007bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60722/0 (socket says 172.21.15.22:60722) 2023-09-20T21:39:40.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.766+0000 7efcbf7fe700 1 -- 172.21.15.22:0/1044305875 learned_addr learned my addr 172.21.15.22:0/1044305875 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:40.865 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.766+0000 7efcbf7fe700 1 -- 172.21.15.22:0/1044305875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb0009550 con 0x7efcc01065b0 2023-09-20T21:39:40.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.766+0000 7efcbf7fe700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc007bc30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009130 tx=0x7efcb0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:40.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.766+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcb0003ab0 con 0x7efcc01065b0 2023-09-20T21:39:40.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.767+0000 7efcc61f3700 1 -- 172.21.15.22:0/1044305875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc007a280 con 0x7efcc01065b0 2023-09-20T21:39:40.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.767+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcb0003c10 con 0x7efcc01065b0 2023-09-20T21:39:40.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.767+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcb00247d0 con 0x7efcc01065b0 2023-09-20T21:39:40.866 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.767+0000 7efcc61f3700 1 -- 172.21.15.22:0/1044305875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc007a6a0 con 0x7efcc01065b0 2023-09-20T21:39:40.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.768+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 45618+0+0 (secure 0 0 0) 0x7efcb001a430 con 0x7efcc01065b0 2023-09-20T21:39:40.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.769+0000 7efcbcff9700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 0x7efca803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:40.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.769+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 --> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7efca803b370 con 0x7efca80387a0 2023-09-20T21:39:40.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.769+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efcb004c5b0 con 0x7efcc01065b0 2023-09-20T21:39:40.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.769+0000 7efcbeffd700 1 -- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 msgr2=0x7efca803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1128310681 2023-09-20T21:39:40.867 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.769+0000 7efcbeffd700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 0x7efca803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-20T21:39:40.868 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.970+0000 7efcbeffd700 1 -- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 msgr2=0x7efca803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1128310681 2023-09-20T21:39:40.868 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:32.970+0000 7efcbeffd700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 0x7efca803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-20T21:39:40.868 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:33.371+0000 7efcbeffd700 1 -- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 msgr2=0x7efca803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1128310681 2023-09-20T21:39:40.868 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:33.371+0000 7efcbeffd700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 0x7efca803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-20T21:39:40.868 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:34.172+0000 7efcbeffd700 1 -- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 msgr2=0x7efca803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1128310681 2023-09-20T21:39:40.868 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:34.172+0000 7efcbeffd700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 0x7efca803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-20T21:39:40.869 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:35.773+0000 7efcbeffd700 1 -- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 msgr2=0x7efca803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1128310681 2023-09-20T21:39:40.869 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:35.774+0000 7efcbeffd700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 0x7efca803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-20T21:39:40.869 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:38.976+0000 7efcbeffd700 1 -- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 msgr2=0x7efca803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/1128310681 2023-09-20T21:39:40.869 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:38.976+0000 7efcbeffd700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 0x7efca803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-09-20T21:39:40.869 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:39.365+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 6) v1 ==== 45385+0+0 (secure 0 0 0) 0x7efcb002d4e0 con 0x7efcc01065b0 2023-09-20T21:39:40.869 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:39.365+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 msgr2=0x7efca803ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:39:40.870 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:39.365+0000 7efcbcff9700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 0x7efca803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:40.870 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.367+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 7) v1 ==== 45512+0+0 (secure 0 0 0) 0x7efcb002c730 con 0x7efcc01065b0 2023-09-20T21:39:40.870 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.367+0000 7efcbcff9700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7efca803bfe0 0x7efca803e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:40.870 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.367+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 --> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7efca803b370 con 0x7efca803bfe0 2023-09-20T21:39:40.870 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.368+0000 7efcbeffd700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7efca803bfe0 0x7efca803e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:40.871 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.369+0000 7efcbeffd700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7efca803bfe0 0x7efca803e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efcb4003a10 tx=0x7efcb4009290 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:40.871 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.370+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 <== mgr.14122 v2:172.21.15.22:6800/3260510802 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7efca803b370 con 0x7efca803bfe0 2023-09-20T21:39:40.871 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.377+0000 7efcc61f3700 1 -- 172.21.15.22:0/1044305875 --> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7efcc007b7d0 con 0x7efca803bfe0 2023-09-20T21:39:40.871 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.377+0000 7efcbcff9700 1 -- 172.21.15.22:0/1044305875 <== mgr.14122 v2:172.21.15.22:6800/3260510802 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7efcc007b7d0 con 0x7efca803bfe0 2023-09-20T21:39:40.871 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.377+0000 7efcc61f3700 1 -- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7efca803bfe0 msgr2=0x7efca803e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:40.871 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.377+0000 7efcc61f3700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7efca803bfe0 0x7efca803e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efcb4003a10 tx=0x7efcb4009290 comp rx=0 tx=0).stop 2023-09-20T21:39:40.872 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.377+0000 7efcc61f3700 1 -- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 msgr2=0x7efcc007bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:40.872 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.377+0000 7efcc61f3700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc007bc30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009130 tx=0x7efcb0011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:40.872 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.378+0000 7efcc61f3700 1 -- 172.21.15.22:0/1044305875 shutdown_connections 2023-09-20T21:39:40.872 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.378+0000 7efcc61f3700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7efca803bfe0 0x7efca803e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:40.872 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.378+0000 7efcc61f3700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcc01065b0 0x7efcc007bc30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:40.873 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.378+0000 7efcc61f3700 1 --2- 172.21.15.22:0/1044305875 >> [v2:172.21.15.22:6800/1128310681,v1:172.21.15.22:6801/1128310681] conn(0x7efca80387a0 0x7efca803ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:40.873 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.378+0000 7efcc61f3700 1 -- 172.21.15.22:0/1044305875 >> 172.21.15.22:0/1044305875 conn(0x7efcc0101b30 msgr2=0x7efcc0103300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:40.873 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.378+0000 7efcc61f3700 1 -- 172.21.15.22:0/1044305875 shutdown_connections 2023-09-20T21:39:40.873 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:40.378+0000 7efcc61f3700 1 -- 172.21.15.22:0/1044305875 wait complete. 2023-09-20T21:39:40.873 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 5 is available 2023-09-20T21:39:40.873 INFO:teuthology.orchestra.run.smithi022.stdout:Setting orchestrator backend to cephadm... 2023-09-20T21:39:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:41 smithi022 ceph-mon[97651]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-20T21:39:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:41 smithi022 ceph-mon[97651]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-20T21:39:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:41 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:41 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:42.044 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.383+0000 7f2ce6439700 1 Processor -- start 2023-09-20T21:39:42.044 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.383+0000 7f2ce6439700 1 -- start start 2023-09-20T21:39:42.045 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.383+0000 7f2ce6439700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:42.045 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.383+0000 7f2ce6439700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce007a890 con 0x7f2ce007be60 2023-09-20T21:39:42.045 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.384+0000 7f2cdffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:42.046 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.384+0000 7f2cdffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41992/0 (socket says 172.21.15.22:41992) 2023-09-20T21:39:42.046 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.384+0000 7f2cdffff700 1 -- 172.21.15.22:0/1313110223 learned_addr learned my addr 172.21.15.22:0/1313110223 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:42.047 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.384+0000 7f2cdffff700 1 -- 172.21.15.22:0/1313110223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce007a9d0 con 0x7f2ce007be60 2023-09-20T21:39:42.047 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.384+0000 7f2cdffff700 1 --2- 172.21.15.22:0/1313110223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce007a2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00098a0 tx=0x7f2cd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3311c77b5c3893e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:42.047 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.385+0000 7f2cdeffd700 1 -- 172.21.15.22:0/1313110223 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd0004160 con 0x7f2ce007be60 2023-09-20T21:39:42.048 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.385+0000 7f2cdeffd700 1 -- 172.21.15.22:0/1313110223 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cd00042c0 con 0x7f2ce007be60 2023-09-20T21:39:42.048 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.385+0000 7f2cdeffd700 1 -- 172.21.15.22:0/1313110223 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd0011a00 con 0x7f2ce007be60 2023-09-20T21:39:42.048 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.385+0000 7f2ce6439700 1 -- 172.21.15.22:0/1313110223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 msgr2=0x7f2ce007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:42.049 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.385+0000 7f2ce6439700 1 --2- 172.21.15.22:0/1313110223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce007a2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00098a0 tx=0x7f2cd0009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:42.049 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.386+0000 7f2ce6439700 1 -- 172.21.15.22:0/1313110223 shutdown_connections 2023-09-20T21:39:42.050 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.386+0000 7f2ce6439700 1 --2- 172.21.15.22:0/1313110223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce007a2c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:42.050 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.386+0000 7f2ce6439700 1 -- 172.21.15.22:0/1313110223 >> 172.21.15.22:0/1313110223 conn(0x7f2ce0101b30 msgr2=0x7f2ce0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:42.050 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.386+0000 7f2ce6439700 1 -- 172.21.15.22:0/1313110223 shutdown_connections 2023-09-20T21:39:42.051 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.386+0000 7f2ce6439700 1 -- 172.21.15.22:0/1313110223 wait complete. 2023-09-20T21:39:42.051 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.387+0000 7f2ce6439700 1 Processor -- start 2023-09-20T21:39:42.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.387+0000 7f2ce6439700 1 -- start start 2023-09-20T21:39:42.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.388+0000 7f2ce6439700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:42.052 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.388+0000 7f2ce6439700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce019ca30 con 0x7f2ce007be60 2023-09-20T21:39:42.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.388+0000 7f2cdffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:42.053 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.388+0000 7f2cdffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42000/0 (socket says 172.21.15.22:42000) 2023-09-20T21:39:42.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.388+0000 7f2cdffff700 1 -- 172.21.15.22:0/798652624 learned_addr learned my addr 172.21.15.22:0/798652624 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:42.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.388+0000 7f2cdffff700 1 -- 172.21.15.22:0/798652624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0009550 con 0x7f2ce007be60 2023-09-20T21:39:42.054 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.389+0000 7f2cdffff700 1 --2- 172.21.15.22:0/798652624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce019c4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009130 tx=0x7f2cd0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:42.055 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.389+0000 7f2cdd7fa700 1 -- 172.21.15.22:0/798652624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd0003b70 con 0x7f2ce007be60 2023-09-20T21:39:42.055 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.389+0000 7f2cdd7fa700 1 -- 172.21.15.22:0/798652624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2cd0003cd0 con 0x7f2ce007be60 2023-09-20T21:39:42.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.389+0000 7f2cdd7fa700 1 -- 172.21.15.22:0/798652624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd00197d0 con 0x7f2ce007be60 2023-09-20T21:39:42.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.389+0000 7f2ce6439700 1 -- 172.21.15.22:0/798652624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce019cc30 con 0x7f2ce007be60 2023-09-20T21:39:42.056 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.389+0000 7f2ce6439700 1 -- 172.21.15.22:0/798652624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce019d050 con 0x7f2ce007be60 2023-09-20T21:39:42.057 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.390+0000 7f2ce6439700 1 -- 172.21.15.22:0/798652624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce0196580 con 0x7f2ce007be60 2023-09-20T21:39:42.057 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.390+0000 7f2cdd7fa700 1 -- 172.21.15.22:0/798652624 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 7) v1 ==== 45512+0+0 (secure 0 0 0) 0x7f2cd001abb0 con 0x7f2ce007be60 2023-09-20T21:39:42.057 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.391+0000 7f2cdd7fa700 1 --2- 172.21.15.22:0/798652624 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2cc80386c0 0x7f2cc803ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:42.058 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.391+0000 7f2cdd7fa700 1 -- 172.21.15.22:0/798652624 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2cd004c880 con 0x7f2ce007be60 2023-09-20T21:39:42.058 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.391+0000 7f2cdf7fe700 1 --2- 172.21.15.22:0/798652624 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2cc80386c0 0x7f2cc803ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:42.058 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.392+0000 7f2cdf7fe700 1 --2- 172.21.15.22:0/798652624 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2cc80386c0 0x7f2cc803ab80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4006fd0 tx=0x7f2cd4006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:42.058 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.396+0000 7f2cdd7fa700 1 -- 172.21.15.22:0/798652624 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2cd001a430 con 0x7f2ce007be60 2023-09-20T21:39:42.059 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.623+0000 7f2ce6439700 1 -- 172.21.15.22:0/798652624 --> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ce002d060 con 0x7f2cc80386c0 2023-09-20T21:39:42.059 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.630+0000 7f2cdd7fa700 1 -- 172.21.15.22:0/798652624 <== mgr.14122 v2:172.21.15.22:6800/3260510802 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2ce002d060 con 0x7f2cc80386c0 2023-09-20T21:39:42.059 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.633+0000 7f2ce6439700 1 -- 172.21.15.22:0/798652624 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2cc80386c0 msgr2=0x7f2cc803ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:42.059 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.633+0000 7f2ce6439700 1 --2- 172.21.15.22:0/798652624 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2cc80386c0 0x7f2cc803ab80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4006fd0 tx=0x7f2cd4006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:42.059 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.633+0000 7f2ce6439700 1 -- 172.21.15.22:0/798652624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 msgr2=0x7f2ce019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:42.060 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.633+0000 7f2ce6439700 1 --2- 172.21.15.22:0/798652624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce019c4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009130 tx=0x7f2cd0011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:42.060 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.633+0000 7f2ce6439700 1 -- 172.21.15.22:0/798652624 shutdown_connections 2023-09-20T21:39:42.060 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.633+0000 7f2ce6439700 1 --2- 172.21.15.22:0/798652624 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2cc80386c0 0x7f2cc803ab80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:42.060 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.633+0000 7f2ce6439700 1 --2- 172.21.15.22:0/798652624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ce007be60 0x7f2ce019c4f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:42.060 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.633+0000 7f2ce6439700 1 -- 172.21.15.22:0/798652624 >> 172.21.15.22:0/798652624 conn(0x7f2ce0101b30 msgr2=0x7f2ce0103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:42.061 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.633+0000 7f2ce6439700 1 -- 172.21.15.22:0/798652624 shutdown_connections 2023-09-20T21:39:42.061 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:41.633+0000 7f2ce6439700 1 -- 172.21.15.22:0/798652624 wait complete. 2023-09-20T21:39:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:42 smithi022 ceph-mon[97651]: [20/Sep/2023:21:39:41] ENGINE Bus STARTING 2023-09-20T21:39:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:42 smithi022 ceph-mon[97651]: [20/Sep/2023:21:39:41] ENGINE Serving on http://172.21.15.22:8765 2023-09-20T21:39:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:42 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:42 smithi022 ceph-mon[97651]: mgrmap e8: x(active, since 2s) 2023-09-20T21:39:42.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:42 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:39:43.219 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout value unchanged 2023-09-20T21:39:43.220 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.547+0000 7f64e475a700 1 Processor -- start 2023-09-20T21:39:43.220 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.548+0000 7f64e475a700 1 -- start start 2023-09-20T21:39:43.220 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.548+0000 7f64e475a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc07a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:43.220 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.548+0000 7f64e475a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc07a870 con 0x7f64dc07be40 2023-09-20T21:39:43.220 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.548+0000 7f64e24f6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc07a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:43.221 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.548+0000 7f64e24f6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc07a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42012/0 (socket says 172.21.15.22:42012) 2023-09-20T21:39:43.221 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.548+0000 7f64e24f6700 1 -- 172.21.15.22:0/886624566 learned_addr learned my addr 172.21.15.22:0/886624566 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:43.221 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.549+0000 7f64e24f6700 1 -- 172.21.15.22:0/886624566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc07a9b0 con 0x7f64dc07be40 2023-09-20T21:39:43.221 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.549+0000 7f64e24f6700 1 --2- 172.21.15.22:0/886624566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc07a2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f64d80098a0 tx=0x7f64d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a27eb78fd0b65a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:43.221 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.549+0000 7f64e14f4700 1 -- 172.21.15.22:0/886624566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64d8004160 con 0x7f64dc07be40 2023-09-20T21:39:43.222 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.549+0000 7f64e14f4700 1 -- 172.21.15.22:0/886624566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64d80042c0 con 0x7f64dc07be40 2023-09-20T21:39:43.222 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.549+0000 7f64e14f4700 1 -- 172.21.15.22:0/886624566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64d8011a00 con 0x7f64dc07be40 2023-09-20T21:39:43.222 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.550+0000 7f64e475a700 1 -- 172.21.15.22:0/886624566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 msgr2=0x7f64dc07a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:43.222 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.550+0000 7f64e475a700 1 --2- 172.21.15.22:0/886624566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc07a2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f64d80098a0 tx=0x7f64d8009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:43.223 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.550+0000 7f64e475a700 1 -- 172.21.15.22:0/886624566 shutdown_connections 2023-09-20T21:39:43.223 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.550+0000 7f64e475a700 1 --2- 172.21.15.22:0/886624566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc07a2a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:43.223 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.550+0000 7f64e475a700 1 -- 172.21.15.22:0/886624566 >> 172.21.15.22:0/886624566 conn(0x7f64dc101b30 msgr2=0x7f64dc103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:43.223 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.550+0000 7f64e475a700 1 -- 172.21.15.22:0/886624566 shutdown_connections 2023-09-20T21:39:43.223 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.550+0000 7f64e475a700 1 -- 172.21.15.22:0/886624566 wait complete. 2023-09-20T21:39:43.224 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.551+0000 7f64e475a700 1 Processor -- start 2023-09-20T21:39:43.224 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.551+0000 7f64e475a700 1 -- start start 2023-09-20T21:39:43.224 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.552+0000 7f64e475a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:43.224 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.552+0000 7f64e475a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc19ca40 con 0x7f64dc07be40 2023-09-20T21:39:43.224 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.552+0000 7f64e24f6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:43.225 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.552+0000 7f64e24f6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42018/0 (socket says 172.21.15.22:42018) 2023-09-20T21:39:43.225 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.552+0000 7f64e24f6700 1 -- 172.21.15.22:0/2320216276 learned_addr learned my addr 172.21.15.22:0/2320216276 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:43.225 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.552+0000 7f64e24f6700 1 -- 172.21.15.22:0/2320216276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d8009550 con 0x7f64dc07be40 2023-09-20T21:39:43.225 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.553+0000 7f64e24f6700 1 --2- 172.21.15.22:0/2320216276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc19c500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009fb0 tx=0x7f64d8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:43.225 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.553+0000 7f64d37fe700 1 -- 172.21.15.22:0/2320216276 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64d8003a40 con 0x7f64dc07be40 2023-09-20T21:39:43.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.553+0000 7f64e475a700 1 -- 172.21.15.22:0/2320216276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64dc19cc40 con 0x7f64dc07be40 2023-09-20T21:39:43.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.553+0000 7f64d37fe700 1 -- 172.21.15.22:0/2320216276 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64d8003ba0 con 0x7f64dc07be40 2023-09-20T21:39:43.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.553+0000 7f64d37fe700 1 -- 172.21.15.22:0/2320216276 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64d801a750 con 0x7f64dc07be40 2023-09-20T21:39:43.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.553+0000 7f64e475a700 1 -- 172.21.15.22:0/2320216276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64dc19d060 con 0x7f64dc07be40 2023-09-20T21:39:43.226 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.554+0000 7f64d37fe700 1 -- 172.21.15.22:0/2320216276 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f64d8024b60 con 0x7f64dc07be40 2023-09-20T21:39:43.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.554+0000 7f64e475a700 1 -- 172.21.15.22:0/2320216276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc062410 con 0x7f64dc07be40 2023-09-20T21:39:43.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.554+0000 7f64d37fe700 1 --2- 172.21.15.22:0/2320216276 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f64c80387a0 0x7f64c803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:43.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.555+0000 7f64d37fe700 1 -- 172.21.15.22:0/2320216276 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f64d804c8a0 con 0x7f64dc07be40 2023-09-20T21:39:43.227 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.555+0000 7f64e1cf5700 1 --2- 172.21.15.22:0/2320216276 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f64c80387a0 0x7f64c803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:43.228 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.555+0000 7f64e1cf5700 1 --2- 172.21.15.22:0/2320216276 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f64c80387a0 0x7f64c803ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f64cc006fd0 tx=0x7f64cc006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:43.228 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.560+0000 7f64d37fe700 1 -- 172.21.15.22:0/2320216276 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f64d8024430 con 0x7f64dc07be40 2023-09-20T21:39:43.228 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.771+0000 7f64e475a700 1 -- 172.21.15.22:0/2320216276 --> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f64dc103ec0 con 0x7f64c80387a0 2023-09-20T21:39:43.228 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.772+0000 7f64d37fe700 1 -- 172.21.15.22:0/2320216276 <== mgr.14122 v2:172.21.15.22:6800/3260510802 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f64dc103ec0 con 0x7f64c80387a0 2023-09-20T21:39:43.228 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.774+0000 7f64e475a700 1 -- 172.21.15.22:0/2320216276 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f64c80387a0 msgr2=0x7f64c803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:43.229 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.774+0000 7f64e475a700 1 --2- 172.21.15.22:0/2320216276 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f64c80387a0 0x7f64c803ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f64cc006fd0 tx=0x7f64cc006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:43.229 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.774+0000 7f64e475a700 1 -- 172.21.15.22:0/2320216276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 msgr2=0x7f64dc19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:43.229 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.774+0000 7f64e475a700 1 --2- 172.21.15.22:0/2320216276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc19c500 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009fb0 tx=0x7f64d8011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:43.229 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.775+0000 7f64e475a700 1 -- 172.21.15.22:0/2320216276 shutdown_connections 2023-09-20T21:39:43.229 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.775+0000 7f64e475a700 1 --2- 172.21.15.22:0/2320216276 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f64c80387a0 0x7f64c803ac60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:43.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.775+0000 7f64e475a700 1 --2- 172.21.15.22:0/2320216276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64dc07be40 0x7f64dc19c500 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:43.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.775+0000 7f64e475a700 1 -- 172.21.15.22:0/2320216276 >> 172.21.15.22:0/2320216276 conn(0x7f64dc101b30 msgr2=0x7f64dc1032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:43.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.775+0000 7f64e475a700 1 -- 172.21.15.22:0/2320216276 shutdown_connections 2023-09-20T21:39:43.230 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:42.775+0000 7f64e475a700 1 -- 172.21.15.22:0/2320216276 wait complete. 2023-09-20T21:39:43.231 INFO:teuthology.orchestra.run.smithi022.stdout:Generating ssh key... 2023-09-20T21:39:43.470 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:43 smithi022 ceph-mon[97651]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:39:43.470 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:43 smithi022 ceph-mon[97651]: [20/Sep/2023:21:39:41] ENGINE Serving on https://172.21.15.22:7150 2023-09-20T21:39:43.471 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:43 smithi022 ceph-mon[97651]: [20/Sep/2023:21:39:41] ENGINE Bus STARTED 2023-09-20T21:39:44.440 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: Generating public/private rsa key pair. 2023-09-20T21:39:44.440 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: Your identification has been saved in /tmp/tmp60xs33p_/key. 2023-09-20T21:39:44.440 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: Your public key has been saved in /tmp/tmp60xs33p_/key.pub. 2023-09-20T21:39:44.441 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: The key fingerprint is: 2023-09-20T21:39:44.441 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: SHA256:2nUIZxw9FnUy2/2V7BI4R7c8cqoxFwwZmZMGkPSxD1U ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:39:44.441 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: The key's randomart image is: 2023-09-20T21:39:44.441 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: +---[RSA 3072]----+ 2023-09-20T21:39:44.441 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: | .oooo=OE+..| 2023-09-20T21:39:44.442 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: | ...+OO +*+| 2023-09-20T21:39:44.442 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: | .+=+.B.B+| 2023-09-20T21:39:44.442 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: | +o.o B +| 2023-09-20T21:39:44.442 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: | S o+.+ ..| 2023-09-20T21:39:44.442 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: | o . .= . | 2023-09-20T21:39:44.443 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: | . . . | 2023-09-20T21:39:44.443 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: | | 2023-09-20T21:39:44.443 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: | | 2023-09-20T21:39:44.443 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: +----[SHA256]-----+ 2023-09-20T21:39:44.599 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.755+0000 7f980d25f700 1 Processor -- start 2023-09-20T21:39:44.599 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.755+0000 7f980d25f700 1 -- start start 2023-09-20T21:39:44.599 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.755+0000 7f980d25f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980807a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:44.599 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.755+0000 7f980d25f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980807a890 con 0x7f980807be60 2023-09-20T21:39:44.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.755+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980807a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:44.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.756+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980807a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42022/0 (socket says 172.21.15.22:42022) 2023-09-20T21:39:44.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.756+0000 7f9806d9d700 1 -- 172.21.15.22:0/2364021428 learned_addr learned my addr 172.21.15.22:0/2364021428 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:44.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.756+0000 7f9806d9d700 1 -- 172.21.15.22:0/2364021428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980807a9d0 con 0x7f980807be60 2023-09-20T21:39:44.600 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.756+0000 7f9806d9d700 1 --2- 172.21.15.22:0/2364021428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980807a2c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f97f80098a0 tx=0x7f97f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0972bc6e725389 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:44.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.756+0000 7f9805d9b700 1 -- 172.21.15.22:0/2364021428 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97f8004160 con 0x7f980807be60 2023-09-20T21:39:44.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.756+0000 7f9805d9b700 1 -- 172.21.15.22:0/2364021428 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97f80042c0 con 0x7f980807be60 2023-09-20T21:39:44.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.756+0000 7f9805d9b700 1 -- 172.21.15.22:0/2364021428 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97f8011a00 con 0x7f980807be60 2023-09-20T21:39:44.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.757+0000 7f980d25f700 1 -- 172.21.15.22:0/2364021428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 msgr2=0x7f980807a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:44.601 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.757+0000 7f980d25f700 1 --2- 172.21.15.22:0/2364021428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980807a2c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f97f80098a0 tx=0x7f97f8009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:44.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.757+0000 7f980d25f700 1 -- 172.21.15.22:0/2364021428 shutdown_connections 2023-09-20T21:39:44.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.757+0000 7f980d25f700 1 --2- 172.21.15.22:0/2364021428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980807a2c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:44.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.757+0000 7f980d25f700 1 -- 172.21.15.22:0/2364021428 >> 172.21.15.22:0/2364021428 conn(0x7f9808101b30 msgr2=0x7f9808103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:44.602 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.757+0000 7f980d25f700 1 -- 172.21.15.22:0/2364021428 shutdown_connections 2023-09-20T21:39:44.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.757+0000 7f980d25f700 1 -- 172.21.15.22:0/2364021428 wait complete. 2023-09-20T21:39:44.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.759+0000 7f980d25f700 1 Processor -- start 2023-09-20T21:39:44.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.759+0000 7f980d25f700 1 -- start start 2023-09-20T21:39:44.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.759+0000 7f980d25f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:44.603 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.759+0000 7f980d25f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819ca30 con 0x7f980807be60 2023-09-20T21:39:44.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.760+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:44.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.760+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42038/0 (socket says 172.21.15.22:42038) 2023-09-20T21:39:44.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.760+0000 7f9806d9d700 1 -- 172.21.15.22:0/2676345767 learned_addr learned my addr 172.21.15.22:0/2676345767 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:44.604 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.760+0000 7f9806d9d700 1 -- 172.21.15.22:0/2676345767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f8009550 con 0x7f980807be60 2023-09-20T21:39:44.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.760+0000 7f9806d9d700 1 --2- 172.21.15.22:0/2676345767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980819c4f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009130 tx=0x7f97f8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:44.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.761+0000 7f97effff700 1 -- 172.21.15.22:0/2676345767 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97f8003b70 con 0x7f980807be60 2023-09-20T21:39:44.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.761+0000 7f97effff700 1 -- 172.21.15.22:0/2676345767 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97f8003cd0 con 0x7f980807be60 2023-09-20T21:39:44.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.761+0000 7f97effff700 1 -- 172.21.15.22:0/2676345767 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97f80197d0 con 0x7f980807be60 2023-09-20T21:39:44.605 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.761+0000 7f980d25f700 1 -- 172.21.15.22:0/2676345767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980819cc30 con 0x7f980807be60 2023-09-20T21:39:44.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.761+0000 7f980d25f700 1 -- 172.21.15.22:0/2676345767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980819d050 con 0x7f980807be60 2023-09-20T21:39:44.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.762+0000 7f97effff700 1 -- 172.21.15.22:0/2676345767 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f97f801abb0 con 0x7f980807be60 2023-09-20T21:39:44.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.762+0000 7f980d25f700 1 -- 172.21.15.22:0/2676345767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9808196580 con 0x7f980807be60 2023-09-20T21:39:44.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.762+0000 7f97effff700 1 --2- 172.21.15.22:0/2676345767 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f97f00387a0 0x7f97f003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:44.606 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.762+0000 7f97effff700 1 -- 172.21.15.22:0/2676345767 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f97f804cb30 con 0x7f980807be60 2023-09-20T21:39:44.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.763+0000 7f980659c700 1 --2- 172.21.15.22:0/2676345767 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f97f00387a0 0x7f97f003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:44.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.763+0000 7f980659c700 1 --2- 172.21.15.22:0/2676345767 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f97f00387a0 0x7f97f003ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f97fc006fd0 tx=0x7f97fc006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:44.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.767+0000 7f97effff700 1 -- 172.21.15.22:0/2676345767 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f97f801a430 con 0x7f980807be60 2023-09-20T21:39:44.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:43.977+0000 7f980d25f700 1 -- 172.21.15.22:0/2676345767 --> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f980802d060 con 0x7f97f00387a0 2023-09-20T21:39:44.607 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.103+0000 7f97effff700 1 -- 172.21.15.22:0/2676345767 <== mgr.14122 v2:172.21.15.22:6800/3260510802 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f980802d060 con 0x7f97f00387a0 2023-09-20T21:39:44.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.105+0000 7f980d25f700 1 -- 172.21.15.22:0/2676345767 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f97f00387a0 msgr2=0x7f97f003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:44.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.105+0000 7f980d25f700 1 --2- 172.21.15.22:0/2676345767 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f97f00387a0 0x7f97f003ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f97fc006fd0 tx=0x7f97fc006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:44.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.105+0000 7f980d25f700 1 -- 172.21.15.22:0/2676345767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 msgr2=0x7f980819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:44.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.105+0000 7f980d25f700 1 --2- 172.21.15.22:0/2676345767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980819c4f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009130 tx=0x7f97f8011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:44.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.106+0000 7f980d25f700 1 -- 172.21.15.22:0/2676345767 shutdown_connections 2023-09-20T21:39:44.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.106+0000 7f980d25f700 1 --2- 172.21.15.22:0/2676345767 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f97f00387a0 0x7f97f003ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:44.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.106+0000 7f980d25f700 1 --2- 172.21.15.22:0/2676345767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f980807be60 0x7f980819c4f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:44.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.106+0000 7f980d25f700 1 -- 172.21.15.22:0/2676345767 >> 172.21.15.22:0/2676345767 conn(0x7f9808101b30 msgr2=0x7f9808103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:44.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.106+0000 7f980d25f700 1 -- 172.21.15.22:0/2676345767 shutdown_connections 2023-09-20T21:39:44.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:44.106+0000 7f980d25f700 1 -- 172.21.15.22:0/2676345767 wait complete. 2023-09-20T21:39:44.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-mon[97651]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:39:44.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:44.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:44 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:45.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:45 smithi022 ceph-mon[97651]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:39:45.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:45 smithi022 ceph-mon[97651]: Generating ssh key... 2023-09-20T21:39:45.779 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3B+lE3MW556Fkpg4qXTcj2XA7hcScu48hvT/yGSvVzFmVJ5KGdiHII8b1o6m4GEZe7ue4GKZ6CvPR7HJhqbOI/dQ9uJo/bMBtJt5A9vFWf4ldnduMoo+fS56cQlnzCBNzkpSXLC1/KV5OGZ4spcfDeOUU5FOCqkl/X7OYv0frMxFZYZl7difgX3V2ATYXmVilPz7fSrEZ4l3C4FRLgn3JGq/GPJjsaZ+72g3r8EpqcDipIIclhhctoSmAhjcYqM9f1bILd+EzrrzDc+YlYYEqmmwTsxEeIC/Gf/kBasW8mBQtLgC5QaeFqgufeGKCyCOkVnQY/tDOvv/pXDjAL8aRu3RRlTCZXfZ1wfE8JpSuGjyMtLOkiaV0BN/nAJOJHNdPJ3FllIQlh3YRx/LO9qOPelMvGrC9AFfIIw5vp2w0sVwBagU9pJ5WO/B1nDElM19CYZoFm7z4nQ7kX576Aioc7Slnd3q7EjKNCpfz5U15ZzGKVb2bdyTB7VqgFq4HIeM= ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:39:45.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.130+0000 7fb90c015700 1 Processor -- start 2023-09-20T21:39:45.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.130+0000 7fb90c015700 1 -- start start 2023-09-20T21:39:45.780 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.131+0000 7fb90c015700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90407a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:45.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.131+0000 7fb90c015700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90407a870 con 0x7fb90407be40 2023-09-20T21:39:45.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.131+0000 7fb909db1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90407a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:45.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.131+0000 7fb909db1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90407a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42040/0 (socket says 172.21.15.22:42040) 2023-09-20T21:39:45.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.131+0000 7fb909db1700 1 -- 172.21.15.22:0/1976254964 learned_addr learned my addr 172.21.15.22:0/1976254964 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:45.781 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.131+0000 7fb909db1700 1 -- 172.21.15.22:0/1976254964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90407a9b0 con 0x7fb90407be40 2023-09-20T21:39:45.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.132+0000 7fb909db1700 1 --2- 172.21.15.22:0/1976254964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90407a2a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb9000098a0 tx=0x7fb900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8212a5d62442c2d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:45.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.132+0000 7fb908daf700 1 -- 172.21.15.22:0/1976254964 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb900004160 con 0x7fb90407be40 2023-09-20T21:39:45.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.132+0000 7fb908daf700 1 -- 172.21.15.22:0/1976254964 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb9000042c0 con 0x7fb90407be40 2023-09-20T21:39:45.782 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.132+0000 7fb908daf700 1 -- 172.21.15.22:0/1976254964 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb900011a00 con 0x7fb90407be40 2023-09-20T21:39:45.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.132+0000 7fb90c015700 1 -- 172.21.15.22:0/1976254964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 msgr2=0x7fb90407a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:45.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.132+0000 7fb90c015700 1 --2- 172.21.15.22:0/1976254964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90407a2a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb9000098a0 tx=0x7fb900009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:45.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.133+0000 7fb90c015700 1 -- 172.21.15.22:0/1976254964 shutdown_connections 2023-09-20T21:39:45.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.133+0000 7fb90c015700 1 --2- 172.21.15.22:0/1976254964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90407a2a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:45.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.133+0000 7fb90c015700 1 -- 172.21.15.22:0/1976254964 >> 172.21.15.22:0/1976254964 conn(0x7fb904101b30 msgr2=0x7fb904103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:45.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.133+0000 7fb90c015700 1 -- 172.21.15.22:0/1976254964 shutdown_connections 2023-09-20T21:39:45.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.133+0000 7fb90c015700 1 -- 172.21.15.22:0/1976254964 wait complete. 2023-09-20T21:39:45.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.134+0000 7fb90c015700 1 Processor -- start 2023-09-20T21:39:45.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.134+0000 7fb90c015700 1 -- start start 2023-09-20T21:39:45.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.134+0000 7fb90c015700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:45.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.134+0000 7fb90c015700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90419ca40 con 0x7fb90407be40 2023-09-20T21:39:45.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.134+0000 7fb909db1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:45.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.134+0000 7fb909db1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42046/0 (socket says 172.21.15.22:42046) 2023-09-20T21:39:45.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.135+0000 7fb909db1700 1 -- 172.21.15.22:0/647558982 learned_addr learned my addr 172.21.15.22:0/647558982 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:45.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.135+0000 7fb909db1700 1 -- 172.21.15.22:0/647558982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb900009550 con 0x7fb90407be40 2023-09-20T21:39:45.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.135+0000 7fb909db1700 1 --2- 172.21.15.22:0/647558982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90419c500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb900009130 tx=0x7fb900011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:45.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.135+0000 7fb8faffd700 1 -- 172.21.15.22:0/647558982 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb900003b70 con 0x7fb90407be40 2023-09-20T21:39:45.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.135+0000 7fb8faffd700 1 -- 172.21.15.22:0/647558982 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb900003cd0 con 0x7fb90407be40 2023-09-20T21:39:45.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.135+0000 7fb90c015700 1 -- 172.21.15.22:0/647558982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90419cc40 con 0x7fb90407be40 2023-09-20T21:39:45.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.135+0000 7fb8faffd700 1 -- 172.21.15.22:0/647558982 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9000197d0 con 0x7fb90407be40 2023-09-20T21:39:45.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.136+0000 7fb90c015700 1 -- 172.21.15.22:0/647558982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90419d060 con 0x7fb90407be40 2023-09-20T21:39:45.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.137+0000 7fb8faffd700 1 -- 172.21.15.22:0/647558982 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45618+0+0 (secure 0 0 0) 0x7fb90001abb0 con 0x7fb90407be40 2023-09-20T21:39:45.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.137+0000 7fb90c015700 1 -- 172.21.15.22:0/647558982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90410c960 con 0x7fb90407be40 2023-09-20T21:39:45.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.137+0000 7fb8faffd700 1 --2- 172.21.15.22:0/647558982 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fb8f00387f0 0x7fb8f003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:45.788 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.137+0000 7fb8faffd700 1 -- 172.21.15.22:0/647558982 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb90004cac0 con 0x7fb90407be40 2023-09-20T21:39:45.788 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.137+0000 7fb9095b0700 1 --2- 172.21.15.22:0/647558982 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fb8f00387f0 0x7fb8f003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:45.788 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.138+0000 7fb9095b0700 1 --2- 172.21.15.22:0/647558982 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fb8f00387f0 0x7fb8f003acb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4006fd0 tx=0x7fb8f4006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:45.788 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.142+0000 7fb8faffd700 1 -- 172.21.15.22:0/647558982 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb90001a430 con 0x7fb90407be40 2023-09-20T21:39:45.788 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.352+0000 7fb90c015700 1 -- 172.21.15.22:0/647558982 --> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb90419faa0 con 0x7fb8f00387f0 2023-09-20T21:39:45.789 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.353+0000 7fb8faffd700 1 -- 172.21.15.22:0/647558982 <== mgr.14122 v2:172.21.15.22:6800/3260510802 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fb90419faa0 con 0x7fb8f00387f0 2023-09-20T21:39:45.789 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.356+0000 7fb90c015700 1 -- 172.21.15.22:0/647558982 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fb8f00387f0 msgr2=0x7fb8f003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:45.789 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.356+0000 7fb90c015700 1 --2- 172.21.15.22:0/647558982 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fb8f00387f0 0x7fb8f003acb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4006fd0 tx=0x7fb8f4006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:45.789 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.356+0000 7fb90c015700 1 -- 172.21.15.22:0/647558982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 msgr2=0x7fb90419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:45.789 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.356+0000 7fb90c015700 1 --2- 172.21.15.22:0/647558982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90419c500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb900009130 tx=0x7fb900011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:45.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.356+0000 7fb90c015700 1 -- 172.21.15.22:0/647558982 shutdown_connections 2023-09-20T21:39:45.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.356+0000 7fb90c015700 1 --2- 172.21.15.22:0/647558982 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fb8f00387f0 0x7fb8f003acb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:45.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.356+0000 7fb90c015700 1 --2- 172.21.15.22:0/647558982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb90407be40 0x7fb90419c500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:45.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.356+0000 7fb90c015700 1 -- 172.21.15.22:0/647558982 >> 172.21.15.22:0/647558982 conn(0x7fb904101b30 msgr2=0x7fb9041032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:45.790 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.356+0000 7fb90c015700 1 -- 172.21.15.22:0/647558982 shutdown_connections 2023-09-20T21:39:45.791 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:45.356+0000 7fb90c015700 1 -- 172.21.15.22:0/647558982 wait complete. 2023-09-20T21:39:45.791 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-09-20T21:39:45.791 INFO:teuthology.orchestra.run.smithi022.stdout:Adding key to root@localhost authorized_keys... 2023-09-20T21:39:45.792 INFO:teuthology.orchestra.run.smithi022.stdout:Adding host smithi022... 2023-09-20T21:39:46.693 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:46 smithi022 ceph-mon[97651]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:39:48.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:48 smithi022 ceph-mon[97651]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:39:48.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:48 smithi022 ceph-mon[97651]: Deploying cephadm binary to smithi022 2023-09-20T21:39:50.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Added host 'smithi022' with addr '172.21.15.22' 2023-09-20T21:39:50.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.349+0000 7f222ed4c700 1 Processor -- start 2023-09-20T21:39:50.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.350+0000 7f222ed4c700 1 -- start start 2023-09-20T21:39:50.271 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.350+0000 7f222ed4c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:50.272 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.350+0000 7f222ed4c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222807a8b0 con 0x7f222807be80 2023-09-20T21:39:50.272 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.350+0000 7f222cae8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:50.272 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.350+0000 7f222cae8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32858/0 (socket says 172.21.15.22:32858) 2023-09-20T21:39:50.272 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.350+0000 7f222cae8700 1 -- 172.21.15.22:0/2979363123 learned_addr learned my addr 172.21.15.22:0/2979363123 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:50.272 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.351+0000 7f222cae8700 1 -- 172.21.15.22:0/2979363123 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222807a9f0 con 0x7f222807be80 2023-09-20T21:39:50.273 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.351+0000 7f222cae8700 1 --2- 172.21.15.22:0/2979363123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222807a2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f221c0098a0 tx=0x7f221c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c3470319cc162 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:50.273 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.351+0000 7f22277fe700 1 -- 172.21.15.22:0/2979363123 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f221c004160 con 0x7f222807be80 2023-09-20T21:39:50.273 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.351+0000 7f22277fe700 1 -- 172.21.15.22:0/2979363123 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f221c0042c0 con 0x7f222807be80 2023-09-20T21:39:50.273 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.351+0000 7f22277fe700 1 -- 172.21.15.22:0/2979363123 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f221c011a00 con 0x7f222807be80 2023-09-20T21:39:50.273 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.352+0000 7f222ed4c700 1 -- 172.21.15.22:0/2979363123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 msgr2=0x7f222807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:50.274 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.352+0000 7f222ed4c700 1 --2- 172.21.15.22:0/2979363123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222807a2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f221c0098a0 tx=0x7f221c009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:50.274 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.352+0000 7f222ed4c700 1 -- 172.21.15.22:0/2979363123 shutdown_connections 2023-09-20T21:39:50.274 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.352+0000 7f222ed4c700 1 --2- 172.21.15.22:0/2979363123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222807a2e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:50.274 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.352+0000 7f222ed4c700 1 -- 172.21.15.22:0/2979363123 >> 172.21.15.22:0/2979363123 conn(0x7f2228101b30 msgr2=0x7f2228103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:50.274 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.352+0000 7f222ed4c700 1 -- 172.21.15.22:0/2979363123 shutdown_connections 2023-09-20T21:39:50.275 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.352+0000 7f222ed4c700 1 -- 172.21.15.22:0/2979363123 wait complete. 2023-09-20T21:39:50.275 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.353+0000 7f222ed4c700 1 Processor -- start 2023-09-20T21:39:50.275 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.353+0000 7f222ed4c700 1 -- start start 2023-09-20T21:39:50.275 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.354+0000 7f222ed4c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:50.275 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.354+0000 7f222ed4c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222819ca40 con 0x7f222807be80 2023-09-20T21:39:50.276 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.354+0000 7f222cae8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:50.276 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.354+0000 7f222cae8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32866/0 (socket says 172.21.15.22:32866) 2023-09-20T21:39:50.276 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.354+0000 7f222cae8700 1 -- 172.21.15.22:0/56403731 learned_addr learned my addr 172.21.15.22:0/56403731 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:50.276 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.354+0000 7f222cae8700 1 -- 172.21.15.22:0/56403731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c009550 con 0x7f222807be80 2023-09-20T21:39:50.276 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.354+0000 7f222cae8700 1 --2- 172.21.15.22:0/56403731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222819c500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f221c009fb0 tx=0x7f221c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:50.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.355+0000 7f2225ffb700 1 -- 172.21.15.22:0/56403731 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f221c003a40 con 0x7f222807be80 2023-09-20T21:39:50.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.355+0000 7f2225ffb700 1 -- 172.21.15.22:0/56403731 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f221c003ba0 con 0x7f222807be80 2023-09-20T21:39:50.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.355+0000 7f222ed4c700 1 -- 172.21.15.22:0/56403731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222819cc40 con 0x7f222807be80 2023-09-20T21:39:50.277 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.355+0000 7f2225ffb700 1 -- 172.21.15.22:0/56403731 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f221c01a750 con 0x7f222807be80 2023-09-20T21:39:50.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.355+0000 7f222ed4c700 1 -- 172.21.15.22:0/56403731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222819d060 con 0x7f222807be80 2023-09-20T21:39:50.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.356+0000 7f222ed4c700 1 -- 172.21.15.22:0/56403731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2228196590 con 0x7f222807be80 2023-09-20T21:39:50.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.356+0000 7f2225ffb700 1 -- 172.21.15.22:0/56403731 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f221c024b60 con 0x7f222807be80 2023-09-20T21:39:50.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.357+0000 7f2225ffb700 1 --2- 172.21.15.22:0/56403731 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2218038750 0x7f221803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:50.278 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.357+0000 7f2225ffb700 1 -- 172.21.15.22:0/56403731 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f221c04c7b0 con 0x7f222807be80 2023-09-20T21:39:50.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.357+0000 7f2227fff700 1 --2- 172.21.15.22:0/56403731 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2218038750 0x7f221803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:50.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.357+0000 7f2227fff700 1 --2- 172.21.15.22:0/56403731 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2218038750 0x7f221803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2214006fd0 tx=0x7f2214006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:50.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.361+0000 7f2225ffb700 1 -- 172.21.15.22:0/56403731 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f221c018530 con 0x7f222807be80 2023-09-20T21:39:50.279 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:46.579+0000 7f222ed4c700 1 -- 172.21.15.22:0/56403731 --> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}) v1 -- 0x7f222802d0a0 con 0x7f2218038750 2023-09-20T21:39:50.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.861+0000 7f2225ffb700 1 -- 172.21.15.22:0/56403731 <== mgr.14122 v2:172.21.15.22:6800/3260510802 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f222802d0a0 con 0x7f2218038750 2023-09-20T21:39:50.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.864+0000 7f222ed4c700 1 -- 172.21.15.22:0/56403731 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2218038750 msgr2=0x7f221803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:50.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.864+0000 7f222ed4c700 1 --2- 172.21.15.22:0/56403731 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2218038750 0x7f221803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2214006fd0 tx=0x7f2214006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:50.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.864+0000 7f222ed4c700 1 -- 172.21.15.22:0/56403731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 msgr2=0x7f222819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:50.280 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.864+0000 7f222ed4c700 1 --2- 172.21.15.22:0/56403731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222819c500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f221c009fb0 tx=0x7f221c011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:50.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.864+0000 7f222ed4c700 1 -- 172.21.15.22:0/56403731 shutdown_connections 2023-09-20T21:39:50.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.864+0000 7f222ed4c700 1 --2- 172.21.15.22:0/56403731 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f2218038750 0x7f221803ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:50.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.864+0000 7f222ed4c700 1 --2- 172.21.15.22:0/56403731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f222807be80 0x7f222819c500 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:50.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.864+0000 7f222ed4c700 1 -- 172.21.15.22:0/56403731 >> 172.21.15.22:0/56403731 conn(0x7f2228101b30 msgr2=0x7f2228103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:50.281 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.864+0000 7f222ed4c700 1 -- 172.21.15.22:0/56403731 shutdown_connections 2023-09-20T21:39:50.282 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:49.865+0000 7f222ed4c700 1 -- 172.21.15.22:0/56403731 wait complete. 2023-09-20T21:39:50.282 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mon service... 2023-09-20T21:39:51.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:50 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:51.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:50 smithi022 ceph-mon[97651]: Added host smithi022 2023-09-20T21:39:51.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:50 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:39:51.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-09-20T21:39:51.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.826+0000 7f0db6d68700 1 Processor -- start 2023-09-20T21:39:51.757 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.826+0000 7f0db6d68700 1 -- start start 2023-09-20T21:39:51.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.826+0000 7f0db6d68700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:51.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.827+0000 7f0db6d68700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db007a8b0 con 0x7f0db007be80 2023-09-20T21:39:51.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.827+0000 7f0db4b04700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:51.758 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.827+0000 7f0db4b04700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32876/0 (socket says 172.21.15.22:32876) 2023-09-20T21:39:51.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.827+0000 7f0db4b04700 1 -- 172.21.15.22:0/150951474 learned_addr learned my addr 172.21.15.22:0/150951474 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:51.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.827+0000 7f0db4b04700 1 -- 172.21.15.22:0/150951474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db007a9f0 con 0x7f0db007be80 2023-09-20T21:39:51.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.828+0000 7f0db4b04700 1 --2- 172.21.15.22:0/150951474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db007a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0da40098a0 tx=0x7f0da4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3527020a5352b05b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:51.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.828+0000 7f0daf7fe700 1 -- 172.21.15.22:0/150951474 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0da4004160 con 0x7f0db007be80 2023-09-20T21:39:51.759 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.828+0000 7f0daf7fe700 1 -- 172.21.15.22:0/150951474 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0da40042c0 con 0x7f0db007be80 2023-09-20T21:39:51.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.828+0000 7f0daf7fe700 1 -- 172.21.15.22:0/150951474 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0da4011a00 con 0x7f0db007be80 2023-09-20T21:39:51.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.828+0000 7f0db6d68700 1 -- 172.21.15.22:0/150951474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 msgr2=0x7f0db007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:51.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.828+0000 7f0db6d68700 1 --2- 172.21.15.22:0/150951474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db007a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0da40098a0 tx=0x7f0da4009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:51.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.829+0000 7f0db6d68700 1 -- 172.21.15.22:0/150951474 shutdown_connections 2023-09-20T21:39:51.760 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.829+0000 7f0db6d68700 1 --2- 172.21.15.22:0/150951474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db007a2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:51.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.829+0000 7f0db6d68700 1 -- 172.21.15.22:0/150951474 >> 172.21.15.22:0/150951474 conn(0x7f0db0101b30 msgr2=0x7f0db0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:51.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.829+0000 7f0db6d68700 1 -- 172.21.15.22:0/150951474 shutdown_connections 2023-09-20T21:39:51.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.829+0000 7f0db6d68700 1 -- 172.21.15.22:0/150951474 wait complete. 2023-09-20T21:39:51.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.830+0000 7f0db6d68700 1 Processor -- start 2023-09-20T21:39:51.761 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.830+0000 7f0db6d68700 1 -- start start 2023-09-20T21:39:51.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.830+0000 7f0db6d68700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:51.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.830+0000 7f0db6d68700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db019ca40 con 0x7f0db007be80 2023-09-20T21:39:51.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.830+0000 7f0db4b04700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:51.762 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.831+0000 7f0db4b04700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32880/0 (socket says 172.21.15.22:32880) 2023-09-20T21:39:51.763 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.831+0000 7f0db4b04700 1 -- 172.21.15.22:0/3369639508 learned_addr learned my addr 172.21.15.22:0/3369639508 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:51.763 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.831+0000 7f0db4b04700 1 -- 172.21.15.22:0/3369639508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4009550 con 0x7f0db007be80 2023-09-20T21:39:51.763 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.831+0000 7f0db4b04700 1 --2- 172.21.15.22:0/3369639508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db019c500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009fb0 tx=0x7f0da4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:51.763 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.831+0000 7f0dadffb700 1 -- 172.21.15.22:0/3369639508 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0da4003a40 con 0x7f0db007be80 2023-09-20T21:39:51.763 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.831+0000 7f0dadffb700 1 -- 172.21.15.22:0/3369639508 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0da4003ba0 con 0x7f0db007be80 2023-09-20T21:39:51.764 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.831+0000 7f0db6d68700 1 -- 172.21.15.22:0/3369639508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db019cc40 con 0x7f0db007be80 2023-09-20T21:39:51.764 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.831+0000 7f0dadffb700 1 -- 172.21.15.22:0/3369639508 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0da401a750 con 0x7f0db007be80 2023-09-20T21:39:51.764 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.832+0000 7f0db6d68700 1 -- 172.21.15.22:0/3369639508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db019d060 con 0x7f0db007be80 2023-09-20T21:39:51.764 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.833+0000 7f0db6d68700 1 -- 172.21.15.22:0/3369639508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db0196590 con 0x7f0db007be80 2023-09-20T21:39:51.764 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.833+0000 7f0dadffb700 1 -- 172.21.15.22:0/3369639508 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f0da4024b60 con 0x7f0db007be80 2023-09-20T21:39:51.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.833+0000 7f0dadffb700 1 --2- 172.21.15.22:0/3369639508 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f0da00387a0 0x7f0da003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:51.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.833+0000 7f0dadffb700 1 -- 172.21.15.22:0/3369639508 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0da404c820 con 0x7f0db007be80 2023-09-20T21:39:51.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.834+0000 7f0daffff700 1 --2- 172.21.15.22:0/3369639508 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f0da00387a0 0x7f0da003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:51.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.834+0000 7f0daffff700 1 --2- 172.21.15.22:0/3369639508 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f0da00387a0 0x7f0da003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c006fd0 tx=0x7f0d9c006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:51.765 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:50.839+0000 7f0dadffb700 1 -- 172.21.15.22:0/3369639508 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0da4024430 con 0x7f0db007be80 2023-09-20T21:39:51.766 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.072+0000 7f0db6d68700 1 -- 172.21.15.22:0/3369639508 --> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0db002d0a0 con 0x7f0da00387a0 2023-09-20T21:39:51.766 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.077+0000 7f0dadffb700 1 -- 172.21.15.22:0/3369639508 <== mgr.14122 v2:172.21.15.22:6800/3260510802 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0db002d0a0 con 0x7f0da00387a0 2023-09-20T21:39:51.766 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.079+0000 7f0db6d68700 1 -- 172.21.15.22:0/3369639508 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f0da00387a0 msgr2=0x7f0da003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:51.766 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.079+0000 7f0db6d68700 1 --2- 172.21.15.22:0/3369639508 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f0da00387a0 0x7f0da003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c006fd0 tx=0x7f0d9c006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:51.766 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.079+0000 7f0db6d68700 1 -- 172.21.15.22:0/3369639508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 msgr2=0x7f0db019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:51.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.079+0000 7f0db6d68700 1 --2- 172.21.15.22:0/3369639508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db019c500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009fb0 tx=0x7f0da4011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:51.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.079+0000 7f0db6d68700 1 -- 172.21.15.22:0/3369639508 shutdown_connections 2023-09-20T21:39:51.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.079+0000 7f0db6d68700 1 --2- 172.21.15.22:0/3369639508 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f0da00387a0 0x7f0da003ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:51.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.079+0000 7f0db6d68700 1 --2- 172.21.15.22:0/3369639508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0db007be80 0x7f0db019c500 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:51.767 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.079+0000 7f0db6d68700 1 -- 172.21.15.22:0/3369639508 >> 172.21.15.22:0/3369639508 conn(0x7f0db0101b30 msgr2=0x7f0db0103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:51.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.080+0000 7f0db6d68700 1 -- 172.21.15.22:0/3369639508 shutdown_connections 2023-09-20T21:39:51.768 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:51.080+0000 7f0db6d68700 1 -- 172.21.15.22:0/3369639508 wait complete. 2023-09-20T21:39:51.768 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mgr service... 2023-09-20T21:39:52.258 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:52 smithi022 ceph-mon[97651]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:39:52.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:52 smithi022 ceph-mon[97651]: Saving service mon spec with placement count:5 2023-09-20T21:39:52.259 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:52 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-09-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.457+0000 7f47058af700 1 Processor -- start 2023-09-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.457+0000 7f47058af700 1 -- start start 2023-09-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.458+0000 7f47058af700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f47001069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.458+0000 7f47058af700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4700106fa0 con 0x7f47001065b0 2023-09-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.458+0000 7f46feffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f47001069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.458+0000 7f46feffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f47001069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32894/0 (socket says 172.21.15.22:32894) 2023-09-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.458+0000 7f46feffd700 1 -- 172.21.15.22:0/2265938657 learned_addr learned my addr 172.21.15.22:0/2265938657 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.459+0000 7f46feffd700 1 -- 172.21.15.22:0/2265938657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47001077b0 con 0x7f47001065b0 2023-09-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.459+0000 7f46feffd700 1 --2- 172.21.15.22:0/2265938657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f47001069d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f46f00098a0 tx=0x7f46f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b853497266ab9cdc server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:53.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.459+0000 7f46fdffb700 1 -- 172.21.15.22:0/2265938657 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0004160 con 0x7f47001065b0 2023-09-20T21:39:53.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.459+0000 7f46fdffb700 1 -- 172.21.15.22:0/2265938657 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46f00042c0 con 0x7f47001065b0 2023-09-20T21:39:53.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.459+0000 7f46fdffb700 1 -- 172.21.15.22:0/2265938657 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0011a00 con 0x7f47001065b0 2023-09-20T21:39:53.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.460+0000 7f47058af700 1 -- 172.21.15.22:0/2265938657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 msgr2=0x7f47001069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:53.242 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.460+0000 7f47058af700 1 --2- 172.21.15.22:0/2265938657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f47001069d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f46f00098a0 tx=0x7f46f0009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:53.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.460+0000 7f47058af700 1 -- 172.21.15.22:0/2265938657 shutdown_connections 2023-09-20T21:39:53.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.460+0000 7f47058af700 1 --2- 172.21.15.22:0/2265938657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f47001069d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:53.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.460+0000 7f47058af700 1 -- 172.21.15.22:0/2265938657 >> 172.21.15.22:0/2265938657 conn(0x7f4700101b30 msgr2=0x7f4700103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:53.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.460+0000 7f47058af700 1 -- 172.21.15.22:0/2265938657 shutdown_connections 2023-09-20T21:39:53.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.460+0000 7f47058af700 1 -- 172.21.15.22:0/2265938657 wait complete. 2023-09-20T21:39:53.243 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.461+0000 7f47058af700 1 Processor -- start 2023-09-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.461+0000 7f47058af700 1 -- start start 2023-09-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.462+0000 7f47058af700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f470019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.462+0000 7f47058af700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470019ca70 con 0x7f47001065b0 2023-09-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.462+0000 7f46feffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f470019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.462+0000 7f46feffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f470019c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32896/0 (socket says 172.21.15.22:32896) 2023-09-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.462+0000 7f46feffd700 1 -- 172.21.15.22:0/3100584824 learned_addr learned my addr 172.21.15.22:0/3100584824 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:53.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.462+0000 7f46feffd700 1 -- 172.21.15.22:0/3100584824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0009550 con 0x7f47001065b0 2023-09-20T21:39:53.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.462+0000 7f46feffd700 1 --2- 172.21.15.22:0/3100584824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f470019c530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009130 tx=0x7f46f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:53.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.463+0000 7f47048ad700 1 -- 172.21.15.22:0/3100584824 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0003ab0 con 0x7f47001065b0 2023-09-20T21:39:53.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.463+0000 7f47048ad700 1 -- 172.21.15.22:0/3100584824 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46f0003c10 con 0x7f47001065b0 2023-09-20T21:39:53.245 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.463+0000 7f47058af700 1 -- 172.21.15.22:0/3100584824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470019cc70 con 0x7f47001065b0 2023-09-20T21:39:53.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.463+0000 7f47048ad700 1 -- 172.21.15.22:0/3100584824 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f00247d0 con 0x7f47001065b0 2023-09-20T21:39:53.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.463+0000 7f47058af700 1 -- 172.21.15.22:0/3100584824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470019d090 con 0x7f47001065b0 2023-09-20T21:39:53.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.464+0000 7f47058af700 1 -- 172.21.15.22:0/3100584824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4700196790 con 0x7f47001065b0 2023-09-20T21:39:53.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.464+0000 7f47048ad700 1 -- 172.21.15.22:0/3100584824 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f46f001abb0 con 0x7f47001065b0 2023-09-20T21:39:53.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.465+0000 7f47048ad700 1 --2- 172.21.15.22:0/3100584824 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f46e8038840 0x7f46e803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:53.246 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.465+0000 7f47048ad700 1 -- 172.21.15.22:0/3100584824 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f46f004c9b0 con 0x7f47001065b0 2023-09-20T21:39:53.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.465+0000 7f46fe7fc700 1 --2- 172.21.15.22:0/3100584824 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f46e8038840 0x7f46e803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:53.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.466+0000 7f46fe7fc700 1 --2- 172.21.15.22:0/3100584824 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f46e8038840 0x7f46e803ad00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f46f4006fd0 tx=0x7f46f4006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:53.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.470+0000 7f47048ad700 1 -- 172.21.15.22:0/3100584824 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f46f0018530 con 0x7f47001065b0 2023-09-20T21:39:53.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.600+0000 7f47048ad700 1 -- 172.21.15.22:0/3100584824 <== mon.0 v2:172.21.15.22:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f004c300 con 0x7f47001065b0 2023-09-20T21:39:53.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.700+0000 7f47058af700 1 -- 172.21.15.22:0/3100584824 --> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f470002d0b0 con 0x7f46e8038840 2023-09-20T21:39:53.247 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.705+0000 7f47048ad700 1 -- 172.21.15.22:0/3100584824 <== mgr.14122 v2:172.21.15.22:6800/3260510802 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f470002d0b0 con 0x7f46e8038840 2023-09-20T21:39:53.248 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.708+0000 7f47058af700 1 -- 172.21.15.22:0/3100584824 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f46e8038840 msgr2=0x7f46e803ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:53.248 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.708+0000 7f47058af700 1 --2- 172.21.15.22:0/3100584824 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f46e8038840 0x7f46e803ad00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f46f4006fd0 tx=0x7f46f4006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:53.248 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.708+0000 7f47058af700 1 -- 172.21.15.22:0/3100584824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 msgr2=0x7f470019c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:53.248 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.708+0000 7f47058af700 1 --2- 172.21.15.22:0/3100584824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f470019c530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009130 tx=0x7f46f0011ad0 comp rx=0 tx=0).stop 2023-09-20T21:39:53.248 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.708+0000 7f47058af700 1 -- 172.21.15.22:0/3100584824 shutdown_connections 2023-09-20T21:39:53.249 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.708+0000 7f47058af700 1 --2- 172.21.15.22:0/3100584824 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f46e8038840 0x7f46e803ad00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:53.249 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.708+0000 7f47058af700 1 --2- 172.21.15.22:0/3100584824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f47001065b0 0x7f470019c530 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:53.249 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.708+0000 7f47058af700 1 -- 172.21.15.22:0/3100584824 >> 172.21.15.22:0/3100584824 conn(0x7f4700101b30 msgr2=0x7f4700103280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:53.249 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.708+0000 7f47058af700 1 -- 172.21.15.22:0/3100584824 shutdown_connections 2023-09-20T21:39:53.249 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:52.708+0000 7f47058af700 1 -- 172.21.15.22:0/3100584824 wait complete. 2023-09-20T21:39:53.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:53 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:53.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:53 smithi022 ceph-mon[97651]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:39:53.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:53 smithi022 ceph-mon[97651]: Saving service mgr spec with placement count:2 2023-09-20T21:39:53.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:53 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:53.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:53 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:54.828 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.817+0000 7f7d76623700 1 Processor -- start 2023-09-20T21:39:54.828 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.817+0000 7f7d76623700 1 -- start start 2023-09-20T21:39:54.829 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.817+0000 7f7d76623700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:54.829 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.817+0000 7f7d76623700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7007a8b0 con 0x7f7d7007be80 2023-09-20T21:39:54.829 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.818+0000 7f7d6ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:54.829 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.818+0000 7f7d6ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32908/0 (socket says 172.21.15.22:32908) 2023-09-20T21:39:54.829 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.818+0000 7f7d6ffff700 1 -- 172.21.15.22:0/930132674 learned_addr learned my addr 172.21.15.22:0/930132674 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:54.830 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.818+0000 7f7d6ffff700 1 -- 172.21.15.22:0/930132674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7007a9f0 con 0x7f7d7007be80 2023-09-20T21:39:54.830 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.819+0000 7f7d6ffff700 1 --2- 172.21.15.22:0/930132674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7007a2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7d600098a0 tx=0x7f7d60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c1a3bbaac4ee87 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:54.830 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.819+0000 7f7d6f7fe700 1 -- 172.21.15.22:0/930132674 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d60004160 con 0x7f7d7007be80 2023-09-20T21:39:54.830 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.819+0000 7f7d6f7fe700 1 -- 172.21.15.22:0/930132674 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d600042c0 con 0x7f7d7007be80 2023-09-20T21:39:54.830 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.819+0000 7f7d6f7fe700 1 -- 172.21.15.22:0/930132674 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d600119b0 con 0x7f7d7007be80 2023-09-20T21:39:54.831 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.819+0000 7f7d76623700 1 -- 172.21.15.22:0/930132674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 msgr2=0x7f7d7007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:54.831 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.819+0000 7f7d76623700 1 --2- 172.21.15.22:0/930132674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7007a2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7d600098a0 tx=0x7f7d60009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:54.831 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.820+0000 7f7d76623700 1 -- 172.21.15.22:0/930132674 shutdown_connections 2023-09-20T21:39:54.831 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.820+0000 7f7d76623700 1 --2- 172.21.15.22:0/930132674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7007a2e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:54.831 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.820+0000 7f7d76623700 1 -- 172.21.15.22:0/930132674 >> 172.21.15.22:0/930132674 conn(0x7f7d70101b30 msgr2=0x7f7d70103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:54.832 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.820+0000 7f7d76623700 1 -- 172.21.15.22:0/930132674 shutdown_connections 2023-09-20T21:39:54.832 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.820+0000 7f7d76623700 1 -- 172.21.15.22:0/930132674 wait complete. 2023-09-20T21:39:54.832 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.821+0000 7f7d76623700 1 Processor -- start 2023-09-20T21:39:54.832 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.821+0000 7f7d76623700 1 -- start start 2023-09-20T21:39:54.832 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.821+0000 7f7d76623700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:54.832 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.821+0000 7f7d76623700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7019cac0 con 0x7f7d7007be80 2023-09-20T21:39:54.833 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.821+0000 7f7d6ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:54.833 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.821+0000 7f7d6ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32924/0 (socket says 172.21.15.22:32924) 2023-09-20T21:39:54.833 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.821+0000 7f7d6ffff700 1 -- 172.21.15.22:0/46063323 learned_addr learned my addr 172.21.15.22:0/46063323 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:54.833 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.822+0000 7f7d6ffff700 1 -- 172.21.15.22:0/46063323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d60009550 con 0x7f7d7007be80 2023-09-20T21:39:54.833 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.822+0000 7f7d6ffff700 1 --2- 172.21.15.22:0/46063323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7019c580 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d60003e60 tx=0x7f7d60011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:54.834 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.822+0000 7f7d6dffb700 1 -- 172.21.15.22:0/46063323 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d60003a60 con 0x7f7d7007be80 2023-09-20T21:39:54.834 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.822+0000 7f7d6dffb700 1 -- 172.21.15.22:0/46063323 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d60003bc0 con 0x7f7d7007be80 2023-09-20T21:39:54.834 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.822+0000 7f7d76623700 1 -- 172.21.15.22:0/46063323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7019ccc0 con 0x7f7d7007be80 2023-09-20T21:39:54.834 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.822+0000 7f7d6dffb700 1 -- 172.21.15.22:0/46063323 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d600196c0 con 0x7f7d7007be80 2023-09-20T21:39:54.835 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.822+0000 7f7d76623700 1 -- 172.21.15.22:0/46063323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7019d0e0 con 0x7f7d7007be80 2023-09-20T21:39:54.835 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.823+0000 7f7d6dffb700 1 -- 172.21.15.22:0/46063323 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f7d6001ab60 con 0x7f7d7007be80 2023-09-20T21:39:54.835 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.823+0000 7f7d76623700 1 -- 172.21.15.22:0/46063323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d701965b0 con 0x7f7d7007be80 2023-09-20T21:39:54.835 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.824+0000 7f7d6dffb700 1 --2- 172.21.15.22:0/46063323 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f7d5c0387d0 0x7f7d5c03ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:54.836 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.824+0000 7f7d6dffb700 1 -- 172.21.15.22:0/46063323 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7d6004cad0 con 0x7f7d7007be80 2023-09-20T21:39:54.836 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.824+0000 7f7d67fff700 1 --2- 172.21.15.22:0/46063323 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f7d5c0387d0 0x7f7d5c03ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:54.836 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.825+0000 7f7d67fff700 1 --2- 172.21.15.22:0/46063323 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f7d5c0387d0 0x7f7d5c03ac90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006fd0 tx=0x7f7d58006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:54.836 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:53.829+0000 7f7d6dffb700 1 -- 172.21.15.22:0/46063323 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7d60010790 con 0x7f7d7007be80 2023-09-20T21:39:54.837 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.046+0000 7f7d76623700 1 -- 172.21.15.22:0/46063323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f7d70062410 con 0x7f7d7007be80 2023-09-20T21:39:54.837 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.049+0000 7f7d6dffb700 1 -- 172.21.15.22:0/46063323 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f7d60018350 con 0x7f7d7007be80 2023-09-20T21:39:54.838 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.052+0000 7f7d76623700 1 -- 172.21.15.22:0/46063323 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f7d5c0387d0 msgr2=0x7f7d5c03ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:54.838 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.052+0000 7f7d76623700 1 --2- 172.21.15.22:0/46063323 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f7d5c0387d0 0x7f7d5c03ac90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006fd0 tx=0x7f7d58006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:54.838 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.052+0000 7f7d76623700 1 -- 172.21.15.22:0/46063323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 msgr2=0x7f7d7019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:54.839 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.052+0000 7f7d76623700 1 --2- 172.21.15.22:0/46063323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7019c580 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d60003e60 tx=0x7f7d60011a30 comp rx=0 tx=0).stop 2023-09-20T21:39:54.839 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.053+0000 7f7d76623700 1 -- 172.21.15.22:0/46063323 shutdown_connections 2023-09-20T21:39:54.840 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.053+0000 7f7d76623700 1 --2- 172.21.15.22:0/46063323 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f7d5c0387d0 0x7f7d5c03ac90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:54.840 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.053+0000 7f7d76623700 1 --2- 172.21.15.22:0/46063323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7d7007be80 0x7f7d7019c580 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:54.840 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.053+0000 7f7d76623700 1 -- 172.21.15.22:0/46063323 >> 172.21.15.22:0/46063323 conn(0x7f7d70101b30 msgr2=0x7f7d70103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:54.841 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.053+0000 7f7d76623700 1 -- 172.21.15.22:0/46063323 shutdown_connections 2023-09-20T21:39:54.841 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:54.053+0000 7f7d76623700 1 -- 172.21.15.22:0/46063323 wait complete. 2023-09-20T21:39:55.362 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:55 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/46063323' entity='client.admin' 2023-09-20T21:39:56.309 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.657+0000 7fbb40e26700 1 Processor -- start 2023-09-20T21:39:56.309 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.657+0000 7fbb40e26700 1 -- start start 2023-09-20T21:39:56.309 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.657+0000 7fbb40e26700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c105280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:56.309 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.658+0000 7fbb40e26700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c079e30 con 0x7fbb3c104e60 2023-09-20T21:39:56.309 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.658+0000 7fbb3a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c105280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:56.310 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.658+0000 7fbb3a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c105280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32930/0 (socket says 172.21.15.22:32930) 2023-09-20T21:39:56.310 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.658+0000 7fbb3a59c700 1 -- 172.21.15.22:0/4085428809 learned_addr learned my addr 172.21.15.22:0/4085428809 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:56.310 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.658+0000 7fbb3a59c700 1 -- 172.21.15.22:0/4085428809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c1057c0 con 0x7fbb3c104e60 2023-09-20T21:39:56.310 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.659+0000 7fbb3a59c700 1 --2- 172.21.15.22:0/4085428809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c105280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098a0 tx=0x7fbb2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125f62201ec911bf server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:56.310 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.659+0000 7fbb3959a700 1 -- 172.21.15.22:0/4085428809 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb2c004160 con 0x7fbb3c104e60 2023-09-20T21:39:56.311 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.659+0000 7fbb3959a700 1 -- 172.21.15.22:0/4085428809 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb2c0042c0 con 0x7fbb3c104e60 2023-09-20T21:39:56.311 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.659+0000 7fbb3959a700 1 -- 172.21.15.22:0/4085428809 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb2c0119b0 con 0x7fbb3c104e60 2023-09-20T21:39:56.311 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.659+0000 7fbb40e26700 1 -- 172.21.15.22:0/4085428809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 msgr2=0x7fbb3c105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:56.311 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.660+0000 7fbb40e26700 1 --2- 172.21.15.22:0/4085428809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c105280 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0098a0 tx=0x7fbb2c009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:56.311 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.660+0000 7fbb40e26700 1 -- 172.21.15.22:0/4085428809 shutdown_connections 2023-09-20T21:39:56.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.660+0000 7fbb40e26700 1 --2- 172.21.15.22:0/4085428809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c105280 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:56.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.660+0000 7fbb40e26700 1 -- 172.21.15.22:0/4085428809 >> 172.21.15.22:0/4085428809 conn(0x7fbb3c100a20 msgr2=0x7fbb3c102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:56.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.660+0000 7fbb40e26700 1 -- 172.21.15.22:0/4085428809 shutdown_connections 2023-09-20T21:39:56.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.660+0000 7fbb40e26700 1 -- 172.21.15.22:0/4085428809 wait complete. 2023-09-20T21:39:56.315 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.661+0000 7fbb40e26700 1 Processor -- start 2023-09-20T21:39:56.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.662+0000 7fbb40e26700 1 -- start start 2023-09-20T21:39:56.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.662+0000 7fbb40e26700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:56.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.662+0000 7fbb40e26700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c19ca20 con 0x7fbb3c104e60 2023-09-20T21:39:56.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.662+0000 7fbb3a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:56.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.662+0000 7fbb3a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c19c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32934/0 (socket says 172.21.15.22:32934) 2023-09-20T21:39:56.316 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.662+0000 7fbb3a59c700 1 -- 172.21.15.22:0/2566749839 learned_addr learned my addr 172.21.15.22:0/2566749839 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:56.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.663+0000 7fbb3a59c700 1 -- 172.21.15.22:0/2566749839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c009550 con 0x7fbb3c104e60 2023-09-20T21:39:56.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.663+0000 7fbb3a59c700 1 --2- 172.21.15.22:0/2566749839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c19c4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c003f00 tx=0x7fbb2c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:56.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.663+0000 7fbb2b7fe700 1 -- 172.21.15.22:0/2566749839 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb2c003b20 con 0x7fbb3c104e60 2023-09-20T21:39:56.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.663+0000 7fbb2b7fe700 1 -- 172.21.15.22:0/2566749839 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb2c003c80 con 0x7fbb3c104e60 2023-09-20T21:39:56.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.663+0000 7fbb2b7fe700 1 -- 172.21.15.22:0/2566749839 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb2c019770 con 0x7fbb3c104e60 2023-09-20T21:39:56.317 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.663+0000 7fbb40e26700 1 -- 172.21.15.22:0/2566749839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c19cc20 con 0x7fbb3c104e60 2023-09-20T21:39:56.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.664+0000 7fbb40e26700 1 -- 172.21.15.22:0/2566749839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c19d040 con 0x7fbb3c104e60 2023-09-20T21:39:56.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.665+0000 7fbb2b7fe700 1 -- 172.21.15.22:0/2566749839 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45618+0+0 (secure 0 0 0) 0x7fbb2c01a430 con 0x7fbb3c104e60 2023-09-20T21:39:56.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.665+0000 7fbb40e26700 1 -- 172.21.15.22:0/2566749839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3c195e20 con 0x7fbb3c104e60 2023-09-20T21:39:56.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.665+0000 7fbb2b7fe700 1 --2- 172.21.15.22:0/2566749839 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fbb240387a0 0x7fbb2403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:56.318 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.665+0000 7fbb2b7fe700 1 -- 172.21.15.22:0/2566749839 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbb2c04cad0 con 0x7fbb3c104e60 2023-09-20T21:39:56.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.666+0000 7fbb39d9b700 1 --2- 172.21.15.22:0/2566749839 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fbb240387a0 0x7fbb2403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:56.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.666+0000 7fbb39d9b700 1 --2- 172.21.15.22:0/2566749839 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fbb240387a0 0x7fbb2403ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbb30006fd0 tx=0x7fbb30006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:56.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.673+0000 7fbb2b7fe700 1 -- 172.21.15.22:0/2566749839 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbb2c02d9a0 con 0x7fbb3c104e60 2023-09-20T21:39:56.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.882+0000 7fbb40e26700 1 -- 172.21.15.22:0/2566749839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fbb3c106670 con 0x7fbb3c104e60 2023-09-20T21:39:56.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.886+0000 7fbb2b7fe700 1 -- 172.21.15.22:0/2566749839 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fbb2c02d350 con 0x7fbb3c104e60 2023-09-20T21:39:56.319 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.890+0000 7fbb40e26700 1 -- 172.21.15.22:0/2566749839 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fbb240387a0 msgr2=0x7fbb2403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:56.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.890+0000 7fbb40e26700 1 --2- 172.21.15.22:0/2566749839 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fbb240387a0 0x7fbb2403ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbb30006fd0 tx=0x7fbb30006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:56.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.890+0000 7fbb40e26700 1 -- 172.21.15.22:0/2566749839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 msgr2=0x7fbb3c19c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:56.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.890+0000 7fbb40e26700 1 --2- 172.21.15.22:0/2566749839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c19c4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c003f00 tx=0x7fbb2c011a30 comp rx=0 tx=0).stop 2023-09-20T21:39:56.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.890+0000 7fbb40e26700 1 -- 172.21.15.22:0/2566749839 shutdown_connections 2023-09-20T21:39:56.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.890+0000 7fbb40e26700 1 --2- 172.21.15.22:0/2566749839 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fbb240387a0 0x7fbb2403ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:56.320 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.890+0000 7fbb40e26700 1 --2- 172.21.15.22:0/2566749839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb3c104e60 0x7fbb3c19c4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:56.321 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.890+0000 7fbb40e26700 1 -- 172.21.15.22:0/2566749839 >> 172.21.15.22:0/2566749839 conn(0x7fbb3c100a20 msgr2=0x7fbb3c190a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:56.321 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.890+0000 7fbb40e26700 1 -- 172.21.15.22:0/2566749839 shutdown_connections 2023-09-20T21:39:56.321 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:55.890+0000 7fbb40e26700 1 -- 172.21.15.22:0/2566749839 wait complete. 2023-09-20T21:39:56.321 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling the dashboard module... 2023-09-20T21:39:56.590 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:56 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:56.591 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:56 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2566749839' entity='client.admin' 2023-09-20T21:39:58.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:58 smithi022 ceph-mon[97651]: from='mgr.14122 172.21.15.22:0/1286897394' entity='mgr.x' 2023-09-20T21:39:58.293 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:58 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2828549837' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-09-20T21:39:58.293 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: ignoring --setuser ceph since I am not root 2023-09-20T21:39:58.293 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: ignoring --setgroup ceph since I am not root 2023-09-20T21:39:58.601 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:58.368+0000 7f5176f4f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-20T21:39:58.601 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:58.479+0000 7f5176f4f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-20T21:39:58.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.992+0000 7f6819cc4700 1 Processor -- start 2023-09-20T21:39:58.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.993+0000 7f6819cc4700 1 -- start start 2023-09-20T21:39:58.808 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.993+0000 7f6819cc4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f68141069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:58.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.993+0000 7f6819cc4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6814106fa0 con 0x7f68141065b0 2023-09-20T21:39:58.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.994+0000 7f68137fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f68141069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:58.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.994+0000 7f68137fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f68141069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50116/0 (socket says 172.21.15.22:50116) 2023-09-20T21:39:58.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.994+0000 7f68137fe700 1 -- 172.21.15.22:0/3058229982 learned_addr learned my addr 172.21.15.22:0/3058229982 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:58.809 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.994+0000 7f68137fe700 1 -- 172.21.15.22:0/3058229982 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68141077b0 con 0x7f68141065b0 2023-09-20T21:39:58.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.994+0000 7f68137fe700 1 --2- 172.21.15.22:0/3058229982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f68141069d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68040098a0 tx=0x7f6804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2311eb464e937b81 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:58.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.994+0000 7f68127fc700 1 -- 172.21.15.22:0/3058229982 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6804004160 con 0x7f68141065b0 2023-09-20T21:39:58.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.995+0000 7f68127fc700 1 -- 172.21.15.22:0/3058229982 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68040042c0 con 0x7f68141065b0 2023-09-20T21:39:58.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.995+0000 7f68127fc700 1 -- 172.21.15.22:0/3058229982 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68040119b0 con 0x7f68141065b0 2023-09-20T21:39:58.810 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.995+0000 7f6819cc4700 1 -- 172.21.15.22:0/3058229982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 msgr2=0x7f68141069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:58.811 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.995+0000 7f6819cc4700 1 --2- 172.21.15.22:0/3058229982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f68141069d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f68040098a0 tx=0x7f6804009bb0 comp rx=0 tx=0).stop 2023-09-20T21:39:58.811 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.995+0000 7f6819cc4700 1 -- 172.21.15.22:0/3058229982 shutdown_connections 2023-09-20T21:39:58.811 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.995+0000 7f6819cc4700 1 --2- 172.21.15.22:0/3058229982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f68141069d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:58.811 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.995+0000 7f6819cc4700 1 -- 172.21.15.22:0/3058229982 >> 172.21.15.22:0/3058229982 conn(0x7f6814101b30 msgr2=0x7f6814103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:58.811 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.995+0000 7f6819cc4700 1 -- 172.21.15.22:0/3058229982 shutdown_connections 2023-09-20T21:39:58.811 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.995+0000 7f6819cc4700 1 -- 172.21.15.22:0/3058229982 wait complete. 2023-09-20T21:39:58.812 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.996+0000 7f6819cc4700 1 Processor -- start 2023-09-20T21:39:58.812 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.997+0000 7f6819cc4700 1 -- start start 2023-09-20T21:39:58.812 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.997+0000 7f6819cc4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f681407bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:58.812 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.997+0000 7f6819cc4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681407c170 con 0x7f68141065b0 2023-09-20T21:39:58.812 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.997+0000 7f68137fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f681407bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:58.812 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.997+0000 7f68137fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f681407bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50126/0 (socket says 172.21.15.22:50126) 2023-09-20T21:39:58.813 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.997+0000 7f68137fe700 1 -- 172.21.15.22:0/2828549837 learned_addr learned my addr 172.21.15.22:0/2828549837 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:39:58.813 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.998+0000 7f68137fe700 1 -- 172.21.15.22:0/2828549837 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6804009550 con 0x7f68141065b0 2023-09-20T21:39:58.813 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.998+0000 7f68137fe700 1 --2- 172.21.15.22:0/2828549837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f681407bc30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6804003e60 tx=0x7f6804011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:58.813 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.998+0000 7f6810ff9700 1 -- 172.21.15.22:0/2828549837 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6804003a60 con 0x7f68141065b0 2023-09-20T21:39:58.813 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.998+0000 7f6810ff9700 1 -- 172.21.15.22:0/2828549837 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6804003bc0 con 0x7f68141065b0 2023-09-20T21:39:58.813 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.998+0000 7f6819cc4700 1 -- 172.21.15.22:0/2828549837 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681407a280 con 0x7f68141065b0 2023-09-20T21:39:58.814 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.998+0000 7f6810ff9700 1 -- 172.21.15.22:0/2828549837 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68040196c0 con 0x7f68141065b0 2023-09-20T21:39:58.814 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.998+0000 7f6819cc4700 1 -- 172.21.15.22:0/2828549837 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681407a6a0 con 0x7f68141065b0 2023-09-20T21:39:58.814 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.999+0000 7f6819cc4700 1 -- 172.21.15.22:0/2828549837 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681407b0b0 con 0x7f68141065b0 2023-09-20T21:39:58.814 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:56.999+0000 7f6810ff9700 1 -- 172.21.15.22:0/2828549837 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 8) v1 ==== 45618+0+0 (secure 0 0 0) 0x7f680401ab60 con 0x7f68141065b0 2023-09-20T21:39:58.814 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:57.000+0000 7f6810ff9700 1 --2- 172.21.15.22:0/2828549837 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f67fc038820 0x7f67fc03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:39:58.815 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:57.000+0000 7f6810ff9700 1 -- 172.21.15.22:0/2828549837 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f680404cad0 con 0x7f68141065b0 2023-09-20T21:39:58.815 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:57.000+0000 7f6812ffd700 1 --2- 172.21.15.22:0/2828549837 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f67fc038820 0x7f67fc03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:39:58.815 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:57.001+0000 7f6812ffd700 1 --2- 172.21.15.22:0/2828549837 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f67fc038820 0x7f67fc03ace0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6808006fd0 tx=0x7f6808006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:39:58.815 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:57.005+0000 7f6810ff9700 1 -- 172.21.15.22:0/2828549837 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6804010790 con 0x7f68141065b0 2023-09-20T21:39:58.815 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:57.288+0000 7f6819cc4700 1 -- 172.21.15.22:0/2828549837 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f681410a9b0 con 0x7f68141065b0 2023-09-20T21:39:58.816 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.039+0000 7f6810ff9700 1 -- 172.21.15.22:0/2828549837 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 9) v1 ==== 45631+0+0 (secure 0 0 0) 0x7f680401a430 con 0x7f68141065b0 2023-09-20T21:39:58.816 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.040+0000 7f6810ff9700 1 -- 172.21.15.22:0/2828549837 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f68040247d0 con 0x7f68141065b0 2023-09-20T21:39:58.816 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.043+0000 7f6819cc4700 1 -- 172.21.15.22:0/2828549837 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f67fc038820 msgr2=0x7f67fc03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:58.816 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.043+0000 7f6819cc4700 1 --2- 172.21.15.22:0/2828549837 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f67fc038820 0x7f67fc03ace0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6808006fd0 tx=0x7f6808006e40 comp rx=0 tx=0).stop 2023-09-20T21:39:58.816 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.043+0000 7f6819cc4700 1 -- 172.21.15.22:0/2828549837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 msgr2=0x7f681407bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:39:58.816 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.043+0000 7f6819cc4700 1 --2- 172.21.15.22:0/2828549837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f681407bc30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6804003e60 tx=0x7f6804011a30 comp rx=0 tx=0).stop 2023-09-20T21:39:58.817 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.043+0000 7f6819cc4700 1 -- 172.21.15.22:0/2828549837 shutdown_connections 2023-09-20T21:39:58.817 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.043+0000 7f6819cc4700 1 --2- 172.21.15.22:0/2828549837 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f67fc038820 0x7f67fc03ace0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:58.817 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.043+0000 7f6819cc4700 1 --2- 172.21.15.22:0/2828549837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68141065b0 0x7f681407bc30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:39:58.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.043+0000 7f6819cc4700 1 -- 172.21.15.22:0/2828549837 >> 172.21.15.22:0/2828549837 conn(0x7f6814101b30 msgr2=0x7f6814103300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:39:58.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.044+0000 7f6819cc4700 1 -- 172.21.15.22:0/2828549837 shutdown_connections 2023-09-20T21:39:58.818 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:58.044+0000 7f6819cc4700 1 -- 172.21.15.22:0/2828549837 wait complete. 2023-09-20T21:39:58.901 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:58.600+0000 7f5176f4f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-20T21:39:58.902 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:58.724+0000 7f5176f4f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-20T21:39:59.189 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:59 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2828549837' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-09-20T21:39:59.190 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:39:59 smithi022 ceph-mon[97651]: mgrmap e9: x(active, since 18s) 2023-09-20T21:39:59.190 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:58.972+0000 7f5176f4f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-20T21:39:59.747 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:59 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:59.458+0000 7f5176f4f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-20T21:39:59.748 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:59 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:59.631+0000 7f5176f4f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-20T21:40:00.039 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:59 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:59.752+0000 7f5176f4f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-20T21:40:00.040 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:39:59 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:39:59.906+0000 7f5176f4f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-20T21:40:00.197 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2023-09-20T21:40:00.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-09-20T21:40:00.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2023-09-20T21:40:00.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-09-20T21:40:00.203 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-09-20T21:40:00.204 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2023-09-20T21:40:00.204 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.468+0000 7f494b7dc700 1 Processor -- start 2023-09-20T21:40:00.204 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.468+0000 7f494b7dc700 1 -- start start 2023-09-20T21:40:00.204 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.469+0000 7f494b7dc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:00.204 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.469+0000 7f494b7dc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494407a890 con 0x7f494407be60 2023-09-20T21:40:00.205 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.469+0000 7f4949578700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:00.205 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.469+0000 7f4949578700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50156/0 (socket says 172.21.15.22:50156) 2023-09-20T21:40:00.205 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.469+0000 7f4949578700 1 -- 172.21.15.22:0/1302836266 learned_addr learned my addr 172.21.15.22:0/1302836266 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:00.205 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.470+0000 7f4949578700 1 -- 172.21.15.22:0/1302836266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494407a9d0 con 0x7f494407be60 2023-09-20T21:40:00.205 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.470+0000 7f4949578700 1 --2- 172.21.15.22:0/1302836266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494407a2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49400098a0 tx=0x7f4940009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e77ffccb696392a server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:00.206 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.470+0000 7f493bfff700 1 -- 172.21.15.22:0/1302836266 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4940004160 con 0x7f494407be60 2023-09-20T21:40:00.206 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.470+0000 7f493bfff700 1 -- 172.21.15.22:0/1302836266 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49400042c0 con 0x7f494407be60 2023-09-20T21:40:00.206 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.470+0000 7f493bfff700 1 -- 172.21.15.22:0/1302836266 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49400119b0 con 0x7f494407be60 2023-09-20T21:40:00.206 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.471+0000 7f494b7dc700 1 -- 172.21.15.22:0/1302836266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 msgr2=0x7f494407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:00.206 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.471+0000 7f494b7dc700 1 --2- 172.21.15.22:0/1302836266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494407a2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49400098a0 tx=0x7f4940009bb0 comp rx=0 tx=0).stop 2023-09-20T21:40:00.207 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.471+0000 7f494b7dc700 1 -- 172.21.15.22:0/1302836266 shutdown_connections 2023-09-20T21:40:00.207 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.471+0000 7f494b7dc700 1 --2- 172.21.15.22:0/1302836266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494407a2c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:00.207 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.471+0000 7f494b7dc700 1 -- 172.21.15.22:0/1302836266 >> 172.21.15.22:0/1302836266 conn(0x7f4944101b30 msgr2=0x7f4944103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:00.207 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.471+0000 7f494b7dc700 1 -- 172.21.15.22:0/1302836266 shutdown_connections 2023-09-20T21:40:00.207 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.471+0000 7f494b7dc700 1 -- 172.21.15.22:0/1302836266 wait complete. 2023-09-20T21:40:00.207 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.472+0000 7f494b7dc700 1 Processor -- start 2023-09-20T21:40:00.208 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.472+0000 7f494b7dc700 1 -- start start 2023-09-20T21:40:00.208 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.473+0000 7f494b7dc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:00.208 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.473+0000 7f494b7dc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494419ca60 con 0x7f494407be60 2023-09-20T21:40:00.208 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.473+0000 7f4949578700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:00.208 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.473+0000 7f4949578700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50168/0 (socket says 172.21.15.22:50168) 2023-09-20T21:40:00.209 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.473+0000 7f4949578700 1 -- 172.21.15.22:0/2730499795 learned_addr learned my addr 172.21.15.22:0/2730499795 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:00.209 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.473+0000 7f4949578700 1 -- 172.21.15.22:0/2730499795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940009550 con 0x7f494407be60 2023-09-20T21:40:00.209 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.473+0000 7f4949578700 1 --2- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494419c520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4940003f00 tx=0x7f4940011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:00.209 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.474+0000 7f493a7fc700 1 -- 172.21.15.22:0/2730499795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4940003b20 con 0x7f494407be60 2023-09-20T21:40:00.209 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.474+0000 7f493a7fc700 1 -- 172.21.15.22:0/2730499795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4940003c80 con 0x7f494407be60 2023-09-20T21:40:00.210 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.474+0000 7f493a7fc700 1 -- 172.21.15.22:0/2730499795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4940019770 con 0x7f494407be60 2023-09-20T21:40:00.210 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.474+0000 7f494b7dc700 1 -- 172.21.15.22:0/2730499795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494419cc60 con 0x7f494407be60 2023-09-20T21:40:00.210 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.474+0000 7f494b7dc700 1 -- 172.21.15.22:0/2730499795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494419d080 con 0x7f494407be60 2023-09-20T21:40:00.210 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.475+0000 7f493a7fc700 1 -- 172.21.15.22:0/2730499795 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) v1 ==== 45631+0+0 (secure 0 0 0) 0x7f494001ab60 con 0x7f494407be60 2023-09-20T21:40:00.210 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.475+0000 7f494b7dc700 1 -- 172.21.15.22:0/2730499795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49441965f0 con 0x7f494407be60 2023-09-20T21:40:00.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.476+0000 7f493a7fc700 1 --2- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f4930038840 0x7f493003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:00.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.476+0000 7f493a7fc700 1 -- 172.21.15.22:0/2730499795 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f494004cac0 con 0x7f494407be60 2023-09-20T21:40:00.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.476+0000 7f4948d77700 1 -- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f4930038840 msgr2=0x7f493003ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3260510802 2023-09-20T21:40:00.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.476+0000 7f4948d77700 1 --2- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f4930038840 0x7f493003ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-20T21:40:00.211 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.481+0000 7f493a7fc700 1 -- 172.21.15.22:0/2730499795 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f494004ee60 con 0x7f494407be60 2023-09-20T21:40:00.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.677+0000 7f4948d77700 1 -- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f4930038840 msgr2=0x7f493003ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3260510802 2023-09-20T21:40:00.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.677+0000 7f4948d77700 1 --2- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f4930038840 0x7f493003ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-20T21:40:00.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.751+0000 7f494b7dc700 1 -- 172.21.15.22:0/2730499795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f4944062410 con 0x7f494407be60 2023-09-20T21:40:00.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.752+0000 7f493a7fc700 1 -- 172.21.15.22:0/2730499795 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f494001a430 con 0x7f494407be60 2023-09-20T21:40:00.212 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.754+0000 7f494b7dc700 1 -- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f4930038840 msgr2=0x7f493003ad00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:40:00.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.754+0000 7f494b7dc700 1 --2- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f4930038840 0x7f493003ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:00.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.754+0000 7f494b7dc700 1 -- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 msgr2=0x7f494419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:00.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.754+0000 7f494b7dc700 1 --2- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494419c520 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4940003f00 tx=0x7f4940011a30 comp rx=0 tx=0).stop 2023-09-20T21:40:00.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.754+0000 7f494b7dc700 1 -- 172.21.15.22:0/2730499795 shutdown_connections 2023-09-20T21:40:00.213 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.755+0000 7f494b7dc700 1 --2- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7f4930038840 0x7f493003ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:00.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.755+0000 7f494b7dc700 1 --2- 172.21.15.22:0/2730499795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f494407be60 0x7f494419c520 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:00.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.755+0000 7f494b7dc700 1 -- 172.21.15.22:0/2730499795 >> 172.21.15.22:0/2730499795 conn(0x7f4944101b30 msgr2=0x7f49441032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:00.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.755+0000 7f494b7dc700 1 -- 172.21.15.22:0/2730499795 shutdown_connections 2023-09-20T21:40:00.214 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:39:59.755+0000 7f494b7dc700 1 -- 172.21.15.22:0/2730499795 wait complete. 2023-09-20T21:40:00.214 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2023-09-20T21:40:00.215 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 9... 2023-09-20T21:40:00.475 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:00 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2730499795' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-09-20T21:40:00.747 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:00 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:00.500+0000 7f5176f4f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-20T21:40:01.747 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:01 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:01.392+0000 7f5176f4f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-20T21:40:01.748 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:01 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:01.503+0000 7f5176f4f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-20T21:40:02.747 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:02 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:02.495+0000 7f5176f4f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-20T21:40:02.748 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:02 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:02.618+0000 7f5176f4f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-20T21:40:03.227 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:02 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:02.849+0000 7f5176f4f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-20T21:40:03.497 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:03.227+0000 7f5176f4f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-20T21:40:03.997 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:03.512+0000 7f5176f4f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-20T21:40:03.998 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:03.712+0000 7f5176f4f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-20T21:40:04.497 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:04 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:04.066+0000 7f5176f4f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-20T21:40:05.468 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:05.030+0000 7f5176f4f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-20T21:40:05.469 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:05.211+0000 7f5176f4f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-20T21:40:05.747 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:05.468+0000 7f5176f4f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-20T21:40:05.748 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:05.620+0000 7f5176f4f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-20T21:40:06.247 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:05.906+0000 7f5176f4f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-20T21:40:07.498 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:07.014+0000 7f5176f4f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-20T21:40:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: Active manager daemon x restarted 2023-09-20T21:40:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: Activating manager daemon x 2023-09-20T21:40:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: osdmap e3: 0 total, 0 up, 0 in 2023-09-20T21:40:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: mgrmap e10: x(active, starting, since 0.00369565s) 2023-09-20T21:40:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:40:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-09-20T21:40:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-09-20T21:40:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-09-20T21:40:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-09-20T21:40:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: Manager daemon x is now available 2023-09-20T21:40:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:08.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-09-20T21:40:08.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-09-20T21:40:08.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:08 smithi022 ceph-mon[97651]: mgrmap e11: x(active, since 1.00737s) 2023-09-20T21:40:08.511 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2023-09-20T21:40:08.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-09-20T21:40:08.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2023-09-20T21:40:08.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2023-09-20T21:40:08.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.757+0000 7fcc46037700 1 Processor -- start 2023-09-20T21:40:08.512 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.757+0000 7fcc46037700 1 -- start start 2023-09-20T21:40:08.513 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.757+0000 7fcc46037700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc40109400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:08.519 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.757+0000 7fcc46037700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40079e90 con 0x7fcc40107010 2023-09-20T21:40:08.520 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.758+0000 7fcc3f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc40109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:08.520 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.758+0000 7fcc3f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc40109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50180/0 (socket says 172.21.15.22:50180) 2023-09-20T21:40:08.520 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.758+0000 7fcc3f7fe700 1 -- 172.21.15.22:0/1437423507 learned_addr learned my addr 172.21.15.22:0/1437423507 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:08.520 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.758+0000 7fcc3f7fe700 1 -- 172.21.15.22:0/1437423507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc40109940 con 0x7fcc40107010 2023-09-20T21:40:08.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.759+0000 7fcc3f7fe700 1 --2- 172.21.15.22:0/1437423507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc40109400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc300098a0 tx=0x7fcc30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc81a10172dfb132 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:08.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.759+0000 7fcc3e7fc700 1 -- 172.21.15.22:0/1437423507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc30004160 con 0x7fcc40107010 2023-09-20T21:40:08.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.759+0000 7fcc3e7fc700 1 -- 172.21.15.22:0/1437423507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc300042c0 con 0x7fcc40107010 2023-09-20T21:40:08.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.759+0000 7fcc3e7fc700 1 -- 172.21.15.22:0/1437423507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc300119b0 con 0x7fcc40107010 2023-09-20T21:40:08.521 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.760+0000 7fcc46037700 1 -- 172.21.15.22:0/1437423507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 msgr2=0x7fcc40109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:08.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.760+0000 7fcc46037700 1 --2- 172.21.15.22:0/1437423507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc40109400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc300098a0 tx=0x7fcc30009bb0 comp rx=0 tx=0).stop 2023-09-20T21:40:08.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.760+0000 7fcc46037700 1 -- 172.21.15.22:0/1437423507 shutdown_connections 2023-09-20T21:40:08.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.760+0000 7fcc46037700 1 --2- 172.21.15.22:0/1437423507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc40109400 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:08.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.760+0000 7fcc46037700 1 -- 172.21.15.22:0/1437423507 >> 172.21.15.22:0/1437423507 conn(0x7fcc40100a20 msgr2=0x7fcc40102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:08.522 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.760+0000 7fcc46037700 1 -- 172.21.15.22:0/1437423507 shutdown_connections 2023-09-20T21:40:08.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.760+0000 7fcc46037700 1 -- 172.21.15.22:0/1437423507 wait complete. 2023-09-20T21:40:08.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.761+0000 7fcc46037700 1 Processor -- start 2023-09-20T21:40:08.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.762+0000 7fcc46037700 1 -- start start 2023-09-20T21:40:08.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.762+0000 7fcc46037700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc4019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:08.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.762+0000 7fcc46037700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4019a810 con 0x7fcc40107010 2023-09-20T21:40:08.523 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.762+0000 7fcc3f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc4019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:08.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.763+0000 7fcc3f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc4019a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50192/0 (socket says 172.21.15.22:50192) 2023-09-20T21:40:08.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.763+0000 7fcc3f7fe700 1 -- 172.21.15.22:0/1200389875 learned_addr learned my addr 172.21.15.22:0/1200389875 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:08.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.763+0000 7fcc3f7fe700 1 -- 172.21.15.22:0/1200389875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc30009550 con 0x7fcc40107010 2023-09-20T21:40:08.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.763+0000 7fcc3f7fe700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc4019a2d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcc30003f00 tx=0x7fcc30011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:08.524 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.763+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc30003b20 con 0x7fcc40107010 2023-09-20T21:40:08.525 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.764+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc30003c80 con 0x7fcc40107010 2023-09-20T21:40:08.525 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.764+0000 7fcc46037700 1 -- 172.21.15.22:0/1200389875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4019aa10 con 0x7fcc40107010 2023-09-20T21:40:08.525 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.764+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc30019770 con 0x7fcc40107010 2023-09-20T21:40:08.525 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.764+0000 7fcc46037700 1 -- 172.21.15.22:0/1200389875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4019ae30 con 0x7fcc40107010 2023-09-20T21:40:08.525 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.765+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 9) v1 ==== 45631+0+0 (secure 0 0 0) 0x7fcc3001a430 con 0x7fcc40107010 2023-09-20T21:40:08.526 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.765+0000 7fcc3cff9700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 0x7fcc2803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:08.526 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.765+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 --> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc2803b410 con 0x7fcc28038840 2023-09-20T21:40:08.526 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.765+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcc3004cbc0 con 0x7fcc40107010 2023-09-20T21:40:08.526 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.765+0000 7fcc3effd700 1 -- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 msgr2=0x7fcc2803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3260510802 2023-09-20T21:40:08.527 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.765+0000 7fcc3effd700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 0x7fcc2803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-20T21:40:08.527 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.967+0000 7fcc3effd700 1 -- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 msgr2=0x7fcc2803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3260510802 2023-09-20T21:40:08.527 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:00.967+0000 7fcc3effd700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 0x7fcc2803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-20T21:40:08.527 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:01.368+0000 7fcc3effd700 1 -- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 msgr2=0x7fcc2803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3260510802 2023-09-20T21:40:08.527 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:01.368+0000 7fcc3effd700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 0x7fcc2803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-20T21:40:08.527 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:02.169+0000 7fcc3effd700 1 -- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 msgr2=0x7fcc2803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3260510802 2023-09-20T21:40:08.528 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:02.169+0000 7fcc3effd700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 0x7fcc2803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-20T21:40:08.528 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:03.770+0000 7fcc3effd700 1 -- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 msgr2=0x7fcc2803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3260510802 2023-09-20T21:40:08.528 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:03.771+0000 7fcc3effd700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 0x7fcc2803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-09-20T21:40:08.528 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:06.974+0000 7fcc3effd700 1 -- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 msgr2=0x7fcc2803ad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.22:6800/3260510802 2023-09-20T21:40:08.528 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:06.974+0000 7fcc3effd700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 0x7fcc2803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-09-20T21:40:08.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:07.018+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mgrmap(e 10) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fcc30019a80 con 0x7fcc40107010 2023-09-20T21:40:08.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:07.018+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 msgr2=0x7fcc2803ad00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:40:08.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:07.018+0000 7fcc3cff9700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 0x7fcc2803ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:08.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.021+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 11) v1 ==== 45525+0+0 (secure 0 0 0) 0x7fcc3000ed40 con 0x7fcc40107010 2023-09-20T21:40:08.529 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.021+0000 7fcc3cff9700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcc2803c180 0x7fcc2803e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:08.530 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.021+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc2803b410 con 0x7fcc2803c180 2023-09-20T21:40:08.530 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.022+0000 7fcc3effd700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcc2803c180 0x7fcc2803e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:08.530 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.023+0000 7fcc3effd700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcc2803c180 0x7fcc2803e570 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc34003a10 tx=0x7fcc34009270 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:08.530 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.024+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fcc2803b410 con 0x7fcc2803c180 2023-09-20T21:40:08.530 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.031+0000 7fcc46037700 1 -- 172.21.15.22:0/1200389875 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fcc4019b610 con 0x7fcc2803c180 2023-09-20T21:40:08.531 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.031+0000 7fcc3cff9700 1 -- 172.21.15.22:0/1200389875 <== mgr.14154 v2:172.21.15.22:6800/718010003 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fcc4019b610 con 0x7fcc2803c180 2023-09-20T21:40:08.531 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.031+0000 7fcc46037700 1 -- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcc2803c180 msgr2=0x7fcc2803e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:08.531 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.032+0000 7fcc46037700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcc2803c180 0x7fcc2803e570 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcc34003a10 tx=0x7fcc34009270 comp rx=0 tx=0).stop 2023-09-20T21:40:08.531 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.032+0000 7fcc46037700 1 -- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 msgr2=0x7fcc4019a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:08.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.032+0000 7fcc46037700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc4019a2d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcc30003f00 tx=0x7fcc30011a30 comp rx=0 tx=0).stop 2023-09-20T21:40:08.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.032+0000 7fcc46037700 1 -- 172.21.15.22:0/1200389875 shutdown_connections 2023-09-20T21:40:08.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.032+0000 7fcc46037700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcc2803c180 0x7fcc2803e570 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:08.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.032+0000 7fcc46037700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcc40107010 0x7fcc4019a2d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:08.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.032+0000 7fcc46037700 1 --2- 172.21.15.22:0/1200389875 >> [v2:172.21.15.22:6800/3260510802,v1:172.21.15.22:6801/3260510802] conn(0x7fcc28038840 0x7fcc2803ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:08.532 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.032+0000 7fcc46037700 1 -- 172.21.15.22:0/1200389875 >> 172.21.15.22:0/1200389875 conn(0x7fcc40100a20 msgr2=0x7fcc401021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:08.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.032+0000 7fcc46037700 1 -- 172.21.15.22:0/1200389875 shutdown_connections 2023-09-20T21:40:08.533 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:08.032+0000 7fcc46037700 1 -- 172.21.15.22:0/1200389875 wait complete. 2023-09-20T21:40:08.533 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 9 is available 2023-09-20T21:40:08.533 INFO:teuthology.orchestra.run.smithi022.stdout:Generating a dashboard self-signed certificate... 2023-09-20T21:40:09.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:09 smithi022 ceph-mon[97651]: [20/Sep/2023:21:40:07] ENGINE Bus STARTING 2023-09-20T21:40:09.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:09 smithi022 ceph-mon[97651]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-09-20T21:40:09.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:09 smithi022 ceph-mon[97651]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-09-20T21:40:09.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:09 smithi022 ceph-mon[97651]: [20/Sep/2023:21:40:08] ENGINE Serving on https://172.21.15.22:7150 2023-09-20T21:40:10.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-09-20T21:40:10.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.215+0000 7f43d9cb0700 1 Processor -- start 2023-09-20T21:40:10.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.215+0000 7f43d9cb0700 1 -- start start 2023-09-20T21:40:10.019 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.216+0000 7f43d9cb0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d41069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:10.019 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.216+0000 7f43d9cb0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d4106fa0 con 0x7f43d41065b0 2023-09-20T21:40:10.019 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.216+0000 7f43d37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d41069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:10.019 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.216+0000 7f43d37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d41069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45342/0 (socket says 172.21.15.22:45342) 2023-09-20T21:40:10.019 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.216+0000 7f43d37fe700 1 -- 172.21.15.22:0/1040913940 learned_addr learned my addr 172.21.15.22:0/1040913940 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:10.020 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.217+0000 7f43d37fe700 1 -- 172.21.15.22:0/1040913940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d41077b0 con 0x7f43d41065b0 2023-09-20T21:40:10.020 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.217+0000 7f43d37fe700 1 --2- 172.21.15.22:0/1040913940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d41069d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f43c40098a0 tx=0x7f43c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4856c926d206296d server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:10.020 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.217+0000 7f43d27fc700 1 -- 172.21.15.22:0/1040913940 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c4004160 con 0x7f43d41065b0 2023-09-20T21:40:10.020 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.217+0000 7f43d27fc700 1 -- 172.21.15.22:0/1040913940 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c40042c0 con 0x7f43d41065b0 2023-09-20T21:40:10.020 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.217+0000 7f43d27fc700 1 -- 172.21.15.22:0/1040913940 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c40119b0 con 0x7f43d41065b0 2023-09-20T21:40:10.021 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.218+0000 7f43d9cb0700 1 -- 172.21.15.22:0/1040913940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 msgr2=0x7f43d41069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:10.021 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.218+0000 7f43d9cb0700 1 --2- 172.21.15.22:0/1040913940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d41069d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f43c40098a0 tx=0x7f43c4009bb0 comp rx=0 tx=0).stop 2023-09-20T21:40:10.021 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.218+0000 7f43d9cb0700 1 -- 172.21.15.22:0/1040913940 shutdown_connections 2023-09-20T21:40:10.021 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.218+0000 7f43d9cb0700 1 --2- 172.21.15.22:0/1040913940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d41069d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:10.021 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.218+0000 7f43d9cb0700 1 -- 172.21.15.22:0/1040913940 >> 172.21.15.22:0/1040913940 conn(0x7f43d4101b30 msgr2=0x7f43d4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:10.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.218+0000 7f43d9cb0700 1 -- 172.21.15.22:0/1040913940 shutdown_connections 2023-09-20T21:40:10.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.218+0000 7f43d9cb0700 1 -- 172.21.15.22:0/1040913940 wait complete. 2023-09-20T21:40:10.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.219+0000 7f43d9cb0700 1 Processor -- start 2023-09-20T21:40:10.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.219+0000 7f43d9cb0700 1 -- start start 2023-09-20T21:40:10.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.219+0000 7f43d9cb0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:10.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.219+0000 7f43d9cb0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d419ca90 con 0x7f43d41065b0 2023-09-20T21:40:10.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.219+0000 7f43d37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:10.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.220+0000 7f43d37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45346/0 (socket says 172.21.15.22:45346) 2023-09-20T21:40:10.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.220+0000 7f43d37fe700 1 -- 172.21.15.22:0/2599644031 learned_addr learned my addr 172.21.15.22:0/2599644031 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:10.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.220+0000 7f43d37fe700 1 -- 172.21.15.22:0/2599644031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4009550 con 0x7f43d41065b0 2023-09-20T21:40:10.024 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.220+0000 7f43d37fe700 1 --2- 172.21.15.22:0/2599644031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d419c550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43c4006af0 tx=0x7f43c4011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:10.024 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.220+0000 7f43d0ff9700 1 -- 172.21.15.22:0/2599644031 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c4003a40 con 0x7f43d41065b0 2023-09-20T21:40:10.024 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.220+0000 7f43d0ff9700 1 -- 172.21.15.22:0/2599644031 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c4003ba0 con 0x7f43d41065b0 2023-09-20T21:40:10.024 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.220+0000 7f43d0ff9700 1 -- 172.21.15.22:0/2599644031 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c4019690 con 0x7f43d41065b0 2023-09-20T21:40:10.024 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.220+0000 7f43d9cb0700 1 -- 172.21.15.22:0/2599644031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d419cc90 con 0x7f43d41065b0 2023-09-20T21:40:10.025 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.221+0000 7f43d9cb0700 1 -- 172.21.15.22:0/2599644031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d419d0b0 con 0x7f43d41065b0 2023-09-20T21:40:10.025 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.222+0000 7f43d0ff9700 1 -- 172.21.15.22:0/2599644031 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 45631+0+0 (secure 0 0 0) 0x7f43c401aa60 con 0x7f43d41065b0 2023-09-20T21:40:10.025 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.222+0000 7f43d9cb0700 1 -- 172.21.15.22:0/2599644031 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d4196850 con 0x7f43d41065b0 2023-09-20T21:40:10.025 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.222+0000 7f43d0ff9700 1 --2- 172.21.15.22:0/2599644031 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f43bc0387f0 0x7f43bc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:10.025 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.222+0000 7f43d0ff9700 1 -- 172.21.15.22:0/2599644031 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f43c404cc30 con 0x7f43d41065b0 2023-09-20T21:40:10.026 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.222+0000 7f43d2ffd700 1 --2- 172.21.15.22:0/2599644031 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f43bc0387f0 0x7f43bc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:10.026 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.223+0000 7f43d2ffd700 1 --2- 172.21.15.22:0/2599644031 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f43bc0387f0 0x7f43bc03acb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f43c8006fd0 tx=0x7f43c8006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:10.026 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.227+0000 7f43d0ff9700 1 -- 172.21.15.22:0/2599644031 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f43c40237a0 con 0x7f43d41065b0 2023-09-20T21:40:10.026 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.463+0000 7f43d9cb0700 1 -- 172.21.15.22:0/2599644031 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f43d402d000 con 0x7f43bc0387f0 2023-09-20T21:40:10.026 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.552+0000 7f43d0ff9700 1 -- 172.21.15.22:0/2599644031 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f43d402d000 con 0x7f43bc0387f0 2023-09-20T21:40:10.027 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.554+0000 7f43d9cb0700 1 -- 172.21.15.22:0/2599644031 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f43bc0387f0 msgr2=0x7f43bc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:10.027 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.554+0000 7f43d9cb0700 1 --2- 172.21.15.22:0/2599644031 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f43bc0387f0 0x7f43bc03acb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f43c8006fd0 tx=0x7f43c8006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:10.027 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.554+0000 7f43d9cb0700 1 -- 172.21.15.22:0/2599644031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 msgr2=0x7f43d419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:10.027 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.554+0000 7f43d9cb0700 1 --2- 172.21.15.22:0/2599644031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d419c550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43c4006af0 tx=0x7f43c4011a10 comp rx=0 tx=0).stop 2023-09-20T21:40:10.027 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.554+0000 7f43d9cb0700 1 -- 172.21.15.22:0/2599644031 shutdown_connections 2023-09-20T21:40:10.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.554+0000 7f43d9cb0700 1 --2- 172.21.15.22:0/2599644031 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f43bc0387f0 0x7f43bc03acb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:10.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.554+0000 7f43d9cb0700 1 --2- 172.21.15.22:0/2599644031 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43d41065b0 0x7f43d419c550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:10.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.554+0000 7f43d9cb0700 1 -- 172.21.15.22:0/2599644031 >> 172.21.15.22:0/2599644031 conn(0x7f43d4101b30 msgr2=0x7f43d41032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:10.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.555+0000 7f43d9cb0700 1 -- 172.21.15.22:0/2599644031 shutdown_connections 2023-09-20T21:40:10.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:09.555+0000 7f43d9cb0700 1 -- 172.21.15.22:0/2599644031 wait complete. 2023-09-20T21:40:10.029 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial admin user... 2023-09-20T21:40:10.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:10 smithi022 ceph-mon[97651]: [20/Sep/2023:21:40:08] ENGINE Serving on http://172.21.15.22:8765 2023-09-20T21:40:10.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:10 smithi022 ceph-mon[97651]: [20/Sep/2023:21:40:08] ENGINE Bus STARTED 2023-09-20T21:40:10.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:10 smithi022 ceph-mon[97651]: mgrmap e12: x(active, since 2s) 2023-09-20T21:40:10.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:10.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:10.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:11.331 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:11 smithi022 ceph-mon[97651]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:40:11.332 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$KTeNiP5J9zrpt/.SPBpEhO6nFgimQpJ9ihNvCG1m3UGL6xCSetCEa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1695246011, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-09-20T21:40:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.666+0000 7f4fa46c0700 1 Processor -- start 2023-09-20T21:40:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.666+0000 7f4fa46c0700 1 -- start start 2023-09-20T21:40:11.608 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.666+0000 7f4fa46c0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.666+0000 7f4fa46c0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c07a890 con 0x7f4f9c07be60 2023-09-20T21:40:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.667+0000 7f4fa245c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.667+0000 7f4fa245c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45354/0 (socket says 172.21.15.22:45354) 2023-09-20T21:40:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.667+0000 7f4fa245c700 1 -- 172.21.15.22:0/2854488950 learned_addr learned my addr 172.21.15.22:0/2854488950 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:11.609 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.667+0000 7f4fa245c700 1 -- 172.21.15.22:0/2854488950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c07a9d0 con 0x7f4f9c07be60 2023-09-20T21:40:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.667+0000 7f4fa245c700 1 --2- 172.21.15.22:0/2854488950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c07a2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f980098a0 tx=0x7f4f98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c353bcfe51acac server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.668+0000 7f4fa145a700 1 -- 172.21.15.22:0/2854488950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f98004160 con 0x7f4f9c07be60 2023-09-20T21:40:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.668+0000 7f4fa145a700 1 -- 172.21.15.22:0/2854488950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f980042c0 con 0x7f4f9c07be60 2023-09-20T21:40:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.668+0000 7f4fa145a700 1 -- 172.21.15.22:0/2854488950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f980119b0 con 0x7f4f9c07be60 2023-09-20T21:40:11.610 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.668+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2854488950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 msgr2=0x7f4f9c07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:11.611 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.668+0000 7f4fa46c0700 1 --2- 172.21.15.22:0/2854488950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c07a2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f980098a0 tx=0x7f4f98009bb0 comp rx=0 tx=0).stop 2023-09-20T21:40:11.611 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.669+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2854488950 shutdown_connections 2023-09-20T21:40:11.611 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.669+0000 7f4fa46c0700 1 --2- 172.21.15.22:0/2854488950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c07a2c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:11.611 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.669+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2854488950 >> 172.21.15.22:0/2854488950 conn(0x7f4f9c101b30 msgr2=0x7f4f9c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:11.611 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.669+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2854488950 shutdown_connections 2023-09-20T21:40:11.612 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.669+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2854488950 wait complete. 2023-09-20T21:40:11.612 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.670+0000 7f4fa46c0700 1 Processor -- start 2023-09-20T21:40:11.612 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.671+0000 7f4fa46c0700 1 -- start start 2023-09-20T21:40:11.612 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.671+0000 7f4fa46c0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:11.612 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.671+0000 7f4fa46c0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c19ca80 con 0x7f4f9c07be60 2023-09-20T21:40:11.617 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.671+0000 7f4fa245c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:11.619 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.671+0000 7f4fa245c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45356/0 (socket says 172.21.15.22:45356) 2023-09-20T21:40:11.619 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.671+0000 7f4fa245c700 1 -- 172.21.15.22:0/2552150278 learned_addr learned my addr 172.21.15.22:0/2552150278 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:11.619 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.672+0000 7f4fa245c700 1 -- 172.21.15.22:0/2552150278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f98009550 con 0x7f4f9c07be60 2023-09-20T21:40:11.619 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.672+0000 7f4fa245c700 1 --2- 172.21.15.22:0/2552150278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c19c540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f98003f00 tx=0x7f4f98011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:11.620 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.672+0000 7f4f937fe700 1 -- 172.21.15.22:0/2552150278 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f98003b20 con 0x7f4f9c07be60 2023-09-20T21:40:11.620 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.672+0000 7f4f937fe700 1 -- 172.21.15.22:0/2552150278 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f98003c80 con 0x7f4f9c07be60 2023-09-20T21:40:11.620 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.672+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2552150278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c19cc80 con 0x7f4f9c07be60 2023-09-20T21:40:11.620 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.672+0000 7f4f937fe700 1 -- 172.21.15.22:0/2552150278 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f98019770 con 0x7f4f9c07be60 2023-09-20T21:40:11.620 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.672+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2552150278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c19d0a0 con 0x7f4f9c07be60 2023-09-20T21:40:11.621 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.673+0000 7f4f937fe700 1 -- 172.21.15.22:0/2552150278 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 45631+0+0 (secure 0 0 0) 0x7f4f980199e0 con 0x7f4f9c07be60 2023-09-20T21:40:11.621 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.673+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2552150278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c1965f0 con 0x7f4f9c07be60 2023-09-20T21:40:11.621 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.674+0000 7f4f937fe700 1 --2- 172.21.15.22:0/2552150278 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f88038840 0x7f4f8803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:11.621 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.674+0000 7f4f937fe700 1 -- 172.21.15.22:0/2552150278 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4f9804cc10 con 0x7f4f9c07be60 2023-09-20T21:40:11.621 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.674+0000 7f4fa1c5b700 1 --2- 172.21.15.22:0/2552150278 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f88038840 0x7f4f8803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:11.622 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.675+0000 7f4fa1c5b700 1 --2- 172.21.15.22:0/2552150278 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f88038840 0x7f4f8803ad00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006fd0 tx=0x7f4f8c006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:11.622 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.681+0000 7f4f937fe700 1 -- 172.21.15.22:0/2552150278 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4f98018b80 con 0x7f4f9c07be60 2023-09-20T21:40:11.622 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:10.921+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2552150278 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4f9c062230 con 0x7f4f88038840 2023-09-20T21:40:11.622 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.168+0000 7f4f937fe700 1 -- 172.21.15.22:0/2552150278 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f4f9c062230 con 0x7f4f88038840 2023-09-20T21:40:11.622 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.170+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2552150278 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f88038840 msgr2=0x7f4f8803ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:11.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.170+0000 7f4fa46c0700 1 --2- 172.21.15.22:0/2552150278 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f88038840 0x7f4f8803ad00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006fd0 tx=0x7f4f8c006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:11.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.170+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2552150278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 msgr2=0x7f4f9c19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:11.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.170+0000 7f4fa46c0700 1 --2- 172.21.15.22:0/2552150278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c19c540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f98003f00 tx=0x7f4f98011a30 comp rx=0 tx=0).stop 2023-09-20T21:40:11.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.170+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2552150278 shutdown_connections 2023-09-20T21:40:11.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.170+0000 7f4fa46c0700 1 --2- 172.21.15.22:0/2552150278 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f88038840 0x7f4f8803ad00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:11.623 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.170+0000 7f4fa46c0700 1 --2- 172.21.15.22:0/2552150278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9c07be60 0x7f4f9c19c540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:11.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.170+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2552150278 >> 172.21.15.22:0/2552150278 conn(0x7f4f9c101b30 msgr2=0x7f4f9c1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:11.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.170+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2552150278 shutdown_connections 2023-09-20T21:40:11.624 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:11.170+0000 7f4fa46c0700 1 -- 172.21.15.22:0/2552150278 wait complete. 2023-09-20T21:40:11.624 INFO:teuthology.orchestra.run.smithi022.stdout:Fetching dashboard port number... 2023-09-20T21:40:12.168 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:12 smithi022 ceph-mon[97651]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:40:12.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:13.105 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 8443 2023-09-20T21:40:13.105 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.167+0000 7f8a70f4c700 1 Processor -- start 2023-09-20T21:40:13.105 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.167+0000 7f8a70f4c700 1 -- start start 2023-09-20T21:40:13.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.168+0000 7f8a70f4c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:13.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.168+0000 7f8a70f4c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c07a890 con 0x7f8a6c07be60 2023-09-20T21:40:13.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.169+0000 7f8a6a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:13.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.169+0000 7f8a6a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45360/0 (socket says 172.21.15.22:45360) 2023-09-20T21:40:13.106 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.169+0000 7f8a6a59c700 1 -- 172.21.15.22:0/2520515639 learned_addr learned my addr 172.21.15.22:0/2520515639 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:13.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.170+0000 7f8a6a59c700 1 -- 172.21.15.22:0/2520515639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c07a9d0 con 0x7f8a6c07be60 2023-09-20T21:40:13.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.170+0000 7f8a6a59c700 1 --2- 172.21.15.22:0/2520515639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c07a2c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c0098a0 tx=0x7f8a5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e52683caf2a1bdc server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:13.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.171+0000 7f8a6959a700 1 -- 172.21.15.22:0/2520515639 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a5c004160 con 0x7f8a6c07be60 2023-09-20T21:40:13.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.171+0000 7f8a6959a700 1 -- 172.21.15.22:0/2520515639 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a5c0042c0 con 0x7f8a6c07be60 2023-09-20T21:40:13.107 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.171+0000 7f8a6959a700 1 -- 172.21.15.22:0/2520515639 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a5c0119b0 con 0x7f8a6c07be60 2023-09-20T21:40:13.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.172+0000 7f8a70f4c700 1 -- 172.21.15.22:0/2520515639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 msgr2=0x7f8a6c07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:13.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.172+0000 7f8a70f4c700 1 --2- 172.21.15.22:0/2520515639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c07a2c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c0098a0 tx=0x7f8a5c009bb0 comp rx=0 tx=0).stop 2023-09-20T21:40:13.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.172+0000 7f8a70f4c700 1 -- 172.21.15.22:0/2520515639 shutdown_connections 2023-09-20T21:40:13.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.172+0000 7f8a70f4c700 1 --2- 172.21.15.22:0/2520515639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c07a2c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:13.108 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.172+0000 7f8a70f4c700 1 -- 172.21.15.22:0/2520515639 >> 172.21.15.22:0/2520515639 conn(0x7f8a6c101b30 msgr2=0x7f8a6c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:13.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.172+0000 7f8a70f4c700 1 -- 172.21.15.22:0/2520515639 shutdown_connections 2023-09-20T21:40:13.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.173+0000 7f8a70f4c700 1 -- 172.21.15.22:0/2520515639 wait complete. 2023-09-20T21:40:13.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.174+0000 7f8a70f4c700 1 Processor -- start 2023-09-20T21:40:13.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.174+0000 7f8a70f4c700 1 -- start start 2023-09-20T21:40:13.109 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.174+0000 7f8a70f4c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:13.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.174+0000 7f8a70f4c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c19ca60 con 0x7f8a6c07be60 2023-09-20T21:40:13.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.175+0000 7f8a6a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:13.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.175+0000 7f8a6a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45370/0 (socket says 172.21.15.22:45370) 2023-09-20T21:40:13.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.175+0000 7f8a6a59c700 1 -- 172.21.15.22:0/415959032 learned_addr learned my addr 172.21.15.22:0/415959032 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:13.110 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.175+0000 7f8a6a59c700 1 -- 172.21.15.22:0/415959032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c009550 con 0x7f8a6c07be60 2023-09-20T21:40:13.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.175+0000 7f8a6a59c700 1 --2- 172.21.15.22:0/415959032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c19c520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c003f00 tx=0x7f8a5c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:13.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.175+0000 7f8a5b7fe700 1 -- 172.21.15.22:0/415959032 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a5c003b20 con 0x7f8a6c07be60 2023-09-20T21:40:13.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.176+0000 7f8a5b7fe700 1 -- 172.21.15.22:0/415959032 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a5c003c80 con 0x7f8a6c07be60 2023-09-20T21:40:13.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.176+0000 7f8a70f4c700 1 -- 172.21.15.22:0/415959032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c19cc60 con 0x7f8a6c07be60 2023-09-20T21:40:13.111 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.176+0000 7f8a5b7fe700 1 -- 172.21.15.22:0/415959032 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a5c019770 con 0x7f8a6c07be60 2023-09-20T21:40:13.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.176+0000 7f8a70f4c700 1 -- 172.21.15.22:0/415959032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c19d080 con 0x7f8a6c07be60 2023-09-20T21:40:13.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.178+0000 7f8a5b7fe700 1 -- 172.21.15.22:0/415959032 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 12) v1 ==== 45631+0+0 (secure 0 0 0) 0x7f8a5c019a00 con 0x7f8a6c07be60 2023-09-20T21:40:13.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.178+0000 7f8a70f4c700 1 -- 172.21.15.22:0/415959032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4c005320 con 0x7f8a6c07be60 2023-09-20T21:40:13.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.178+0000 7f8a5b7fe700 1 --2- 172.21.15.22:0/415959032 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a54038840 0x7f8a5403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:13.112 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.178+0000 7f8a5b7fe700 1 -- 172.21.15.22:0/415959032 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a5c04cc00 con 0x7f8a6c07be60 2023-09-20T21:40:13.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.179+0000 7f8a69d9b700 1 --2- 172.21.15.22:0/415959032 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a54038840 0x7f8a5403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:13.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.180+0000 7f8a69d9b700 1 --2- 172.21.15.22:0/415959032 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a54038840 0x7f8a5403ad00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006fd0 tx=0x7f8a60006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:13.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.184+0000 7f8a5b7fe700 1 -- 172.21.15.22:0/415959032 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8a5c018b80 con 0x7f8a6c07be60 2023-09-20T21:40:13.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.421+0000 7f8a70f4c700 1 -- 172.21.15.22:0/415959032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f8a4c005b50 con 0x7f8a6c07be60 2023-09-20T21:40:13.113 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.421+0000 7f8a5b7fe700 1 -- 172.21.15.22:0/415959032 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8a5c04ce60 con 0x7f8a6c07be60 2023-09-20T21:40:13.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.424+0000 7f8a70f4c700 1 -- 172.21.15.22:0/415959032 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a54038840 msgr2=0x7f8a5403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:13.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.424+0000 7f8a70f4c700 1 --2- 172.21.15.22:0/415959032 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a54038840 0x7f8a5403ad00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006fd0 tx=0x7f8a60006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:13.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.424+0000 7f8a70f4c700 1 -- 172.21.15.22:0/415959032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 msgr2=0x7f8a6c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:13.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.424+0000 7f8a70f4c700 1 --2- 172.21.15.22:0/415959032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c19c520 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c003f00 tx=0x7f8a5c011a30 comp rx=0 tx=0).stop 2023-09-20T21:40:13.114 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.424+0000 7f8a70f4c700 1 -- 172.21.15.22:0/415959032 shutdown_connections 2023-09-20T21:40:13.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.424+0000 7f8a70f4c700 1 --2- 172.21.15.22:0/415959032 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a54038840 0x7f8a5403ad00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:13.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.424+0000 7f8a70f4c700 1 --2- 172.21.15.22:0/415959032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a6c07be60 0x7f8a6c19c520 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:13.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.424+0000 7f8a70f4c700 1 -- 172.21.15.22:0/415959032 >> 172.21.15.22:0/415959032 conn(0x7f8a6c101b30 msgr2=0x7f8a6c1032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:13.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.424+0000 7f8a70f4c700 1 -- 172.21.15.22:0/415959032 shutdown_connections 2023-09-20T21:40:13.115 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:12.424+0000 7f8a70f4c700 1 -- 172.21.15.22:0/415959032 wait complete. 2023-09-20T21:40:13.121 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-09-20T21:40:13.122 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2023-09-20T21:40:13.138 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-09-20T21:40:13.138 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2023-09-20T21:40:13.138 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2023-09-20T21:40:13.139 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-09-20T21:40:13.140 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph Dashboard is now available at: 2023-09-20T21:40:13.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:13.141 INFO:teuthology.orchestra.run.smithi022.stdout: URL: https://smithi022.front.sepia.ceph.com:8443/ 2023-09-20T21:40:13.141 INFO:teuthology.orchestra.run.smithi022.stdout: User: admin 2023-09-20T21:40:13.141 INFO:teuthology.orchestra.run.smithi022.stdout: Password: pa5zqf8tmj 2023-09-20T21:40:13.141 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:13.142 INFO:teuthology.orchestra.run.smithi022.stdout:Saving cluster configuration to /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config directory 2023-09-20T21:40:13.142 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling autotune for osd_memory_target 2023-09-20T21:40:13.394 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:13 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/415959032' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-09-20T21:40:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.663+0000 7f4fa06d5700 1 Processor -- start 2023-09-20T21:40:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.663+0000 7f4fa06d5700 1 -- start start 2023-09-20T21:40:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.663+0000 7f4fa06d5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9807a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:14.324 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.663+0000 7f4fa06d5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9807a890 con 0x7f4f9807be60 2023-09-20T21:40:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.664+0000 7f4f9e471700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9807a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.664+0000 7f4f9e471700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9807a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45376/0 (socket says 172.21.15.22:45376) 2023-09-20T21:40:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.664+0000 7f4f9e471700 1 -- 172.21.15.22:0/1646849109 learned_addr learned my addr 172.21.15.22:0/1646849109 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.664+0000 7f4f9e471700 1 -- 172.21.15.22:0/1646849109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9807a9d0 con 0x7f4f9807be60 2023-09-20T21:40:14.325 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.664+0000 7f4f9e471700 1 --2- 172.21.15.22:0/1646849109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9807a2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4f940098a0 tx=0x7f4f94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bc5315daf5b75d server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.664+0000 7f4f9d46f700 1 -- 172.21.15.22:0/1646849109 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f94004160 con 0x7f4f9807be60 2023-09-20T21:40:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.665+0000 7f4f9d46f700 1 -- 172.21.15.22:0/1646849109 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f940042c0 con 0x7f4f9807be60 2023-09-20T21:40:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.665+0000 7f4f9d46f700 1 -- 172.21.15.22:0/1646849109 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f940119b0 con 0x7f4f9807be60 2023-09-20T21:40:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.665+0000 7f4fa06d5700 1 -- 172.21.15.22:0/1646849109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 msgr2=0x7f4f9807a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:14.326 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.665+0000 7f4fa06d5700 1 --2- 172.21.15.22:0/1646849109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9807a2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4f940098a0 tx=0x7f4f94009bb0 comp rx=0 tx=0).stop 2023-09-20T21:40:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.665+0000 7f4fa06d5700 1 -- 172.21.15.22:0/1646849109 shutdown_connections 2023-09-20T21:40:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.665+0000 7f4fa06d5700 1 --2- 172.21.15.22:0/1646849109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9807a2c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.665+0000 7f4fa06d5700 1 -- 172.21.15.22:0/1646849109 >> 172.21.15.22:0/1646849109 conn(0x7f4f98101b30 msgr2=0x7f4f98103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.665+0000 7f4fa06d5700 1 -- 172.21.15.22:0/1646849109 shutdown_connections 2023-09-20T21:40:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.665+0000 7f4fa06d5700 1 -- 172.21.15.22:0/1646849109 wait complete. 2023-09-20T21:40:14.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.666+0000 7f4fa06d5700 1 Processor -- start 2023-09-20T21:40:14.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.666+0000 7f4fa06d5700 1 -- start start 2023-09-20T21:40:14.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.667+0000 7f4fa06d5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:14.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.667+0000 7f4fa06d5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9819ca60 con 0x7f4f9807be60 2023-09-20T21:40:14.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.667+0000 7f4f9e471700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:14.328 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.667+0000 7f4f9e471700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9819c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45390/0 (socket says 172.21.15.22:45390) 2023-09-20T21:40:14.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.667+0000 7f4f9e471700 1 -- 172.21.15.22:0/694305386 learned_addr learned my addr 172.21.15.22:0/694305386 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:14.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.667+0000 7f4f9e471700 1 -- 172.21.15.22:0/694305386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f94009550 con 0x7f4f9807be60 2023-09-20T21:40:14.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.668+0000 7f4f9e471700 1 --2- 172.21.15.22:0/694305386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9819c520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4f94003f00 tx=0x7f4f94011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:14.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.668+0000 7f4f8f7fe700 1 -- 172.21.15.22:0/694305386 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f94003b20 con 0x7f4f9807be60 2023-09-20T21:40:14.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.668+0000 7f4f8f7fe700 1 -- 172.21.15.22:0/694305386 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f94003c80 con 0x7f4f9807be60 2023-09-20T21:40:14.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.668+0000 7f4f8f7fe700 1 -- 172.21.15.22:0/694305386 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f94019770 con 0x7f4f9807be60 2023-09-20T21:40:14.330 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.668+0000 7f4fa06d5700 1 -- 172.21.15.22:0/694305386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9819cc60 con 0x7f4f9807be60 2023-09-20T21:40:14.330 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.668+0000 7f4fa06d5700 1 -- 172.21.15.22:0/694305386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9819d080 con 0x7f4f9807be60 2023-09-20T21:40:14.330 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.669+0000 7f4fa06d5700 1 -- 172.21.15.22:0/694305386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f981965f0 con 0x7f4f9807be60 2023-09-20T21:40:14.330 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.669+0000 7f4f8f7fe700 1 -- 172.21.15.22:0/694305386 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7f4f9401ab60 con 0x7f4f9807be60 2023-09-20T21:40:14.330 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.670+0000 7f4f8f7fe700 1 --2- 172.21.15.22:0/694305386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f840388e0 0x7f4f8403ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:14.330 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.670+0000 7f4f8f7fe700 1 -- 172.21.15.22:0/694305386 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4f9404cd20 con 0x7f4f9807be60 2023-09-20T21:40:14.331 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.670+0000 7f4f9dc70700 1 --2- 172.21.15.22:0/694305386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f840388e0 0x7f4f8403ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:14.331 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.670+0000 7f4f9dc70700 1 --2- 172.21.15.22:0/694305386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f840388e0 0x7f4f8403ada0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4f88006fd0 tx=0x7f4f88006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:14.331 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.675+0000 7f4f8f7fe700 1 -- 172.21.15.22:0/694305386 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4f9401a430 con 0x7f4f9807be60 2023-09-20T21:40:14.331 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.883+0000 7f4fa06d5700 1 -- 172.21.15.22:0/694305386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f4f98062410 con 0x7f4f9807be60 2023-09-20T21:40:14.331 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.883+0000 7f4f8f7fe700 1 -- 172.21.15.22:0/694305386 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f4f9402db40 con 0x7f4f9807be60 2023-09-20T21:40:14.332 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.886+0000 7f4fa06d5700 1 -- 172.21.15.22:0/694305386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f840388e0 msgr2=0x7f4f8403ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:14.332 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.886+0000 7f4fa06d5700 1 --2- 172.21.15.22:0/694305386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f840388e0 0x7f4f8403ada0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4f88006fd0 tx=0x7f4f88006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:14.332 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.886+0000 7f4fa06d5700 1 -- 172.21.15.22:0/694305386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 msgr2=0x7f4f9819c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:14.332 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.886+0000 7f4fa06d5700 1 --2- 172.21.15.22:0/694305386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9819c520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4f94003f00 tx=0x7f4f94011a30 comp rx=0 tx=0).stop 2023-09-20T21:40:14.332 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.886+0000 7f4fa06d5700 1 -- 172.21.15.22:0/694305386 shutdown_connections 2023-09-20T21:40:14.332 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.886+0000 7f4fa06d5700 1 --2- 172.21.15.22:0/694305386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f840388e0 0x7f4f8403ada0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:14.333 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.886+0000 7f4fa06d5700 1 --2- 172.21.15.22:0/694305386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f9807be60 0x7f4f9819c520 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:14.333 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.886+0000 7f4fa06d5700 1 -- 172.21.15.22:0/694305386 >> 172.21.15.22:0/694305386 conn(0x7f4f98101b30 msgr2=0x7f4f981032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:14.333 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.886+0000 7f4fa06d5700 1 -- 172.21.15.22:0/694305386 shutdown_connections 2023-09-20T21:40:14.333 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:13.886+0000 7f4fa06d5700 1 -- 172.21.15.22:0/694305386 wait complete. 2023-09-20T21:40:14.574 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:14 smithi022 ceph-mon[97651]: mgrmap e13: x(active, since 6s) 2023-09-20T21:40:15.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.862+0000 7fa0b826e700 1 Processor -- start 2023-09-20T21:40:15.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.862+0000 7fa0b826e700 1 -- start start 2023-09-20T21:40:15.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.863+0000 7fa0b826e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:15.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.863+0000 7fa0b826e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b007a890 con 0x7fa0b007be60 2023-09-20T21:40:15.711 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.863+0000 7fa0b600a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:15.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.863+0000 7fa0b600a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45406/0 (socket says 172.21.15.22:45406) 2023-09-20T21:40:15.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.863+0000 7fa0b600a700 1 -- 172.21.15.22:0/3732109797 learned_addr learned my addr 172.21.15.22:0/3732109797 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:15.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.864+0000 7fa0b600a700 1 -- 172.21.15.22:0/3732109797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b007a9d0 con 0x7fa0b007be60 2023-09-20T21:40:15.712 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.864+0000 7fa0b600a700 1 --2- 172.21.15.22:0/3732109797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b007a2c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0098a0 tx=0x7fa0ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb23a15f53e35141 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:15.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.864+0000 7fa0b5008700 1 -- 172.21.15.22:0/3732109797 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac004160 con 0x7fa0b007be60 2023-09-20T21:40:15.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.864+0000 7fa0b5008700 1 -- 172.21.15.22:0/3732109797 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ac0042c0 con 0x7fa0b007be60 2023-09-20T21:40:15.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.864+0000 7fa0b5008700 1 -- 172.21.15.22:0/3732109797 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac0119b0 con 0x7fa0b007be60 2023-09-20T21:40:15.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.865+0000 7fa0b826e700 1 -- 172.21.15.22:0/3732109797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 msgr2=0x7fa0b007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:15.713 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.865+0000 7fa0b826e700 1 --2- 172.21.15.22:0/3732109797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b007a2c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0098a0 tx=0x7fa0ac009bb0 comp rx=0 tx=0).stop 2023-09-20T21:40:15.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.865+0000 7fa0b826e700 1 -- 172.21.15.22:0/3732109797 shutdown_connections 2023-09-20T21:40:15.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.865+0000 7fa0b826e700 1 --2- 172.21.15.22:0/3732109797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b007a2c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:15.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.865+0000 7fa0b826e700 1 -- 172.21.15.22:0/3732109797 >> 172.21.15.22:0/3732109797 conn(0x7fa0b0101b30 msgr2=0x7fa0b0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:15.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.865+0000 7fa0b826e700 1 -- 172.21.15.22:0/3732109797 shutdown_connections 2023-09-20T21:40:15.714 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.865+0000 7fa0b826e700 1 -- 172.21.15.22:0/3732109797 wait complete. 2023-09-20T21:40:15.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.866+0000 7fa0b826e700 1 Processor -- start 2023-09-20T21:40:15.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.866+0000 7fa0b826e700 1 -- start start 2023-09-20T21:40:15.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.866+0000 7fa0b826e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:15.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.866+0000 7fa0b826e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b019ca60 con 0x7fa0b007be60 2023-09-20T21:40:15.715 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.867+0000 7fa0b600a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:15.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.867+0000 7fa0b600a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45410/0 (socket says 172.21.15.22:45410) 2023-09-20T21:40:15.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.867+0000 7fa0b600a700 1 -- 172.21.15.22:0/157564818 learned_addr learned my addr 172.21.15.22:0/157564818 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:15.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.867+0000 7fa0b600a700 1 -- 172.21.15.22:0/157564818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac009550 con 0x7fa0b007be60 2023-09-20T21:40:15.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.867+0000 7fa0b600a700 1 --2- 172.21.15.22:0/157564818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b019c520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006af0 tx=0x7fa0ac011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:15.716 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.867+0000 7fa0a77fe700 1 -- 172.21.15.22:0/157564818 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac003a40 con 0x7fa0b007be60 2023-09-20T21:40:15.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.868+0000 7fa0a77fe700 1 -- 172.21.15.22:0/157564818 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ac003ba0 con 0x7fa0b007be60 2023-09-20T21:40:15.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.868+0000 7fa0a77fe700 1 -- 172.21.15.22:0/157564818 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac0196c0 con 0x7fa0b007be60 2023-09-20T21:40:15.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.868+0000 7fa0b826e700 1 -- 172.21.15.22:0/157564818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b019cc60 con 0x7fa0b007be60 2023-09-20T21:40:15.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.868+0000 7fa0b826e700 1 -- 172.21.15.22:0/157564818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b019d080 con 0x7fa0b007be60 2023-09-20T21:40:15.717 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.869+0000 7fa0b826e700 1 -- 172.21.15.22:0/157564818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b01965f0 con 0x7fa0b007be60 2023-09-20T21:40:15.718 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.869+0000 7fa0a77fe700 1 -- 172.21.15.22:0/157564818 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7fa0ac01aa80 con 0x7fa0b007be60 2023-09-20T21:40:15.718 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.869+0000 7fa0a77fe700 1 --2- 172.21.15.22:0/157564818 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa09c038890 0x7fa09c03ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:15.718 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.869+0000 7fa0a77fe700 1 -- 172.21.15.22:0/157564818 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa0ac04cd90 con 0x7fa0b007be60 2023-09-20T21:40:15.718 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.869+0000 7fa0b5809700 1 --2- 172.21.15.22:0/157564818 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa09c038890 0x7fa09c03ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:15.719 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.870+0000 7fa0b5809700 1 --2- 172.21.15.22:0/157564818 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa09c038890 0x7fa09c03ad50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0006fd0 tx=0x7fa0a0006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:15.719 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:14.874+0000 7fa0a77fe700 1 -- 172.21.15.22:0/157564818 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa0ac052020 con 0x7fa0b007be60 2023-09-20T21:40:15.719 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.196+0000 7fa0b826e700 1 -- 172.21.15.22:0/157564818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fa0b0062410 con 0x7fa0b007be60 2023-09-20T21:40:15.719 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.198+0000 7fa0a77fe700 1 -- 172.21.15.22:0/157564818 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fa0ac01a430 con 0x7fa0b007be60 2023-09-20T21:40:15.720 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.201+0000 7fa0b826e700 1 -- 172.21.15.22:0/157564818 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa09c038890 msgr2=0x7fa09c03ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:15.720 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.201+0000 7fa0b826e700 1 --2- 172.21.15.22:0/157564818 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa09c038890 0x7fa09c03ad50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0006fd0 tx=0x7fa0a0006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:15.720 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.201+0000 7fa0b826e700 1 -- 172.21.15.22:0/157564818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 msgr2=0x7fa0b019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:15.720 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.201+0000 7fa0b826e700 1 --2- 172.21.15.22:0/157564818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b019c520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006af0 tx=0x7fa0ac011a10 comp rx=0 tx=0).stop 2023-09-20T21:40:15.720 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.201+0000 7fa0b826e700 1 -- 172.21.15.22:0/157564818 shutdown_connections 2023-09-20T21:40:15.721 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.201+0000 7fa0b826e700 1 --2- 172.21.15.22:0/157564818 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa09c038890 0x7fa09c03ad50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:15.721 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.201+0000 7fa0b826e700 1 --2- 172.21.15.22:0/157564818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b007be60 0x7fa0b019c520 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:15.721 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.201+0000 7fa0b826e700 1 -- 172.21.15.22:0/157564818 >> 172.21.15.22:0/157564818 conn(0x7fa0b0101b30 msgr2=0x7fa0b01032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:15.721 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.201+0000 7fa0b826e700 1 -- 172.21.15.22:0/157564818 shutdown_connections 2023-09-20T21:40:15.721 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr 2023-09-20T21:40:15.201+0000 7fa0b826e700 1 -- 172.21.15.22:0/157564818 wait complete. 2023-09-20T21:40:15.722 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-09-20T21:40:15.722 INFO:teuthology.orchestra.run.smithi022.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-09-20T21:40:15.722 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:15.722 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-09-20T21:40:15.723 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:15.723 INFO:teuthology.orchestra.run.smithi022.stdout:Or, if you are only running a single cluster on this host: 2023-09-20T21:40:15.723 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:15.723 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-09-20T21:40:15.723 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:15.723 INFO:teuthology.orchestra.run.smithi022.stdout:Please consider enabling telemetry to help improve Ceph: 2023-09-20T21:40:15.724 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:15.724 INFO:teuthology.orchestra.run.smithi022.stdout: ceph telemetry on 2023-09-20T21:40:15.724 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:15.724 INFO:teuthology.orchestra.run.smithi022.stdout:For more information see: 2023-09-20T21:40:15.724 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:15.725 INFO:teuthology.orchestra.run.smithi022.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-09-20T21:40:15.725 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:15.725 INFO:teuthology.orchestra.run.smithi022.stdout:Bootstrap complete. 2023-09-20T21:40:15.762 INFO:tasks.cephadm:Fetching config... 2023-09-20T21:40:15.763 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:40:15.763 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-09-20T21:40:15.846 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-09-20T21:40:15.846 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:40:15.847 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-09-20T21:40:15.931 INFO:tasks.cephadm:Fetching mon keyring... 2023-09-20T21:40:15.932 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:40:15.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/keyring of=/dev/stdout 2023-09-20T21:40:16.041 INFO:tasks.cephadm:Fetching pub ssh key... 2023-09-20T21:40:16.042 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:40:16.042 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-09-20T21:40:16.127 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-09-20T21:40:16.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3B+lE3MW556Fkpg4qXTcj2XA7hcScu48hvT/yGSvVzFmVJ5KGdiHII8b1o6m4GEZe7ue4GKZ6CvPR7HJhqbOI/dQ9uJo/bMBtJt5A9vFWf4ldnduMoo+fS56cQlnzCBNzkpSXLC1/KV5OGZ4spcfDeOUU5FOCqkl/X7OYv0frMxFZYZl7difgX3V2ATYXmVilPz7fSrEZ4l3C4FRLgn3JGq/GPJjsaZ+72g3r8EpqcDipIIclhhctoSmAhjcYqM9f1bILd+EzrrzDc+YlYYEqmmwTsxEeIC/Gf/kBasW8mBQtLgC5QaeFqgufeGKCyCOkVnQY/tDOvv/pXDjAL8aRu3RRlTCZXfZ1wfE8JpSuGjyMtLOkiaV0BN/nAJOJHNdPJ3FllIQlh3YRx/LO9qOPelMvGrC9AFfIIw5vp2w0sVwBagU9pJ5WO/B1nDElM19CYZoFm7z4nQ7kX576Aioc7Slnd3q7EjKNCpfz5U15ZzGKVb2bdyTB7VqgFq4HIeM= ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-20T21:40:16.272 INFO:teuthology.orchestra.run.smithi022.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3B+lE3MW556Fkpg4qXTcj2XA7hcScu48hvT/yGSvVzFmVJ5KGdiHII8b1o6m4GEZe7ue4GKZ6CvPR7HJhqbOI/dQ9uJo/bMBtJt5A9vFWf4ldnduMoo+fS56cQlnzCBNzkpSXLC1/KV5OGZ4spcfDeOUU5FOCqkl/X7OYv0frMxFZYZl7difgX3V2ATYXmVilPz7fSrEZ4l3C4FRLgn3JGq/GPJjsaZ+72g3r8EpqcDipIIclhhctoSmAhjcYqM9f1bILd+EzrrzDc+YlYYEqmmwTsxEeIC/Gf/kBasW8mBQtLgC5QaeFqgufeGKCyCOkVnQY/tDOvv/pXDjAL8aRu3RRlTCZXfZ1wfE8JpSuGjyMtLOkiaV0BN/nAJOJHNdPJ3FllIQlh3YRx/LO9qOPelMvGrC9AFfIIw5vp2w0sVwBagU9pJ5WO/B1nDElM19CYZoFm7z4nQ7kX576Aioc7Slnd3q7EjKNCpfz5U15ZzGKVb2bdyTB7VqgFq4HIeM= ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:40:16.295 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:16 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/157564818' entity='client.admin' 2023-09-20T21:40:16.304 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3B+lE3MW556Fkpg4qXTcj2XA7hcScu48hvT/yGSvVzFmVJ5KGdiHII8b1o6m4GEZe7ue4GKZ6CvPR7HJhqbOI/dQ9uJo/bMBtJt5A9vFWf4ldnduMoo+fS56cQlnzCBNzkpSXLC1/KV5OGZ4spcfDeOUU5FOCqkl/X7OYv0frMxFZYZl7difgX3V2ATYXmVilPz7fSrEZ4l3C4FRLgn3JGq/GPJjsaZ+72g3r8EpqcDipIIclhhctoSmAhjcYqM9f1bILd+EzrrzDc+YlYYEqmmwTsxEeIC/Gf/kBasW8mBQtLgC5QaeFqgufeGKCyCOkVnQY/tDOvv/pXDjAL8aRu3RRlTCZXfZ1wfE8JpSuGjyMtLOkiaV0BN/nAJOJHNdPJ3FllIQlh3YRx/LO9qOPelMvGrC9AFfIIw5vp2w0sVwBagU9pJ5WO/B1nDElM19CYZoFm7z4nQ7kX576Aioc7Slnd3q7EjKNCpfz5U15ZzGKVb2bdyTB7VqgFq4HIeM= ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-20T21:40:16.350 INFO:teuthology.orchestra.run.smithi089.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3B+lE3MW556Fkpg4qXTcj2XA7hcScu48hvT/yGSvVzFmVJ5KGdiHII8b1o6m4GEZe7ue4GKZ6CvPR7HJhqbOI/dQ9uJo/bMBtJt5A9vFWf4ldnduMoo+fS56cQlnzCBNzkpSXLC1/KV5OGZ4spcfDeOUU5FOCqkl/X7OYv0frMxFZYZl7difgX3V2ATYXmVilPz7fSrEZ4l3C4FRLgn3JGq/GPJjsaZ+72g3r8EpqcDipIIclhhctoSmAhjcYqM9f1bILd+EzrrzDc+YlYYEqmmwTsxEeIC/Gf/kBasW8mBQtLgC5QaeFqgufeGKCyCOkVnQY/tDOvv/pXDjAL8aRu3RRlTCZXfZ1wfE8JpSuGjyMtLOkiaV0BN/nAJOJHNdPJ3FllIQlh3YRx/LO9qOPelMvGrC9AFfIIw5vp2w0sVwBagU9pJ5WO/B1nDElM19CYZoFm7z4nQ7kX576Aioc7Slnd3q7EjKNCpfz5U15ZzGKVb2bdyTB7VqgFq4HIeM= ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:40:16.363 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3B+lE3MW556Fkpg4qXTcj2XA7hcScu48hvT/yGSvVzFmVJ5KGdiHII8b1o6m4GEZe7ue4GKZ6CvPR7HJhqbOI/dQ9uJo/bMBtJt5A9vFWf4ldnduMoo+fS56cQlnzCBNzkpSXLC1/KV5OGZ4spcfDeOUU5FOCqkl/X7OYv0frMxFZYZl7difgX3V2ATYXmVilPz7fSrEZ4l3C4FRLgn3JGq/GPJjsaZ+72g3r8EpqcDipIIclhhctoSmAhjcYqM9f1bILd+EzrrzDc+YlYYEqmmwTsxEeIC/Gf/kBasW8mBQtLgC5QaeFqgufeGKCyCOkVnQY/tDOvv/pXDjAL8aRu3RRlTCZXfZ1wfE8JpSuGjyMtLOkiaV0BN/nAJOJHNdPJ3FllIQlh3YRx/LO9qOPelMvGrC9AFfIIw5vp2w0sVwBagU9pJ5WO/B1nDElM19CYZoFm7z4nQ7kX576Aioc7Slnd3q7EjKNCpfz5U15ZzGKVb2bdyTB7VqgFq4HIeM= ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-09-20T21:40:16.411 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3B+lE3MW556Fkpg4qXTcj2XA7hcScu48hvT/yGSvVzFmVJ5KGdiHII8b1o6m4GEZe7ue4GKZ6CvPR7HJhqbOI/dQ9uJo/bMBtJt5A9vFWf4ldnduMoo+fS56cQlnzCBNzkpSXLC1/KV5OGZ4spcfDeOUU5FOCqkl/X7OYv0frMxFZYZl7difgX3V2ATYXmVilPz7fSrEZ4l3C4FRLgn3JGq/GPJjsaZ+72g3r8EpqcDipIIclhhctoSmAhjcYqM9f1bILd+EzrrzDc+YlYYEqmmwTsxEeIC/Gf/kBasW8mBQtLgC5QaeFqgufeGKCyCOkVnQY/tDOvv/pXDjAL8aRu3RRlTCZXfZ1wfE8JpSuGjyMtLOkiaV0BN/nAJOJHNdPJ3FllIQlh3YRx/LO9qOPelMvGrC9AFfIIw5vp2w0sVwBagU9pJ5WO/B1nDElM19CYZoFm7z4nQ7kX576Aioc7Slnd3q7EjKNCpfz5U15ZzGKVb2bdyTB7VqgFq4HIeM= ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:40:16.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-09-20T21:40:17.170 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:40:18.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.958+0000 7eff355eb700 1 -- 172.21.15.22:0/740008134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff30074e10 msgr2=0x7eff30073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:18.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.958+0000 7eff355eb700 1 --2- 172.21.15.22:0/740008134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff30074e10 0x7eff30073270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7eff20009910 tx=0x7eff20009c20 comp rx=0 tx=0).stop 2023-09-20T21:40:18.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.959+0000 7eff355eb700 1 -- 172.21.15.22:0/740008134 shutdown_connections 2023-09-20T21:40:18.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.959+0000 7eff355eb700 1 --2- 172.21.15.22:0/740008134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff30074e10 0x7eff30073270 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:18.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.959+0000 7eff355eb700 1 -- 172.21.15.22:0/740008134 >> 172.21.15.22:0/740008134 conn(0x7eff300fbe70 msgr2=0x7eff300fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:18.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.959+0000 7eff355eb700 1 -- 172.21.15.22:0/740008134 shutdown_connections 2023-09-20T21:40:18.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.959+0000 7eff355eb700 1 -- 172.21.15.22:0/740008134 wait complete. 2023-09-20T21:40:18.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.960+0000 7eff355eb700 1 Processor -- start 2023-09-20T21:40:18.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.960+0000 7eff355eb700 1 -- start start 2023-09-20T21:40:18.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.960+0000 7eff355eb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff30074e10 0x7eff30197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:18.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.960+0000 7eff355eb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff30198170 con 0x7eff30074e10 2023-09-20T21:40:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.960+0000 7eff2effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff30074e10 0x7eff30197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.960+0000 7eff2effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff30074e10 0x7eff30197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32914/0 (socket says 172.21.15.22:32914) 2023-09-20T21:40:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.960+0000 7eff2effd700 1 -- 172.21.15.22:0/2590631465 learned_addr learned my addr 172.21.15.22:0/2590631465 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.961+0000 7eff2effd700 1 -- 172.21.15.22:0/2590631465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff200095f0 con 0x7eff30074e10 2023-09-20T21:40:18.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.961+0000 7eff2effd700 1 --2- 172.21.15.22:0/2590631465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff30074e10 0x7eff30197c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff20004f40 tx=0x7eff200057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:18.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.961+0000 7eff17fff700 1 -- 172.21.15.22:0/2590631465 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff2001d070 con 0x7eff30074e10 2023-09-20T21:40:18.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.961+0000 7eff17fff700 1 -- 172.21.15.22:0/2590631465 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff2000bc10 con 0x7eff30074e10 2023-09-20T21:40:18.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.961+0000 7eff17fff700 1 -- 172.21.15.22:0/2590631465 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff20003a70 con 0x7eff30074e10 2023-09-20T21:40:18.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.961+0000 7eff355eb700 1 -- 172.21.15.22:0/2590631465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff30198370 con 0x7eff30074e10 2023-09-20T21:40:18.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.961+0000 7eff355eb700 1 -- 172.21.15.22:0/2590631465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff30198810 con 0x7eff30074e10 2023-09-20T21:40:18.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.962+0000 7eff17fff700 1 -- 172.21.15.22:0/2590631465 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7eff20003ce0 con 0x7eff30074e10 2023-09-20T21:40:18.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.962+0000 7eff355eb700 1 -- 172.21.15.22:0/2590631465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff30191610 con 0x7eff30074e10 2023-09-20T21:40:18.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.963+0000 7eff17fff700 1 --2- 172.21.15.22:0/2590631465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7eff18038890 0x7eff1803ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:18.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.963+0000 7eff17fff700 1 -- 172.21.15.22:0/2590631465 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7eff2004c670 con 0x7eff30074e10 2023-09-20T21:40:18.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.963+0000 7eff2e7fc700 1 --2- 172.21.15.22:0/2590631465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7eff18038890 0x7eff1803ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:18.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.964+0000 7eff2e7fc700 1 --2- 172.21.15.22:0/2590631465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7eff18038890 0x7eff1803ad50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7eff24006fd0 tx=0x7eff24006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:18.969+0000 7eff17fff700 1 -- 172.21.15.22:0/2590631465 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7eff2002ae20 con 0x7eff30074e10 2023-09-20T21:40:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.196+0000 7eff355eb700 1 -- 172.21.15.22:0/2590631465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7eff30108850 con 0x7eff30074e10 2023-09-20T21:40:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.199+0000 7eff17fff700 1 -- 172.21.15.22:0/2590631465 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7eff20026070 con 0x7eff30074e10 2023-09-20T21:40:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.201+0000 7eff355eb700 1 -- 172.21.15.22:0/2590631465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7eff18038890 msgr2=0x7eff1803ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:19.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.201+0000 7eff355eb700 1 --2- 172.21.15.22:0/2590631465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7eff18038890 0x7eff1803ad50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7eff24006fd0 tx=0x7eff24006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:19.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.201+0000 7eff355eb700 1 -- 172.21.15.22:0/2590631465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff30074e10 msgr2=0x7eff30197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:19.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.201+0000 7eff355eb700 1 --2- 172.21.15.22:0/2590631465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff30074e10 0x7eff30197c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff20004f40 tx=0x7eff200057c0 comp rx=0 tx=0).stop 2023-09-20T21:40:19.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.201+0000 7eff355eb700 1 -- 172.21.15.22:0/2590631465 shutdown_connections 2023-09-20T21:40:19.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.201+0000 7eff355eb700 1 --2- 172.21.15.22:0/2590631465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7eff18038890 0x7eff1803ad50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:19.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.201+0000 7eff355eb700 1 --2- 172.21.15.22:0/2590631465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff30074e10 0x7eff30197c30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:19.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.201+0000 7eff355eb700 1 -- 172.21.15.22:0/2590631465 >> 172.21.15.22:0/2590631465 conn(0x7eff300fbe70 msgr2=0x7eff300fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:19.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.202+0000 7eff355eb700 1 -- 172.21.15.22:0/2590631465 shutdown_connections 2023-09-20T21:40:19.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:19.202+0000 7eff355eb700 1 -- 172.21.15.22:0/2590631465 wait complete. 2023-09-20T21:40:19.646 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:19.646 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:19.646 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:40:19.646 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:19.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:19.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:19.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:19 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2590631465' entity='client.admin' 2023-09-20T21:40:19.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:19.648 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:19.713 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-09-20T21:40:19.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-09-20T21:40:20.471 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:40:22.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.158+0000 7f6e02311700 1 -- 172.21.15.22:0/3790769565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dfc101360 msgr2=0x7f6dfc103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:22.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.158+0000 7f6e02311700 1 --2- 172.21.15.22:0/3790769565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dfc101360 0x7f6dfc103750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009910 tx=0x7f6dec009c20 comp rx=0 tx=0).stop 2023-09-20T21:40:22.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.158+0000 7f6e02311700 1 -- 172.21.15.22:0/3790769565 shutdown_connections 2023-09-20T21:40:22.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.158+0000 7f6e02311700 1 --2- 172.21.15.22:0/3790769565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dfc101360 0x7f6dfc103750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:22.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.158+0000 7f6e02311700 1 -- 172.21.15.22:0/3790769565 >> 172.21.15.22:0/3790769565 conn(0x7f6dfc0fad70 msgr2=0x7f6dfc0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:22.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.158+0000 7f6e02311700 1 -- 172.21.15.22:0/3790769565 shutdown_connections 2023-09-20T21:40:22.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.158+0000 7f6e02311700 1 -- 172.21.15.22:0/3790769565 wait complete. 2023-09-20T21:40:22.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.159+0000 7f6e02311700 1 Processor -- start 2023-09-20T21:40:22.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.159+0000 7f6e02311700 1 -- start start 2023-09-20T21:40:22.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.160+0000 7f6e02311700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dfc101360 0x7f6dfc100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:22.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.160+0000 7f6e02311700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc101200 con 0x7f6dfc101360 2023-09-20T21:40:22.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.160+0000 7f6dfbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dfc101360 0x7f6dfc100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:22.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.160+0000 7f6dfbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dfc101360 0x7f6dfc100cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32932/0 (socket says 172.21.15.22:32932) 2023-09-20T21:40:22.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.160+0000 7f6dfbfff700 1 -- 172.21.15.22:0/3907715618 learned_addr learned my addr 172.21.15.22:0/3907715618 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:22.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.160+0000 7f6dfbfff700 1 -- 172.21.15.22:0/3907715618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec0095f0 con 0x7f6dfc101360 2023-09-20T21:40:22.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.161+0000 7f6dfbfff700 1 --2- 172.21.15.22:0/3907715618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dfc101360 0x7f6dfc100cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005f50 tx=0x7f6dec0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:22.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.161+0000 7f6df97fa700 1 -- 172.21.15.22:0/3907715618 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dec01d070 con 0x7f6dfc101360 2023-09-20T21:40:22.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.161+0000 7f6df97fa700 1 -- 172.21.15.22:0/3907715618 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dec022470 con 0x7f6dfc101360 2023-09-20T21:40:22.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.161+0000 7f6df97fa700 1 -- 172.21.15.22:0/3907715618 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dec00f5f0 con 0x7f6dfc101360 2023-09-20T21:40:22.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.161+0000 7f6e02311700 1 -- 172.21.15.22:0/3907715618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc0ff310 con 0x7f6dfc101360 2023-09-20T21:40:22.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.161+0000 7f6e02311700 1 -- 172.21.15.22:0/3907715618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc0ff7b0 con 0x7f6dfc101360 2023-09-20T21:40:22.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.162+0000 7f6df97fa700 1 -- 172.21.15.22:0/3907715618 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7f6dec022850 con 0x7f6dfc101360 2023-09-20T21:40:22.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.162+0000 7f6e02311700 1 -- 172.21.15.22:0/3907715618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc062410 con 0x7f6dfc101360 2023-09-20T21:40:22.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.162+0000 7f6df97fa700 1 --2- 172.21.15.22:0/3907715618 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6de4038890 0x7f6de403ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:22.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.162+0000 7f6df97fa700 1 -- 172.21.15.22:0/3907715618 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6dec04cb00 con 0x7f6dfc101360 2023-09-20T21:40:22.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.163+0000 7f6dfb7fe700 1 --2- 172.21.15.22:0/3907715618 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6de4038890 0x7f6de403ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:22.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.163+0000 7f6dfb7fe700 1 --2- 172.21.15.22:0/3907715618 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6de4038890 0x7f6de403ad50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006fd0 tx=0x7f6df0006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:22.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.167+0000 7f6df97fa700 1 -- 172.21.15.22:0/3907715618 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6dec02bba0 con 0x7f6dfc101360 2023-09-20T21:40:22.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.381+0000 7f6e02311700 1 -- 172.21.15.22:0/3907715618 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dfc18f0f0 con 0x7f6de4038890 2023-09-20T21:40:22.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.385+0000 7f6df97fa700 1 -- 172.21.15.22:0/3907715618 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6dfc18f0f0 con 0x7f6de4038890 2023-09-20T21:40:22.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.387+0000 7f6e02311700 1 -- 172.21.15.22:0/3907715618 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6de4038890 msgr2=0x7f6de403ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:22.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.387+0000 7f6e02311700 1 --2- 172.21.15.22:0/3907715618 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6de4038890 0x7f6de403ad50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006fd0 tx=0x7f6df0006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.387+0000 7f6e02311700 1 -- 172.21.15.22:0/3907715618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dfc101360 msgr2=0x7f6dfc100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.387+0000 7f6e02311700 1 --2- 172.21.15.22:0/3907715618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dfc101360 0x7f6dfc100cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005f50 tx=0x7f6dec0057c0 comp rx=0 tx=0).stop 2023-09-20T21:40:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.387+0000 7f6e02311700 1 -- 172.21.15.22:0/3907715618 shutdown_connections 2023-09-20T21:40:22.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.387+0000 7f6e02311700 1 --2- 172.21.15.22:0/3907715618 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6de4038890 0x7f6de403ad50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:22.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.387+0000 7f6e02311700 1 --2- 172.21.15.22:0/3907715618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dfc101360 0x7f6dfc100cc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:22.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.387+0000 7f6e02311700 1 -- 172.21.15.22:0/3907715618 >> 172.21.15.22:0/3907715618 conn(0x7f6dfc0fad70 msgr2=0x7f6dfc0fbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:22.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.388+0000 7f6e02311700 1 -- 172.21.15.22:0/3907715618 shutdown_connections 2023-09-20T21:40:22.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:22.388+0000 7f6e02311700 1 -- 172.21.15.22:0/3907715618 wait complete. 2023-09-20T21:40:22.801 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi089 2023-09-20T21:40:22.802 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:40:22.802 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/etc/ceph/ceph.conf 2023-09-20T21:40:22.822 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:40:22.823 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:40:22.882 INFO:tasks.cephadm:Adding host smithi089 to orchestrator... 2023-09-20T21:40:22.883 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch host add smithi089 2023-09-20T21:40:23.634 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:40:23.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:23 smithi022 ceph-mon[97651]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:40:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:40:23.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:23 smithi022 ceph-mon[97651]: Updating smithi022:/etc/ceph/ceph.conf 2023-09-20T21:40:23.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:23 smithi022 ceph-mon[97651]: Updating smithi022:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:40:24.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:24 smithi022 ceph-mon[97651]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:40:24.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:24 smithi022 ceph-mon[97651]: Updating smithi022:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-20T21:40:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.113+0000 7f789e380700 1 -- 172.21.15.22:0/1076640752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7898106870 msgr2=0x7f7898106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.113+0000 7f789e380700 1 --2- 172.21.15.22:0/1076640752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7898106870 0x7f7898106c50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7888009910 tx=0x7f7888009c20 comp rx=0 tx=0).stop 2023-09-20T21:40:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.114+0000 7f789e380700 1 -- 172.21.15.22:0/1076640752 shutdown_connections 2023-09-20T21:40:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.114+0000 7f789e380700 1 --2- 172.21.15.22:0/1076640752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7898106870 0x7f7898106c50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.114+0000 7f789e380700 1 -- 172.21.15.22:0/1076640752 >> 172.21.15.22:0/1076640752 conn(0x7f78980fc3a0 msgr2=0x7f78980fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.114+0000 7f789e380700 1 -- 172.21.15.22:0/1076640752 shutdown_connections 2023-09-20T21:40:25.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.114+0000 7f789e380700 1 -- 172.21.15.22:0/1076640752 wait complete. 2023-09-20T21:40:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.115+0000 7f789e380700 1 Processor -- start 2023-09-20T21:40:25.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.115+0000 7f789e380700 1 -- start start 2023-09-20T21:40:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.115+0000 7f789e380700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7898106870 0x7f7898197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.115+0000 7f789e380700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7898198470 con 0x7f7898106870 2023-09-20T21:40:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.116+0000 7f7897fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7898106870 0x7f7898197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.116+0000 7f7897fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7898106870 0x7f7898197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32944/0 (socket says 172.21.15.22:32944) 2023-09-20T21:40:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.116+0000 7f7897fff700 1 -- 172.21.15.22:0/78269127 learned_addr learned my addr 172.21.15.22:0/78269127 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.116+0000 7f7897fff700 1 -- 172.21.15.22:0/78269127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880095f0 con 0x7f7898106870 2023-09-20T21:40:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.116+0000 7f7897fff700 1 --2- 172.21.15.22:0/78269127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7898106870 0x7f7898197f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7888006010 tx=0x7f7888004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.116+0000 7f78957fa700 1 -- 172.21.15.22:0/78269127 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788801d070 con 0x7f7898106870 2023-09-20T21:40:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.116+0000 7f78957fa700 1 -- 172.21.15.22:0/78269127 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f788800bbb0 con 0x7f7898106870 2023-09-20T21:40:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.116+0000 7f789e380700 1 -- 172.21.15.22:0/78269127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78981986d0 con 0x7f7898106870 2023-09-20T21:40:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.116+0000 7f78957fa700 1 -- 172.21.15.22:0/78269127 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7888003930 con 0x7f7898106870 2023-09-20T21:40:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.117+0000 7f789e380700 1 -- 172.21.15.22:0/78269127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7898198b50 con 0x7f7898106870 2023-09-20T21:40:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.118+0000 7f789e380700 1 -- 172.21.15.22:0/78269127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7898062410 con 0x7f7898106870 2023-09-20T21:40:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.118+0000 7f78957fa700 1 -- 172.21.15.22:0/78269127 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7f788800bd20 con 0x7f7898106870 2023-09-20T21:40:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.118+0000 7f78957fa700 1 --2- 172.21.15.22:0/78269127 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7880038840 0x7f788003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.118+0000 7f78957fa700 1 -- 172.21.15.22:0/78269127 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f788804c6f0 con 0x7f7898106870 2023-09-20T21:40:25.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.118+0000 7f78977fe700 1 --2- 172.21.15.22:0/78269127 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7880038840 0x7f788003ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:25.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.119+0000 7f78977fe700 1 --2- 172.21.15.22:0/78269127 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7880038840 0x7f788003ad00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f788c006fd0 tx=0x7f788c006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:25.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.123+0000 7f78957fa700 1 -- 172.21.15.22:0/78269127 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7888029770 con 0x7f7898106870 2023-09-20T21:40:25.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:25.340+0000 7f789e380700 1 -- 172.21.15.22:0/78269127 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi089", "target": ["mon-mgr", ""]}) v1 -- 0x7f789810acf0 con 0x7f7880038840 2023-09-20T21:40:25.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:25.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:25.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:26.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:26 smithi022 ceph-mon[97651]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi089", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:40:26.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:26 smithi022 ceph-mon[97651]: Deploying cephadm binary to smithi089 2023-09-20T21:40:27.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.806+0000 7f78957fa700 1 -- 172.21.15.22:0/78269127 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f789810acf0 con 0x7f7880038840 2023-09-20T21:40:27.807 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi089' with addr '172.21.15.89' 2023-09-20T21:40:27.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.809+0000 7f789e380700 1 -- 172.21.15.22:0/78269127 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7880038840 msgr2=0x7f788003ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:27.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.809+0000 7f789e380700 1 --2- 172.21.15.22:0/78269127 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7880038840 0x7f788003ad00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f788c006fd0 tx=0x7f788c006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:27.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.809+0000 7f789e380700 1 -- 172.21.15.22:0/78269127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7898106870 msgr2=0x7f7898197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.809+0000 7f789e380700 1 --2- 172.21.15.22:0/78269127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7898106870 0x7f7898197f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7888006010 tx=0x7f7888004ff0 comp rx=0 tx=0).stop 2023-09-20T21:40:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.809+0000 7f789e380700 1 -- 172.21.15.22:0/78269127 shutdown_connections 2023-09-20T21:40:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.809+0000 7f789e380700 1 --2- 172.21.15.22:0/78269127 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7880038840 0x7f788003ad00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.809+0000 7f789e380700 1 --2- 172.21.15.22:0/78269127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7898106870 0x7f7898197f30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.809+0000 7f789e380700 1 -- 172.21.15.22:0/78269127 >> 172.21.15.22:0/78269127 conn(0x7f78980fc3a0 msgr2=0x7f7898108cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.809+0000 7f789e380700 1 -- 172.21.15.22:0/78269127 shutdown_connections 2023-09-20T21:40:27.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:27.809+0000 7f789e380700 1 -- 172.21.15.22:0/78269127 wait complete. 2023-09-20T21:40:28.215 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:28 smithi022 ceph-mon[97651]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:28.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:28.216 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:28.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-20T21:40:28.965 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:40:29.302 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:29 smithi022 ceph-mon[97651]: Added host smithi089 2023-09-20T21:40:29.302 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:29 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:29.302 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:29 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:30.204 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:30 smithi022 ceph-mon[97651]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:30.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.453+0000 7faaa8953700 1 -- 172.21.15.22:0/2844472628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa4106870 msgr2=0x7faaa4106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:30.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.453+0000 7faaa8953700 1 --2- 172.21.15.22:0/2844472628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa4106870 0x7faaa4106c50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faa94009910 tx=0x7faa94009c20 comp rx=0 tx=0).stop 2023-09-20T21:40:30.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.453+0000 7faaa8953700 1 -- 172.21.15.22:0/2844472628 shutdown_connections 2023-09-20T21:40:30.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.453+0000 7faaa8953700 1 --2- 172.21.15.22:0/2844472628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa4106870 0x7faaa4106c50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:30.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.453+0000 7faaa8953700 1 -- 172.21.15.22:0/2844472628 >> 172.21.15.22:0/2844472628 conn(0x7faaa40fc3a0 msgr2=0x7faaa40fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:30.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.453+0000 7faaa8953700 1 -- 172.21.15.22:0/2844472628 shutdown_connections 2023-09-20T21:40:30.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.453+0000 7faaa8953700 1 -- 172.21.15.22:0/2844472628 wait complete. 2023-09-20T21:40:30.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.454+0000 7faaa8953700 1 Processor -- start 2023-09-20T21:40:30.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.454+0000 7faaa8953700 1 -- start start 2023-09-20T21:40:30.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.455+0000 7faaa8953700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa4106870 0x7faaa4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:30.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.455+0000 7faaa8953700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa4198470 con 0x7faaa4106870 2023-09-20T21:40:30.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.455+0000 7faaa259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa4106870 0x7faaa4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:30.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.455+0000 7faaa259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa4106870 0x7faaa4197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52728/0 (socket says 172.21.15.22:52728) 2023-09-20T21:40:30.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.455+0000 7faaa259c700 1 -- 172.21.15.22:0/2096842522 learned_addr learned my addr 172.21.15.22:0/2096842522 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:30.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.455+0000 7faaa259c700 1 -- 172.21.15.22:0/2096842522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa940095f0 con 0x7faaa4106870 2023-09-20T21:40:30.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.455+0000 7faaa259c700 1 --2- 172.21.15.22:0/2096842522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa4106870 0x7faaa4197f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faa94006010 tx=0x7faa94004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:30.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.456+0000 7faa937fe700 1 -- 172.21.15.22:0/2096842522 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa9401d070 con 0x7faaa4106870 2023-09-20T21:40:30.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.456+0000 7faaa8953700 1 -- 172.21.15.22:0/2096842522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa41986d0 con 0x7faaa4106870 2023-09-20T21:40:30.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.456+0000 7faa937fe700 1 -- 172.21.15.22:0/2096842522 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa9400bbb0 con 0x7faaa4106870 2023-09-20T21:40:30.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.456+0000 7faa937fe700 1 -- 172.21.15.22:0/2096842522 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa94003930 con 0x7faaa4106870 2023-09-20T21:40:30.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.456+0000 7faaa8953700 1 -- 172.21.15.22:0/2096842522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa4198b50 con 0x7faaa4106870 2023-09-20T21:40:30.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.457+0000 7faa937fe700 1 -- 172.21.15.22:0/2096842522 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7faa9400b6f0 con 0x7faaa4106870 2023-09-20T21:40:30.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.457+0000 7faaa8953700 1 -- 172.21.15.22:0/2096842522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa4062410 con 0x7faaa4106870 2023-09-20T21:40:30.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.457+0000 7faa937fe700 1 --2- 172.21.15.22:0/2096842522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faa8c038890 0x7faa8c03ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:30.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.457+0000 7faa937fe700 1 -- 172.21.15.22:0/2096842522 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faa9404c620 con 0x7faaa4106870 2023-09-20T21:40:30.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.458+0000 7faaa1d9b700 1 --2- 172.21.15.22:0/2096842522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faa8c038890 0x7faa8c03ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:30.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.458+0000 7faaa1d9b700 1 --2- 172.21.15.22:0/2096842522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faa8c038890 0x7faa8c03ad50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faa98006fd0 tx=0x7faa98006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:30.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.463+0000 7faa937fe700 1 -- 172.21.15.22:0/2096842522 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7faa94029540 con 0x7faaa4106870 2023-09-20T21:40:30.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.678+0000 7faaa8953700 1 -- 172.21.15.22:0/2096842522 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faaa40008d0 con 0x7faa8c038890 2023-09-20T21:40:30.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.679+0000 7faa937fe700 1 -- 172.21.15.22:0/2096842522 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7faaa40008d0 con 0x7faa8c038890 2023-09-20T21:40:30.680 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:30.680 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}] 2023-09-20T21:40:30.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.682+0000 7faaa8953700 1 -- 172.21.15.22:0/2096842522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faa8c038890 msgr2=0x7faa8c03ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:30.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.682+0000 7faaa8953700 1 --2- 172.21.15.22:0/2096842522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faa8c038890 0x7faa8c03ad50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faa98006fd0 tx=0x7faa98006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:30.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.682+0000 7faaa8953700 1 -- 172.21.15.22:0/2096842522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa4106870 msgr2=0x7faaa4197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:30.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.682+0000 7faaa8953700 1 --2- 172.21.15.22:0/2096842522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa4106870 0x7faaa4197f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faa94006010 tx=0x7faa94004ff0 comp rx=0 tx=0).stop 2023-09-20T21:40:30.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.682+0000 7faaa8953700 1 -- 172.21.15.22:0/2096842522 shutdown_connections 2023-09-20T21:40:30.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.682+0000 7faaa8953700 1 --2- 172.21.15.22:0/2096842522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faa8c038890 0x7faa8c03ad50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.682+0000 7faaa8953700 1 --2- 172.21.15.22:0/2096842522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa4106870 0x7faaa4197f30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.682+0000 7faaa8953700 1 -- 172.21.15.22:0/2096842522 >> 172.21.15.22:0/2096842522 conn(0x7faaa40fc3a0 msgr2=0x7faaa4108ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.682+0000 7faaa8953700 1 -- 172.21.15.22:0/2096842522 shutdown_connections 2023-09-20T21:40:30.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:30.682+0000 7faaa8953700 1 -- 172.21.15.22:0/2096842522 wait complete. 2023-09-20T21:40:31.141 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2023-09-20T21:40:31.142 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:40:31.142 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2023-09-20T21:40:31.163 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:40:31.163 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:40:31.223 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2023-09-20T21:40:31.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch host add smithi123 2023-09-20T21:40:31.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:31.958 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:40:32.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:32 smithi022 ceph-mon[97651]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:40:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:32 smithi022 ceph-mon[97651]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:33.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.459+0000 7f1bd23b7700 1 -- 172.21.15.22:0/4192215757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bcc106870 msgr2=0x7f1bcc106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.459+0000 7f1bd23b7700 1 --2- 172.21.15.22:0/4192215757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bcc106870 0x7f1bcc106c50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009910 tx=0x7f1bbc009c20 comp rx=0 tx=0).stop 2023-09-20T21:40:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.459+0000 7f1bd23b7700 1 -- 172.21.15.22:0/4192215757 shutdown_connections 2023-09-20T21:40:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.459+0000 7f1bd23b7700 1 --2- 172.21.15.22:0/4192215757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bcc106870 0x7f1bcc106c50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.459+0000 7f1bd23b7700 1 -- 172.21.15.22:0/4192215757 >> 172.21.15.22:0/4192215757 conn(0x7f1bcc0fc3a0 msgr2=0x7f1bcc0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.459+0000 7f1bd23b7700 1 -- 172.21.15.22:0/4192215757 shutdown_connections 2023-09-20T21:40:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.459+0000 7f1bd23b7700 1 -- 172.21.15.22:0/4192215757 wait complete. 2023-09-20T21:40:33.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.460+0000 7f1bd23b7700 1 Processor -- start 2023-09-20T21:40:33.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.461+0000 7f1bd23b7700 1 -- start start 2023-09-20T21:40:33.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.461+0000 7f1bd23b7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bcc106870 0x7f1bcc195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:33.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.461+0000 7f1bd23b7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc196280 con 0x7f1bcc106870 2023-09-20T21:40:33.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.461+0000 7f1bcbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bcc106870 0x7f1bcc195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:33.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.461+0000 7f1bcbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bcc106870 0x7f1bcc195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52746/0 (socket says 172.21.15.22:52746) 2023-09-20T21:40:33.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.461+0000 7f1bcbfff700 1 -- 172.21.15.22:0/1100078910 learned_addr learned my addr 172.21.15.22:0/1100078910 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:33.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.461+0000 7f1bcbfff700 1 -- 172.21.15.22:0/1100078910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc0095f0 con 0x7f1bcc106870 2023-09-20T21:40:33.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.462+0000 7f1bcbfff700 1 --2- 172.21.15.22:0/1100078910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bcc106870 0x7f1bcc195d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc006010 tx=0x7f1bbc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:33.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.462+0000 7f1bc97fa700 1 -- 172.21.15.22:0/1100078910 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bbc01d070 con 0x7f1bcc106870 2023-09-20T21:40:33.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.462+0000 7f1bc97fa700 1 -- 172.21.15.22:0/1100078910 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bbc00bbb0 con 0x7f1bcc106870 2023-09-20T21:40:33.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.462+0000 7f1bc97fa700 1 -- 172.21.15.22:0/1100078910 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bbc003930 con 0x7f1bcc106870 2023-09-20T21:40:33.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.462+0000 7f1bd23b7700 1 -- 172.21.15.22:0/1100078910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc1964e0 con 0x7f1bcc106870 2023-09-20T21:40:33.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.462+0000 7f1bd23b7700 1 -- 172.21.15.22:0/1100078910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc196960 con 0x7f1bcc106870 2023-09-20T21:40:33.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.463+0000 7f1bc97fa700 1 -- 172.21.15.22:0/1100078910 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7f1bbc00bd20 con 0x7f1bcc106870 2023-09-20T21:40:33.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.463+0000 7f1bd23b7700 1 -- 172.21.15.22:0/1100078910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc062410 con 0x7f1bcc106870 2023-09-20T21:40:33.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.464+0000 7f1bc97fa700 1 --2- 172.21.15.22:0/1100078910 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bb4038890 0x7f1bb403ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:33.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.464+0000 7f1bc97fa700 1 -- 172.21.15.22:0/1100078910 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1bbc04c690 con 0x7f1bcc106870 2023-09-20T21:40:33.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.464+0000 7f1bcb7fe700 1 --2- 172.21.15.22:0/1100078910 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bb4038890 0x7f1bb403ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:33.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.465+0000 7f1bcb7fe700 1 --2- 172.21.15.22:0/1100078910 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bb4038890 0x7f1bb403ad50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0006fd0 tx=0x7f1bc0006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.469+0000 7f1bc97fa700 1 -- 172.21.15.22:0/1100078910 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1bbc029360 con 0x7f1bcc106870 2023-09-20T21:40:33.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:33.684+0000 7f1bd23b7700 1 -- 172.21.15.22:0/1100078910 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bcc0008d0 con 0x7f1bb4038890 2023-09-20T21:40:34.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:34 smithi022 ceph-mon[97651]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:34.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:34.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:34.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:34.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:40:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:40:35.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:35 smithi022 ceph-mon[97651]: Updating smithi089:/etc/ceph/ceph.conf 2023-09-20T21:40:35.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:35 smithi022 ceph-mon[97651]: Updating smithi089:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:40:35.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:35 smithi022 ceph-mon[97651]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:40:35.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:35 smithi022 ceph-mon[97651]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:40:35.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:35 smithi022 ceph-mon[97651]: Updating smithi089:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-20T21:40:35.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:35.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:35.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.143+0000 7f1bc97fa700 1 -- 172.21.15.22:0/1100078910 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f1bcc0008d0 con 0x7f1bb4038890 2023-09-20T21:40:36.144 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi123' with addr '172.21.15.123' 2023-09-20T21:40:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.146+0000 7f1bd23b7700 1 -- 172.21.15.22:0/1100078910 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bb4038890 msgr2=0x7f1bb403ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.146+0000 7f1bd23b7700 1 --2- 172.21.15.22:0/1100078910 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bb4038890 0x7f1bb403ad50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0006fd0 tx=0x7f1bc0006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.146+0000 7f1bd23b7700 1 -- 172.21.15.22:0/1100078910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bcc106870 msgr2=0x7f1bcc195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.146+0000 7f1bd23b7700 1 --2- 172.21.15.22:0/1100078910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bcc106870 0x7f1bcc195d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc006010 tx=0x7f1bbc004ff0 comp rx=0 tx=0).stop 2023-09-20T21:40:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.146+0000 7f1bd23b7700 1 -- 172.21.15.22:0/1100078910 shutdown_connections 2023-09-20T21:40:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.146+0000 7f1bd23b7700 1 --2- 172.21.15.22:0/1100078910 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bb4038890 0x7f1bb403ad50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.146+0000 7f1bd23b7700 1 --2- 172.21.15.22:0/1100078910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bcc106870 0x7f1bcc195d40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.146+0000 7f1bd23b7700 1 -- 172.21.15.22:0/1100078910 >> 172.21.15.22:0/1100078910 conn(0x7f1bcc0fc3a0 msgr2=0x7f1bcc108d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.146+0000 7f1bd23b7700 1 -- 172.21.15.22:0/1100078910 shutdown_connections 2023-09-20T21:40:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:36.146+0000 7f1bd23b7700 1 -- 172.21.15.22:0/1100078910 wait complete. 2023-09-20T21:40:36.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:36 smithi022 ceph-mon[97651]: Deploying cephadm binary to smithi123 2023-09-20T21:40:36.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:36 smithi022 ceph-mon[97651]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:36.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:36 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:36.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:36 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:36.540 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch host ls --format=json 2023-09-20T21:40:37.276 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:40:37.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:37 smithi022 ceph-mon[97651]: Added host smithi123 2023-09-20T21:40:37.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:38.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:38 smithi022 ceph-mon[97651]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:38.419 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:38 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:38.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.668+0000 7f1759da0700 1 -- 172.21.15.22:0/3138204312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1754106870 msgr2=0x7f1754106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:38.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.668+0000 7f1759da0700 1 --2- 172.21.15.22:0/3138204312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1754106870 0x7f1754106c50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1744009910 tx=0x7f1744009c20 comp rx=0 tx=0).stop 2023-09-20T21:40:38.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.668+0000 7f1759da0700 1 -- 172.21.15.22:0/3138204312 shutdown_connections 2023-09-20T21:40:38.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.668+0000 7f1759da0700 1 --2- 172.21.15.22:0/3138204312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1754106870 0x7f1754106c50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:38.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.668+0000 7f1759da0700 1 -- 172.21.15.22:0/3138204312 >> 172.21.15.22:0/3138204312 conn(0x7f17540fc3a0 msgr2=0x7f17540fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:38.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.669+0000 7f1759da0700 1 -- 172.21.15.22:0/3138204312 shutdown_connections 2023-09-20T21:40:38.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.669+0000 7f1759da0700 1 -- 172.21.15.22:0/3138204312 wait complete. 2023-09-20T21:40:38.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.670+0000 7f1759da0700 1 Processor -- start 2023-09-20T21:40:38.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.670+0000 7f1759da0700 1 -- start start 2023-09-20T21:40:38.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.670+0000 7f1759da0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1754106870 0x7f17540730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:38.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.670+0000 7f1759da0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1754075060 con 0x7f1754106870 2023-09-20T21:40:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.670+0000 7f17537fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1754106870 0x7f17540730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.670+0000 7f17537fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1754106870 0x7f17540730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59160/0 (socket says 172.21.15.22:59160) 2023-09-20T21:40:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.670+0000 7f17537fe700 1 -- 172.21.15.22:0/4184420748 learned_addr learned my addr 172.21.15.22:0/4184420748 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.671+0000 7f17537fe700 1 -- 172.21.15.22:0/4184420748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440095f0 con 0x7f1754106870 2023-09-20T21:40:38.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.671+0000 7f17537fe700 1 --2- 172.21.15.22:0/4184420748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1754106870 0x7f17540730e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1744006010 tx=0x7f1744004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.671+0000 7f1750ff9700 1 -- 172.21.15.22:0/4184420748 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f174401d070 con 0x7f1754106870 2023-09-20T21:40:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.671+0000 7f1759da0700 1 -- 172.21.15.22:0/4184420748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1754073620 con 0x7f1754106870 2023-09-20T21:40:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.671+0000 7f1750ff9700 1 -- 172.21.15.22:0/4184420748 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f174400bc10 con 0x7f1754106870 2023-09-20T21:40:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.671+0000 7f1750ff9700 1 -- 172.21.15.22:0/4184420748 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1744003910 con 0x7f1754106870 2023-09-20T21:40:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.671+0000 7f1759da0700 1 -- 172.21.15.22:0/4184420748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1754073aa0 con 0x7f1754106870 2023-09-20T21:40:38.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.672+0000 7f1750ff9700 1 -- 172.21.15.22:0/4184420748 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7f174400b710 con 0x7f1754106870 2023-09-20T21:40:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.673+0000 7f1759da0700 1 -- 172.21.15.22:0/4184420748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1754104b20 con 0x7f1754106870 2023-09-20T21:40:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.673+0000 7f1750ff9700 1 --2- 172.21.15.22:0/4184420748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f173c038890 0x7f173c03ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.673+0000 7f1750ff9700 1 -- 172.21.15.22:0/4184420748 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f174404c690 con 0x7f1754106870 2023-09-20T21:40:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.673+0000 7f1752ffd700 1 --2- 172.21.15.22:0/4184420748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f173c038890 0x7f173c03ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:38.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.674+0000 7f1752ffd700 1 --2- 172.21.15.22:0/4184420748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f173c038890 0x7f173c03ad50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1748006fd0 tx=0x7f1748006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:38.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.678+0000 7f1750ff9700 1 -- 172.21.15.22:0/4184420748 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1744029540 con 0x7f1754106870 2023-09-20T21:40:38.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.896+0000 7f1759da0700 1 -- 172.21.15.22:0/4184420748 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1754073da0 con 0x7f173c038890 2023-09-20T21:40:38.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.897+0000 7f1750ff9700 1 -- 172.21.15.22:0/4184420748 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f1754073da0 con 0x7f173c038890 2023-09-20T21:40:38.898 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:40:38.898 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2023-09-20T21:40:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.899+0000 7f1759da0700 1 -- 172.21.15.22:0/4184420748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f173c038890 msgr2=0x7f173c03ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.899+0000 7f1759da0700 1 --2- 172.21.15.22:0/4184420748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f173c038890 0x7f173c03ad50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1748006fd0 tx=0x7f1748006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.899+0000 7f1759da0700 1 -- 172.21.15.22:0/4184420748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1754106870 msgr2=0x7f17540730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.899+0000 7f1759da0700 1 --2- 172.21.15.22:0/4184420748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1754106870 0x7f17540730e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1744006010 tx=0x7f1744004ff0 comp rx=0 tx=0).stop 2023-09-20T21:40:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.899+0000 7f1759da0700 1 -- 172.21.15.22:0/4184420748 shutdown_connections 2023-09-20T21:40:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.899+0000 7f1759da0700 1 --2- 172.21.15.22:0/4184420748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f173c038890 0x7f173c03ad50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.899+0000 7f1759da0700 1 --2- 172.21.15.22:0/4184420748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1754106870 0x7f17540730e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.899+0000 7f1759da0700 1 -- 172.21.15.22:0/4184420748 >> 172.21.15.22:0/4184420748 conn(0x7f17540fc3a0 msgr2=0x7f1754108d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:38.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.899+0000 7f1759da0700 1 -- 172.21.15.22:0/4184420748 shutdown_connections 2023-09-20T21:40:38.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:38.900+0000 7f1759da0700 1 -- 172.21.15.22:0/4184420748 wait complete. 2023-09-20T21:40:39.359 INFO:tasks.cephadm:Setting crush tunables to default 2023-09-20T21:40:39.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd crush tunables default 2023-09-20T21:40:39.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:39.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:39 smithi022 ceph-mon[97651]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:40:39.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:39 smithi022 ceph-mon[97651]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:40:41.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.604+0000 7f85225cf700 1 -- 172.21.15.22:0/2349042945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f851c106850 msgr2=0x7f851c106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:41.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.604+0000 7f85225cf700 1 --2- 172.21.15.22:0/2349042945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f851c106850 0x7f851c106c30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f850c009910 tx=0x7f850c009c20 comp rx=0 tx=0).stop 2023-09-20T21:40:41.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.604+0000 7f85225cf700 1 -- 172.21.15.22:0/2349042945 shutdown_connections 2023-09-20T21:40:41.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.604+0000 7f85225cf700 1 --2- 172.21.15.22:0/2349042945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f851c106850 0x7f851c106c30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:41.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.604+0000 7f85225cf700 1 -- 172.21.15.22:0/2349042945 >> 172.21.15.22:0/2349042945 conn(0x7f851c0fc340 msgr2=0x7f851c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:41.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.604+0000 7f85225cf700 1 -- 172.21.15.22:0/2349042945 shutdown_connections 2023-09-20T21:40:41.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.605+0000 7f85225cf700 1 -- 172.21.15.22:0/2349042945 wait complete. 2023-09-20T21:40:41.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.605+0000 7f85225cf700 1 Processor -- start 2023-09-20T21:40:41.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.606+0000 7f85225cf700 1 -- start start 2023-09-20T21:40:41.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.606+0000 7f85225cf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f851c106850 0x7f851c0730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:41.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.606+0000 7f85225cf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0750b0 con 0x7f851c106850 2023-09-20T21:40:41.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.606+0000 7f851bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f851c106850 0x7f851c0730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:41.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.606+0000 7f851bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f851c106850 0x7f851c0730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59166/0 (socket says 172.21.15.22:59166) 2023-09-20T21:40:41.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.606+0000 7f851bfff700 1 -- 172.21.15.22:0/1660139695 learned_addr learned my addr 172.21.15.22:0/1660139695 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:40:41.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.607+0000 7f851bfff700 1 -- 172.21.15.22:0/1660139695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c0095f0 con 0x7f851c106850 2023-09-20T21:40:41.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.607+0000 7f851bfff700 1 --2- 172.21.15.22:0/1660139695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f851c106850 0x7f851c0730e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f850c006010 tx=0x7f850c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:41.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.607+0000 7f85197fa700 1 -- 172.21.15.22:0/1660139695 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f850c01d070 con 0x7f851c106850 2023-09-20T21:40:41.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.607+0000 7f85197fa700 1 -- 172.21.15.22:0/1660139695 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850c00bc10 con 0x7f851c106850 2023-09-20T21:40:41.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.607+0000 7f85225cf700 1 -- 172.21.15.22:0/1660139695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c0736e0 con 0x7f851c106850 2023-09-20T21:40:41.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.607+0000 7f85197fa700 1 -- 172.21.15.22:0/1660139695 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f850c003910 con 0x7f851c106850 2023-09-20T21:40:41.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.607+0000 7f85225cf700 1 -- 172.21.15.22:0/1660139695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c073b60 con 0x7f851c106850 2023-09-20T21:40:41.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.608+0000 7f85197fa700 1 -- 172.21.15.22:0/1660139695 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7f850c003bd0 con 0x7f851c106850 2023-09-20T21:40:41.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.608+0000 7f85225cf700 1 -- 172.21.15.22:0/1660139695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c104b00 con 0x7f851c106850 2023-09-20T21:40:41.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.609+0000 7f85197fa700 1 --2- 172.21.15.22:0/1660139695 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8504038840 0x7f850403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:41.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.609+0000 7f85197fa700 1 -- 172.21.15.22:0/1660139695 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f850c04c770 con 0x7f851c106850 2023-09-20T21:40:41.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.609+0000 7f851b7fe700 1 --2- 172.21.15.22:0/1660139695 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8504038840 0x7f850403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:41.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.610+0000 7f851b7fe700 1 --2- 172.21.15.22:0/1660139695 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8504038840 0x7f850403ad00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:41.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.617+0000 7f85197fa700 1 -- 172.21.15.22:0/1660139695 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f850c029950 con 0x7f851c106850 2023-09-20T21:40:41.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:41.831+0000 7f85225cf700 1 -- 172.21.15.22:0/1660139695 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f851c073ee0 con 0x7f851c106850 2023-09-20T21:40:42.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.479+0000 7f85197fa700 1 -- 172.21.15.22:0/1660139695 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f850c026020 con 0x7f851c106850 2023-09-20T21:40:42.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.481+0000 7f85225cf700 1 -- 172.21.15.22:0/1660139695 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8504038840 msgr2=0x7f850403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:42.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.481+0000 7f85225cf700 1 --2- 172.21.15.22:0/1660139695 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8504038840 0x7f850403ad00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:42.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.481+0000 7f85225cf700 1 -- 172.21.15.22:0/1660139695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f851c106850 msgr2=0x7f851c0730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:42.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.481+0000 7f85225cf700 1 --2- 172.21.15.22:0/1660139695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f851c106850 0x7f851c0730e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f850c006010 tx=0x7f850c004ff0 comp rx=0 tx=0).stop 2023-09-20T21:40:42.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.481+0000 7f85225cf700 1 -- 172.21.15.22:0/1660139695 shutdown_connections 2023-09-20T21:40:42.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.481+0000 7f85225cf700 1 --2- 172.21.15.22:0/1660139695 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8504038840 0x7f850403ad00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:42.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.481+0000 7f85225cf700 1 --2- 172.21.15.22:0/1660139695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f851c106850 0x7f851c0730e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:42.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.481+0000 7f85225cf700 1 -- 172.21.15.22:0/1660139695 >> 172.21.15.22:0/1660139695 conn(0x7f851c0fc340 msgr2=0x7f851c108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:42.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.482+0000 7f85225cf700 1 -- 172.21.15.22:0/1660139695 shutdown_connections 2023-09-20T21:40:42.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:40:42.482+0000 7f85225cf700 1 -- 172.21.15.22:0/1660139695 wait complete. 2023-09-20T21:40:42.486 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2023-09-20T21:40:42.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:42 smithi022 ceph-mon[97651]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:42.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:42.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:42.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:42.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:42.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:40:42.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:42.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:40:42.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:42 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1660139695' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-20T21:40:42.868 INFO:tasks.cephadm:Adding mon.a on smithi022 2023-09-20T21:40:42.869 INFO:tasks.cephadm:Adding mon.b on smithi089 2023-09-20T21:40:42.869 INFO:tasks.cephadm:Adding mon.c on smithi123 2023-09-20T21:40:42.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch apply mon '3;smithi022:172.21.15.22=a;smithi089:172.21.15.89=b;smithi123:172.21.15.123=c' 2023-09-20T21:40:43.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:43 smithi022 ceph-mon[97651]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-20T21:40:43.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:43 smithi022 ceph-mon[97651]: Updating smithi123:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:40:43.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:43 smithi022 ceph-mon[97651]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:40:43.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:43 smithi022 ceph-mon[97651]: Updating smithi123:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-20T21:40:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:43 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1660139695' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-20T21:40:43.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:43 smithi022 ceph-mon[97651]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:40:43.453 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:40:44.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:44 smithi022 ceph-mon[97651]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:44.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.548+0000 7f4f36011700 1 -- 172.21.15.123:0/1447837817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f301057d0 msgr2=0x7f4f30105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:44.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.548+0000 7f4f36011700 1 --2- 172.21.15.123:0/1447837817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f301057d0 0x7f4f30105bb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009910 tx=0x7f4f24009c20 comp rx=0 tx=0).stop 2023-09-20T21:40:44.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.548+0000 7f4f36011700 1 -- 172.21.15.123:0/1447837817 shutdown_connections 2023-09-20T21:40:44.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.548+0000 7f4f36011700 1 --2- 172.21.15.123:0/1447837817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f301057d0 0x7f4f30105bb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:44.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.548+0000 7f4f36011700 1 -- 172.21.15.123:0/1447837817 >> 172.21.15.123:0/1447837817 conn(0x7f4f300752e0 msgr2=0x7f4f300756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:44.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.548+0000 7f4f36011700 1 -- 172.21.15.123:0/1447837817 shutdown_connections 2023-09-20T21:40:44.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.548+0000 7f4f36011700 1 -- 172.21.15.123:0/1447837817 wait complete. 2023-09-20T21:40:44.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.549+0000 7f4f36011700 1 Processor -- start 2023-09-20T21:40:44.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.549+0000 7f4f36011700 1 -- start start 2023-09-20T21:40:44.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.549+0000 7f4f36011700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f301057d0 0x7f4f30196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:44.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.549+0000 7f4f36011700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f301970e0 con 0x7f4f301057d0 2023-09-20T21:40:44.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.549+0000 7f4f2f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f301057d0 0x7f4f30196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:44.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.549+0000 7f4f2f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f301057d0 0x7f4f30196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.123:57238/0 (socket says 172.21.15.123:57238) 2023-09-20T21:40:44.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.549+0000 7f4f2f7fe700 1 -- 172.21.15.123:0/4152422905 learned_addr learned my addr 172.21.15.123:0/4152422905 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-20T21:40:44.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.550+0000 7f4f2f7fe700 1 -- 172.21.15.123:0/4152422905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f240095f0 con 0x7f4f301057d0 2023-09-20T21:40:44.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.550+0000 7f4f2f7fe700 1 --2- 172.21.15.123:0/4152422905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f301057d0 0x7f4f30196ba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f24006010 tx=0x7f4f24004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:44.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.550+0000 7f4f2cff9700 1 -- 172.21.15.123:0/4152422905 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f2401d070 con 0x7f4f301057d0 2023-09-20T21:40:44.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.550+0000 7f4f2cff9700 1 -- 172.21.15.123:0/4152422905 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f240044a0 con 0x7f4f301057d0 2023-09-20T21:40:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.550+0000 7f4f36011700 1 -- 172.21.15.123:0/4152422905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f301972e0 con 0x7f4f301057d0 2023-09-20T21:40:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.550+0000 7f4f2cff9700 1 -- 172.21.15.123:0/4152422905 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f24003a70 con 0x7f4f301057d0 2023-09-20T21:40:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.550+0000 7f4f36011700 1 -- 172.21.15.123:0/4152422905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f30197700 con 0x7f4f301057d0 2023-09-20T21:40:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.551+0000 7f4f2cff9700 1 -- 172.21.15.123:0/4152422905 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7f4f24003cb0 con 0x7f4f301057d0 2023-09-20T21:40:44.555 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.551+0000 7f4f36011700 1 -- 172.21.15.123:0/4152422905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f30103a10 con 0x7f4f301057d0 2023-09-20T21:40:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.551+0000 7f4f2cff9700 1 --2- 172.21.15.123:0/4152422905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f18038840 0x7f4f1803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.551+0000 7f4f2cff9700 1 -- 172.21.15.123:0/4152422905 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4f2404c810 con 0x7f4f301057d0 2023-09-20T21:40:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.551+0000 7f4f2effd700 1 --2- 172.21.15.123:0/4152422905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f18038840 0x7f4f1803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:44.556 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.552+0000 7f4f2effd700 1 --2- 172.21.15.123:0/4152422905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f18038840 0x7f4f1803ad00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4f20006fd0 tx=0x7f4f20006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:44.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.557+0000 7f4f2cff9700 1 -- 172.21.15.123:0/4152422905 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4f24029950 con 0x7f4f301057d0 2023-09-20T21:40:44.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.755+0000 7f4f36011700 1 -- 172.21.15.123:0/4152422905 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi089:172.21.15.89=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f30068760 con 0x7f4f18038840 2023-09-20T21:40:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.762+0000 7f4f2cff9700 1 -- 172.21.15.123:0/4152422905 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4f30068760 con 0x7f4f18038840 2023-09-20T21:40:44.763 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mon update... 2023-09-20T21:40:44.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.764+0000 7f4f36011700 1 -- 172.21.15.123:0/4152422905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f18038840 msgr2=0x7f4f1803ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:44.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.764+0000 7f4f36011700 1 --2- 172.21.15.123:0/4152422905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f18038840 0x7f4f1803ad00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4f20006fd0 tx=0x7f4f20006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:44.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.764+0000 7f4f36011700 1 -- 172.21.15.123:0/4152422905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f301057d0 msgr2=0x7f4f30196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.764+0000 7f4f36011700 1 --2- 172.21.15.123:0/4152422905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f301057d0 0x7f4f30196ba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f24006010 tx=0x7f4f24004ff0 comp rx=0 tx=0).stop 2023-09-20T21:40:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.764+0000 7f4f36011700 1 -- 172.21.15.123:0/4152422905 shutdown_connections 2023-09-20T21:40:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.764+0000 7f4f36011700 1 --2- 172.21.15.123:0/4152422905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4f18038840 0x7f4f1803ad00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.764+0000 7f4f36011700 1 --2- 172.21.15.123:0/4152422905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f301057d0 0x7f4f30196ba0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.764+0000 7f4f36011700 1 -- 172.21.15.123:0/4152422905 >> 172.21.15.123:0/4152422905 conn(0x7f4f300752e0 msgr2=0x7f4f300ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.764+0000 7f4f36011700 1 -- 172.21.15.123:0/4152422905 shutdown_connections 2023-09-20T21:40:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:44.764+0000 7f4f36011700 1 -- 172.21.15.123:0/4152422905 wait complete. 2023-09-20T21:40:45.217 DEBUG:teuthology.orchestra.run.smithi089:mon.b> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.b.service 2023-09-20T21:40:45.219 DEBUG:teuthology.orchestra.run.smithi123:mon.c> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.c.service 2023-09-20T21:40:45.222 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-20T21:40:45.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph mon dump -f json 2023-09-20T21:40:45.252 INFO:journalctl@ceph.mon.c.smithi123.stdout:-- Logs begin at Wed 2023-09-20 21:17:48 UTC. -- 2023-09-20T21:40:45.255 INFO:journalctl@ceph.mon.b.smithi089.stdout:-- Logs begin at Wed 2023-09-20 21:17:56 UTC. -- 2023-09-20T21:40:45.816 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:40:46.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi089:172.21.15.89=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:40:46.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi089:172.21.15.89=b;smithi123:172.21.15.123=c;count:3 2023-09-20T21:40:46.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:46.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:46.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:46.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:40:46.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:46.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:40:46.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:46.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: Deploying daemon mon.c on smithi123 2023-09-20T21:40:46.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:45 smithi022 ceph-mon[97651]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:47.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.680+0000 7ff4dbd00700 1 -- 172.21.15.123:0/259790764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41000e0 msgr2=0x7ff4d41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:47.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.680+0000 7ff4dbd00700 1 --2- 172.21.15.123:0/259790764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41000e0 0x7ff4d41004c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009910 tx=0x7ff4c4009c20 comp rx=0 tx=0).stop 2023-09-20T21:40:47.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.680+0000 7ff4dbd00700 1 -- 172.21.15.123:0/259790764 shutdown_connections 2023-09-20T21:40:47.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.680+0000 7ff4dbd00700 1 --2- 172.21.15.123:0/259790764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41000e0 0x7ff4d41004c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:47.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.680+0000 7ff4dbd00700 1 -- 172.21.15.123:0/259790764 >> 172.21.15.123:0/259790764 conn(0x7ff4d4075e40 msgr2=0x7ff4d4076250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:47.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.681+0000 7ff4dbd00700 1 -- 172.21.15.123:0/259790764 shutdown_connections 2023-09-20T21:40:47.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.681+0000 7ff4dbd00700 1 -- 172.21.15.123:0/259790764 wait complete. 2023-09-20T21:40:47.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.681+0000 7ff4dbd00700 1 Processor -- start 2023-09-20T21:40:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.681+0000 7ff4dbd00700 1 -- start start 2023-09-20T21:40:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.682+0000 7ff4dbd00700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41000e0 0x7ff4d4196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.682+0000 7ff4dbd00700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d41970a0 con 0x7ff4d41000e0 2023-09-20T21:40:47.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.682+0000 7ff4d9a9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41000e0 0x7ff4d4196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:47.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.682+0000 7ff4d9a9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41000e0 0x7ff4d4196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.123:57244/0 (socket says 172.21.15.123:57244) 2023-09-20T21:40:47.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.682+0000 7ff4d9a9c700 1 -- 172.21.15.123:0/2485684905 learned_addr learned my addr 172.21.15.123:0/2485684905 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-20T21:40:47.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.682+0000 7ff4d9a9c700 1 -- 172.21.15.123:0/2485684905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c40095f0 con 0x7ff4d41000e0 2023-09-20T21:40:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.683+0000 7ff4d9a9c700 1 --2- 172.21.15.123:0/2485684905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41000e0 0x7ff4d4196b60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4004f40 tx=0x7ff4c4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.683+0000 7ff4caffd700 1 -- 172.21.15.123:0/2485684905 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4c401d070 con 0x7ff4d41000e0 2023-09-20T21:40:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.683+0000 7ff4dbd00700 1 -- 172.21.15.123:0/2485684905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d41972a0 con 0x7ff4d41000e0 2023-09-20T21:40:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.683+0000 7ff4caffd700 1 -- 172.21.15.123:0/2485684905 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c4004370 con 0x7ff4d41000e0 2023-09-20T21:40:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.683+0000 7ff4dbd00700 1 -- 172.21.15.123:0/2485684905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d4197740 con 0x7ff4d41000e0 2023-09-20T21:40:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.683+0000 7ff4caffd700 1 -- 172.21.15.123:0/2485684905 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4c40039b0 con 0x7ff4d41000e0 2023-09-20T21:40:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.684+0000 7ff4dbd00700 1 -- 172.21.15.123:0/2485684905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d4103a60 con 0x7ff4d41000e0 2023-09-20T21:40:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.684+0000 7ff4caffd700 1 -- 172.21.15.123:0/2485684905 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7ff4c4003c70 con 0x7ff4d41000e0 2023-09-20T21:40:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.684+0000 7ff4caffd700 1 --2- 172.21.15.123:0/2485684905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4c0038840 0x7ff4c003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.684+0000 7ff4caffd700 1 -- 172.21.15.123:0/2485684905 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff4c404c670 con 0x7ff4d41000e0 2023-09-20T21:40:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.684+0000 7ff4d929b700 1 --2- 172.21.15.123:0/2485684905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4c0038840 0x7ff4c003ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.685+0000 7ff4d929b700 1 --2- 172.21.15.123:0/2485684905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4c0038840 0x7ff4c003ad00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0006fd0 tx=0x7ff4d0006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:47.690 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:47.690+0000 7ff4caffd700 1 -- 172.21.15.123:0/2485684905 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff4c4029360 con 0x7ff4d41000e0 2023-09-20T21:40:48.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.009+0000 7ff4dbd00700 1 -- 172.21.15.123:0/2485684905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff4d40ff390 con 0x7ff4d41000e0 2023-09-20T21:40:48.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.010+0000 7ff4caffd700 1 -- 172.21.15.123:0/2485684905 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7ff4c4026070 con 0x7ff4d41000e0 2023-09-20T21:40:48.010 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:40:48.011 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","modified":"2023-09-20T21:39:03.785697Z","created":"2023-09-20T21:39:03.785697Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-09-20T21:40:48.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.012+0000 7ff4dbd00700 1 -- 172.21.15.123:0/2485684905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4c0038840 msgr2=0x7ff4c003ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:48.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.012+0000 7ff4dbd00700 1 --2- 172.21.15.123:0/2485684905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4c0038840 0x7ff4c003ad00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0006fd0 tx=0x7ff4d0006e40 comp rx=0 tx=0).stop 2023-09-20T21:40:48.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.012+0000 7ff4dbd00700 1 -- 172.21.15.123:0/2485684905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41000e0 msgr2=0x7ff4d4196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:48.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.012+0000 7ff4dbd00700 1 --2- 172.21.15.123:0/2485684905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41000e0 0x7ff4d4196b60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4004f40 tx=0x7ff4c4005020 comp rx=0 tx=0).stop 2023-09-20T21:40:48.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.012+0000 7ff4dbd00700 1 -- 172.21.15.123:0/2485684905 shutdown_connections 2023-09-20T21:40:48.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.012+0000 7ff4dbd00700 1 --2- 172.21.15.123:0/2485684905 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4c0038840 0x7ff4c003ad00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:48.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.012+0000 7ff4dbd00700 1 --2- 172.21.15.123:0/2485684905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4d41000e0 0x7ff4d4196b60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:48.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.012+0000 7ff4dbd00700 1 -- 172.21.15.123:0/2485684905 >> 172.21.15.123:0/2485684905 conn(0x7ff4d4075e40 msgr2=0x7ff4d40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:48.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.012+0000 7ff4dbd00700 1 -- 172.21.15.123:0/2485684905 shutdown_connections 2023-09-20T21:40:48.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:48.012+0000 7ff4dbd00700 1 -- 172.21.15.123:0/2485684905 wait complete. 2023-09-20T21:40:48.014 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2023-09-20T21:40:48.039 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:47 smithi123 systemd[1]: Starting Ceph mon.c for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:40:48.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:48 smithi022 ceph-mon[97651]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:48 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/2485684905' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-20T21:40:48.676 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 podman[134391]: 2023-09-20T21:40:48.677 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: set uid:gid to 167:167 (ceph:ceph) 2023-09-20T21:40:48.677 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: ceph version 18.0.0-6213-gbf8a061f (bf8a061ff16ac1d393cf108f740d92dfa49f2e54) reef (dev), process ceph-mon, pid 7 2023-09-20T21:40:48.677 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pidfile_write: ignore empty --pid-file 2023-09-20T21:40:48.677 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: load: jerasure load: lrc 2023-09-20T21:40:48.678 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: RocksDB version: 7.9.2 2023-09-20T21:40:48.678 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Git sha 0 2023-09-20T21:40:48.678 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Compile date 2023-09-20 08:00:43 2023-09-20T21:40:48.678 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: DB SUMMARY 2023-09-20T21:40:48.678 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: DB Session ID: Q0RAN0S1V1OMP9S3FC8S 2023-09-20T21:40:48.679 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: CURRENT file: CURRENT 2023-09-20T21:40:48.679 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: IDENTITY file: IDENTITY 2023-09-20T21:40:48.679 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-20T21:40:48.679 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-09-20T21:40:48.679 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-09-20T21:40:48.679 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.error_if_exists: 0 2023-09-20T21:40:48.680 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.create_if_missing: 0 2023-09-20T21:40:48.680 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.paranoid_checks: 1 2023-09-20T21:40:48.680 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-20T21:40:48.680 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-20T21:40:48.680 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-20T21:40:48.681 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.env: 0x56461223fda0 2023-09-20T21:40:48.681 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.fs: PosixFileSystem 2023-09-20T21:40:48.681 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.info_log: 0x56461468f580 2023-09-20T21:40:48.681 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_file_opening_threads: 16 2023-09-20T21:40:48.681 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.statistics: (nil) 2023-09-20T21:40:48.682 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.use_fsync: 0 2023-09-20T21:40:48.682 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_log_file_size: 0 2023-09-20T21:40:48.682 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-20T21:40:48.682 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-20T21:40:48.682 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.keep_log_file_num: 1000 2023-09-20T21:40:48.682 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.recycle_log_file_num: 0 2023-09-20T21:40:48.683 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.allow_fallocate: 1 2023-09-20T21:40:48.683 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.allow_mmap_reads: 0 2023-09-20T21:40:48.683 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.allow_mmap_writes: 0 2023-09-20T21:40:48.683 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.use_direct_reads: 0 2023-09-20T21:40:48.683 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-20T21:40:48.684 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.create_missing_column_families: 0 2023-09-20T21:40:48.684 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.db_log_dir: 2023-09-20T21:40:48.684 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.wal_dir: 2023-09-20T21:40:48.684 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-20T21:40:48.684 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-20T21:40:48.684 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-20T21:40:48.685 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-20T21:40:48.685 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-20T21:40:48.685 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-20T21:40:48.685 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.advise_random_on_open: 1 2023-09-20T21:40:48.685 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.db_write_buffer_size: 0 2023-09-20T21:40:48.686 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.write_buffer_manager: 0x5646146474a0 2023-09-20T21:40:48.686 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-20T21:40:48.686 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-20T21:40:48.686 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-20T21:40:48.686 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.rate_limiter: (nil) 2023-09-20T21:40:48.686 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-20T21:40:48.687 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.wal_recovery_mode: 2 2023-09-20T21:40:48.687 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.enable_thread_tracking: 0 2023-09-20T21:40:48.687 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.enable_pipelined_write: 0 2023-09-20T21:40:48.687 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.unordered_write: 0 2023-09-20T21:40:48.687 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-20T21:40:48.688 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-20T21:40:48.688 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-20T21:40:48.688 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-20T21:40:48.688 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.row_cache: None 2023-09-20T21:40:48.688 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.wal_filter: None 2023-09-20T21:40:48.688 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-20T21:40:48.689 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.allow_ingest_behind: 0 2023-09-20T21:40:48.689 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.two_write_queues: 0 2023-09-20T21:40:48.689 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.manual_wal_flush: 0 2023-09-20T21:40:48.689 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.wal_compression: 0 2023-09-20T21:40:48.689 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.atomic_flush: 0 2023-09-20T21:40:48.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-20T21:40:48.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-20T21:40:48.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-20T21:40:48.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.log_readahead_size: 0 2023-09-20T21:40:48.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-20T21:40:48.690 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.best_efforts_recovery: 0 2023-09-20T21:40:48.691 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-20T21:40:48.691 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-20T21:40:48.691 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.allow_data_in_errors: 0 2023-09-20T21:40:48.691 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.db_host_id: __hostname__ 2023-09-20T21:40:48.691 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-20T21:40:48.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_background_jobs: 2 2023-09-20T21:40:48.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_background_compactions: -1 2023-09-20T21:40:48.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_subcompactions: 1 2023-09-20T21:40:48.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-20T21:40:48.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-20T21:40:48.693 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-20T21:40:48.693 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_total_wal_size: 0 2023-09-20T21:40:48.693 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-20T21:40:48.693 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-20T21:40:48.693 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-20T21:40:48.693 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-20T21:40:48.694 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_open_files: -1 2023-09-20T21:40:48.694 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bytes_per_sync: 0 2023-09-20T21:40:48.694 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-20T21:40:48.694 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-20T21:40:48.694 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_readahead_size: 0 2023-09-20T21:40:48.695 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_background_flushes: -1 2023-09-20T21:40:48.695 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Compression algorithms supported: 2023-09-20T21:40:48.695 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-20T21:40:48.695 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: kZSTD supported: 0 2023-09-20T21:40:48.695 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: kXpressCompression supported: 0 2023-09-20T21:40:48.696 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: kLZ4HCCompression supported: 1 2023-09-20T21:40:48.696 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: kZlibCompression supported: 1 2023-09-20T21:40:48.696 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: kSnappyCompression supported: 1 2023-09-20T21:40:48.696 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: kLZ4Compression supported: 1 2023-09-20T21:40:48.696 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: kBZip2Compression supported: 0 2023-09-20T21:40:48.696 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-20T21:40:48.697 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-20T21:40:48.697 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-09-20T21:40:48.697 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-20T21:40:48.697 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-20T21:40:48.697 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.merge_operator: 2023-09-20T21:40:48.697 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_filter: None 2023-09-20T21:40:48.698 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_filter_factory: None 2023-09-20T21:40:48.698 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.sst_partitioner_factory: None 2023-09-20T21:40:48.698 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-20T21:40:48.698 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-20T21:40:48.698 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56461468f680) 2023-09-20T21:40:48.698 INFO:journalctl@ceph.mon.c.smithi123.stdout: cache_index_and_filter_blocks: 1 2023-09-20T21:40:48.699 INFO:journalctl@ceph.mon.c.smithi123.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-20T21:40:48.699 INFO:journalctl@ceph.mon.c.smithi123.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-20T21:40:48.699 INFO:journalctl@ceph.mon.c.smithi123.stdout: pin_top_level_index_and_filter: 1 2023-09-20T21:40:48.699 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_type: 0 2023-09-20T21:40:48.699 INFO:journalctl@ceph.mon.c.smithi123.stdout: data_block_index_type: 0 2023-09-20T21:40:48.699 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_shortening: 1 2023-09-20T21:40:48.700 INFO:journalctl@ceph.mon.c.smithi123.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-20T21:40:48.700 INFO:journalctl@ceph.mon.c.smithi123.stdout: checksum: 4 2023-09-20T21:40:48.700 INFO:journalctl@ceph.mon.c.smithi123.stdout: no_block_cache: 0 2023-09-20T21:40:48.700 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache: 0x5646146e51f0 2023-09-20T21:40:48.700 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_name: BinnedLRUCache 2023-09-20T21:40:48.701 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_options: 2023-09-20T21:40:48.701 INFO:journalctl@ceph.mon.c.smithi123.stdout: capacity : 536870912 2023-09-20T21:40:48.701 INFO:journalctl@ceph.mon.c.smithi123.stdout: num_shard_bits : 4 2023-09-20T21:40:48.701 INFO:journalctl@ceph.mon.c.smithi123.stdout: strict_capacity_limit : 0 2023-09-20T21:40:48.701 INFO:journalctl@ceph.mon.c.smithi123.stdout: high_pri_pool_ratio: 0.000 2023-09-20T21:40:48.701 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_compressed: (nil) 2023-09-20T21:40:48.701 INFO:journalctl@ceph.mon.c.smithi123.stdout: persistent_cache: (nil) 2023-09-20T21:40:48.702 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_size: 4096 2023-09-20T21:40:48.702 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_size_deviation: 10 2023-09-20T21:40:48.702 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_restart_interval: 16 2023-09-20T21:40:48.702 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_block_restart_interval: 1 2023-09-20T21:40:48.702 INFO:journalctl@ceph.mon.c.smithi123.stdout: metadata_block_size: 4096 2023-09-20T21:40:48.703 INFO:journalctl@ceph.mon.c.smithi123.stdout: partition_filters: 0 2023-09-20T21:40:48.703 INFO:journalctl@ceph.mon.c.smithi123.stdout: use_delta_encoding: 1 2023-09-20T21:40:48.703 INFO:journalctl@ceph.mon.c.smithi123.stdout: filter_policy: bloomfilter 2023-09-20T21:40:48.703 INFO:journalctl@ceph.mon.c.smithi123.stdout: whole_key_filtering: 1 2023-09-20T21:40:48.703 INFO:journalctl@ceph.mon.c.smithi123.stdout: verify_compression: 0 2023-09-20T21:40:48.704 INFO:journalctl@ceph.mon.c.smithi123.stdout: read_amp_bytes_per_bit: 0 2023-09-20T21:40:48.704 INFO:journalctl@ceph.mon.c.smithi123.stdout: format_version: 5 2023-09-20T21:40:48.704 INFO:journalctl@ceph.mon.c.smithi123.stdout: enable_index_compression: 1 2023-09-20T21:40:48.704 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_align: 0 2023-09-20T21:40:48.704 INFO:journalctl@ceph.mon.c.smithi123.stdout: max_auto_readahead_size: 262144 2023-09-20T21:40:48.704 INFO:journalctl@ceph.mon.c.smithi123.stdout: prepopulate_block_cache: 0 2023-09-20T21:40:48.705 INFO:journalctl@ceph.mon.c.smithi123.stdout: initial_auto_readahead_size: 8192 2023-09-20T21:40:48.705 INFO:journalctl@ceph.mon.c.smithi123.stdout: num_file_reads_for_auto_readahead: 2 2023-09-20T21:40:48.705 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.write_buffer_size: 33554432 2023-09-20T21:40:48.705 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_write_buffer_number: 2 2023-09-20T21:40:48.705 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compression: NoCompression 2023-09-20T21:40:48.706 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bottommost_compression: Disabled 2023-09-20T21:40:48.706 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.prefix_extractor: nullptr 2023-09-20T21:40:48.706 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-20T21:40:48.706 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.num_levels: 7 2023-09-20T21:40:48.706 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-20T21:40:48.707 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-20T21:40:48.707 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-20T21:40:48.707 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-20T21:40:48.707 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-20T21:40:48.707 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-20T21:40:48.707 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-20T21:40:48.709 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-20T21:40:48.709 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-20T21:40:48.709 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-20T21:40:48.709 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-20T21:40:48.709 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-20T21:40:48.710 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-20T21:40:48.710 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compression_opts.level: 32767 2023-09-20T21:40:48.710 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compression_opts.strategy: 0 2023-09-20T21:40:48.710 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-20T21:40:48.710 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 bash[134391]: cf9392da2285e1802b9e90577810b9d49d57fddc14832ea0c0ef44dc29f018ac 2023-09-20T21:40:48.710 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-20T21:40:48.711 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-20T21:40:48.711 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-20T21:40:48.711 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compression_opts.enabled: false 2023-09-20T21:40:48.711 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-20T21:40:48.711 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-20T21:40:48.712 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-20T21:40:48.712 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-20T21:40:48.712 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.target_file_size_base: 67108864 2023-09-20T21:40:48.712 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-20T21:40:48.712 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-20T21:40:48.712 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-20T21:40:48.713 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-20T21:40:48.713 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-20T21:40:48.713 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-20T21:40:48.713 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-20T21:40:48.713 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-20T21:40:48.713 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-20T21:40:48.714 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-20T21:40:48.714 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-20T21:40:48.714 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-20T21:40:48.714 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-20T21:40:48.714 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-20T21:40:48.715 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.arena_block_size: 1048576 2023-09-20T21:40:48.715 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-20T21:40:48.715 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-20T21:40:48.715 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.disable_auto_compactions: 0 2023-09-20T21:40:48.715 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-20T21:40:48.715 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-20T21:40:48.716 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-20T21:40:48.716 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-20T21:40:48.716 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-20T21:40:48.716 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-20T21:40:48.716 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-20T21:40:48.717 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-20T21:40:48.717 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-20T21:40:48.717 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-20T21:40:48.717 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.table_properties_collectors: 2023-09-20T21:40:48.717 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.inplace_update_support: 0 2023-09-20T21:40:48.717 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-20T21:40:48.718 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-20T21:40:48.718 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-20T21:40:48.718 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-20T21:40:48.718 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.bloom_locality: 0 2023-09-20T21:40:48.718 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.max_successive_merges: 0 2023-09-20T21:40:48.719 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-20T21:40:48.719 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.paranoid_file_checks: 0 2023-09-20T21:40:48.719 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.force_consistency_checks: 1 2023-09-20T21:40:48.719 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.report_bg_io_stats: 0 2023-09-20T21:40:48.719 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.ttl: 2592000 2023-09-20T21:40:48.719 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-20T21:40:48.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-20T21:40:48.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-20T21:40:48.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.enable_blob_files: false 2023-09-20T21:40:48.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.min_blob_size: 0 2023-09-20T21:40:48.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.blob_file_size: 268435456 2023-09-20T21:40:48.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-20T21:40:48.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-20T21:40:48.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-20T21:40:48.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-20T21:40:48.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-20T21:40:48.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.blob_file_starting_level: 0 2023-09-20T21:40:48.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-20T21:40:48.722 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-20T21:40:48.722 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-20T21:40:48.722 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bbea83e0-8d91-4f07-b5d5-99e10ec3cc85 2023-09-20T21:40:48.722 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695246048650298, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-20T21:40:48.722 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-20T21:40:48.723 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695246048650696, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1695246048, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bbea83e0-8d91-4f07-b5d5-99e10ec3cc85", "db_session_id": "Q0RAN0S1V1OMP9S3FC8S", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-20T21:40:48.723 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695246048650762, "job": 1, "event": "recovery_finished"} 2023-09-20T21:40:48.723 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-20T21:40:48.723 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-20T21:40:48.723 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564614782000 2023-09-20T21:40:48.724 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: DB pointer 0x56461476e000 2023-09-20T21:40:48.724 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-20T21:40:48.724 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-20T21:40:48.724 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** DB Stats ** 2023-09-20T21:40:48.724 INFO:journalctl@ceph.mon.c.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-20T21:40:48.724 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-20T21:40:48.725 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-20T21:40:48.725 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-20T21:40:48.725 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-20T21:40:48.725 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-20T21:40:48.726 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-20T21:40:48.726 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-20T21:40:48.726 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** Compaction Stats [default] ** 2023-09-20T21:40:48.726 INFO:journalctl@ceph.mon.c.smithi123.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-20T21:40:48.726 INFO:journalctl@ceph.mon.c.smithi123.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-20T21:40:48.726 INFO:journalctl@ceph.mon.c.smithi123.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-20T21:40:48.727 INFO:journalctl@ceph.mon.c.smithi123.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-20T21:40:48.727 INFO:journalctl@ceph.mon.c.smithi123.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-20T21:40:48.727 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-20T21:40:48.727 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** Compaction Stats [default] ** 2023-09-20T21:40:48.727 INFO:journalctl@ceph.mon.c.smithi123.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-20T21:40:48.728 INFO:journalctl@ceph.mon.c.smithi123.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-20T21:40:48.728 INFO:journalctl@ceph.mon.c.smithi123.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-09-20T21:40:48.728 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-20T21:40:48.728 INFO:journalctl@ceph.mon.c.smithi123.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-20T21:40:48.728 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-20T21:40:48.728 INFO:journalctl@ceph.mon.c.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-20T21:40:48.729 INFO:journalctl@ceph.mon.c.smithi123.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-20T21:40:48.729 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-20T21:40:48.729 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-20T21:40:48.729 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-20T21:40:48.729 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-20T21:40:48.729 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-20T21:40:48.730 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-20T21:40:48.730 INFO:journalctl@ceph.mon.c.smithi123.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-20T21:40:48.730 INFO:journalctl@ceph.mon.c.smithi123.stdout: Block cache BinnedLRUCache@0x5646146e51f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-09-20T21:40:48.730 INFO:journalctl@ceph.mon.c.smithi123.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-20T21:40:48.730 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-20T21:40:48.731 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-20T21:40:48.731 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-09-20T21:40:48.731 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: using public_addr v2:172.21.15.123:0/0 -> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2023-09-20T21:40:48.731 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: starting mon.c rank -1 at public addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] at bind addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:40:48.731 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(???) e0 preinit fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:40:48.732 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).mds e1 new map 2023-09-20T21:40:48.732 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).mds e1 print_map 2023-09-20T21:40:48.732 INFO:journalctl@ceph.mon.c.smithi123.stdout: e1 2023-09-20T21:40:48.732 INFO:journalctl@ceph.mon.c.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-20T21:40:48.732 INFO:journalctl@ceph.mon.c.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-20T21:40:48.732 INFO:journalctl@ceph.mon.c.smithi123.stdout: legacy client fscid: -1 2023-09-20T21:40:48.733 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2023-09-20T21:40:48.733 INFO:journalctl@ceph.mon.c.smithi123.stdout: No filesystems configured 2023-09-20T21:40:48.733 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-20T21:40:48.733 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-20T21:40:48.733 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-20T21:40:48.734 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-20T21:40:48.734 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-20T21:40:48.734 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-20T21:40:48.734 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-20T21:40:48.734 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-20T21:40:48.734 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-20T21:40:48.735 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-20T21:40:48.735 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.735 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.735 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:40:48.735 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:40:48.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:48.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:40:48.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Updating smithi089:/etc/ceph/ceph.conf 2023-09-20T21:40:48.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Updating smithi089:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:40:48.738 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:40:48.738 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:40:48.738 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Updating smithi089:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-20T21:40:48.738 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.738 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.739 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.739 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Deploying cephadm binary to smithi123 2023-09-20T21:40:48.739 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.739 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.739 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:48.739 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Added host smithi123 2023-09-20T21:40:48.740 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.740 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.740 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.740 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.740 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:40:48.740 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.741 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.741 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.741 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.741 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.741 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:40:48.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:48.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:40:48.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1660139695' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-20T21:40:48.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-20T21:40:48.742 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Updating smithi123:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:40:48.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:40:48.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Updating smithi123:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-20T21:40:48.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.743 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1660139695' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-20T21:40:48.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:40:48.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi089:172.21.15.89=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:40:48.744 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi089:172.21.15.89=b;smithi123:172.21.15.123=c;count:3 2023-09-20T21:40:48.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:48.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:48.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:40:48.745 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:48.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:40:48.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:48.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: Deploying daemon mon.c on smithi123 2023-09-20T21:40:48.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:48.746 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/2485684905' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-20T21:40:48.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-20T21:40:48.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-09-20T21:40:48.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta expand map: {default=false} 2023-09-20T21:40:48.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta from 'false' to 'false' 2023-09-20T21:40:48.747 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta expanded map: {default=false} 2023-09-20T21:40:48.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta expand map: {default=info} 2023-09-20T21:40:48.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta from 'info' to 'info' 2023-09-20T21:40:48.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta expanded map: {default=info} 2023-09-20T21:40:48.748 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta expand map: {default=daemon} 2023-09-20T21:40:49.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-20T21:40:49.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta expanded map: {default=daemon} 2023-09-20T21:40:49.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta expand map: {default=debug} 2023-09-20T21:40:49.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta from 'debug' to 'debug' 2023-09-20T21:40:49.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 ceph-mon[134433]: expand_channel_meta expanded map: {default=debug} 2023-09-20T21:40:49.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:48 smithi123 systemd[1]: Started Ceph mon.c for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:40:49.472 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-20T21:40:49.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph mon dump -f json 2023-09-20T21:40:50.030 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:40:54.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:40:54.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: mon.a calling monitor election 2023-09-20T21:40:54.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:54.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: mon.c calling monitor election 2023-09-20T21:40:54.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:54.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:54.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-20T21:40:54.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-20T21:40:54.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: fsmap 2023-09-20T21:40:54.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:40:54.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: mgrmap e13: x(active, since 46s) 2023-09-20T21:40:54.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: overall HEALTH_OK 2023-09-20T21:40:54.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:54.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:54.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:54.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:40:54.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:54.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:40:54.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: mon.a calling monitor election 2023-09-20T21:40:54.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:54.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: mon.c calling monitor election 2023-09-20T21:40:54.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:54.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:54.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:54.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-20T21:40:54.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-20T21:40:54.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: fsmap 2023-09-20T21:40:54.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:40:54.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: mgrmap e13: x(active, since 46s) 2023-09-20T21:40:54.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: overall HEALTH_OK 2023-09-20T21:40:54.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:54.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:54.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:54.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:40:54.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:54.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.266+0000 7fa422da8700 1 -- 172.21.15.123:0/914421166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c0035c0 msgr2=0x7fa40c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:54.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.266+0000 7fa422da8700 1 --2- 172.21.15.123:0/914421166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c0035c0 0x7fa40c005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0767d0 tx=0x7fa41000feb0 comp rx=0 tx=0).stop 2023-09-20T21:40:54.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.266+0000 7fa422da8700 1 -- 172.21.15.123:0/914421166 shutdown_connections 2023-09-20T21:40:54.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.266+0000 7fa422da8700 1 --2- 172.21.15.123:0/914421166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa40c0035c0 0x7fa40c005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:54.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.266+0000 7fa422da8700 1 -- 172.21.15.123:0/914421166 >> 172.21.15.123:0/914421166 conn(0x7fa41c075390 msgr2=0x7fa41c0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:54.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.266+0000 7fa422da8700 1 -- 172.21.15.123:0/914421166 shutdown_connections 2023-09-20T21:40:54.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.266+0000 7fa422da8700 1 -- 172.21.15.123:0/914421166 wait complete. 2023-09-20T21:40:54.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.267+0000 7fa422da8700 1 Processor -- start 2023-09-20T21:40:54.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.267+0000 7fa422da8700 1 -- start start 2023-09-20T21:40:54.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa422da8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c0035c0 0x7fa41c194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:54.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa422da8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41c1952f0 0x7fa41c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:54.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa422da8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c195880 con 0x7fa41c1952f0 2023-09-20T21:40:54.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa422da8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c1959f0 con 0x7fa40c0035c0 2023-09-20T21:40:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa420b44700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c0035c0 0x7fa41c194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa420b44700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c0035c0 0x7fa41c194db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33778/0 (socket says 172.21.15.123:33778) 2023-09-20T21:40:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa420b44700 1 -- 172.21.15.123:0/3152829642 learned_addr learned my addr 172.21.15.123:0/3152829642 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-20T21:40:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa420b44700 1 -- 172.21.15.123:0/3152829642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c0035c0 msgr2=0x7fa41c194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-09-20T21:40:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa420b44700 1 -- 172.21.15.123:0/3152829642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c0035c0 msgr2=0x7fa41c194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-20T21:40:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa420b44700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c0035c0 0x7fa41c194db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-20T21:40:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa420b44700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c0035c0 0x7fa41c194db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-20T21:40:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa41bfff700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41c1952f0 0x7fa41c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa41bfff700 1 -- 172.21.15.123:0/3152829642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c0035c0 msgr2=0x7fa41c194db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:40:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa41bfff700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c0035c0 0x7fa41c194db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.268+0000 7fa41bfff700 1 -- 172.21.15.123:0/3152829642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41000f900 con 0x7fa41c1952f0 2023-09-20T21:40:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.269+0000 7fa41bfff700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41c1952f0 0x7fa41c199760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa40800b4d0 tx=0x7fa40800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.269+0000 7fa419ffb700 1 -- 172.21.15.123:0/3152829642 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa408011630 con 0x7fa41c1952f0 2023-09-20T21:40:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.269+0000 7fa419ffb700 1 -- 172.21.15.123:0/3152829642 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa408011c50 con 0x7fa41c1952f0 2023-09-20T21:40:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.269+0000 7fa422da8700 1 -- 172.21.15.123:0/3152829642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41c199d00 con 0x7fa41c1952f0 2023-09-20T21:40:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.269+0000 7fa419ffb700 1 -- 172.21.15.123:0/3152829642 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa408010d90 con 0x7fa41c1952f0 2023-09-20T21:40:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.269+0000 7fa422da8700 1 -- 172.21.15.123:0/3152829642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c19a250 con 0x7fa41c1952f0 2023-09-20T21:40:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.270+0000 7fa419ffb700 1 -- 172.21.15.123:0/3152829642 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7fa40801a400 con 0x7fa41c1952f0 2023-09-20T21:40:54.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.270+0000 7fa419ffb700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa400038680 0x7fa40003ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:40:54.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.270+0000 7fa419ffb700 1 -- 172.21.15.123:0/3152829642 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa40804aec0 con 0x7fa41c1952f0 2023-09-20T21:40:54.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.271+0000 7fa422da8700 1 -- 172.21.15.123:0/3152829642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41c19a580 con 0x7fa41c1952f0 2023-09-20T21:40:54.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.271+0000 7fa420b44700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa400038680 0x7fa40003ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:40:54.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.271+0000 7fa420b44700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa400038680 0x7fa40003ab40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa410010010 tx=0x7fa410001e50 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:40:54.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.276+0000 7fa419ffb700 1 -- 172.21.15.123:0/3152829642 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa4080193b0 con 0x7fa41c1952f0 2023-09-20T21:40:54.547 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.546+0000 7fa422da8700 1 -- 172.21.15.123:0/3152829642 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa41c062410 con 0x7fa41c1952f0 2023-09-20T21:40:54.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.547+0000 7fa419ffb700 1 -- 172.21.15.123:0/3152829642 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fa4080172c0 con 0x7fa41c1952f0 2023-09-20T21:40:54.548 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:40:54.548 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":2,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","modified":"2023-09-20T21:40:48.680996Z","created":"2023-09-20T21:39:03.785697Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-20T21:40:54.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 -- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa400038680 msgr2=0x7fa40003ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:54.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa400038680 0x7fa40003ab40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa410010010 tx=0x7fa410001e50 comp rx=0 tx=0).stop 2023-09-20T21:40:54.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 -- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41c1952f0 msgr2=0x7fa41c199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:40:54.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41c1952f0 0x7fa41c199760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa40800b4d0 tx=0x7fa40800b890 comp rx=0 tx=0).stop 2023-09-20T21:40:54.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 -- 172.21.15.123:0/3152829642 shutdown_connections 2023-09-20T21:40:54.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa400038680 0x7fa40003ab40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:54.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c0035c0 0x7fa41c194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:54.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 --2- 172.21.15.123:0/3152829642 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa41c1952f0 0x7fa41c199760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:40:54.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 -- 172.21.15.123:0/3152829642 >> 172.21.15.123:0/3152829642 conn(0x7fa41c075390 msgr2=0x7fa41c18dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:40:54.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 -- 172.21.15.123:0/3152829642 shutdown_connections 2023-09-20T21:40:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:40:54.550+0000 7fa422da8700 1 -- 172.21.15.123:0/3152829642 wait complete. 2023-09-20T21:40:54.553 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 2 2023-09-20T21:40:54.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:54 smithi123 ceph-mon[134433]: Deploying daemon mon.b on smithi089 2023-09-20T21:40:54.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:54 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/3152829642' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-20T21:40:54.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:54 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:55.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:54 smithi022 ceph-mon[97651]: Deploying daemon mon.b on smithi089 2023-09-20T21:40:55.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:54 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/3152829642' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-20T21:40:55.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:54 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:55.980 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-09-20T21:40:55.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph mon dump -f json 2023-09-20T21:40:55.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:40:55 smithi022 ceph-mon[97651]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:55.998 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:40:55 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:40:55.681+0000 7f5130602700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-20T21:40:56.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:40:55 smithi123 ceph-mon[134433]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:56.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 systemd[1]: Starting Ceph mon.b for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:40:56.549 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:40:56.769 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 podman[134050]: 2023-09-20T21:40:56.769 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: set uid:gid to 167:167 (ceph:ceph) 2023-09-20T21:40:56.770 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: ceph version 18.0.0-6213-gbf8a061f (bf8a061ff16ac1d393cf108f740d92dfa49f2e54) reef (dev), process ceph-mon, pid 7 2023-09-20T21:40:56.770 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pidfile_write: ignore empty --pid-file 2023-09-20T21:40:56.770 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: load: jerasure load: lrc 2023-09-20T21:40:56.770 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: RocksDB version: 7.9.2 2023-09-20T21:40:56.771 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Git sha 0 2023-09-20T21:40:56.771 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Compile date 2023-09-20 08:00:43 2023-09-20T21:40:56.771 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: DB SUMMARY 2023-09-20T21:40:56.771 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: DB Session ID: 1032QL8RY2ELE0Z6I0YC 2023-09-20T21:40:56.771 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: CURRENT file: CURRENT 2023-09-20T21:40:56.771 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: IDENTITY file: IDENTITY 2023-09-20T21:40:56.772 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-09-20T21:40:56.772 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-09-20T21:40:56.772 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-09-20T21:40:56.772 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.error_if_exists: 0 2023-09-20T21:40:56.772 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.create_if_missing: 0 2023-09-20T21:40:56.773 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.paranoid_checks: 1 2023-09-20T21:40:56.773 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.flush_verify_memtable_count: 1 2023-09-20T21:40:56.773 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-09-20T21:40:56.773 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-09-20T21:40:56.773 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.env: 0x55a3dd740da0 2023-09-20T21:40:56.774 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.fs: PosixFileSystem 2023-09-20T21:40:56.774 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.info_log: 0x55a3de7d7580 2023-09-20T21:40:56.774 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_file_opening_threads: 16 2023-09-20T21:40:56.774 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.statistics: (nil) 2023-09-20T21:40:56.774 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.use_fsync: 0 2023-09-20T21:40:56.775 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_log_file_size: 0 2023-09-20T21:40:56.775 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-09-20T21:40:56.775 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.log_file_time_to_roll: 0 2023-09-20T21:40:56.775 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.keep_log_file_num: 1000 2023-09-20T21:40:56.775 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.recycle_log_file_num: 0 2023-09-20T21:40:56.776 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.allow_fallocate: 1 2023-09-20T21:40:56.776 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.allow_mmap_reads: 0 2023-09-20T21:40:56.776 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.allow_mmap_writes: 0 2023-09-20T21:40:56.776 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.use_direct_reads: 0 2023-09-20T21:40:56.776 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-09-20T21:40:56.776 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.create_missing_column_families: 0 2023-09-20T21:40:56.777 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.db_log_dir: 2023-09-20T21:40:56.777 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.wal_dir: 2023-09-20T21:40:56.777 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.table_cache_numshardbits: 6 2023-09-20T21:40:56.777 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.WAL_ttl_seconds: 0 2023-09-20T21:40:56.777 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.WAL_size_limit_MB: 0 2023-09-20T21:40:56.778 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-09-20T21:40:56.778 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-09-20T21:40:56.778 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.is_fd_close_on_exec: 1 2023-09-20T21:40:56.778 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.advise_random_on_open: 1 2023-09-20T21:40:56.778 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.db_write_buffer_size: 0 2023-09-20T21:40:56.779 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.write_buffer_manager: 0x55a3de78f4a0 2023-09-20T21:40:56.779 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-09-20T21:40:56.779 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-09-20T21:40:56.779 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.use_adaptive_mutex: 0 2023-09-20T21:40:56.779 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.rate_limiter: (nil) 2023-09-20T21:40:56.779 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-09-20T21:40:56.780 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.wal_recovery_mode: 2 2023-09-20T21:40:56.780 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.enable_thread_tracking: 0 2023-09-20T21:40:56.780 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.enable_pipelined_write: 0 2023-09-20T21:40:56.780 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.unordered_write: 0 2023-09-20T21:40:56.780 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-09-20T21:40:56.780 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-09-20T21:40:56.781 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-09-20T21:40:56.781 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-09-20T21:40:56.781 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.row_cache: None 2023-09-20T21:40:56.781 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.wal_filter: None 2023-09-20T21:40:56.781 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-09-20T21:40:56.782 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.allow_ingest_behind: 0 2023-09-20T21:40:56.782 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.two_write_queues: 0 2023-09-20T21:40:56.782 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.manual_wal_flush: 0 2023-09-20T21:40:56.782 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.wal_compression: 0 2023-09-20T21:40:56.782 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.atomic_flush: 0 2023-09-20T21:40:56.782 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-09-20T21:40:56.783 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.persist_stats_to_disk: 0 2023-09-20T21:40:56.783 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.write_dbid_to_manifest: 0 2023-09-20T21:40:56.783 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.log_readahead_size: 0 2023-09-20T21:40:56.783 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-09-20T21:40:56.783 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.best_efforts_recovery: 0 2023-09-20T21:40:56.784 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-09-20T21:40:56.784 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-09-20T21:40:56.784 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.allow_data_in_errors: 0 2023-09-20T21:40:56.784 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.db_host_id: __hostname__ 2023-09-20T21:40:56.784 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.enforce_single_del_contracts: true 2023-09-20T21:40:56.784 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_background_jobs: 2 2023-09-20T21:40:56.785 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_background_compactions: -1 2023-09-20T21:40:56.785 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_subcompactions: 1 2023-09-20T21:40:56.785 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-09-20T21:40:56.785 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-09-20T21:40:56.785 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.delayed_write_rate : 16777216 2023-09-20T21:40:56.786 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_total_wal_size: 0 2023-09-20T21:40:56.786 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-09-20T21:40:56.786 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.stats_dump_period_sec: 600 2023-09-20T21:40:56.786 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.stats_persist_period_sec: 600 2023-09-20T21:40:56.786 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-09-20T21:40:56.787 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_open_files: -1 2023-09-20T21:40:56.787 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bytes_per_sync: 0 2023-09-20T21:40:56.787 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.wal_bytes_per_sync: 0 2023-09-20T21:40:56.787 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.strict_bytes_per_sync: 0 2023-09-20T21:40:56.787 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_readahead_size: 0 2023-09-20T21:40:56.787 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_background_flushes: -1 2023-09-20T21:40:56.788 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Compression algorithms supported: 2023-09-20T21:40:56.788 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-09-20T21:40:56.788 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: kZSTD supported: 0 2023-09-20T21:40:56.788 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: kXpressCompression supported: 0 2023-09-20T21:40:56.788 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: kLZ4HCCompression supported: 1 2023-09-20T21:40:56.788 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: kZlibCompression supported: 1 2023-09-20T21:40:56.789 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: kSnappyCompression supported: 1 2023-09-20T21:40:56.789 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: kLZ4Compression supported: 1 2023-09-20T21:40:56.789 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: kBZip2Compression supported: 0 2023-09-20T21:40:56.789 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Fast CRC32 supported: Supported on x86 2023-09-20T21:40:56.789 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: DMutex implementation: pthread_mutex_t 2023-09-20T21:40:56.790 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-09-20T21:40:56.790 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-09-20T21:40:56.790 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-09-20T21:40:56.790 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.merge_operator: 2023-09-20T21:40:56.790 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_filter: None 2023-09-20T21:40:56.790 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_filter_factory: None 2023-09-20T21:40:56.791 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.sst_partitioner_factory: None 2023-09-20T21:40:56.791 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.memtable_factory: SkipListFactory 2023-09-20T21:40:56.791 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.table_factory: BlockBasedTable 2023-09-20T21:40:56.791 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 bash[134050]: 125e4a072e721af6b857623c4d533f2635b150d0755f112db6a1de6f1fab371e 2023-09-20T21:40:56.791 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a3de7d7680) 2023-09-20T21:40:56.792 INFO:journalctl@ceph.mon.b.smithi089.stdout: cache_index_and_filter_blocks: 1 2023-09-20T21:40:56.792 INFO:journalctl@ceph.mon.b.smithi089.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-09-20T21:40:56.792 INFO:journalctl@ceph.mon.b.smithi089.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-09-20T21:40:56.792 INFO:journalctl@ceph.mon.b.smithi089.stdout: pin_top_level_index_and_filter: 1 2023-09-20T21:40:56.792 INFO:journalctl@ceph.mon.b.smithi089.stdout: index_type: 0 2023-09-20T21:40:56.793 INFO:journalctl@ceph.mon.b.smithi089.stdout: data_block_index_type: 0 2023-09-20T21:40:56.793 INFO:journalctl@ceph.mon.b.smithi089.stdout: index_shortening: 1 2023-09-20T21:40:56.793 INFO:journalctl@ceph.mon.b.smithi089.stdout: data_block_hash_table_util_ratio: 0.750000 2023-09-20T21:40:56.793 INFO:journalctl@ceph.mon.b.smithi089.stdout: checksum: 4 2023-09-20T21:40:56.793 INFO:journalctl@ceph.mon.b.smithi089.stdout: no_block_cache: 0 2023-09-20T21:40:56.793 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_cache: 0x55a3de82d1f0 2023-09-20T21:40:56.794 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_cache_name: BinnedLRUCache 2023-09-20T21:40:56.794 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_cache_options: 2023-09-20T21:40:56.794 INFO:journalctl@ceph.mon.b.smithi089.stdout: capacity : 536870912 2023-09-20T21:40:56.794 INFO:journalctl@ceph.mon.b.smithi089.stdout: num_shard_bits : 4 2023-09-20T21:40:56.794 INFO:journalctl@ceph.mon.b.smithi089.stdout: strict_capacity_limit : 0 2023-09-20T21:40:56.794 INFO:journalctl@ceph.mon.b.smithi089.stdout: high_pri_pool_ratio: 0.000 2023-09-20T21:40:56.795 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_cache_compressed: (nil) 2023-09-20T21:40:56.795 INFO:journalctl@ceph.mon.b.smithi089.stdout: persistent_cache: (nil) 2023-09-20T21:40:56.795 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_size: 4096 2023-09-20T21:40:56.795 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_size_deviation: 10 2023-09-20T21:40:56.795 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_restart_interval: 16 2023-09-20T21:40:56.796 INFO:journalctl@ceph.mon.b.smithi089.stdout: index_block_restart_interval: 1 2023-09-20T21:40:56.796 INFO:journalctl@ceph.mon.b.smithi089.stdout: metadata_block_size: 4096 2023-09-20T21:40:56.796 INFO:journalctl@ceph.mon.b.smithi089.stdout: partition_filters: 0 2023-09-20T21:40:56.796 INFO:journalctl@ceph.mon.b.smithi089.stdout: use_delta_encoding: 1 2023-09-20T21:40:56.796 INFO:journalctl@ceph.mon.b.smithi089.stdout: filter_policy: bloomfilter 2023-09-20T21:40:56.796 INFO:journalctl@ceph.mon.b.smithi089.stdout: whole_key_filtering: 1 2023-09-20T21:40:56.797 INFO:journalctl@ceph.mon.b.smithi089.stdout: verify_compression: 0 2023-09-20T21:40:56.797 INFO:journalctl@ceph.mon.b.smithi089.stdout: read_amp_bytes_per_bit: 0 2023-09-20T21:40:56.797 INFO:journalctl@ceph.mon.b.smithi089.stdout: format_version: 5 2023-09-20T21:40:56.797 INFO:journalctl@ceph.mon.b.smithi089.stdout: enable_index_compression: 1 2023-09-20T21:40:56.797 INFO:journalctl@ceph.mon.b.smithi089.stdout: block_align: 0 2023-09-20T21:40:56.797 INFO:journalctl@ceph.mon.b.smithi089.stdout: max_auto_readahead_size: 262144 2023-09-20T21:40:56.798 INFO:journalctl@ceph.mon.b.smithi089.stdout: prepopulate_block_cache: 0 2023-09-20T21:40:56.798 INFO:journalctl@ceph.mon.b.smithi089.stdout: initial_auto_readahead_size: 8192 2023-09-20T21:40:56.798 INFO:journalctl@ceph.mon.b.smithi089.stdout: num_file_reads_for_auto_readahead: 2 2023-09-20T21:40:56.798 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.write_buffer_size: 33554432 2023-09-20T21:40:56.798 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_write_buffer_number: 2 2023-09-20T21:40:56.799 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compression: NoCompression 2023-09-20T21:40:56.799 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bottommost_compression: Disabled 2023-09-20T21:40:56.799 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.prefix_extractor: nullptr 2023-09-20T21:40:56.799 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-09-20T21:40:56.799 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.num_levels: 7 2023-09-20T21:40:56.799 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-09-20T21:40:56.800 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-09-20T21:40:56.800 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-09-20T21:40:56.800 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-09-20T21:40:56.800 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-09-20T21:40:56.800 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-09-20T21:40:56.800 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-09-20T21:40:56.801 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-09-20T21:40:56.801 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-09-20T21:40:56.801 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-09-20T21:40:56.801 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-09-20T21:40:56.801 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-09-20T21:40:56.802 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compression_opts.window_bits: -14 2023-09-20T21:40:56.802 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compression_opts.level: 32767 2023-09-20T21:40:56.802 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compression_opts.strategy: 0 2023-09-20T21:40:56.802 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-09-20T21:40:56.802 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-09-20T21:40:56.803 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-09-20T21:40:56.803 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-09-20T21:40:56.803 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compression_opts.enabled: false 2023-09-20T21:40:56.803 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-09-20T21:40:56.803 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-09-20T21:40:56.804 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-09-20T21:40:56.804 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-09-20T21:40:56.804 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.target_file_size_base: 67108864 2023-09-20T21:40:56.804 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.target_file_size_multiplier: 1 2023-09-20T21:40:56.804 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-09-20T21:40:56.804 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-09-20T21:40:56.805 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-09-20T21:40:56.805 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-09-20T21:40:56.805 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-09-20T21:40:56.805 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-09-20T21:40:56.805 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-09-20T21:40:56.806 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-09-20T21:40:56.806 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-09-20T21:40:56.806 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-09-20T21:40:56.806 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-09-20T21:40:56.806 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-09-20T21:40:56.807 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-09-20T21:40:56.807 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.arena_block_size: 1048576 2023-09-20T21:40:56.807 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-09-20T21:40:56.807 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-09-20T21:40:56.807 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.disable_auto_compactions: 0 2023-09-20T21:40:56.808 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-09-20T21:40:56.808 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-09-20T21:40:56.809 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-09-20T21:40:56.809 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-09-20T21:40:56.809 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-09-20T21:40:56.809 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-09-20T21:40:56.809 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-09-20T21:40:56.809 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-09-20T21:40:56.810 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-09-20T21:40:56.810 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-09-20T21:40:56.810 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.table_properties_collectors: 2023-09-20T21:40:56.810 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.inplace_update_support: 0 2023-09-20T21:40:56.810 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.inplace_update_num_locks: 10000 2023-09-20T21:40:56.811 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-09-20T21:40:56.811 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-09-20T21:40:56.811 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.memtable_huge_page_size: 0 2023-09-20T21:40:56.811 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.bloom_locality: 0 2023-09-20T21:40:56.811 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.max_successive_merges: 0 2023-09-20T21:40:56.811 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.optimize_filters_for_hits: 0 2023-09-20T21:40:56.812 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.paranoid_file_checks: 0 2023-09-20T21:40:56.812 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.force_consistency_checks: 1 2023-09-20T21:40:56.812 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.report_bg_io_stats: 0 2023-09-20T21:40:56.812 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.ttl: 2592000 2023-09-20T21:40:56.812 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.periodic_compaction_seconds: 0 2023-09-20T21:40:56.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-09-20T21:40:56.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-09-20T21:40:56.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.enable_blob_files: false 2023-09-20T21:40:56.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.min_blob_size: 0 2023-09-20T21:40:56.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.blob_file_size: 268435456 2023-09-20T21:40:56.813 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.blob_compression_type: NoCompression 2023-09-20T21:40:56.814 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.enable_blob_garbage_collection: false 2023-09-20T21:40:56.814 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-09-20T21:40:56.814 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-09-20T21:40:56.814 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-09-20T21:40:56.814 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.blob_file_starting_level: 0 2023-09-20T21:40:56.815 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-09-20T21:40:56.815 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-09-20T21:40:56.815 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-09-20T21:40:56.815 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6e6b61e0-22ae-4e8c-811c-537c6924b885 2023-09-20T21:40:56.815 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695246056745801, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-09-20T21:40:56.815 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-09-20T21:40:56.816 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695246056746348, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1695246056, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6e6b61e0-22ae-4e8c-811c-537c6924b885", "db_session_id": "1032QL8RY2ELE0Z6I0YC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-09-20T21:40:56.816 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: EVENT_LOG_v1 {"time_micros": 1695246056746438, "job": 1, "event": "recovery_finished"} 2023-09-20T21:40:56.816 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-09-20T21:40:56.816 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-09-20T21:40:56.816 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a3de8ca000 2023-09-20T21:40:56.817 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: DB pointer 0x55a3de8b6000 2023-09-20T21:40:56.817 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-09-20T21:40:56.817 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-09-20T21:40:56.817 INFO:journalctl@ceph.mon.b.smithi089.stdout: ** DB Stats ** 2023-09-20T21:40:56.817 INFO:journalctl@ceph.mon.b.smithi089.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-20T21:40:56.817 INFO:journalctl@ceph.mon.b.smithi089.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-09-20T21:40:56.818 INFO:journalctl@ceph.mon.b.smithi089.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-20T21:40:56.818 INFO:journalctl@ceph.mon.b.smithi089.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-20T21:40:56.818 INFO:journalctl@ceph.mon.b.smithi089.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-09-20T21:40:56.818 INFO:journalctl@ceph.mon.b.smithi089.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-09-20T21:40:56.818 INFO:journalctl@ceph.mon.b.smithi089.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-09-20T21:40:56.818 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2023-09-20T21:40:56.819 INFO:journalctl@ceph.mon.b.smithi089.stdout: ** Compaction Stats [default] ** 2023-09-20T21:40:56.819 INFO:journalctl@ceph.mon.b.smithi089.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-20T21:40:56.819 INFO:journalctl@ceph.mon.b.smithi089.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-09-20T21:40:56.819 INFO:journalctl@ceph.mon.b.smithi089.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-20T21:40:56.819 INFO:journalctl@ceph.mon.b.smithi089.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-20T21:40:56.820 INFO:journalctl@ceph.mon.b.smithi089.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-20T21:40:56.820 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2023-09-20T21:40:56.820 INFO:journalctl@ceph.mon.b.smithi089.stdout: ** Compaction Stats [default] ** 2023-09-20T21:40:56.820 INFO:journalctl@ceph.mon.b.smithi089.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-09-20T21:40:56.820 INFO:journalctl@ceph.mon.b.smithi089.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-09-20T21:40:56.820 INFO:journalctl@ceph.mon.b.smithi089.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-09-20T21:40:56.821 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2023-09-20T21:40:56.821 INFO:journalctl@ceph.mon.b.smithi089.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-09-20T21:40:56.821 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2023-09-20T21:40:56.821 INFO:journalctl@ceph.mon.b.smithi089.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-09-20T21:40:56.821 INFO:journalctl@ceph.mon.b.smithi089.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-09-20T21:40:56.821 INFO:journalctl@ceph.mon.b.smithi089.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-09-20T21:40:56.822 INFO:journalctl@ceph.mon.b.smithi089.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-09-20T21:40:56.822 INFO:journalctl@ceph.mon.b.smithi089.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-09-20T21:40:56.822 INFO:journalctl@ceph.mon.b.smithi089.stdout: AddFile(Keys): cumulative 0, interval 0 2023-09-20T21:40:56.822 INFO:journalctl@ceph.mon.b.smithi089.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-20T21:40:56.822 INFO:journalctl@ceph.mon.b.smithi089.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-09-20T21:40:56.823 INFO:journalctl@ceph.mon.b.smithi089.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-09-20T21:40:56.823 INFO:journalctl@ceph.mon.b.smithi089.stdout: Block cache BinnedLRUCache@0x55a3de82d1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-09-20T21:40:56.823 INFO:journalctl@ceph.mon.b.smithi089.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-09-20T21:40:56.823 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2023-09-20T21:40:56.823 INFO:journalctl@ceph.mon.b.smithi089.stdout: ** File Read Latency Histogram By Level [default] ** 2023-09-20T21:40:56.824 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-09-20T21:40:56.824 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: using public_addr v2:172.21.15.89:0/0 -> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] 2023-09-20T21:40:56.824 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: starting mon.b rank -1 at public addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] at bind addrs [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:40:56.824 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(???) e0 preinit fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:40:56.824 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).mds e1 new map 2023-09-20T21:40:56.824 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).mds e1 print_map 2023-09-20T21:40:56.825 INFO:journalctl@ceph.mon.b.smithi089.stdout: e1 2023-09-20T21:40:56.825 INFO:journalctl@ceph.mon.b.smithi089.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-09-20T21:40:56.825 INFO:journalctl@ceph.mon.b.smithi089.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-20T21:40:56.825 INFO:journalctl@ceph.mon.b.smithi089.stdout: legacy client fscid: -1 2023-09-20T21:40:56.825 INFO:journalctl@ceph.mon.b.smithi089.stdout: 2023-09-20T21:40:56.826 INFO:journalctl@ceph.mon.b.smithi089.stdout: No filesystems configured 2023-09-20T21:40:56.826 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-09-20T21:40:56.826 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-09-20T21:40:56.826 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-09-20T21:40:56.826 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-09-20T21:40:56.826 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-09-20T21:40:56.827 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-09-20T21:40:57.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-09-20T21:40:57.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-20T21:40:57.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-20T21:40:57.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-09-20T21:40:57.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:40:57.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:40:57.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:57.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:40:57.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Updating smithi089:/etc/ceph/ceph.conf 2023-09-20T21:40:57.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Updating smithi089:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:40:57.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:40:57.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:40:57.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Updating smithi089:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-20T21:40:57.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Deploying cephadm binary to smithi123 2023-09-20T21:40:57.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.117 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:57.117 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Added host smithi123 2023-09-20T21:40:57.117 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.117 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.117 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.117 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:40:57.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.119 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.119 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.119 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:40:57.119 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:57.119 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:40:57.119 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1660139695' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-09-20T21:40:57.120 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-20T21:40:57.120 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Updating smithi123:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:40:57.120 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:40:57.120 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Updating smithi123:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.client.admin.keyring 2023-09-20T21:40:57.120 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.120 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.121 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.121 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1660139695' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-09-20T21:40:57.121 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:40:57.121 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.121 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi089:172.21.15.89=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:40:57.122 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi089:172.21.15.89=b;smithi123:172.21.15.123=c;count:3 2023-09-20T21:40:57.122 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.122 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:40:57.122 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:57.122 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:40:57.123 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.123 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:40:57.123 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:57.123 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Deploying daemon mon.c on smithi123 2023-09-20T21:40:57.123 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.123 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.124 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/2485684905' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-20T21:40:57.124 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:40:57.124 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.a calling monitor election 2023-09-20T21:40:57.124 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:57.124 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:57.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:57.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.c calling monitor election 2023-09-20T21:40:57.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:57.125 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:57.126 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.126 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:57.126 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-20T21:40:57.126 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-20T21:40:57.126 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: fsmap 2023-09-20T21:40:57.127 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:40:57.127 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mgrmap e13: x(active, since 46s) 2023-09-20T21:40:57.127 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: overall HEALTH_OK 2023-09-20T21:40:57.127 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.127 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.127 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:40:57.128 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:40:57.128 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:40:57.128 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: Deploying daemon mon.b on smithi089 2023-09-20T21:40:57.128 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/3152829642' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-20T21:40:57.128 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:40:57.128 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:40:57.129 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-09-20T21:40:57.129 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-09-20T21:40:57.129 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta expand map: {default=false} 2023-09-20T21:40:57.129 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta from 'false' to 'false' 2023-09-20T21:40:57.129 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta expanded map: {default=false} 2023-09-20T21:40:57.130 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta expand map: {default=info} 2023-09-20T21:40:57.130 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta from 'info' to 'info' 2023-09-20T21:40:57.130 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta expanded map: {default=info} 2023-09-20T21:40:57.130 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta expand map: {default=daemon} 2023-09-20T21:40:57.130 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta from 'daemon' to 'daemon' 2023-09-20T21:40:57.130 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta expanded map: {default=daemon} 2023-09-20T21:40:57.131 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta expand map: {default=debug} 2023-09-20T21:40:57.131 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta from 'debug' to 'debug' 2023-09-20T21:40:57.131 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 ceph-mon[134090]: expand_channel_meta expanded map: {default=debug} 2023-09-20T21:40:57.131 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:40:56 smithi089 systemd[1]: Started Ceph mon.b for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:41:02.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:41:02.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: mon.a calling monitor election 2023-09-20T21:41:02.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: mon.c calling monitor election 2023-09-20T21:41:02.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:41:02.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:02.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:02.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:02.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-20T21:41:02.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-20T21:41:02.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: fsmap 2023-09-20T21:41:02.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:41:02.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: mgrmap e13: x(active, since 54s) 2023-09-20T21:41:02.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-20T21:41:02.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-20T21:41:02.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-20T21:41:02.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: mon.b (rank 2) addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] is down (out of quorum) 2023-09-20T21:41:02.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:02.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:02.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:02.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:02.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.127+0000 7f638e8ec700 1 -- 172.21.15.123:0/16819967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63700035c0 msgr2=0x7f6370005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:02.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.127+0000 7f638e8ec700 1 --2- 172.21.15.123:0/16819967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63700035c0 0x7f6370005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f63880767d0 tx=0x7f637800fe60 comp rx=0 tx=0).stop 2023-09-20T21:41:02.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.127+0000 7f638e8ec700 1 -- 172.21.15.123:0/16819967 shutdown_connections 2023-09-20T21:41:02.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.127+0000 7f638e8ec700 1 --2- 172.21.15.123:0/16819967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63700035c0 0x7f6370005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:02.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.127+0000 7f638e8ec700 1 -- 172.21.15.123:0/16819967 >> 172.21.15.123:0/16819967 conn(0x7f6388075390 msgr2=0x7f63880757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:41:02.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.127+0000 7f638e8ec700 1 -- 172.21.15.123:0/16819967 shutdown_connections 2023-09-20T21:41:02.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.127+0000 7f638e8ec700 1 -- 172.21.15.123:0/16819967 wait complete. 2023-09-20T21:41:02.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.128+0000 7f638e8ec700 1 Processor -- start 2023-09-20T21:41:02.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.128+0000 7f638e8ec700 1 -- start start 2023-09-20T21:41:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f638e8ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63700035c0 0x7f638818cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f638e8ec700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638818d280 0x7f638818d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f638e8ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881a21e0 0x7f63881a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f638e8ec700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881a4c20 con 0x7f638818d280 2023-09-20T21:41:02.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f638e8ec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881a4d90 con 0x7f63881a21e0 2023-09-20T21:41:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f638e8ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881a4f50 con 0x7f63700035c0 2023-09-20T21:41:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f6387fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63700035c0 0x7f638818cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f6387fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63700035c0 0x7f638818cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33804/0 (socket says 172.21.15.123:33804) 2023-09-20T21:41:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f6387fff700 1 -- 172.21.15.123:0/1309938459 learned_addr learned my addr 172.21.15.123:0/1309938459 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-20T21:41:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f638ce89700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881a21e0 0x7f63881a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:02.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f6387fff700 1 -- 172.21.15.123:0/1309938459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63700035c0 msgr2=0x7f638818cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-09-20T21:41:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f6387fff700 1 -- 172.21.15.123:0/1309938459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63700035c0 msgr2=0x7f638818cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-20T21:41:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f6387fff700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63700035c0 0x7f638818cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-20T21:41:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f6387fff700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63700035c0 0x7f638818cd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-20T21:41:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f63877fe700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638818d280 0x7f638818d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f638ce89700 1 -- 172.21.15.123:0/1309938459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881a21e0 msgr2=0x7f63881a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-09-20T21:41:02.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f638ce89700 1 -- 172.21.15.123:0/1309938459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881a21e0 msgr2=0x7f63881a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-20T21:41:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.129+0000 7f638ce89700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881a21e0 0x7f63881a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-20T21:41:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f638ce89700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881a21e0 0x7f63881a45d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-20T21:41:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f63877fe700 1 -- 172.21.15.123:0/1309938459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63700035c0 msgr2=0x7f638818cd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:41:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f63877fe700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63700035c0 0x7f638818cd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:02.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f63877fe700 1 -- 172.21.15.123:0/1309938459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881a21e0 msgr2=0x7f63881a45d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:41:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f63877fe700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881a21e0 0x7f63881a45d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f63877fe700 1 -- 172.21.15.123:0/1309938459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637800f8b0 con 0x7f638818d280 2023-09-20T21:41:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f63877fe700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638818d280 0x7f638818d700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f637c00b550 tx=0x7f637c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:41:02.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f63857fa700 1 -- 172.21.15.123:0/1309938459 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c004230 con 0x7f638818d280 2023-09-20T21:41:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f63857fa700 1 -- 172.21.15.123:0/1309938459 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f637c004390 con 0x7f638818d280 2023-09-20T21:41:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f638e8ec700 1 -- 172.21.15.123:0/1309938459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63881a5230 con 0x7f638818d280 2023-09-20T21:41:02.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f63857fa700 1 -- 172.21.15.123:0/1309938459 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c011af0 con 0x7f638818d280 2023-09-20T21:41:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.130+0000 7f638e8ec700 1 -- 172.21.15.123:0/1309938459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63881a5780 con 0x7f638818d280 2023-09-20T21:41:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.131+0000 7f63857fa700 1 -- 172.21.15.123:0/1309938459 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7f637c011c50 con 0x7f638818d280 2023-09-20T21:41:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.132+0000 7f63857fa700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6374038d10 0x7f637403b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.131+0000 7f638e8ec700 1 -- 172.21.15.123:0/1309938459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6388062410 con 0x7f638818d280 2023-09-20T21:41:02.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.132+0000 7f63857fa700 1 -- 172.21.15.123:0/1309938459 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f637c04b4c0 con 0x7f638818d280 2023-09-20T21:41:02.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.132+0000 7f6387fff700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6374038d10 0x7f637403b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:02.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.132+0000 7f6387fff700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6374038d10 0x7f637403b1d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6378015040 tx=0x7f6378001e50 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:41:02.140 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.140+0000 7f63857fa700 1 -- 172.21.15.123:0/1309938459 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f637c01a400 con 0x7f638818d280 2023-09-20T21:41:02.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:41:02.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: mon.a calling monitor election 2023-09-20T21:41:02.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: mon.c calling monitor election 2023-09-20T21:41:02.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:41:02.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:02.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:02.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:02.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:02.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-20T21:41:02.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-20T21:41:02.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: fsmap 2023-09-20T21:41:02.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:41:02.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: mgrmap e13: x(active, since 54s) 2023-09-20T21:41:02.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-20T21:41:02.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-20T21:41:02.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-20T21:41:02.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: mon.b (rank 2) addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] is down (out of quorum) 2023-09-20T21:41:02.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:02.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:02.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:02.254 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:02.419 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.419+0000 7f638e8ec700 1 -- 172.21.15.123:0/1309938459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6388107690 con 0x7f638818d280 2023-09-20T21:41:02.420 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.420+0000 7f63857fa700 1 -- 172.21.15.123:0/1309938459 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7f637c00f2e0 con 0x7f638818d280 2023-09-20T21:41:02.421 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-09-20T21:41:02.421 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","modified":"2023-09-20T21:40:56.778477Z","created":"2023-09-20T21:39:03.785697Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-09-20T21:41:02.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 -- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6374038d10 msgr2=0x7f637403b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:02.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6374038d10 0x7f637403b1d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6378015040 tx=0x7f6378001e50 comp rx=0 tx=0).stop 2023-09-20T21:41:02.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 -- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638818d280 msgr2=0x7f638818d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:02.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638818d280 0x7f638818d700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f637c00b550 tx=0x7f637c00b910 comp rx=0 tx=0).stop 2023-09-20T21:41:02.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 -- 172.21.15.123:0/1309938459 shutdown_connections 2023-09-20T21:41:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6374038d10 0x7f637403b1d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881a21e0 0x7f63881a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63700035c0 0x7f638818cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 --2- 172.21.15.123:0/1309938459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f638818d280 0x7f638818d700 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:02.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 -- 172.21.15.123:0/1309938459 >> 172.21.15.123:0/1309938459 conn(0x7f6388075390 msgr2=0x7f6388101390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:41:02.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 -- 172.21.15.123:0/1309938459 shutdown_connections 2023-09-20T21:41:02.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:02.422+0000 7f638e8ec700 1 -- 172.21.15.123:0/1309938459 wait complete. 2023-09-20T21:41:02.425 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 3 2023-09-20T21:41:02.813 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-09-20T21:41:02.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph config generate-minimal-conf 2023-09-20T21:41:03.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:02 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:03.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:02 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:03.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:02 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/1309938459' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-20T21:41:03.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:02 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:03.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:02 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:03.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:02 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:03.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:02 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/1309938459' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-20T21:41:03.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:02 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:03.556 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:41:04.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:41:04.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: mon.a calling monitor election 2023-09-20T21:41:04.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: mon.c calling monitor election 2023-09-20T21:41:04.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:04.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:41:04.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:04.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:04.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:04.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:04.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:04.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:04.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:04.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:04.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-09-20T21:41:04.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-20T21:41:04.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: fsmap 2023-09-20T21:41:04.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:41:04.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: mgrmap e13: x(active, since 54s) 2023-09-20T21:41:04.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-09-20T21:41:04.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-09-20T21:41:04.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-09-20T21:41:04.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: mon.b (rank 2) addr [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] is down (out of quorum) 2023-09-20T21:41:04.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:04.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:04.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:04.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:04.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:04.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:04.117 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/1309938459' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-09-20T21:41:04.117 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:04.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: mon.b calling monitor election 2023-09-20T21:41:04.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: Updating smithi022:/etc/ceph/ceph.conf 2023-09-20T21:41:04.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: Updating smithi089:/etc/ceph/ceph.conf 2023-09-20T21:41:04.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-20T21:41:04.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: mon.b calling monitor election 2023-09-20T21:41:04.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: mon.c calling monitor election 2023-09-20T21:41:04.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: mon.a calling monitor election 2023-09-20T21:41:04.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-20T21:41:04.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-20T21:41:04.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: fsmap 2023-09-20T21:41:04.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:41:04.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: mgrmap e13: x(active, since 56s) 2023-09-20T21:41:04.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-20T21:41:04.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: Cluster is now healthy 2023-09-20T21:41:04.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: overall HEALTH_OK 2023-09-20T21:41:04.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: Updating smithi089:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:41:04.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: Updating smithi123:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:41:04.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:04.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:04.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:04.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:04.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:04 smithi022 ceph-mon[97651]: Updating smithi022:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:41:05.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: mon.b calling monitor election 2023-09-20T21:41:05.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: Updating smithi022:/etc/ceph/ceph.conf 2023-09-20T21:41:05.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: Updating smithi089:/etc/ceph/ceph.conf 2023-09-20T21:41:05.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-20T21:41:05.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: mon.b calling monitor election 2023-09-20T21:41:05.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: mon.c calling monitor election 2023-09-20T21:41:05.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: mon.a calling monitor election 2023-09-20T21:41:05.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-20T21:41:05.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-20T21:41:05.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: fsmap 2023-09-20T21:41:05.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:41:05.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: mgrmap e13: x(active, since 56s) 2023-09-20T21:41:05.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-20T21:41:05.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: Cluster is now healthy 2023-09-20T21:41:05.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: overall HEALTH_OK 2023-09-20T21:41:05.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: Updating smithi089:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:41:05.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: Updating smithi123:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:41:05.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:04 smithi123 ceph-mon[134433]: Updating smithi022:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:41:05.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: mon.b calling monitor election 2023-09-20T21:41:05.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: Updating smithi022:/etc/ceph/ceph.conf 2023-09-20T21:41:05.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: Updating smithi089:/etc/ceph/ceph.conf 2023-09-20T21:41:05.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: Updating smithi123:/etc/ceph/ceph.conf 2023-09-20T21:41:05.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: mon.b calling monitor election 2023-09-20T21:41:05.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: mon.c calling monitor election 2023-09-20T21:41:05.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: mon.a calling monitor election 2023-09-20T21:41:05.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-09-20T21:41:05.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0],c=[v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0]} removed_ranks: {} 2023-09-20T21:41:05.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: fsmap 2023-09-20T21:41:05.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: osdmap e4: 0 total, 0 up, 0 in 2023-09-20T21:41:05.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: mgrmap e13: x(active, since 56s) 2023-09-20T21:41:05.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-09-20T21:41:05.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: Cluster is now healthy 2023-09-20T21:41:05.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: overall HEALTH_OK 2023-09-20T21:41:05.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: Updating smithi089:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:41:05.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: Updating smithi123:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:41:05.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:04 smithi089 ceph-mon[134090]: Updating smithi022:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/config/ceph.conf 2023-09-20T21:41:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.118+0000 7f88ef495700 1 -- 172.21.15.22:0/3947772577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88e8073180 msgr2=0x7f88e8073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:05.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.118+0000 7f88ef495700 1 --2- 172.21.15.22:0/3947772577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88e8073180 0x7f88e8073560 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009910 tx=0x7f88e4009c20 comp rx=0 tx=0).stop 2023-09-20T21:41:05.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.118+0000 7f88ef495700 1 -- 172.21.15.22:0/3947772577 shutdown_connections 2023-09-20T21:41:05.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.118+0000 7f88ef495700 1 --2- 172.21.15.22:0/3947772577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88e8073180 0x7f88e8073560 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:05.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.118+0000 7f88ef495700 1 -- 172.21.15.22:0/3947772577 >> 172.21.15.22:0/3947772577 conn(0x7f88e80fc370 msgr2=0x7f88e80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:41:05.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.118+0000 7f88ef495700 1 -- 172.21.15.22:0/3947772577 shutdown_connections 2023-09-20T21:41:05.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.118+0000 7f88ef495700 1 -- 172.21.15.22:0/3947772577 wait complete. 2023-09-20T21:41:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.119+0000 7f88ef495700 1 Processor -- start 2023-09-20T21:41:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.119+0000 7f88ef495700 1 -- start start 2023-09-20T21:41:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ef495700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e8073180 0x7f88e819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ef495700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88e819c9d0 0x7f88e81a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ef495700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88e81a1380 0x7f88e81a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ef495700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e819d0c0 con 0x7f88e81a1380 2023-09-20T21:41:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ef495700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81a90f0 con 0x7f88e8073180 2023-09-20T21:41:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ef495700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81a9230 con 0x7f88e819c9d0 2023-09-20T21:41:05.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ed231700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e8073180 0x7f88e819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88eda32700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88e81a1380 0x7f88e81a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ed231700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e8073180 0x7f88e819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:48340/0 (socket says 172.21.15.22:48340) 2023-09-20T21:41:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ed231700 1 -- 172.21.15.22:0/2092465830 learned_addr learned my addr 172.21.15.22:0/2092465830 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:41:05.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88eca30700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88e819c9d0 0x7f88e81a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ed231700 1 -- 172.21.15.22:0/2092465830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88e819c9d0 msgr2=0x7f88e81a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ed231700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88e819c9d0 0x7f88e81a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ed231700 1 -- 172.21.15.22:0/2092465830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88e81a1380 msgr2=0x7f88e81a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ed231700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88e81a1380 0x7f88e81a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88ed231700 1 -- 172.21.15.22:0/2092465830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e40095f0 con 0x7f88e8073180 2023-09-20T21:41:05.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.120+0000 7f88eda32700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88e81a1380 0x7f88e81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:41:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.121+0000 7f88ed231700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e8073180 0x7f88e819c490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f88e4004cd0 tx=0x7f88e4004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:41:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.121+0000 7f88de7fc700 1 -- 172.21.15.22:0/2092465830 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e401d070 con 0x7f88e8073180 2023-09-20T21:41:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.121+0000 7f88de7fc700 1 -- 172.21.15.22:0/2092465830 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88e400be20 con 0x7f88e8073180 2023-09-20T21:41:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.121+0000 7f88de7fc700 1 -- 172.21.15.22:0/2092465830 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4022d10 con 0x7f88e8073180 2023-09-20T21:41:05.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.121+0000 7f88ef495700 1 -- 172.21.15.22:0/2092465830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e81a9510 con 0x7f88e8073180 2023-09-20T21:41:05.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.121+0000 7f88ef495700 1 -- 172.21.15.22:0/2092465830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e81a99c0 con 0x7f88e8073180 2023-09-20T21:41:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.122+0000 7f88de7fc700 1 -- 172.21.15.22:0/2092465830 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7f88e4022470 con 0x7f88e8073180 2023-09-20T21:41:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.122+0000 7f88ef495700 1 -- 172.21.15.22:0/2092465830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e8062410 con 0x7f88e8073180 2023-09-20T21:41:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.123+0000 7f88de7fc700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f88d4038d10 0x7f88d403b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.123+0000 7f88de7fc700 1 -- 172.21.15.22:0/2092465830 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f88e404ca10 con 0x7f88e8073180 2023-09-20T21:41:05.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.123+0000 7f88eca30700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f88d4038d10 0x7f88d403b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.124+0000 7f88eca30700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f88d4038d10 0x7f88d403b1d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009ad0 tx=0x7f88d8009210 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:41:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.128+0000 7f88de7fc700 1 -- 172.21.15.22:0/2092465830 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f88e400edb0 con 0x7f88e8073180 2023-09-20T21:41:05.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.346+0000 7f88ef495700 1 -- 172.21.15.22:0/2092465830 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f88e819d7f0 con 0x7f88e8073180 2023-09-20T21:41:05.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.346+0000 7f88de7fc700 1 -- 172.21.15.22:0/2092465830 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f88e4027030 con 0x7f88e8073180 2023-09-20T21:41:05.347 INFO:teuthology.orchestra.run.smithi022.stdout:# minimal ceph.conf for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:41:05.347 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2023-09-20T21:41:05.348 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:41:05.348 INFO:teuthology.orchestra.run.smithi022.stdout: mon_host = [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2023-09-20T21:41:05.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 -- 172.21.15.22:0/2092465830 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f88d4038d10 msgr2=0x7f88d403b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:05.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f88d4038d10 0x7f88d403b1d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009ad0 tx=0x7f88d8009210 comp rx=0 tx=0).stop 2023-09-20T21:41:05.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 -- 172.21.15.22:0/2092465830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e8073180 msgr2=0x7f88e819c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:05.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e8073180 0x7f88e819c490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f88e4004cd0 tx=0x7f88e4004d00 comp rx=0 tx=0).stop 2023-09-20T21:41:05.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 -- 172.21.15.22:0/2092465830 shutdown_connections 2023-09-20T21:41:05.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88e81a1380 0x7f88e81a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:05.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f88d4038d10 0x7f88d403b1d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:05.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f88e8073180 0x7f88e819c490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:05.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 --2- 172.21.15.22:0/2092465830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88e819c9d0 0x7f88e81a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:05.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 -- 172.21.15.22:0/2092465830 >> 172.21.15.22:0/2092465830 conn(0x7f88e80fc370 msgr2=0x7f88e80fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:41:05.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 -- 172.21.15.22:0/2092465830 shutdown_connections 2023-09-20T21:41:05.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:05.349+0000 7f88ef495700 1 -- 172.21.15.22:0/2092465830 wait complete. 2023-09-20T21:41:05.877 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-09-20T21:41:05.877 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:41:05.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.conf 2023-09-20T21:41:05.948 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T21:41:05.778+0000 7f5130602700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-20T21:41:05.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:05.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.949 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.950 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.950 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:05.950 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: Reconfiguring mon.a (unknown last config time)... 2023-09-20T21:41:05.950 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:41:05.950 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-20T21:41:05.951 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:05.951 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: Reconfiguring daemon mon.a on smithi022 2023-09-20T21:41:05.951 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:05.951 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:05 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2092465830' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:05.965 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:41:05.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:41:06.077 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:41:06.078 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.conf 2023-09-20T21:41:06.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:06.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: Reconfiguring mon.a (unknown last config time)... 2023-09-20T21:41:06.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:41:06.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-20T21:41:06.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:06.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: Reconfiguring daemon mon.a on smithi022 2023-09-20T21:41:06.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:06.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:05 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2092465830' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:06.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:41:06.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:06.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: Reconfiguring mon.a (unknown last config time)... 2023-09-20T21:41:06.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:41:06.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-20T21:41:06.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:06.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: Reconfiguring daemon mon.a on smithi022 2023-09-20T21:41:06.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:06.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:05 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2092465830' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:06.114 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:41:06.114 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:41:06.185 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:41:06.185 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2023-09-20T21:41:06.221 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:41:06.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-09-20T21:41:06.294 INFO:tasks.cephadm:Adding mgr.x on smithi022 2023-09-20T21:41:06.295 INFO:tasks.cephadm:Adding mgr.y on smithi089 2023-09-20T21:41:06.295 INFO:tasks.cephadm:Adding mgr.z on smithi123 2023-09-20T21:41:06.295 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch apply mgr '3;smithi022=x;smithi089=y;smithi123=z' 2023-09-20T21:41:06.906 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:41:07.722 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:07.723 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:07.723 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:07 smithi089 ceph-mon[134090]: Reconfiguring mon.b (monmap changed)... 2023-09-20T21:41:07.723 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:41:07.723 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-20T21:41:07.724 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:07.724 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:07 smithi089 ceph-mon[134090]: Reconfiguring daemon mon.b on smithi089 2023-09-20T21:41:07.724 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:07 smithi089 ceph-mon[134090]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:07 smithi123 ceph-mon[134433]: Reconfiguring mon.b (monmap changed)... 2023-09-20T21:41:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:41:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-20T21:41:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:07 smithi123 ceph-mon[134433]: Reconfiguring daemon mon.b on smithi089 2023-09-20T21:41:07.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:07 smithi123 ceph-mon[134433]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:07 smithi022 ceph-mon[97651]: Reconfiguring mon.b (monmap changed)... 2023-09-20T21:41:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:41:07.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-20T21:41:07.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:07.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:07 smithi022 ceph-mon[97651]: Reconfiguring daemon mon.b on smithi089 2023-09-20T21:41:07.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:07 smithi022 ceph-mon[97651]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:08.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.077+0000 7f27c361a700 1 -- 172.21.15.123:0/889938564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27bc0ff900 msgr2=0x7f27bc0ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.077+0000 7f27c361a700 1 --2- 172.21.15.123:0/889938564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27bc0ff900 0x7f27bc0ffd20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009910 tx=0x7f27b8009c20 comp rx=0 tx=0).stop 2023-09-20T21:41:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.077+0000 7f27c361a700 1 -- 172.21.15.123:0/889938564 shutdown_connections 2023-09-20T21:41:08.081 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.077+0000 7f27c361a700 1 --2- 172.21.15.123:0/889938564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27bc0ff900 0x7f27bc0ffd20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.077+0000 7f27c361a700 1 -- 172.21.15.123:0/889938564 >> 172.21.15.123:0/889938564 conn(0x7f27bc0fae80 msgr2=0x7f27bc0fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:41:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.077+0000 7f27c361a700 1 -- 172.21.15.123:0/889938564 shutdown_connections 2023-09-20T21:41:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.078+0000 7f27c361a700 1 -- 172.21.15.123:0/889938564 wait complete. 2023-09-20T21:41:08.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.078+0000 7f27c361a700 1 Processor -- start 2023-09-20T21:41:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.078+0000 7f27c361a700 1 -- start start 2023-09-20T21:41:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.078+0000 7f27c361a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27bc0ff900 0x7f27bc069950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.078+0000 7f27c361a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27bc1989a0 0x7f27bc19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.078+0000 7f27c361a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc19b320 0x7f27bc19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:08.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.078+0000 7f27c361a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc19df60 con 0x7f27bc1989a0 2023-09-20T21:41:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.078+0000 7f27c361a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc19e0d0 con 0x7f27bc0ff900 2023-09-20T21:41:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.078+0000 7f27c361a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc19e290 con 0x7f27bc19b320 2023-09-20T21:41:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.078+0000 7f27c1bb7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc19b320 0x7f27bc19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c1bb7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc19b320 0x7f27bc19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33818/0 (socket says 172.21.15.123:33818) 2023-09-20T21:41:08.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c1bb7700 1 -- 172.21.15.123:0/2401293096 learned_addr learned my addr 172.21.15.123:0/2401293096 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-20T21:41:08.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c13b6700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27bc0ff900 0x7f27bc069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:08.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c1bb7700 1 -- 172.21.15.123:0/2401293096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27bc0ff900 msgr2=0x7f27bc069950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:08.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c1bb7700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27bc0ff900 0x7f27bc069950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:08.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c1bb7700 1 -- 172.21.15.123:0/2401293096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27bc1989a0 msgr2=0x7f27bc19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:08.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c1bb7700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27bc1989a0 0x7f27bc19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:08.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c1bb7700 1 -- 172.21.15.123:0/2401293096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b80095f0 con 0x7f27bc19b320 2023-09-20T21:41:08.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c1bb7700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc19b320 0x7f27bc19d7c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f27b400ba50 tx=0x7f27b400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:41:08.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27b27fc700 1 -- 172.21.15.123:0/2401293096 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b400cab0 con 0x7f27bc19b320 2023-09-20T21:41:08.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27b27fc700 1 -- 172.21.15.123:0/2401293096 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27b400cc10 con 0x7f27bc19b320 2023-09-20T21:41:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c361a700 1 -- 172.21.15.123:0/2401293096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27bc19e570 con 0x7f27bc19b320 2023-09-20T21:41:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27b27fc700 1 -- 172.21.15.123:0/2401293096 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b40075b0 con 0x7f27bc19b320 2023-09-20T21:41:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.079+0000 7f27c361a700 1 -- 172.21.15.123:0/2401293096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27bc19eaf0 con 0x7f27bc19b320 2023-09-20T21:41:08.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.080+0000 7f27b27fc700 1 -- 172.21.15.123:0/2401293096 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7f27b4007710 con 0x7f27bc19b320 2023-09-20T21:41:08.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.080+0000 7f27b27fc700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27a8038cc0 0x7f27a803b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:08.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.080+0000 7f27b27fc700 1 -- 172.21.15.123:0/2401293096 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f27b404bb80 con 0x7f27bc19b320 2023-09-20T21:41:08.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.080+0000 7f27c361a700 1 -- 172.21.15.123:0/2401293096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27bc18e3c0 con 0x7f27bc19b320 2023-09-20T21:41:08.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.080+0000 7f27c13b6700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27a8038cc0 0x7f27a803b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:08.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.081+0000 7f27c13b6700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27a8038cc0 0x7f27a803b180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27b8005e00 tx=0x7f27b8005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:41:08.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.085+0000 7f27b27fc700 1 -- 172.21.15.123:0/2401293096 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f27b401b8b0 con 0x7f27bc19b320 2023-09-20T21:41:08.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.301+0000 7f27c361a700 1 -- 172.21.15.123:0/2401293096 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi089=y;smithi123=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f27bc02d0a0 con 0x7f27a8038cc0 2023-09-20T21:41:08.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.309+0000 7f27b27fc700 1 -- 172.21.15.123:0/2401293096 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f27bc02d0a0 con 0x7f27a8038cc0 2023-09-20T21:41:08.309 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mgr update... 2023-09-20T21:41:08.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 -- 172.21.15.123:0/2401293096 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27a8038cc0 msgr2=0x7f27a803b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:08.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27a8038cc0 0x7f27a803b180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27b8005e00 tx=0x7f27b8005fd0 comp rx=0 tx=0).stop 2023-09-20T21:41:08.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 -- 172.21.15.123:0/2401293096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc19b320 msgr2=0x7f27bc19d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:08.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc19b320 0x7f27bc19d7c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f27b400ba50 tx=0x7f27b400be10 comp rx=0 tx=0).stop 2023-09-20T21:41:08.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 -- 172.21.15.123:0/2401293096 shutdown_connections 2023-09-20T21:41:08.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27bc19b320 0x7f27bc19d7c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:08.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27a8038cc0 0x7f27a803b180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:08.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27bc0ff900 0x7f27bc069950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:08.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 --2- 172.21.15.123:0/2401293096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27bc1989a0 0x7f27bc19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:08.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 -- 172.21.15.123:0/2401293096 >> 172.21.15.123:0/2401293096 conn(0x7f27bc0fae80 msgr2=0x7f27bc0fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:41:08.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 -- 172.21.15.123:0/2401293096 shutdown_connections 2023-09-20T21:41:08.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:41:08.311+0000 7f27c361a700 1 -- 172.21.15.123:0/2401293096 wait complete. 2023-09-20T21:41:08.749 DEBUG:teuthology.orchestra.run.smithi089:mgr.y> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.y.service 2023-09-20T21:41:08.752 DEBUG:teuthology.orchestra.run.smithi123:mgr.z> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.z.service 2023-09-20T21:41:08.754 INFO:tasks.cephadm:Deploying OSDs... 2023-09-20T21:41:08.755 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:41:08.755 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2023-09-20T21:41:08.784 INFO:journalctl@ceph.mgr.z.smithi123.stdout:-- Logs begin at Wed 2023-09-20 21:17:48 UTC. -- 2023-09-20T21:41:08.786 INFO:journalctl@ceph.mgr.y.smithi089.stdout:-- Logs begin at Wed 2023-09-20 21:17:56 UTC. -- 2023-09-20T21:41:08.803 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-20T21:41:08.804 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2023-09-20T21:41:08.851 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-20T21:41:08.851 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:08.851 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-09-20T21:41:08.852 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:08.852 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:08.852 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-09-20 21:40:19.902963684 +0000 2023-09-20T21:41:08.852 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-09-20 21:34:09.606731468 +0000 2023-09-20T21:41:08.853 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-09-20 21:34:09.606731468 +0000 2023-09-20T21:41:08.853 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2023-09-20 21:34:09.583731739 +0000 2023-09-20T21:41:08.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-20T21:41:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-09-20T21:41:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-09-20T21:41:08.922 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000202573 s, 2.5 MB/s 2023-09-20T21:41:08.925 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-20T21:41:08.971 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2023-09-20T21:41:08.991 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:08 smithi123 ceph-mon[134433]: Reconfiguring mon.c (monmap changed)... 2023-09-20T21:41:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:41:08.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-20T21:41:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:08 smithi123 ceph-mon[134433]: Reconfiguring daemon mon.c on smithi123 2023-09-20T21:41:08.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:08.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:08.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:08.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:08 smithi022 ceph-mon[97651]: Reconfiguring mon.c (monmap changed)... 2023-09-20T21:41:08.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:41:08.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-20T21:41:08.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:08.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:08 smithi022 ceph-mon[97651]: Reconfiguring daemon mon.c on smithi123 2023-09-20T21:41:08.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:09.045 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-20T21:41:09.045 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:09.045 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-09-20T21:41:09.046 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:09.046 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:09.046 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-09-20 21:40:19.986962690 +0000 2023-09-20T21:41:09.046 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-09-20 21:34:10.471721298 +0000 2023-09-20T21:41:09.046 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-09-20 21:34:10.471721298 +0000 2023-09-20T21:41:09.047 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2023-09-20 21:34:10.448721568 +0000 2023-09-20T21:41:09.047 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-20T21:41:09.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:09.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:09.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:08 smithi089 ceph-mon[134090]: Reconfiguring mon.c (monmap changed)... 2023-09-20T21:41:09.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-09-20T21:41:09.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-09-20T21:41:09.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:09.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:08 smithi089 ceph-mon[134090]: Reconfiguring daemon mon.c on smithi123 2023-09-20T21:41:09.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:09.115 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-09-20T21:41:09.115 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-09-20T21:41:09.116 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000190882 s, 2.7 MB/s 2023-09-20T21:41:09.118 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-20T21:41:09.164 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2023-09-20T21:41:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-20T21:41:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:09.249 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-09-20T21:41:09.249 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:09.249 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:09.249 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-09-20 21:40:20.062961790 +0000 2023-09-20T21:41:09.249 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-09-20 21:34:11.352710939 +0000 2023-09-20T21:41:09.250 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-09-20 21:34:11.352710939 +0000 2023-09-20T21:41:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2023-09-20 21:34:11.329711209 +0000 2023-09-20T21:41:09.250 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-20T21:41:09.354 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-09-20T21:41:09.354 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-09-20T21:41:09.354 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000201893 s, 2.5 MB/s 2023-09-20T21:41:09.357 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-20T21:41:09.402 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2023-09-20T21:41:09.488 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-20T21:41:09.489 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:09.489 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-09-20T21:41:09.489 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:09.490 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:09.490 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-09-20 21:40:20.139960878 +0000 2023-09-20T21:41:09.490 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-09-20 21:34:12.173701286 +0000 2023-09-20T21:41:09.490 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-09-20 21:34:12.173701286 +0000 2023-09-20T21:41:09.490 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2023-09-20 21:34:12.150701556 +0000 2023-09-20T21:41:09.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-20T21:41:09.592 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-09-20T21:41:09.592 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-09-20T21:41:09.592 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000175004 s, 2.9 MB/s 2023-09-20T21:41:09.594 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-20T21:41:09.641 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2023-09-20T21:41:09.641 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2023-09-20T21:41:09.662 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-20T21:41:09.662 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2023-09-20T21:41:09.723 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-20T21:41:09.723 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:09.723 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 381691 Links: 1 2023-09-20T21:41:09.723 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:09.724 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:09.724 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-09-20 21:40:56.202181536 +0000 2023-09-20T21:41:09.724 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-09-20 21:31:41.531947611 +0000 2023-09-20T21:41:09.724 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-09-20 21:31:41.531947611 +0000 2023-09-20T21:41:09.724 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-09-20T21:41:09.725 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-20T21:41:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-09-20T21:41:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-09-20T21:41:09.793 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000315117 s, 1.6 MB/s 2023-09-20T21:41:09.795 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-20T21:41:09.855 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2023-09-20T21:41:09.914 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-20T21:41:09.915 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:09.915 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 387896 Links: 1 2023-09-20T21:41:09.915 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:09.915 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:09.915 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-09-20 21:40:56.202181536 +0000 2023-09-20T21:41:09.916 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-09-20 21:31:42.042936353 +0000 2023-09-20T21:41:09.916 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-09-20 21:31:42.042936353 +0000 2023-09-20T21:41:09.916 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-09-20T21:41:09.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-20T21:41:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-09-20T21:41:09.982 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-09-20T21:41:09.983 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000234398 s, 2.2 MB/s 2023-09-20T21:41:09.984 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-20T21:41:10.043 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2023-09-20T21:41:10.101 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-20T21:41:10.102 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:10.102 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 388004 Links: 1 2023-09-20T21:41:10.102 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:10.103 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:10.103 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-09-20 21:40:56.203181514 +0000 2023-09-20T21:41:10.103 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-09-20 21:31:42.545925271 +0000 2023-09-20T21:41:10.103 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-09-20 21:31:42.545925271 +0000 2023-09-20T21:41:10.103 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-09-20T21:41:10.104 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-20T21:41:10.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi089=y;smithi123=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:41:10.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: Saving service mgr spec with placement smithi022=x;smithi089=y;smithi123=z;count:3 2023-09-20T21:41:10.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:10.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:10.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:10.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:10.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:10.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:10.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:10.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-20T21:41:10.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-20T21:41:10.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-20T21:41:10.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:10.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:10 smithi089 ceph-mon[134090]: Deploying daemon mgr.z on smithi123 2023-09-20T21:41:10.139 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-09-20T21:41:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-09-20T21:41:10.140 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000229668 s, 2.2 MB/s 2023-09-20T21:41:10.141 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-20T21:41:10.199 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2023-09-20T21:41:10.257 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-20T21:41:10.257 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:10.257 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 6h/6d Inode: 386047 Links: 1 2023-09-20T21:41:10.258 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:10.258 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:10.258 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2023-09-20 21:40:56.203181514 +0000 2023-09-20T21:41:10.258 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2023-09-20 21:31:43.051914123 +0000 2023-09-20T21:41:10.258 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2023-09-20 21:31:43.051914123 +0000 2023-09-20T21:41:10.259 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: - 2023-09-20T21:41:10.259 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-20T21:41:10.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi089=y;smithi123=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:41:10.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: Saving service mgr spec with placement smithi022=x;smithi089=y;smithi123=z;count:3 2023-09-20T21:41:10.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:10.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:10.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:10.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:10.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:10.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:10.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:10.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-20T21:41:10.315 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-20T21:41:10.315 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-20T21:41:10.315 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:10.315 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:10 smithi123 ceph-mon[134433]: Deploying daemon mgr.z on smithi123 2023-09-20T21:41:10.323 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2023-09-20T21:41:10.323 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2023-09-20T21:41:10.323 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000278756 s, 1.8 MB/s 2023-09-20T21:41:10.324 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-20T21:41:10.382 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-09-20T21:41:10.382 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2023-09-20T21:41:10.400 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-20T21:41:10.400 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2023-09-20T21:41:10.464 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-20T21:41:10.464 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:10.465 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 390192 Links: 1 2023-09-20T21:41:10.465 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:10.465 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:10.465 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-20 21:41:10.358997673 +0000 2023-09-20T21:41:10.465 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-20 21:30:47.506458114 +0000 2023-09-20T21:41:10.466 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-20 21:30:47.506458114 +0000 2023-09-20T21:41:10.466 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-20T21:41:10.466 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-20T21:41:10.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=x;smithi089=y;smithi123=z", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:41:10.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: Saving service mgr spec with placement smithi022=x;smithi089=y;smithi123=z;count:3 2023-09-20T21:41:10.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:10.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:10.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:10.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:10.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:10.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:10.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:10.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-20T21:41:10.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-20T21:41:10.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-20T21:41:10.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:10.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:10 smithi022 ceph-mon[97651]: Deploying daemon mgr.z on smithi123 2023-09-20T21:41:10.534 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-20T21:41:10.535 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-20T21:41:10.535 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000225635 s, 2.3 MB/s 2023-09-20T21:41:10.536 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-20T21:41:10.593 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:10 smithi123 systemd[1]: Starting Ceph mgr.z for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:41:10.594 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2023-09-20T21:41:10.652 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-20T21:41:10.653 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:10.653 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 385509 Links: 1 2023-09-20T21:41:10.653 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:10.653 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:10.653 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-20 21:41:10.483994799 +0000 2023-09-20T21:41:10.654 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-20 21:30:48.048445643 +0000 2023-09-20T21:41:10.654 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-20 21:30:48.048445643 +0000 2023-09-20T21:41:10.654 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-20T21:41:10.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-20T21:41:10.722 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-20T21:41:10.722 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-20T21:41:10.722 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000194372 s, 2.6 MB/s 2023-09-20T21:41:10.723 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-20T21:41:10.780 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2023-09-20T21:41:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-20T21:41:10.838 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:10.838 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 393260 Links: 1 2023-09-20T21:41:10.839 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:10.839 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:10.839 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-20 21:41:10.483994799 +0000 2023-09-20T21:41:10.839 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-20 21:30:48.570433631 +0000 2023-09-20T21:41:10.839 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-20 21:30:48.570433631 +0000 2023-09-20T21:41:10.840 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-20T21:41:10.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-20T21:41:10.904 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-20T21:41:10.905 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-20T21:41:10.905 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000219499 s, 2.3 MB/s 2023-09-20T21:41:10.906 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-20T21:41:10.963 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2023-09-20T21:41:11.021 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-20T21:41:11.021 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-20T21:41:11.021 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 391571 Links: 1 2023-09-20T21:41:11.022 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-20T21:41:11.022 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-09-20T21:41:11.022 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-09-20 21:41:10.483994799 +0000 2023-09-20T21:41:11.022 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-09-20 21:30:49.069422149 +0000 2023-09-20T21:41:11.022 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-09-20 21:30:49.069422149 +0000 2023-09-20T21:41:11.023 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-09-20T21:41:11.023 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-20T21:41:11.057 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:10 smithi123 podman[136370]: 2023-09-20T21:41:11.082 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-09-20T21:41:11.082 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-09-20T21:41:11.083 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000212876 s, 2.4 MB/s 2023-09-20T21:41:11.084 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-20T21:41:11.143 INFO:tasks.cephadm:Deploying osd.0 on smithi022 with /dev/vg_nvme/lv_4... 2023-09-20T21:41:11.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-20T21:41:11.343 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:11 smithi123 bash[136370]: 2c7eb7e1ca691875bf981f6525c64a66ed0182f250e1dfa4f3e9224f886536a4 2023-09-20T21:41:11.343 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:11 smithi123 ceph-mgr[136534]: -- 172.21.15.123:0/1538437775 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c4321dd1e0 con 0x55c4321b3000 2023-09-20T21:41:11.344 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:11 smithi123 systemd[1]: Started Ceph mgr.z for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:41:11.344 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:11.191+0000 7f0536a6c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-20T21:41:11.344 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:11.271+0000 7f0536a6c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-20T21:41:11.622 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:11.360+0000 7f0536a6c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-20T21:41:11.623 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:11.452+0000 7f0536a6c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-20T21:41:11.623 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:11.621+0000 7f0536a6c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-20T21:41:11.880 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:41:12.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-mon[134433]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:12.298 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:12.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:12.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:12.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-20T21:41:12.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-20T21:41:12.299 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-20T21:41:12.300 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:12.300 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:11.984+0000 7f0536a6c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-20T21:41:12.300 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:12.113+0000 7f0536a6c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-20T21:41:12.300 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:12.203+0000 7f0536a6c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-20T21:41:12.300 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:12.297+0000 7f0536a6c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-20T21:41:12.450 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:12 smithi022 ceph-mon[97651]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:12.450 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:12.451 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:12.451 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:12.451 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-20T21:41:12.451 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-20T21:41:12.451 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-20T21:41:12.451 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:12.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:12 smithi089 ceph-mon[134090]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:12.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:12.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:12.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:12.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-20T21:41:12.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-09-20T21:41:12.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-20T21:41:12.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:12.860 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:12 smithi089 systemd[1]: Starting Ceph mgr.y for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:41:13.093 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:12 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:12.713+0000 7f0536a6c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-20T21:41:13.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:13 smithi089 ceph-mon[134090]: Deploying daemon mgr.y on smithi089 2023-09-20T21:41:13.264 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:13 smithi089 podman[135529]: 2023-09-20T21:41:13.451 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:13 smithi123 ceph-mon[134433]: Deploying daemon mgr.y on smithi089 2023-09-20T21:41:13.452 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:13 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:13.369+0000 7f0536a6c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-20T21:41:13.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:13 smithi022 ceph-mon[97651]: Deploying daemon mgr.y on smithi089 2023-09-20T21:41:13.568 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:13 smithi089 bash[135529]: 963b524478915b0455f392dbd359ee27d0a16e7410661f5637e99266c7b0b14c 2023-09-20T21:41:13.568 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:13 smithi089 ceph-mgr[135569]: -- 172.21.15.89:0/3846204109 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56477af991e0 con 0x56477af6e800 2023-09-20T21:41:13.568 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:13 smithi089 systemd[1]: Started Ceph mgr.y for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:41:13.568 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:13.384+0000 7fd5fd5e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-20T21:41:13.569 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:13.478+0000 7fd5fd5e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-20T21:41:13.569 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:13.567+0000 7fd5fd5e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-20T21:41:13.841 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:13.662+0000 7fd5fd5e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-20T21:41:13.843 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:13 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:13.450+0000 7f0536a6c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-20T21:41:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-20T21:41:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-20T21:41:13.922 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2023-09-20T21:41:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2023-09-20T21:41:13.922 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0490233 s, 214 MB/s 2023-09-20T21:41:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2023-09-20T21:41:14.110 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:13.841+0000 7fd5fd5e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-20T21:41:14.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:14 smithi022 ceph-mon[97651]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:14.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:14.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi022:vg_nvme/lv_4 2023-09-20T21:41:14.453 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:14.228+0000 7fd5fd5e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-20T21:41:14.453 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:14.363+0000 7fd5fd5e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-20T21:41:14.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:14 smithi089 ceph-mon[134090]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:14.453 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.454 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:14.532 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:14 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:14.250+0000 7f0536a6c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-20T21:41:14.532 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:14 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:14.346+0000 7f0536a6c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-20T21:41:14.533 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:14 smithi123 ceph-mon[134433]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:14.533 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.533 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.533 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.533 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:14.534 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:14.828 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:14 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:14.530+0000 7f0536a6c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-20T21:41:14.828 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:14 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:14.826+0000 7f0536a6c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-20T21:41:14.860 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:14.452+0000 7fd5fd5e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-20T21:41:14.860 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:14.548+0000 7fd5fd5e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-20T21:41:15.093 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:15 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:15.064+0000 7f0536a6c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-20T21:41:15.101 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:41:15.360 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:15.001+0000 7fd5fd5e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-20T21:41:15.482 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:15 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:15.215+0000 7f0536a6c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-20T21:41:15.843 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:15 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:15.480+0000 7f0536a6c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-20T21:41:16.110 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:15.677+0000 7fd5fd5e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-20T21:41:16.111 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:15.758+0000 7fd5fd5e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-20T21:41:16.352 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:16.352 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.352 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.353 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.353 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.353 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:16.353 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:16.353 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-20T21:41:16.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-20T21:41:16.354 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:16.498 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:16.177+0000 7f0536a6c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-20T21:41:16.498 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:16.307+0000 7f0536a6c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-20T21:41:16.498 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:16.496+0000 7f0536a6c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-20T21:41:16.498 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:16.499 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.499 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.499 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.499 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.499 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:16.500 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:16.500 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.500 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-20T21:41:16.500 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-20T21:41:16.500 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:16.524 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:16.524 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.524 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.525 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.525 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.525 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:16.525 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:16.525 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:16.526 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-09-20T21:41:16.526 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-09-20T21:41:16.526 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:16.783 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:16.523+0000 7fd5fd5e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-20T21:41:16.783 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:16.612+0000 7fd5fd5e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-20T21:41:16.826 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:16.610+0000 7f0536a6c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-20T21:41:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.044+0000 7ff9bcba3700 1 -- 172.21.15.22:0/3663045814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b8100a90 msgr2=0x7ff9b8100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.044+0000 7ff9bcba3700 1 --2- 172.21.15.22:0/3663045814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b8100a90 0x7ff9b8100f10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000b0a0 tx=0x7ff9b000b3b0 comp rx=0 tx=0).stop 2023-09-20T21:41:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.044+0000 7ff9bcba3700 1 -- 172.21.15.22:0/3663045814 shutdown_connections 2023-09-20T21:41:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.044+0000 7ff9bcba3700 1 --2- 172.21.15.22:0/3663045814 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b8100a90 0x7ff9b8100f10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.044+0000 7ff9bcba3700 1 --2- 172.21.15.22:0/3663045814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9b80ff770 0x7ff9b80ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.044+0000 7ff9bcba3700 1 --2- 172.21.15.22:0/3663045814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9b81000d0 0x7ff9b8100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.044+0000 7ff9bcba3700 1 -- 172.21.15.22:0/3663045814 >> 172.21.15.22:0/3663045814 conn(0x7ff9b80fb2f0 msgr2=0x7ff9b80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:41:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.044+0000 7ff9bcba3700 1 -- 172.21.15.22:0/3663045814 shutdown_connections 2023-09-20T21:41:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.044+0000 7ff9bcba3700 1 -- 172.21.15.22:0/3663045814 wait complete. 2023-09-20T21:41:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.045+0000 7ff9bcba3700 1 Processor -- start 2023-09-20T21:41:17.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.045+0000 7ff9bcba3700 1 -- start start 2023-09-20T21:41:17.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9bcba3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9b80ff770 0x7ff9b8198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:17.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9bcba3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b81000d0 0x7ff9b8199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:17.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9bcba3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9b8100a90 0x7ff9b81a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:17.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9bcba3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81a09d0 con 0x7ff9b81000d0 2023-09-20T21:41:17.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9bcba3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81a0b10 con 0x7ff9b80ff770 2023-09-20T21:41:17.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9bcba3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81a0ca0 con 0x7ff9b8100a90 2023-09-20T21:41:17.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9b80ff770 0x7ff9b8198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:17.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9b5d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b81000d0 0x7ff9b8199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:17.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9b8100a90 0x7ff9b81a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9b8100a90 0x7ff9b81a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:35710/0 (socket says 172.21.15.22:35710) 2023-09-20T21:41:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9b80ff770 0x7ff9b8198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:41926/0 (socket says 172.21.15.22:41926) 2023-09-20T21:41:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9b659c700 1 -- 172.21.15.22:0/361975394 learned_addr learned my addr 172.21.15.22:0/361975394 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:41:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9b5d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b81000d0 0x7ff9b8199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53840/0 (socket says 172.21.15.22:53840) 2023-09-20T21:41:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9b5d9b700 1 -- 172.21.15.22:0/361975394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9b8100a90 msgr2=0x7ff9b81a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9b5d9b700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9b8100a90 0x7ff9b81a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.046+0000 7ff9b5d9b700 1 -- 172.21.15.22:0/361975394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9b80ff770 msgr2=0x7ff9b8198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.047+0000 7ff9b5d9b700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9b80ff770 0x7ff9b8198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.047+0000 7ff9b5d9b700 1 -- 172.21.15.22:0/361975394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b0009d00 con 0x7ff9b81000d0 2023-09-20T21:41:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.047+0000 7ff9b659c700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9b80ff770 0x7ff9b8198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:41:17.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.047+0000 7ff9b5d9b700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b81000d0 0x7ff9b8199380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00ea20 tx=0x7ff9ac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:41:17.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.047+0000 7ff9a77fe700 1 -- 172.21.15.22:0/361975394 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00cbd0 con 0x7ff9b81000d0 2023-09-20T21:41:17.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.047+0000 7ff9a77fe700 1 -- 172.21.15.22:0/361975394 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9ac00ee60 con 0x7ff9b81000d0 2023-09-20T21:41:17.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.047+0000 7ff9bcba3700 1 -- 172.21.15.22:0/361975394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b81a0f80 con 0x7ff9b81000d0 2023-09-20T21:41:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.047+0000 7ff9a77fe700 1 -- 172.21.15.22:0/361975394 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac018680 con 0x7ff9b81000d0 2023-09-20T21:41:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.047+0000 7ff9bcba3700 1 -- 172.21.15.22:0/361975394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b81a13f0 con 0x7ff9b81000d0 2023-09-20T21:41:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.048+0000 7ff9a77fe700 1 -- 172.21.15.22:0/361975394 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 13) v1 ==== 45674+0+0 (secure 0 0 0) 0x7ff9ac0187e0 con 0x7ff9b81000d0 2023-09-20T21:41:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.048+0000 7ff9bcba3700 1 -- 172.21.15.22:0/361975394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b810a550 con 0x7ff9b81000d0 2023-09-20T21:41:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.049+0000 7ff9a77fe700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9a0038a70 0x7ff9a003af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.049+0000 7ff9a77fe700 1 -- 172.21.15.22:0/361975394 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff9ac014070 con 0x7ff9b81000d0 2023-09-20T21:41:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.049+0000 7ff9b659c700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9a0038a70 0x7ff9a003af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:17.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.049+0000 7ff9b659c700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9a0038a70 0x7ff9a003af30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8011550 tx=0x7ff9a8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:41:17.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.054+0000 7ff9a77fe700 1 -- 172.21.15.22:0/361975394 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff9ac01f900 con 0x7ff9b81000d0 2023-09-20T21:41:17.057 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:16.782+0000 7fd5fd5e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-20T21:41:17.093 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:16.825+0000 7f0536a6c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-20T21:41:17.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:17.279+0000 7ff9bcba3700 1 -- 172.21.15.22:0/361975394 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9b802d230 con 0x7ff9a0038a70 2023-09-20T21:41:17.360 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:17 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:17.057+0000 7fd5fd5e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-20T21:41:17.360 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:17 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:17.281+0000 7fd5fd5e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-20T21:41:17.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:17 smithi089 ceph-mon[134090]: Reconfiguring mgr.x (unknown last config time)... 2023-09-20T21:41:17.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:17 smithi089 ceph-mon[134090]: Reconfiguring daemon mgr.x on smithi022 2023-09-20T21:41:17.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:17.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:17.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:17.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:17 smithi022 ceph-mon[97651]: Reconfiguring mgr.x (unknown last config time)... 2023-09-20T21:41:17.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:17 smithi022 ceph-mon[97651]: Reconfiguring daemon mgr.x on smithi022 2023-09-20T21:41:17.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:17.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:17.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:17.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:17 smithi123 ceph-mon[134433]: Reconfiguring mgr.x (unknown last config time)... 2023-09-20T21:41:17.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:17 smithi123 ceph-mon[134433]: Reconfiguring daemon mgr.x on smithi022 2023-09-20T21:41:17.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:17.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:17.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:17.690 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:17 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:17.430+0000 7fd5fd5e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-20T21:41:18.093 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 21:41:17 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z[136448]: 2023-09-20T21:41:17.688+0000 7f0536a6c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-20T21:41:18.110 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:17 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:17.689+0000 7fd5fd5e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-20T21:41:18.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:18.464+0000 7ff9a77fe700 1 -- 172.21.15.22:0/361975394 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mgrmap(e 14) v1 ==== 90869+0+0 (secure 0 0 0) 0x7ff9ac017510 con 0x7ff9b81000d0 2023-09-20T21:41:18.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:41:18.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:41:18.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:18.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: Standby manager daemon z started 2023-09-20T21:41:18.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-20T21:41:18.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-20T21:41:18.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-20T21:41:18.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:18 smithi022 ceph-mon[97651]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-20T21:41:18.516 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:18.383+0000 7fd5fd5e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-20T21:41:18.516 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:18.516 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:18.516 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:18.517 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:18.517 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:18.517 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:41:18.517 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:41:18.517 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:18.517 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: Standby manager daemon z started 2023-09-20T21:41:18.518 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-20T21:41:18.518 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-20T21:41:18.518 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-20T21:41:18.518 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-mon[134090]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-20T21:41:18.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:18.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:18.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:18.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:18.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:18.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:41:18.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:41:18.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:18.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: Standby manager daemon z started 2023-09-20T21:41:18.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-09-20T21:41:18.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-20T21:41:18.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-09-20T21:41:18.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:18 smithi123 ceph-mon[134433]: from='mgr.? 172.21.15.123:0/699576758' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-20T21:41:18.818 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:18.515+0000 7fd5fd5e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-20T21:41:18.818 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:18.705+0000 7fd5fd5e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-20T21:41:19.110 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:18.817+0000 7fd5fd5e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-20T21:41:19.111 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:19 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:19.032+0000 7fd5fd5e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-20T21:41:19.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:19 smithi022 ceph-mon[97651]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:41:19.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:19 smithi022 ceph-mon[97651]: mgrmap e14: x(active, since 71s), standbys: z 2023-09-20T21:41:19.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-20T21:41:19.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:19 smithi123 ceph-mon[134433]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:41:19.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:19 smithi123 ceph-mon[134433]: mgrmap e14: x(active, since 71s), standbys: z 2023-09-20T21:41:19.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-20T21:41:19.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:19 smithi089 ceph-mon[134090]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:41:19.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:19 smithi089 ceph-mon[134090]: mgrmap e14: x(active, since 71s), standbys: z 2023-09-20T21:41:19.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-09-20T21:41:20.202 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 21:41:19 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y[135545]: 2023-09-20T21:41:19.891+0000 7fd5fd5e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-20T21:41:20.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:20.472+0000 7ff9a77fe700 1 -- 172.21.15.22:0/361975394 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mgrmap(e 15) v1 ==== 136064+0+0 (secure 0 0 0) 0x7ff9ac014230 con 0x7ff9b81000d0 2023-09-20T21:41:20.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:20 smithi022 ceph-mon[97651]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:20 smithi022 ceph-mon[97651]: Standby manager daemon y started 2023-09-20T21:41:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:20 smithi022 ceph-mon[97651]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-20T21:41:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:20 smithi022 ceph-mon[97651]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-20T21:41:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:20 smithi022 ceph-mon[97651]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-20T21:41:20.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:20 smithi022 ceph-mon[97651]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-20T21:41:20.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:20 smithi123 ceph-mon[134433]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:20.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:20 smithi123 ceph-mon[134433]: Standby manager daemon y started 2023-09-20T21:41:20.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:20 smithi123 ceph-mon[134433]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-20T21:41:20.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:20 smithi123 ceph-mon[134433]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-20T21:41:20.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:20 smithi123 ceph-mon[134433]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-20T21:41:20.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:20 smithi123 ceph-mon[134433]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-20T21:41:20.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:20 smithi089 ceph-mon[134090]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:20 smithi089 ceph-mon[134090]: Standby manager daemon y started 2023-09-20T21:41:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:20 smithi089 ceph-mon[134090]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-09-20T21:41:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:20 smithi089 ceph-mon[134090]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-09-20T21:41:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:20 smithi089 ceph-mon[134090]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-09-20T21:41:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:20 smithi089 ceph-mon[134090]: from='mgr.? 172.21.15.89:0/1593410825' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-09-20T21:41:21.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:21 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2019493247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bdd85c1-5ec1-40a9-be24-e1c0701d30a2"}]: dispatch 2023-09-20T21:41:21.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:21 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bdd85c1-5ec1-40a9-be24-e1c0701d30a2"}]: dispatch 2023-09-20T21:41:21.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:21 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6bdd85c1-5ec1-40a9-be24-e1c0701d30a2"}]': finished 2023-09-20T21:41:21.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:21 smithi022 ceph-mon[97651]: osdmap e5: 1 total, 0 up, 1 in 2023-09-20T21:41:21.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:21.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:21 smithi022 ceph-mon[97651]: mgrmap e15: x(active, since 73s), standbys: y, z 2023-09-20T21:41:21.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-20T21:41:21.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:21 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4213227958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:41:21.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:21 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2019493247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bdd85c1-5ec1-40a9-be24-e1c0701d30a2"}]: dispatch 2023-09-20T21:41:21.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:21 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bdd85c1-5ec1-40a9-be24-e1c0701d30a2"}]: dispatch 2023-09-20T21:41:21.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:21 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6bdd85c1-5ec1-40a9-be24-e1c0701d30a2"}]': finished 2023-09-20T21:41:21.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:21 smithi123 ceph-mon[134433]: osdmap e5: 1 total, 0 up, 1 in 2023-09-20T21:41:21.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:21.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:21 smithi123 ceph-mon[134433]: mgrmap e15: x(active, since 73s), standbys: y, z 2023-09-20T21:41:21.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-20T21:41:21.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:21 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4213227958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:41:21.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:21 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2019493247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bdd85c1-5ec1-40a9-be24-e1c0701d30a2"}]: dispatch 2023-09-20T21:41:21.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:21 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6bdd85c1-5ec1-40a9-be24-e1c0701d30a2"}]: dispatch 2023-09-20T21:41:21.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:21 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6bdd85c1-5ec1-40a9-be24-e1c0701d30a2"}]': finished 2023-09-20T21:41:21.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:21 smithi089 ceph-mon[134090]: osdmap e5: 1 total, 0 up, 1 in 2023-09-20T21:41:21.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:21.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:21 smithi089 ceph-mon[134090]: mgrmap e15: x(active, since 73s), standbys: y, z 2023-09-20T21:41:21.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-09-20T21:41:21.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:21 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4213227958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:41:22.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:22 smithi022 ceph-mon[97651]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:22.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:22 smithi123 ceph-mon[134433]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:22.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:22 smithi089 ceph-mon[134090]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:23.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:23 smithi022 ceph-mon[97651]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:23.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:23 smithi123 ceph-mon[134433]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:23.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:23 smithi089 ceph-mon[134090]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:26.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:26 smithi022 ceph-mon[97651]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:26.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:26 smithi123 ceph-mon[134433]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:26.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:26 smithi089 ceph-mon[134090]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:28.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:28 smithi123 ceph-mon[134433]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:28.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-20T21:41:28.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:28.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:28 smithi089 ceph-mon[134090]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:28.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-20T21:41:28.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:28.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:28 smithi022 ceph-mon[97651]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:28.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-09-20T21:41:28.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:29.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:29 smithi089 ceph-mon[134090]: Deploying daemon osd.0 on smithi022 2023-09-20T21:41:29.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:29 smithi022 ceph-mon[97651]: Deploying daemon osd.0 on smithi022 2023-09-20T21:41:29.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:29 smithi123 ceph-mon[134433]: Deploying daemon osd.0 on smithi022 2023-09-20T21:41:30.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:30 smithi022 ceph-mon[97651]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:30.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:30 smithi089 ceph-mon[134090]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:30.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:30 smithi123 ceph-mon[134433]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:32.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:32 smithi089 ceph-mon[134090]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:32 smithi022 ceph-mon[97651]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:32.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:32 smithi123 ceph-mon[134433]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:33.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:33 smithi022 ceph-mon[97651]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:33.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:33 smithi123 ceph-mon[134433]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:33.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:33 smithi089 ceph-mon[134090]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:34.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:34.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:34.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:34.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:34.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:34.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:34.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:34.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:34.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:35.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:35 smithi022 ceph-mon[97651]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:35.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:35 smithi123 ceph-mon[134433]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:35.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:35 smithi089 ceph-mon[134090]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.795+0000 7ff9a77fe700 1 -- 172.21.15.22:0/361975394 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff9b802d230 con 0x7ff9a0038a70 2023-09-20T21:41:37.795 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 0 on host 'smithi022' 2023-09-20T21:41:37.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.797+0000 7ff9bcba3700 1 -- 172.21.15.22:0/361975394 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9a0038a70 msgr2=0x7ff9a003af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:37.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.797+0000 7ff9bcba3700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9a0038a70 0x7ff9a003af30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8011550 tx=0x7ff9a8009260 comp rx=0 tx=0).stop 2023-09-20T21:41:37.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.797+0000 7ff9bcba3700 1 -- 172.21.15.22:0/361975394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b81000d0 msgr2=0x7ff9b8199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.797+0000 7ff9bcba3700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b81000d0 0x7ff9b8199380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00ea20 tx=0x7ff9ac00ed30 comp rx=0 tx=0).stop 2023-09-20T21:41:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.797+0000 7ff9bcba3700 1 -- 172.21.15.22:0/361975394 shutdown_connections 2023-09-20T21:41:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.797+0000 7ff9bcba3700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9b8100a90 0x7ff9b81a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.797+0000 7ff9bcba3700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9a0038a70 0x7ff9a003af30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.797+0000 7ff9bcba3700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff9b80ff770 0x7ff9b8198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.797+0000 7ff9bcba3700 1 --2- 172.21.15.22:0/361975394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9b81000d0 0x7ff9b8199380 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.797+0000 7ff9bcba3700 1 -- 172.21.15.22:0/361975394 >> 172.21.15.22:0/361975394 conn(0x7ff9b80fb2f0 msgr2=0x7ff9b80fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:41:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.798+0000 7ff9bcba3700 1 -- 172.21.15.22:0/361975394 shutdown_connections 2023-09-20T21:41:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:41:37.798+0000 7ff9bcba3700 1 -- 172.21.15.22:0/361975394 wait complete. 2023-09-20T21:41:37.996 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:37.996 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:37.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:37.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:37.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:37.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:37 smithi022 ceph-mon[97651]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:37.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:37.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:37.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:38.327 DEBUG:teuthology.orchestra.run.smithi022:osd.0> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.0.service 2023-09-20T21:41:38.332 INFO:tasks.cephadm:Deploying osd.1 on smithi089 with /dev/vg_nvme/lv_4... 2023-09-20T21:41:38.332 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-20T21:41:38.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:38.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:38.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:38.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:38.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:38.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:37 smithi123 ceph-mon[134433]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:38.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:38.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:38.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:38.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:38.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:38.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:38.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:38.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:38.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:37 smithi089 ceph-mon[134090]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:38.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:38.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:38.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:38.408 INFO:journalctl@ceph.osd.0.smithi022.stdout:-- Logs begin at Wed 2023-09-20 21:22:06 UTC. -- 2023-09-20T21:41:38.894 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T21:41:39.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:38 smithi022 ceph-mon[97651]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-20T21:41:39.316 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:38 smithi089 ceph-mon[134090]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-20T21:41:39.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:38 smithi123 ceph-mon[134433]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-09-20T21:41:40.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:39 smithi089 ceph-mon[134090]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-20T21:41:40.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:39 smithi089 ceph-mon[134090]: osdmap e6: 1 total, 0 up, 1 in 2023-09-20T21:41:40.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:40.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:39 smithi089 ceph-mon[134090]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:41:40.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:39 smithi089 ceph-mon[134090]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:40.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:39 smithi022 ceph-mon[97651]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-20T21:41:40.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:39 smithi022 ceph-mon[97651]: osdmap e6: 1 total, 0 up, 1 in 2023-09-20T21:41:40.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:40.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:39 smithi022 ceph-mon[97651]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:41:40.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:39 smithi022 ceph-mon[97651]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:40.249 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 21:41:39 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-0[116162]: 2023-09-20T21:41:39.940+0000 7fb6c328c700 -1 osd.0 0 waiting for initial osdmap 2023-09-20T21:41:40.249 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 21:41:39 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-0[116162]: 2023-09-20T21:41:39.944+0000 7fb6bd07d700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:41:40.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:39 smithi123 ceph-mon[134433]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-09-20T21:41:40.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:39 smithi123 ceph-mon[134433]: osdmap e6: 1 total, 0 up, 1 in 2023-09-20T21:41:40.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:40.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:39 smithi123 ceph-mon[134433]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:41:40.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:39 smithi123 ceph-mon[134433]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:40.515 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-20T21:41:40.516 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-20T21:41:40.516 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2023-09-20T21:41:40.516 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2023-09-20T21:41:40.516 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109555 s, 957 MB/s 2023-09-20T21:41:40.517 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2023-09-20T21:41:40.947 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi089:vg_nvme/lv_4 2023-09-20T21:41:41.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:40 smithi022 ceph-mon[97651]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:41:41.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:40 smithi022 ceph-mon[97651]: osdmap e7: 1 total, 0 up, 1 in 2023-09-20T21:41:41.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:40 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:41.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:40 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:41.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:40 smithi123 ceph-mon[134433]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:41:41.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:40 smithi123 ceph-mon[134433]: osdmap e7: 1 total, 0 up, 1 in 2023-09-20T21:41:41.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:40 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:41.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:40 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:41.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:40 smithi089 ceph-mon[134090]: from='osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:41:41.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:40 smithi089 ceph-mon[134090]: osdmap e7: 1 total, 0 up, 1 in 2023-09-20T21:41:41.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:40 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:41.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:40 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:41.538 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T21:41:42.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:41 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:41:42.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:41 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:41:42.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:41 smithi089 ceph-mon[134090]: osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] boot 2023-09-20T21:41:42.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:41 smithi089 ceph-mon[134090]: osdmap e8: 1 total, 1 up, 1 in 2023-09-20T21:41:42.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:41 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:42.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:41 smithi089 ceph-mon[134090]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:42.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:41 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:41:42.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:41 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:41:42.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:41 smithi022 ceph-mon[97651]: osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] boot 2023-09-20T21:41:42.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:41 smithi022 ceph-mon[97651]: osdmap e8: 1 total, 1 up, 1 in 2023-09-20T21:41:42.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:41 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:42.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:41 smithi022 ceph-mon[97651]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:42.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:41 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:41:42.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:41 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:41:42.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:41 smithi123 ceph-mon[134433]: osd.0 [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] boot 2023-09-20T21:41:42.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:41 smithi123 ceph-mon[134433]: osdmap e8: 1 total, 1 up, 1 in 2023-09-20T21:41:42.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:41 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-09-20T21:41:42.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:41 smithi123 ceph-mon[134433]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-09-20T21:41:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 -- 172.21.15.89:0/4066532483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc101940 msgr2=0x7f96bc1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 --2- 172.21.15.89:0/4066532483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc101940 0x7f96bc1081f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96b400b0d0 tx=0x7f96b400b3e0 comp rx=0 tx=0).stop 2023-09-20T21:41:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 -- 172.21.15.89:0/4066532483 shutdown_connections 2023-09-20T21:41:42.950 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 --2- 172.21.15.89:0/4066532483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc101940 0x7f96bc1081f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:42.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 --2- 172.21.15.89:0/4066532483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96bc0ffda0 0x7f96bc1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:42.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 --2- 172.21.15.89:0/4066532483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96bc100fa0 0x7f96bc101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:42.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 -- 172.21.15.89:0/4066532483 >> 172.21.15.89:0/4066532483 conn(0x7f96bc0fb360 msgr2=0x7f96bc0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:41:42.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 -- 172.21.15.89:0/4066532483 shutdown_connections 2023-09-20T21:41:42.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 -- 172.21.15.89:0/4066532483 wait complete. 2023-09-20T21:41:42.951 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 Processor -- start 2023-09-20T21:41:42.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 -- start start 2023-09-20T21:41:42.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96bc0ffda0 0x7f96bc195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:42.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.948+0000 7f96c3b86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc100fa0 0x7f96bc196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:42.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c3b86700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96bc101940 0x7f96bc19d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:42.952 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c3b86700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc19d680 con 0x7f96bc101940 2023-09-20T21:41:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c3b86700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc19d7c0 con 0x7f96bc100fa0 2023-09-20T21:41:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c3b86700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc19d950 con 0x7f96bc0ffda0 2023-09-20T21:41:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c1121700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc100fa0 0x7f96bc196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c1121700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc100fa0 0x7f96bc196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55008/0 (socket says 172.21.15.89:55008) 2023-09-20T21:41:42.953 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c1121700 1 -- 172.21.15.89:0/3615696854 learned_addr learned my addr 172.21.15.89:0/3615696854 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-20T21:41:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c1922700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96bc0ffda0 0x7f96bc195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c2123700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96bc101940 0x7f96bc19d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c1121700 1 -- 172.21.15.89:0/3615696854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96bc0ffda0 msgr2=0x7f96bc195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c1121700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96bc0ffda0 0x7f96bc195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:42.954 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c1121700 1 -- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96bc101940 msgr2=0x7f96bc19d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c1121700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96bc101940 0x7f96bc19d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c1121700 1 -- 172.21.15.89:0/3615696854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4009d00 con 0x7f96bc100fa0 2023-09-20T21:41:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c2123700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96bc101940 0x7f96bc19d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:41:42.955 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c1121700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc100fa0 0x7f96bc196030 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00d980 tx=0x7f96ac00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:41:42.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96b2ffd700 1 -- 172.21.15.89:0/3615696854 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac004a30 con 0x7f96bc100fa0 2023-09-20T21:41:42.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96b2ffd700 1 -- 172.21.15.89:0/3615696854 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96ac0054e0 con 0x7f96bc100fa0 2023-09-20T21:41:42.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c3b86700 1 -- 172.21.15.89:0/3615696854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc19dc30 con 0x7f96bc100fa0 2023-09-20T21:41:42.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96b2ffd700 1 -- 172.21.15.89:0/3615696854 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac009c70 con 0x7f96bc100fa0 2023-09-20T21:41:42.956 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.949+0000 7f96c3b86700 1 -- 172.21.15.89:0/3615696854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc19e0e0 con 0x7f96bc100fa0 2023-09-20T21:41:42.957 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.950+0000 7f96c3b86700 1 -- 172.21.15.89:0/3615696854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc1072b0 con 0x7f96bc100fa0 2023-09-20T21:41:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.951+0000 7f96b2ffd700 1 -- 172.21.15.89:0/3615696854 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 15) v1 ==== 136064+0+0 (secure 0 0 0) 0x7f96ac004c20 con 0x7f96bc100fa0 2023-09-20T21:41:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.952+0000 7f96b2ffd700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f96a80a1270 0x7f96a80a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:41:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.952+0000 7f96c1922700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f96a80a1270 0x7f96a80a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:41:42.958 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.953+0000 7f96b2ffd700 1 -- 172.21.15.89:0/3615696854 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f96ac09a8a0 con 0x7f96bc100fa0 2023-09-20T21:41:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.953+0000 7f96c1922700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f96a80a1270 0x7f96a80a3730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f96bc100e00 tx=0x7f96b800a3b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:41:42.959 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:42.958+0000 7f96b2ffd700 1 -- 172.21.15.89:0/3615696854 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f96ac0964e0 con 0x7f96bc100fa0 2023-09-20T21:41:43.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:42 smithi089 ceph-mon[134090]: osdmap e9: 1 total, 1 up, 1 in 2023-09-20T21:41:43.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:42 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi022 2023-09-20T21:41:43.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:41:43.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:42 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi022 to 17272M 2023-09-20T21:41:43.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:43.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:43.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:43.151+0000 7f96c3b86700 1 -- 172.21.15.89:0/3615696854 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f96bc02d260 con 0x7f96a80a1270 2023-09-20T21:41:43.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:42 smithi022 ceph-mon[97651]: osdmap e9: 1 total, 1 up, 1 in 2023-09-20T21:41:43.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:42 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi022 2023-09-20T21:41:43.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:41:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:42 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi022 to 17272M 2023-09-20T21:41:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:43.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:42 smithi123 ceph-mon[134433]: osdmap e9: 1 total, 1 up, 1 in 2023-09-20T21:41:43.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:42 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi022 2023-09-20T21:41:43.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:41:43.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:42 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi022 to 17272M 2023-09-20T21:41:43.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:43.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:43.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:41:43.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:44.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:43 smithi022 ceph-mon[97651]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:44.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:43 smithi022 ceph-mon[97651]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:41:44.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:41:44.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:41:44.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:44.259 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:43 smithi089 ceph-mon[134090]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:44.260 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:43 smithi089 ceph-mon[134090]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:41:44.260 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:43 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:41:44.260 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:43 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:41:44.260 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:43 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:44.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:43 smithi123 ceph-mon[134433]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:44.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:43 smithi123 ceph-mon[134433]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:41:44.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:43 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:41:44.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:43 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:41:44.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:43 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:46.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:46 smithi123 ceph-mon[134433]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:46.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:46 smithi123 ceph-mon[134433]: from='client.? 172.21.15.89:0/2351002272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd88d1d-b602-4722-a8b3-4ef144e4da4c"}]: dispatch 2023-09-20T21:41:46.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:46 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd88d1d-b602-4722-a8b3-4ef144e4da4c"}]: dispatch 2023-09-20T21:41:46.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:46 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acd88d1d-b602-4722-a8b3-4ef144e4da4c"}]': finished 2023-09-20T21:41:46.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:46 smithi123 ceph-mon[134433]: osdmap e10: 2 total, 1 up, 2 in 2023-09-20T21:41:46.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:46 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:41:46.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:46 smithi089 ceph-mon[134090]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:46.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:46 smithi089 ceph-mon[134090]: from='client.? 172.21.15.89:0/2351002272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd88d1d-b602-4722-a8b3-4ef144e4da4c"}]: dispatch 2023-09-20T21:41:46.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:46 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd88d1d-b602-4722-a8b3-4ef144e4da4c"}]: dispatch 2023-09-20T21:41:46.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:46 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acd88d1d-b602-4722-a8b3-4ef144e4da4c"}]': finished 2023-09-20T21:41:46.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:46 smithi089 ceph-mon[134090]: osdmap e10: 2 total, 1 up, 2 in 2023-09-20T21:41:46.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:46 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:41:46.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:46 smithi022 ceph-mon[97651]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:46.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:46 smithi022 ceph-mon[97651]: from='client.? 172.21.15.89:0/2351002272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd88d1d-b602-4722-a8b3-4ef144e4da4c"}]: dispatch 2023-09-20T21:41:46.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:46 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd88d1d-b602-4722-a8b3-4ef144e4da4c"}]: dispatch 2023-09-20T21:41:46.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:46 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acd88d1d-b602-4722-a8b3-4ef144e4da4c"}]': finished 2023-09-20T21:41:46.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:46 smithi022 ceph-mon[97651]: osdmap e10: 2 total, 1 up, 2 in 2023-09-20T21:41:46.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:46 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:41:47.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:47 smithi089 ceph-mon[134090]: from='client.? 172.21.15.89:0/3298110848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:41:47.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:47 smithi022 ceph-mon[97651]: from='client.? 172.21.15.89:0/3298110848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:41:47.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:47 smithi123 ceph-mon[134433]: from='client.? 172.21.15.89:0/3298110848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:41:48.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:48 smithi089 ceph-mon[134090]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:48.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:48 smithi022 ceph-mon[97651]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:48.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:48 smithi123 ceph-mon[134433]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:50.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:50 smithi089 ceph-mon[134090]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:50.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:50 smithi022 ceph-mon[97651]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:50.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:50 smithi123 ceph-mon[134433]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:52.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:52 smithi089 ceph-mon[134090]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:52.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:52 smithi022 ceph-mon[97651]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:52 smithi123 ceph-mon[134433]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:53.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:53 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-20T21:41:53.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:53 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:53.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:53 smithi089 ceph-mon[134090]: Deploying daemon osd.1 on smithi089 2023-09-20T21:41:53.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-20T21:41:53.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:53.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:53 smithi022 ceph-mon[97651]: Deploying daemon osd.1 on smithi089 2023-09-20T21:41:53.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-09-20T21:41:53.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:41:53.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:53 smithi123 ceph-mon[134433]: Deploying daemon osd.1 on smithi089 2023-09-20T21:41:54.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:54 smithi089 ceph-mon[134090]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:54.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:54 smithi022 ceph-mon[97651]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:54.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:54 smithi123 ceph-mon[134433]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:56.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:56 smithi089 ceph-mon[134090]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:56.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:56 smithi022 ceph-mon[97651]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:56.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:56 smithi123 ceph-mon[134433]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:58.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:58.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:58.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:58.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:41:57 smithi089 ceph-mon[134090]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:58.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:58.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:58.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:58.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:41:57 smithi022 ceph-mon[97651]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:58.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:58.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:41:58.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:41:58.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:41:57 smithi123 ceph-mon[134433]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:41:59.868 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.867+0000 7f96b2ffd700 1 -- 172.21.15.89:0/3615696854 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f96bc02d260 con 0x7f96a80a1270 2023-09-20T21:41:59.868 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 1 on host 'smithi089' 2023-09-20T21:41:59.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 -- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f96a80a1270 msgr2=0x7f96a80a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:59.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f96a80a1270 0x7f96a80a3730 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f96bc100e00 tx=0x7f96b800a3b0 comp rx=0 tx=0).stop 2023-09-20T21:41:59.870 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 -- 172.21.15.89:0/3615696854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc100fa0 msgr2=0x7f96bc196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:41:59.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc100fa0 0x7f96bc196030 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00d980 tx=0x7f96ac00dc90 comp rx=0 tx=0).stop 2023-09-20T21:41:59.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 -- 172.21.15.89:0/3615696854 shutdown_connections 2023-09-20T21:41:59.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96bc101940 0x7f96bc19d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:59.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96bc0ffda0 0x7f96bc195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:59.871 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f96a80a1270 0x7f96a80a3730 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:59.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 --2- 172.21.15.89:0/3615696854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc100fa0 0x7f96bc196030 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:41:59.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 -- 172.21.15.89:0/3615696854 >> 172.21.15.89:0/3615696854 conn(0x7f96bc0fb360 msgr2=0x7f96bc1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:41:59.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 -- 172.21.15.89:0/3615696854 shutdown_connections 2023-09-20T21:41:59.872 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:41:59.869+0000 7f96c3b86700 1 -- 172.21.15.89:0/3615696854 wait complete. 2023-09-20T21:42:00.313 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:00 smithi089 ceph-mon[134090]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:42:00.313 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:00 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.313 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:00 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.313 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:00 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:00.314 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:00 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:00.314 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:00 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.314 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:00 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.314 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:00 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:00.314 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:00 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.321 DEBUG:teuthology.orchestra.run.smithi089:osd.1> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.1.service 2023-09-20T21:42:00.325 INFO:tasks.cephadm:Deploying osd.2 on smithi123 with /dev/vg_nvme/lv_4... 2023-09-20T21:42:00.326 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-09-20T21:42:00.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:00 smithi123 ceph-mon[134433]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:42:00.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:00 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:00 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:00 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:00.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:00 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:00.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:00 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:00 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:00 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:00.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:00 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.356 INFO:journalctl@ceph.osd.1.smithi089.stdout:-- Logs begin at Wed 2023-09-20 21:17:56 UTC. -- 2023-09-20T21:42:00.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:00 smithi022 ceph-mon[97651]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:42:00.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:00 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:00 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:00 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:00.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:00 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:00.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:00 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:00 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:00 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:00.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:00 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:00.892 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:42:01.059 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:00 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-1[139100]: 2023-09-20T21:42:00.868+0000 7f283434f640 -1 osd.1 0 log_to_monitors true 2023-09-20T21:42:01.329 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:01 smithi089 ceph-mon[134090]: from='osd.1 [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-20T21:42:01.330 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:01 smithi089 ceph-mon[134090]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-20T21:42:01.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:01 smithi123 ceph-mon[134433]: from='osd.1 [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-20T21:42:01.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:01 smithi123 ceph-mon[134433]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-20T21:42:01.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:01 smithi022 ceph-mon[97651]: from='osd.1 [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-20T21:42:01.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:01 smithi022 ceph-mon[97651]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-09-20T21:42:02.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:02 smithi123 ceph-mon[134433]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:42:02.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:02 smithi123 ceph-mon[134433]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-20T21:42:02.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:02 smithi123 ceph-mon[134433]: osdmap e11: 2 total, 1 up, 2 in 2023-09-20T21:42:02.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:02 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:02.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:02 smithi123 ceph-mon[134433]: from='osd.1 [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:42:02.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:02 smithi123 ceph-mon[134433]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:42:02.360 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:02 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-1[139100]: 2023-09-20T21:42:02.073+0000 7f28299c7700 -1 osd.1 0 waiting for initial osdmap 2023-09-20T21:42:02.361 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:02 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-1[139100]: 2023-09-20T21:42:02.076+0000 7f2821fb5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:42:02.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:02 smithi089 ceph-mon[134090]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:42:02.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:02 smithi089 ceph-mon[134090]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-20T21:42:02.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:02 smithi089 ceph-mon[134090]: osdmap e11: 2 total, 1 up, 2 in 2023-09-20T21:42:02.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:02 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:02.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:02 smithi089 ceph-mon[134090]: from='osd.1 [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:42:02.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:02 smithi089 ceph-mon[134090]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:42:02.406 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-09-20T21:42:02.406 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-09-20T21:42:02.406 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-09-20T21:42:02.407 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-09-20T21:42:02.407 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011441 s, 917 MB/s 2023-09-20T21:42:02.407 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-09-20T21:42:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:02 smithi022 ceph-mon[97651]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:42:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:02 smithi022 ceph-mon[97651]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-09-20T21:42:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:02 smithi022 ceph-mon[97651]: osdmap e11: 2 total, 1 up, 2 in 2023-09-20T21:42:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:02 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:02.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:02 smithi022 ceph-mon[97651]: from='osd.1 [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:42:02.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:02 smithi022 ceph-mon[97651]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:42:02.842 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2023-09-20T21:42:03.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:42:03.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: osdmap e12: 2 total, 1 up, 2 in 2023-09-20T21:42:03.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:03.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:03.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:03.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:03.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:03.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:42:03.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: osdmap e12: 2 total, 1 up, 2 in 2023-09-20T21:42:03.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:03.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:03.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:03.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:03.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:03.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.439 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:42:03.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:42:03.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: osdmap e12: 2 total, 1 up, 2 in 2023-09-20T21:42:03.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:03.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:03.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:03.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:03.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:03.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:03.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:04.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:04 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:42:04.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:04 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:42:04.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:04 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi089 2023-09-20T21:42:04.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:04 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi089 to 17168M 2023-09-20T21:42:04.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:04 smithi123 ceph-mon[134433]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:42:04.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:04 smithi123 ceph-mon[134433]: osd.1 [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] boot 2023-09-20T21:42:04.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:04 smithi123 ceph-mon[134433]: osdmap e13: 2 total, 2 up, 2 in 2023-09-20T21:42:04.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:04 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:04.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:04 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:42:04.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:04 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:42:04.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:04 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi089 2023-09-20T21:42:04.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:04 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi089 to 17168M 2023-09-20T21:42:04.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:04 smithi089 ceph-mon[134090]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:42:04.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:04 smithi089 ceph-mon[134090]: osd.1 [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] boot 2023-09-20T21:42:04.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:04 smithi089 ceph-mon[134090]: osdmap e13: 2 total, 2 up, 2 in 2023-09-20T21:42:04.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:04 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:04.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:04 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:42:04.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:04 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:42:04.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:04 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi089 2023-09-20T21:42:04.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:04 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi089 to 17168M 2023-09-20T21:42:04.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:04 smithi022 ceph-mon[97651]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-09-20T21:42:04.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:04 smithi022 ceph-mon[97651]: osd.1 [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] boot 2023-09-20T21:42:04.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:04 smithi022 ceph-mon[97651]: osdmap e13: 2 total, 2 up, 2 in 2023-09-20T21:42:04.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:04 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-09-20T21:42:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.631+0000 7fb7331b2700 1 -- 172.21.15.123:0/2628388853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c1000d0 msgr2=0x7fb72c100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.631+0000 7fb7331b2700 1 --2- 172.21.15.123:0/2628388853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c1000d0 0x7fb72c100550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb718009990 tx=0x7fb718009ca0 comp rx=0 tx=0).stop 2023-09-20T21:42:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.631+0000 7fb7331b2700 1 -- 172.21.15.123:0/2628388853 shutdown_connections 2023-09-20T21:42:04.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.631+0000 7fb7331b2700 1 --2- 172.21.15.123:0/2628388853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c100a90 0x7fb72c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.631+0000 7fb7331b2700 1 --2- 172.21.15.123:0/2628388853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb72c0ff770 0x7fb72c0ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.631+0000 7fb7331b2700 1 --2- 172.21.15.123:0/2628388853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c1000d0 0x7fb72c100550 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.631+0000 7fb7331b2700 1 -- 172.21.15.123:0/2628388853 >> 172.21.15.123:0/2628388853 conn(0x7fb72c0fb2f0 msgr2=0x7fb72c0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:42:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.631+0000 7fb7331b2700 1 -- 172.21.15.123:0/2628388853 shutdown_connections 2023-09-20T21:42:04.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.631+0000 7fb7331b2700 1 -- 172.21.15.123:0/2628388853 wait complete. 2023-09-20T21:42:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb7331b2700 1 Processor -- start 2023-09-20T21:42:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb7331b2700 1 -- start start 2023-09-20T21:42:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb7331b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c0ff770 0x7fb72c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb7331b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c1000d0 0x7fb72c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:04.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb7331b2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb72c100a90 0x7fb72c1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:04.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb7331b2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c1a0880 con 0x7fb72c100a90 2023-09-20T21:42:04.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb7331b2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c1a09c0 con 0x7fb72c1000d0 2023-09-20T21:42:04.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb7331b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c1a0b80 con 0x7fb72c0ff770 2023-09-20T21:42:04.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb730f4e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c0ff770 0x7fb72c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:04.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb730f4e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c0ff770 0x7fb72c198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33846/0 (socket says 172.21.15.123:33846) 2023-09-20T21:42:04.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb730f4e700 1 -- 172.21.15.123:0/3465368989 learned_addr learned my addr 172.21.15.123:0/3465368989 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-20T21:42:04.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb73174f700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb72c100a90 0x7fb72c1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:04.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.632+0000 7fb72bfff700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c1000d0 0x7fb72c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:04.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb730f4e700 1 -- 172.21.15.123:0/3465368989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c1000d0 msgr2=0x7fb72c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:04.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb730f4e700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c1000d0 0x7fb72c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb730f4e700 1 -- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb72c100a90 msgr2=0x7fb72c1a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb730f4e700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb72c100a90 0x7fb72c1a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb730f4e700 1 -- 172.21.15.123:0/3465368989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7180095f0 con 0x7fb72c0ff770 2023-09-20T21:42:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb73174f700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb72c100a90 0x7fb72c1a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:42:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb72bfff700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c1000d0 0x7fb72c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:42:04.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb730f4e700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c0ff770 0x7fb72c198d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb72000d950 tx=0x7fb72000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:42:04.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb729ffb700 1 -- 172.21.15.123:0/3465368989 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720009b40 con 0x7fb72c0ff770 2023-09-20T21:42:04.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb729ffb700 1 -- 172.21.15.123:0/3465368989 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb720009ca0 con 0x7fb72c0ff770 2023-09-20T21:42:04.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb7331b2700 1 -- 172.21.15.123:0/3465368989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c1a0e60 con 0x7fb72c0ff770 2023-09-20T21:42:04.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb729ffb700 1 -- 172.21.15.123:0/3465368989 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72000f620 con 0x7fb72c0ff770 2023-09-20T21:42:04.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.633+0000 7fb7331b2700 1 -- 172.21.15.123:0/3465368989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c1a1280 con 0x7fb72c0ff770 2023-09-20T21:42:04.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.634+0000 7fb7331b2700 1 -- 172.21.15.123:0/3465368989 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c10a550 con 0x7fb72c0ff770 2023-09-20T21:42:04.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.635+0000 7fb729ffb700 1 -- 172.21.15.123:0/3465368989 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) v1 ==== 136064+0+0 (secure 0 0 0) 0x7fb720010460 con 0x7fb72c0ff770 2023-09-20T21:42:04.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.635+0000 7fb729ffb700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb71c0a12f0 0x7fb71c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:04.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.636+0000 7fb729ffb700 1 -- 172.21.15.123:0/3465368989 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fb7200cb670 con 0x7fb72c0ff770 2023-09-20T21:42:04.642 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.636+0000 7fb72bfff700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb71c0a12f0 0x7fb71c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:04.642 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.637+0000 7fb72bfff700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb71c0a12f0 0x7fb71c0a37b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb718005f50 tx=0x7fb718005e80 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:42:04.642 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.640+0000 7fb729ffb700 1 -- 172.21.15.123:0/3465368989 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb720096590 con 0x7fb72c0ff770 2023-09-20T21:42:04.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:04.834+0000 7fb7331b2700 1 -- 172.21.15.123:0/3465368989 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb72c02d270 con 0x7fb71c0a12f0 2023-09-20T21:42:05.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:05 smithi123 ceph-mon[134433]: osdmap e14: 2 total, 2 up, 2 in 2023-09-20T21:42:05.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:42:05.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:42:05.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:05.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:05 smithi089 ceph-mon[134090]: osdmap e14: 2 total, 2 up, 2 in 2023-09-20T21:42:05.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:42:05.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:42:05.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:05.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:05 smithi022 ceph-mon[97651]: osdmap e14: 2 total, 2 up, 2 in 2023-09-20T21:42:05.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:42:05.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:42:05.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:06.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:06 smithi123 ceph-mon[134433]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:42:06.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:06 smithi123 ceph-mon[134433]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:06.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:06 smithi089 ceph-mon[134090]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:42:06.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:06 smithi089 ceph-mon[134090]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:06.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:06 smithi022 ceph-mon[97651]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:42:06.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:06 smithi022 ceph-mon[97651]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:08.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:08 smithi089 ceph-mon[134090]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:08.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:08 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/1721602563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28865364-a5f4-426f-98a6-19cb758151a0"}]: dispatch 2023-09-20T21:42:08.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:08 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28865364-a5f4-426f-98a6-19cb758151a0"}]: dispatch 2023-09-20T21:42:08.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:08 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28865364-a5f4-426f-98a6-19cb758151a0"}]': finished 2023-09-20T21:42:08.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:08 smithi089 ceph-mon[134090]: osdmap e15: 3 total, 2 up, 3 in 2023-09-20T21:42:08.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:08.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:08 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/1879775639' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:42:08.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:08 smithi022 ceph-mon[97651]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:08.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:08 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/1721602563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28865364-a5f4-426f-98a6-19cb758151a0"}]: dispatch 2023-09-20T21:42:08.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:08 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28865364-a5f4-426f-98a6-19cb758151a0"}]: dispatch 2023-09-20T21:42:08.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:08 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28865364-a5f4-426f-98a6-19cb758151a0"}]': finished 2023-09-20T21:42:08.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:08 smithi022 ceph-mon[97651]: osdmap e15: 3 total, 2 up, 3 in 2023-09-20T21:42:08.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:08.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:08 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/1879775639' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:42:08.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:08 smithi123 ceph-mon[134433]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:08.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:08 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/1721602563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28865364-a5f4-426f-98a6-19cb758151a0"}]: dispatch 2023-09-20T21:42:08.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:08 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28865364-a5f4-426f-98a6-19cb758151a0"}]: dispatch 2023-09-20T21:42:08.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:08 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28865364-a5f4-426f-98a6-19cb758151a0"}]': finished 2023-09-20T21:42:08.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:08 smithi123 ceph-mon[134433]: osdmap e15: 3 total, 2 up, 3 in 2023-09-20T21:42:08.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:08.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:08 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/1879775639' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:42:10.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:10 smithi089 ceph-mon[134090]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:10.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:10 smithi022 ceph-mon[97651]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:10.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:10 smithi123 ceph-mon[134433]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:12.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:12 smithi123 ceph-mon[134433]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:12.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:12 smithi089 ceph-mon[134090]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:12.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:12 smithi022 ceph-mon[97651]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:13.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:13 smithi123 ceph-mon[134433]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:13.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:13 smithi022 ceph-mon[97651]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:13.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:13 smithi089 ceph-mon[134090]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:14.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-20T21:42:14.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:14.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:14 smithi123 ceph-mon[134433]: Deploying daemon osd.2 on smithi123 2023-09-20T21:42:14.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-20T21:42:14.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:14.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:14 smithi022 ceph-mon[97651]: Deploying daemon osd.2 on smithi123 2023-09-20T21:42:14.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-09-20T21:42:14.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:14.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:14 smithi089 ceph-mon[134090]: Deploying daemon osd.2 on smithi123 2023-09-20T21:42:15.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:15 smithi022 ceph-mon[97651]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:15.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:15 smithi123 ceph-mon[134433]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:15.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:15 smithi089 ceph-mon[134090]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:18.155 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:18 smithi123 ceph-mon[134433]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:18.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:18 smithi089 ceph-mon[134090]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:18.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:18 smithi022 ceph-mon[97651]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:19.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:19.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:19.306 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:19.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:19.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:19.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:19.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:19.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:19.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:20.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:20 smithi123 ceph-mon[134433]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:20.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:20.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:20.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:20.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:20.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:20 smithi022 ceph-mon[97651]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:20.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:20.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:20.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:20 smithi089 ceph-mon[134090]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:20.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:20.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.798+0000 7fb729ffb700 1 -- 172.21.15.123:0/3465368989 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb72c02d270 con 0x7fb71c0a12f0 2023-09-20T21:42:20.799 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 2 on host 'smithi123' 2023-09-20T21:42:20.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 -- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb71c0a12f0 msgr2=0x7fb71c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:20.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb71c0a12f0 0x7fb71c0a37b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb718005f50 tx=0x7fb718005e80 comp rx=0 tx=0).stop 2023-09-20T21:42:20.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 -- 172.21.15.123:0/3465368989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c0ff770 msgr2=0x7fb72c198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:20.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c0ff770 0x7fb72c198d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb72000d950 tx=0x7fb72000dd10 comp rx=0 tx=0).stop 2023-09-20T21:42:20.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 -- 172.21.15.123:0/3465368989 shutdown_connections 2023-09-20T21:42:20.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb72c100a90 0x7fb72c1a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:20.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb72c0ff770 0x7fb72c198d30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:20.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb71c0a12f0 0x7fb71c0a37b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:20.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 --2- 172.21.15.123:0/3465368989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb72c1000d0 0x7fb72c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:20.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 -- 172.21.15.123:0/3465368989 >> 172.21.15.123:0/3465368989 conn(0x7fb72c0fb2f0 msgr2=0x7fb72c0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:42:20.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 -- 172.21.15.123:0/3465368989 shutdown_connections 2023-09-20T21:42:20.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:42:20.801+0000 7fb7331b2700 1 -- 172.21.15.123:0/3465368989 wait complete. 2023-09-20T21:42:21.213 DEBUG:teuthology.orchestra.run.smithi123:osd.2> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.2.service 2023-09-20T21:42:21.215 INFO:tasks.cephadm:Deploying osd.3 on smithi022 with /dev/vg_nvme/lv_3... 2023-09-20T21:42:21.215 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-20T21:42:21.243 INFO:journalctl@ceph.osd.2.smithi123.stdout:-- Logs begin at Wed 2023-09-20 21:17:48 UTC. -- 2023-09-20T21:42:21.956 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:42:22.080 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:22.080 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:22.080 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:22.080 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:21 smithi123 ceph-mon[134433]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:22.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:22.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:22.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:22.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:21 smithi089 ceph-mon[134090]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:22.230 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:22.230 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:22.231 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:22.231 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:21 smithi022 ceph-mon[97651]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:22.343 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-2[140228]: 2023-09-20T21:42:22.157+0000 7f253316b640 -1 osd.2 0 log_to_monitors true 2023-09-20T21:42:22.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:22 smithi123 ceph-mon[134433]: from='osd.2 [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-20T21:42:22.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:22 smithi123 ceph-mon[134433]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-20T21:42:23.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:22 smithi089 ceph-mon[134090]: from='osd.2 [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-20T21:42:23.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:22 smithi089 ceph-mon[134090]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-20T21:42:23.201 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:22 smithi022 ceph-mon[97651]: from='osd.2 [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-20T21:42:23.201 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:22 smithi022 ceph-mon[97651]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-09-20T21:42:23.944 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-20T21:42:23.945 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-20T21:42:23.945 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2023-09-20T21:42:23.945 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2023-09-20T21:42:23.945 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0426251 s, 246 MB/s 2023-09-20T21:42:23.945 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2023-09-20T21:42:24.093 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-2[140228]: 2023-09-20T21:42:23.803+0000 7f25287e3700 -1 osd.2 0 waiting for initial osdmap 2023-09-20T21:42:24.093 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-2[140228]: 2023-09-20T21:42:23.808+0000 7f2523dd7700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:42:24.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:23 smithi123 ceph-mon[134433]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-20T21:42:24.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:23 smithi123 ceph-mon[134433]: osdmap e16: 3 total, 2 up, 3 in 2023-09-20T21:42:24.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:23 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:24.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:23 smithi123 ceph-mon[134433]: from='osd.2 [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:42:24.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:23 smithi123 ceph-mon[134433]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:42:24.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:23 smithi123 ceph-mon[134433]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:24.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:23 smithi089 ceph-mon[134090]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-20T21:42:24.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:23 smithi089 ceph-mon[134090]: osdmap e16: 3 total, 2 up, 3 in 2023-09-20T21:42:24.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:23 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:24.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:23 smithi089 ceph-mon[134090]: from='osd.2 [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:42:24.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:23 smithi089 ceph-mon[134090]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:42:24.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:23 smithi089 ceph-mon[134090]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:24.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:23 smithi022 ceph-mon[97651]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-09-20T21:42:24.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:23 smithi022 ceph-mon[97651]: osdmap e16: 3 total, 2 up, 3 in 2023-09-20T21:42:24.128 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:24.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:23 smithi022 ceph-mon[97651]: from='osd.2 [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:42:24.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:23 smithi022 ceph-mon[97651]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:42:24.129 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:23 smithi022 ceph-mon[97651]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:24.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi022:vg_nvme/lv_3 2023-09-20T21:42:25.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:42:25.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: osdmap e17: 3 total, 2 up, 3 in 2023-09-20T21:42:25.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:25.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:25.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi123 2023-09-20T21:42:25.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:25.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi123 to 17168M 2023-09-20T21:42:25.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:25.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:25.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:42:25.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: osdmap e17: 3 total, 2 up, 3 in 2023-09-20T21:42:25.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:25.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:25.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi123 2023-09-20T21:42:25.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:25.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi123 to 17168M 2023-09-20T21:42:25.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:25.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:25.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:42:25.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:42:25.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: osdmap e17: 3 total, 2 up, 3 in 2023-09-20T21:42:25.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:25.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:25.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi123 2023-09-20T21:42:25.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:25.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi123 to 17168M 2023-09-20T21:42:25.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:25.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:25.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:25.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:26.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:25 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:42:26.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:25 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:42:26.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:25 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:26.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:25 smithi123 ceph-mon[134433]: osd.2 [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] boot 2023-09-20T21:42:26.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:25 smithi123 ceph-mon[134433]: osdmap e18: 3 total, 3 up, 3 in 2023-09-20T21:42:26.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:25 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:26.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:25 smithi123 ceph-mon[134433]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:26.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:25 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-20T21:42:26.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:25 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:42:26.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:25 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:42:26.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:25 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:26.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:25 smithi089 ceph-mon[134090]: osd.2 [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] boot 2023-09-20T21:42:26.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:25 smithi089 ceph-mon[134090]: osdmap e18: 3 total, 3 up, 3 in 2023-09-20T21:42:26.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:25 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:26.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:25 smithi089 ceph-mon[134090]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:26.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:25 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-20T21:42:26.204 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:25 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:42:26.204 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:25 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:42:26.205 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:26.205 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:25 smithi022 ceph-mon[97651]: osd.2 [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] boot 2023-09-20T21:42:26.205 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:25 smithi022 ceph-mon[97651]: osdmap e18: 3 total, 3 up, 3 in 2023-09-20T21:42:26.205 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-09-20T21:42:26.205 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:25 smithi022 ceph-mon[97651]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-09-20T21:42:26.205 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-09-20T21:42:26.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.811+0000 7f67f62a6700 1 -- 172.21.15.22:0/3918162820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67f0101fd0 msgr2=0x7f67f0102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.811+0000 7f67f62a6700 1 --2- 172.21.15.22:0/3918162820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67f0101fd0 0x7f67f0102450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009990 tx=0x7f67e4009ca0 comp rx=0 tx=0).stop 2023-09-20T21:42:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.811+0000 7f67f62a6700 1 -- 172.21.15.22:0/3918162820 shutdown_connections 2023-09-20T21:42:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.811+0000 7f67f62a6700 1 --2- 172.21.15.22:0/3918162820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67f0102990 0x7f67f01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.811+0000 7f67f62a6700 1 --2- 172.21.15.22:0/3918162820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f0100e70 0x7f67f0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.811+0000 7f67f62a6700 1 --2- 172.21.15.22:0/3918162820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67f0101fd0 0x7f67f0102450 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.811+0000 7f67f62a6700 1 -- 172.21.15.22:0/3918162820 >> 172.21.15.22:0/3918162820 conn(0x7f67f00fc3d0 msgr2=0x7f67f00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:42:26.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.811+0000 7f67f62a6700 1 -- 172.21.15.22:0/3918162820 shutdown_connections 2023-09-20T21:42:26.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.811+0000 7f67f62a6700 1 -- 172.21.15.22:0/3918162820 wait complete. 2023-09-20T21:42:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.812+0000 7f67f62a6700 1 Processor -- start 2023-09-20T21:42:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.812+0000 7f67f62a6700 1 -- start start 2023-09-20T21:42:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67f62a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f0100e70 0x7f67f0074b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67f62a6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67f0101fd0 0x7f67f00731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67f62a6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67f0102990 0x7f67f0073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67f62a6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f0073e90 con 0x7f67f0101fd0 2023-09-20T21:42:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67f62a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f0074000 con 0x7f67f0100e70 2023-09-20T21:42:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67f62a6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f01a0be0 con 0x7f67f0102990 2023-09-20T21:42:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67effff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f0100e70 0x7f67f0074b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:26.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67effff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f0100e70 0x7f67f0074b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:45428/0 (socket says 172.21.15.22:45428) 2023-09-20T21:42:26.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67effff700 1 -- 172.21.15.22:0/4190677157 learned_addr learned my addr 172.21.15.22:0/4190677157 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:42:26.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67ef7fe700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67f0101fd0 0x7f67f00731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:26.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67f4843700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67f0102990 0x7f67f0073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:26.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67effff700 1 -- 172.21.15.22:0/4190677157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67f0102990 msgr2=0x7f67f0073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67effff700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67f0102990 0x7f67f0073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67effff700 1 -- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67f0101fd0 msgr2=0x7f67f00731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67effff700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67f0101fd0 0x7f67f00731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67effff700 1 -- 172.21.15.22:0/4190677157 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e40095f0 con 0x7f67f0100e70 2023-09-20T21:42:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.813+0000 7f67ef7fe700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67f0101fd0 0x7f67f00731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:42:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.814+0000 7f67effff700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f0100e70 0x7f67f0074b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e9a0 tx=0x7f67e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:42:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.814+0000 7f67ed7fa700 1 -- 172.21.15.22:0/4190677157 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0009790 con 0x7f67f0100e70 2023-09-20T21:42:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.814+0000 7f67ed7fa700 1 -- 172.21.15.22:0/4190677157 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67e0010430 con 0x7f67f0100e70 2023-09-20T21:42:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.814+0000 7f67f62a6700 1 -- 172.21.15.22:0/4190677157 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f01a0ec0 con 0x7f67f0100e70 2023-09-20T21:42:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.814+0000 7f67ed7fa700 1 -- 172.21.15.22:0/4190677157 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0018680 con 0x7f67f0100e70 2023-09-20T21:42:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.814+0000 7f67f62a6700 1 -- 172.21.15.22:0/4190677157 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f01a12d0 con 0x7f67f0100e70 2023-09-20T21:42:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.815+0000 7f67f62a6700 1 -- 172.21.15.22:0/4190677157 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f010c700 con 0x7f67f0100e70 2023-09-20T21:42:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.817+0000 7f67ed7fa700 1 -- 172.21.15.22:0/4190677157 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 15) v1 ==== 136064+0+0 (secure 0 0 0) 0x7f67e00105a0 con 0x7f67f0100e70 2023-09-20T21:42:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.818+0000 7f67ed7fa700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67d80a1270 0x7f67d80a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:26.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.819+0000 7f67ed7fa700 1 -- 172.21.15.22:0/4190677157 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f67e0014070 con 0x7f67f0100e70 2023-09-20T21:42:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.819+0000 7f67ef7fe700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67d80a1270 0x7f67d80a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.819+0000 7f67ef7fe700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67d80a1270 0x7f67d80a3730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f67e400b2e0 tx=0x7f67e4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:42:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:26.821+0000 7f67ed7fa700 1 -- 172.21.15.22:0/4190677157 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f67e00963a0 con 0x7f67f0100e70 2023-09-20T21:42:27.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:27.062+0000 7f67f62a6700 1 -- 172.21.15.22:0/4190677157 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f67f0061220 con 0x7f67d80a1270 2023-09-20T21:42:27.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:26 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-20T21:42:27.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:26 smithi123 ceph-mon[134433]: osdmap e19: 3 total, 3 up, 3 in 2023-09-20T21:42:27.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:26 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-20T21:42:27.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:26 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-20T21:42:27.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:26 smithi089 ceph-mon[134090]: osdmap e19: 3 total, 3 up, 3 in 2023-09-20T21:42:27.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:26 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-20T21:42:27.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:26 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-09-20T21:42:27.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:26 smithi022 ceph-mon[97651]: osdmap e19: 3 total, 3 up, 3 in 2023-09-20T21:42:27.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:26 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-09-20T21:42:27.747 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 21:42:27 smithi022 sudo[122246]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-20T21:42:27.748 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 21:42:27 smithi022 sudo[122246]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:27.748 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 21:42:27 smithi022 sudo[122246]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:27.748 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 21:42:27 smithi022 sudo[122246]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:27.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:27.855+0000 7f67ed7fa700 1 -- 172.21.15.22:0/4190677157 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f67e0010880 con 0x7f67f0100e70 2023-09-20T21:42:28.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-20T21:42:28.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:27 smithi089 ceph-mon[134090]: osdmap e20: 3 total, 3 up, 3 in 2023-09-20T21:42:28.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:27 smithi089 ceph-mon[134090]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:28.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:27 smithi089 ceph-mon[134090]: from='client.24182 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:42:28.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:42:28.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:42:28.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:28.182 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 21:42:28 smithi022 sudo[122290]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-20T21:42:28.183 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 21:42:28 smithi022 sudo[122290]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:28.183 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 21:42:28 smithi022 sudo[122290]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:28.183 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 21:42:28 smithi022 sudo[122290]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:28.184 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-20T21:42:28.184 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:27 smithi022 ceph-mon[97651]: osdmap e20: 3 total, 3 up, 3 in 2023-09-20T21:42:28.185 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:27 smithi022 ceph-mon[97651]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:28.185 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:27 smithi022 ceph-mon[97651]: from='client.24182 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:42:28.185 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:42:28.186 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:42:28.186 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:28.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-09-20T21:42:28.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:27 smithi123 ceph-mon[134433]: osdmap e20: 3 total, 3 up, 3 in 2023-09-20T21:42:28.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:27 smithi123 ceph-mon[134433]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:28.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:27 smithi123 ceph-mon[134433]: from='client.24182 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:42:28.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:42:28.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:42:28.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:28.579 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:28 smithi089 sudo[144429]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-20T21:42:28.579 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:28 smithi089 sudo[144429]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:28.579 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:28 smithi089 sudo[144429]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:28.579 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:28 smithi089 sudo[144429]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:28.847 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:28 smithi089 sudo[144432]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-20T21:42:28.847 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:28 smithi089 sudo[144432]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:28.847 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:28 smithi089 sudo[144432]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:28.848 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 21:42:28 smithi089 sudo[144432]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:29.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:28 smithi123 ceph-mon[134433]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-09-20T21:42:29.103 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:28 smithi123 ceph-mon[134433]: osdmap e21: 3 total, 3 up, 3 in 2023-09-20T21:42:29.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:29.104 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:29.104 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:28 smithi123 sudo[144860]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-20T21:42:29.104 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:28 smithi123 sudo[144860]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:29.105 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:28 smithi123 sudo[144860]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:29.105 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:28 smithi123 sudo[144860]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:29.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:28 smithi089 ceph-mon[134090]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-09-20T21:42:29.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:28 smithi089 ceph-mon[134090]: osdmap e21: 3 total, 3 up, 3 in 2023-09-20T21:42:29.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:29.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:29.114 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:28 smithi022 ceph-mon[97651]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-09-20T21:42:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:28 smithi022 ceph-mon[97651]: osdmap e21: 3 total, 3 up, 3 in 2023-09-20T21:42:29.115 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:29.116 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:29.593 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:29 smithi123 sudo[144863]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-20T21:42:29.593 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:29 smithi123 sudo[144863]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:29.594 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:29 smithi123 sudo[144863]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:29.594 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 21:42:29 smithi123 sudo[144863]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:29.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:29 smithi022 sudo[122403]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-20T21:42:29.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:29 smithi022 sudo[122403]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:29.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:29 smithi022 sudo[122403]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:29.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:29 smithi022 sudo[122403]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:30.483 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:30.483 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:30 smithi123 sudo[144866]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-20T21:42:30.484 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:30 smithi123 sudo[144866]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:30.484 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:30 smithi123 sudo[144866]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:30.484 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:30 smithi123 sudo[144866]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:30.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:30 smithi022 sudo[122435]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-20T21:42:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:30 smithi022 sudo[122435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:30 smithi022 sudo[122435]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:30 smithi022 sudo[122435]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:30.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:30.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:30 smithi123 sudo[144869]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-20T21:42:30.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:30 smithi123 sudo[144869]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:30.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:30 smithi123 sudo[144869]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:30.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:30 smithi123 sudo[144869]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:31.213 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:30 smithi089 sudo[144435]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-09-20T21:42:31.213 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:30 smithi089 sudo[144435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:31.213 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:30 smithi089 sudo[144435]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:31.213 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:30 smithi089 sudo[144435]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:31.213 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 sudo[144438]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-09-20T21:42:31.214 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 sudo[144438]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-09-20T21:42:31.214 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 sudo[144438]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-09-20T21:42:31.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:31.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-20T21:42:31.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1986991369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6"}]: dispatch 2023-09-20T21:42:31.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-20T21:42:31.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:42:31.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:42:31.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:42:31.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6"}]: dispatch 2023-09-20T21:42:31.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6"}]': finished 2023-09-20T21:42:31.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: osdmap e22: 4 total, 3 up, 4 in 2023-09-20T21:42:31.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:31.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-20T21:42:31.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:42:31.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:42:31.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:42:31.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-20T21:42:31.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-20T21:42:31.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:42:31.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:42:31.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:42:31.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1740059230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:42:31.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:31.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:31 smithi022 ceph-mon[97651]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-20T21:42:31.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-20T21:42:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1986991369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6"}]: dispatch 2023-09-20T21:42:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-20T21:42:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:42:31.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:42:31.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:42:31.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6"}]: dispatch 2023-09-20T21:42:31.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6"}]': finished 2023-09-20T21:42:31.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: osdmap e22: 4 total, 3 up, 4 in 2023-09-20T21:42:31.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:31.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-20T21:42:31.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:42:31.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:42:31.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:42:31.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-20T21:42:31.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-20T21:42:31.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:42:31.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:42:31.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:42:31.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1740059230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:42:31.598 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:31.598 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:31 smithi123 ceph-mon[134433]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-20T21:42:31.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 sudo[144438]: pam_unix(sudo:session): session closed for user root 2023-09-20T21:42:31.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:31.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-20T21:42:31.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1986991369' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6"}]: dispatch 2023-09-20T21:42:31.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-20T21:42:31.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:42:31.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:42:31.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:42:31.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6"}]: dispatch 2023-09-20T21:42:31.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6"}]': finished 2023-09-20T21:42:31.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: osdmap e22: 4 total, 3 up, 4 in 2023-09-20T21:42:31.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:31.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-20T21:42:31.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:42:31.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:42:31.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:42:31.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-20T21:42:31.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-09-20T21:42:31.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-09-20T21:42:31.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-09-20T21:42:31.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-09-20T21:42:31.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1740059230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:42:31.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:31.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:31 smithi089 ceph-mon[134090]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-09-20T21:42:33.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:33 smithi022 ceph-mon[97651]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:33.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:33 smithi123 ceph-mon[134433]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:33.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:33 smithi089 ceph-mon[134090]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:36.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:36 smithi022 ceph-mon[97651]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-20T21:42:36.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:36 smithi089 ceph-mon[134090]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-20T21:42:36.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:36 smithi123 ceph-mon[134433]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-09-20T21:42:38.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:38 smithi089 ceph-mon[134090]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.6 KiB/s rd, 115 KiB/s wr, 8 op/s 2023-09-20T21:42:38.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:38 smithi022 ceph-mon[97651]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.6 KiB/s rd, 115 KiB/s wr, 8 op/s 2023-09-20T21:42:38.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:38 smithi123 ceph-mon[134433]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.6 KiB/s rd, 115 KiB/s wr, 8 op/s 2023-09-20T21:42:39.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-20T21:42:39.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:39.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:39 smithi089 ceph-mon[134090]: Deploying daemon osd.3 on smithi022 2023-09-20T21:42:39.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-20T21:42:39.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:39.400 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:39 smithi022 ceph-mon[97651]: Deploying daemon osd.3 on smithi022 2023-09-20T21:42:39.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-09-20T21:42:39.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:39.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:39 smithi123 ceph-mon[134433]: Deploying daemon osd.3 on smithi022 2023-09-20T21:42:40.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:40 smithi089 ceph-mon[134090]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 89 KiB/s wr, 6 op/s 2023-09-20T21:42:40.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:40 smithi022 ceph-mon[97651]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 89 KiB/s wr, 6 op/s 2023-09-20T21:42:40.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:40 smithi123 ceph-mon[134433]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 89 KiB/s wr, 6 op/s 2023-09-20T21:42:42.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:42 smithi022 ceph-mon[97651]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 5 op/s 2023-09-20T21:42:42.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:42 smithi123 ceph-mon[134433]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 5 op/s 2023-09-20T21:42:42.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:42 smithi089 ceph-mon[134090]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 5 op/s 2023-09-20T21:42:43.616 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:43 smithi022 ceph-mon[97651]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-09-20T21:42:43.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:43 smithi123 ceph-mon[134433]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-09-20T21:42:43.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:43 smithi089 ceph-mon[134090]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-09-20T21:42:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:45.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:45 smithi022 ceph-mon[97651]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-09-20T21:42:46.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:45 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:46.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:45 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:46.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:45 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:46.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:45 smithi123 ceph-mon[134433]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-09-20T21:42:46.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:45 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:46.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:45 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:46.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:45 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:46.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:45 smithi089 ceph-mon[134090]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-09-20T21:42:48.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.226+0000 7f67ed7fa700 1 -- 172.21.15.22:0/4190677157 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f67f0061220 con 0x7f67d80a1270 2023-09-20T21:42:48.227 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 3 on host 'smithi022' 2023-09-20T21:42:48.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 -- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67d80a1270 msgr2=0x7f67d80a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:48.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67d80a1270 0x7f67d80a3730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f67e400b2e0 tx=0x7f67e4005fb0 comp rx=0 tx=0).stop 2023-09-20T21:42:48.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 -- 172.21.15.22:0/4190677157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f0100e70 msgr2=0x7f67f0074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:48.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f0100e70 0x7f67f0074b80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e9a0 tx=0x7f67e000ecb0 comp rx=0 tx=0).stop 2023-09-20T21:42:48.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 -- 172.21.15.22:0/4190677157 shutdown_connections 2023-09-20T21:42:48.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67f0102990 0x7f67f0073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:48.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67d80a1270 0x7f67d80a3730 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:48.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f67f0100e70 0x7f67f0074b80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:48.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 --2- 172.21.15.22:0/4190677157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67f0101fd0 0x7f67f00731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:48.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 -- 172.21.15.22:0/4190677157 >> 172.21.15.22:0/4190677157 conn(0x7f67f00fc3d0 msgr2=0x7f67f00fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:42:48.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 -- 172.21.15.22:0/4190677157 shutdown_connections 2023-09-20T21:42:48.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:42:48.229+0000 7f67f62a6700 1 -- 172.21.15.22:0/4190677157 wait complete. 2023-09-20T21:42:48.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:48 smithi089 ceph-mon[134090]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:48.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:48 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:48.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:48 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:48.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:48 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:48.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:48 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:48.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:48 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:48.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:48 smithi022 ceph-mon[97651]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:48.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:48 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:48.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:48 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:48.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:48 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:48.425 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:48 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:48.425 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:48 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:48.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:48 smithi123 ceph-mon[134433]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:48.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:48.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:48.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:48.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:48.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:48.699 DEBUG:teuthology.orchestra.run.smithi022:osd.3> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.3.service 2023-09-20T21:42:48.704 INFO:tasks.cephadm:Deploying osd.4 on smithi089 with /dev/vg_nvme/lv_3... 2023-09-20T21:42:48.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-20T21:42:48.792 INFO:journalctl@ceph.osd.3.smithi022.stdout:-- Logs begin at Wed 2023-09-20 21:22:06 UTC. -- 2023-09-20T21:42:48.998 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 21:42:48 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3[124679]: 2023-09-20T21:42:48.813+0000 7f4e8c355640 -1 osd.3 0 log_to_monitors true 2023-09-20T21:42:49.268 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T21:42:49.472 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:49.473 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:49.473 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:49.473 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:49 smithi089 ceph-mon[134090]: from='osd.3 [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-20T21:42:49.473 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:49 smithi089 ceph-mon[134090]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-20T21:42:49.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:49.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:49.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:49.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:49 smithi022 ceph-mon[97651]: from='osd.3 [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-20T21:42:49.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:49 smithi022 ceph-mon[97651]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-20T21:42:49.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:49.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:49.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:42:49.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:49 smithi123 ceph-mon[134433]: from='osd.3 [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-20T21:42:49.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:49 smithi123 ceph-mon[134433]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-09-20T21:42:50.248 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 21:42:50 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3[124679]: 2023-09-20T21:42:50.210+0000 7f4e819cd700 -1 osd.3 0 waiting for initial osdmap 2023-09-20T21:42:50.248 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 21:42:50 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3[124679]: 2023-09-20T21:42:50.217+0000 7f4e797ba700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:42:50.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:50 smithi022 ceph-mon[97651]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:50.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:50 smithi022 ceph-mon[97651]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-20T21:42:50.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:50 smithi022 ceph-mon[97651]: osdmap e23: 4 total, 3 up, 4 in 2023-09-20T21:42:50.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:50 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:50.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:50 smithi022 ceph-mon[97651]: from='osd.3 [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:42:50.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:50 smithi022 ceph-mon[97651]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:42:50.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:50 smithi123 ceph-mon[134433]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:50.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:50 smithi123 ceph-mon[134433]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-20T21:42:50.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:50 smithi123 ceph-mon[134433]: osdmap e23: 4 total, 3 up, 4 in 2023-09-20T21:42:50.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:50 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:50.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:50 smithi123 ceph-mon[134433]: from='osd.3 [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:42:50.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:50 smithi123 ceph-mon[134433]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:42:50.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:50 smithi089 ceph-mon[134090]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:50.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:50 smithi089 ceph-mon[134090]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-09-20T21:42:50.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:50 smithi089 ceph-mon[134090]: osdmap e23: 4 total, 3 up, 4 in 2023-09-20T21:42:50.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:50 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:50.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:50 smithi089 ceph-mon[134090]: from='osd.3 [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:42:50.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:50 smithi089 ceph-mon[134090]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:42:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-20T21:42:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-20T21:42:50.777 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2023-09-20T21:42:50.777 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2023-09-20T21:42:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109155 s, 961 MB/s 2023-09-20T21:42:50.778 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 2023-09-20T21:42:50.778 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2023-09-20T21:42:51.312 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi089:vg_nvme/lv_3 2023-09-20T21:42:51.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:51 smithi022 ceph-mon[97651]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:42:51.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:51 smithi022 ceph-mon[97651]: osdmap e24: 4 total, 3 up, 4 in 2023-09-20T21:42:51.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:51 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:51.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:51 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:51.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:51 smithi022 ceph-mon[97651]: osd.3 [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] boot 2023-09-20T21:42:51.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:51 smithi022 ceph-mon[97651]: osdmap e25: 4 total, 4 up, 4 in 2023-09-20T21:42:51.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:51 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:51.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:51 smithi123 ceph-mon[134433]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:42:51.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:51 smithi123 ceph-mon[134433]: osdmap e24: 4 total, 3 up, 4 in 2023-09-20T21:42:51.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:51 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:51.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:51 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:51.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:51 smithi123 ceph-mon[134433]: osd.3 [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] boot 2023-09-20T21:42:51.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:51 smithi123 ceph-mon[134433]: osdmap e25: 4 total, 4 up, 4 in 2023-09-20T21:42:51.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:51 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:51.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:51 smithi089 ceph-mon[134090]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:42:51.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:51 smithi089 ceph-mon[134090]: osdmap e24: 4 total, 3 up, 4 in 2023-09-20T21:42:51.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:51 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:51.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:51 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:51.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:51 smithi089 ceph-mon[134090]: osd.3 [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] boot 2023-09-20T21:42:51.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:51 smithi089 ceph-mon[134090]: osdmap e25: 4 total, 4 up, 4 in 2023-09-20T21:42:51.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:51 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-09-20T21:42:51.866 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T21:42:52.349 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:42:52.350 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:42:52.350 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:52.350 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.350 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.350 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:52.351 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:52.351 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.351 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:52.351 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:52.351 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.352 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:52 smithi089 ceph-mon[134090]: osdmap e26: 4 total, 4 up, 4 in 2023-09-20T21:42:52.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:42:52.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:42:52.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:52.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:52.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:52.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:52.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:52.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:52 smithi022 ceph-mon[97651]: osdmap e26: 4 total, 4 up, 4 in 2023-09-20T21:42:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:42:52.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:42:52.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-09-20T21:42:52.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:52.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:42:52.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:52.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:42:52.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:42:52.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:52 smithi123 ceph-mon[134433]: osdmap e26: 4 total, 4 up, 4 in 2023-09-20T21:42:53.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 -- 172.21.15.89:0/4061722966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27e4102990 msgr2=0x7f27e41092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:53.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 --2- 172.21.15.89:0/4061722966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27e4102990 0x7f27e41092b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00b0a0 tx=0x7f27dc00b3b0 comp rx=0 tx=0).stop 2023-09-20T21:42:53.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 -- 172.21.15.89:0/4061722966 shutdown_connections 2023-09-20T21:42:53.056 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 --2- 172.21.15.89:0/4061722966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27e4102990 0x7f27e41092b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:53.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 --2- 172.21.15.89:0/4061722966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27e4100e70 0x7f27e4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:53.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 --2- 172.21.15.89:0/4061722966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e4101fd0 0x7f27e4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:53.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 -- 172.21.15.89:0/4061722966 >> 172.21.15.89:0/4061722966 conn(0x7f27e40fc3d0 msgr2=0x7f27e40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:42:53.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 -- 172.21.15.89:0/4061722966 shutdown_connections 2023-09-20T21:42:53.057 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 -- 172.21.15.89:0/4061722966 wait complete. 2023-09-20T21:42:53.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 Processor -- start 2023-09-20T21:42:53.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.053+0000 7f27ebcd1700 1 -- start start 2023-09-20T21:42:53.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27ebcd1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27e4100e70 0x7f27e4196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:53.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27ebcd1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27e4101fd0 0x7f27e41970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:53.058 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27ebcd1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e4102990 0x7f27e419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:53.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27ebcd1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e419e700 con 0x7f27e4101fd0 2023-09-20T21:42:53.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27ebcd1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e419e840 con 0x7f27e4100e70 2023-09-20T21:42:53.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27ebcd1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e419e9d0 con 0x7f27e4102990 2023-09-20T21:42:53.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27e9a6d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27e4100e70 0x7f27e4196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:53.059 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27e9a6d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27e4100e70 0x7f27e4196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55084/0 (socket says 172.21.15.89:55084) 2023-09-20T21:42:53.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27e9a6d700 1 -- 172.21.15.89:0/2939850799 learned_addr learned my addr 172.21.15.89:0/2939850799 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-20T21:42:53.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27ea26e700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e4102990 0x7f27e419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:53.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27e9a6d700 1 -- 172.21.15.89:0/2939850799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e4102990 msgr2=0x7f27e419e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:53.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27e9a6d700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e4102990 0x7f27e419e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:53.060 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27e9a6d700 1 -- 172.21.15.89:0/2939850799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27e4101fd0 msgr2=0x7f27e41970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:42:53.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27e9a6d700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27e4101fd0 0x7f27e41970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:42:53.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27e9a6d700 1 -- 172.21.15.89:0/2939850799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc009d00 con 0x7f27e4100e70 2023-09-20T21:42:53.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27e9a6d700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27e4100e70 0x7f27e4196b70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c9b0 tx=0x7f27d400ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:42:53.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27daffd700 1 -- 172.21.15.89:0/2939850799 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4007b00 con 0x7f27e4100e70 2023-09-20T21:42:53.061 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27daffd700 1 -- 172.21.15.89:0/2939850799 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d4007c60 con 0x7f27e4100e70 2023-09-20T21:42:53.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27ebcd1700 1 -- 172.21.15.89:0/2939850799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e419ecb0 con 0x7f27e4100e70 2023-09-20T21:42:53.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27daffd700 1 -- 172.21.15.89:0/2939850799 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4018660 con 0x7f27e4100e70 2023-09-20T21:42:53.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.054+0000 7f27ebcd1700 1 -- 172.21.15.89:0/2939850799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e419f1b0 con 0x7f27e4100e70 2023-09-20T21:42:53.062 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.055+0000 7f27ebcd1700 1 -- 172.21.15.89:0/2939850799 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e4107cb0 con 0x7f27e4100e70 2023-09-20T21:42:53.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.057+0000 7f27daffd700 1 -- 172.21.15.89:0/2939850799 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f27d400f450 con 0x7f27e4100e70 2023-09-20T21:42:53.063 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.057+0000 7f27daffd700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27d00a1300 0x7f27d00a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:42:53.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.057+0000 7f27daffd700 1 -- 172.21.15.89:0/2939850799 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f27d40cbbe0 con 0x7f27e4100e70 2023-09-20T21:42:53.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.058+0000 7f27e926c700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27d00a1300 0x7f27d00a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:42:53.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.058+0000 7f27e926c700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27d00a1300 0x7f27d00a37c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27e0005950 tx=0x7f27e00058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:42:53.064 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.060+0000 7f27daffd700 1 -- 172.21.15.89:0/2939850799 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f27d4096540 con 0x7f27e4100e70 2023-09-20T21:42:53.259 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:42:53.258+0000 7f27ebcd1700 1 -- 172.21.15.89:0/2939850799 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f27e402d2d0 con 0x7f27d00a1300 2023-09-20T21:42:53.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:53 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi022 2023-09-20T21:42:53.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:53 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi022 to 8636M 2023-09-20T21:42:53.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:53 smithi022 ceph-mon[97651]: osdmap e27: 4 total, 4 up, 4 in 2023-09-20T21:42:53.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:53 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi022 2023-09-20T21:42:53.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:53 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi022 to 8636M 2023-09-20T21:42:53.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:53 smithi123 ceph-mon[134433]: osdmap e27: 4 total, 4 up, 4 in 2023-09-20T21:42:53.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:53 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi022 2023-09-20T21:42:53.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:53 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi022 to 8636M 2023-09-20T21:42:53.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:53 smithi089 ceph-mon[134090]: osdmap e27: 4 total, 4 up, 4 in 2023-09-20T21:42:54.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:54 smithi089 ceph-mon[134090]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:42:54.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:54 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:42:54.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:54 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:42:54.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:54 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:54.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:54 smithi022 ceph-mon[97651]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:42:54.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:54 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:42:54.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:54 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:42:54.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:54 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:54.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:54 smithi123 ceph-mon[134433]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:42:54.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:54 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:42:54.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:54 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:42:54.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:54 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:42:55.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:55 smithi022 ceph-mon[97651]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:42:55.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:55 smithi022 ceph-mon[97651]: osdmap e28: 4 total, 4 up, 4 in 2023-09-20T21:42:55.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:55 smithi123 ceph-mon[134433]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:42:55.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:55 smithi123 ceph-mon[134433]: osdmap e28: 4 total, 4 up, 4 in 2023-09-20T21:42:55.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:55 smithi089 ceph-mon[134090]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:42:55.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:55 smithi089 ceph-mon[134090]: osdmap e28: 4 total, 4 up, 4 in 2023-09-20T21:42:56.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:56 smithi022 ceph-mon[97651]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-20T21:42:56.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:56 smithi022 ceph-mon[97651]: osdmap e29: 4 total, 4 up, 4 in 2023-09-20T21:42:56.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:56 smithi022 ceph-mon[97651]: from='client.? 172.21.15.89:0/1057224226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f26e77-aca2-47d8-8158-4d080560576e"}]: dispatch 2023-09-20T21:42:56.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:56 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f26e77-aca2-47d8-8158-4d080560576e"}]: dispatch 2023-09-20T21:42:56.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:56 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4f26e77-aca2-47d8-8158-4d080560576e"}]': finished 2023-09-20T21:42:56.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:56 smithi022 ceph-mon[97651]: osdmap e30: 5 total, 4 up, 5 in 2023-09-20T21:42:56.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:56 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:42:56.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:56 smithi022 ceph-mon[97651]: from='client.? 172.21.15.89:0/3494329068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:42:56.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:56 smithi123 ceph-mon[134433]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-20T21:42:56.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:56 smithi123 ceph-mon[134433]: osdmap e29: 4 total, 4 up, 4 in 2023-09-20T21:42:56.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:56 smithi123 ceph-mon[134433]: from='client.? 172.21.15.89:0/1057224226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f26e77-aca2-47d8-8158-4d080560576e"}]: dispatch 2023-09-20T21:42:56.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:56 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f26e77-aca2-47d8-8158-4d080560576e"}]: dispatch 2023-09-20T21:42:56.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:56 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4f26e77-aca2-47d8-8158-4d080560576e"}]': finished 2023-09-20T21:42:56.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:56 smithi123 ceph-mon[134433]: osdmap e30: 5 total, 4 up, 5 in 2023-09-20T21:42:56.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:56 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:42:56.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:56 smithi123 ceph-mon[134433]: from='client.? 172.21.15.89:0/3494329068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:42:56.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:56 smithi089 ceph-mon[134090]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-20T21:42:56.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:56 smithi089 ceph-mon[134090]: osdmap e29: 4 total, 4 up, 4 in 2023-09-20T21:42:56.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:56 smithi089 ceph-mon[134090]: from='client.? 172.21.15.89:0/1057224226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f26e77-aca2-47d8-8158-4d080560576e"}]: dispatch 2023-09-20T21:42:56.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:56 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f26e77-aca2-47d8-8158-4d080560576e"}]: dispatch 2023-09-20T21:42:56.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:56 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4f26e77-aca2-47d8-8158-4d080560576e"}]': finished 2023-09-20T21:42:56.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:56 smithi089 ceph-mon[134090]: osdmap e30: 5 total, 4 up, 5 in 2023-09-20T21:42:56.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:56 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:42:56.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:56 smithi089 ceph-mon[134090]: from='client.? 172.21.15.89:0/3494329068' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:42:57.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:42:57 smithi022 ceph-mon[97651]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-20T21:42:57.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:42:57 smithi123 ceph-mon[134433]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-20T21:42:57.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:42:57 smithi089 ceph-mon[134090]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-09-20T21:43:00.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:00 smithi089 ceph-mon[134090]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-09-20T21:43:00.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:00 smithi022 ceph-mon[97651]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-09-20T21:43:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:00 smithi123 ceph-mon[134433]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-09-20T21:43:02.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:02 smithi089 ceph-mon[134090]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:02 smithi022 ceph-mon[97651]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:02 smithi123 ceph-mon[134433]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:03.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-20T21:43:03.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:03.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-20T21:43:03.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:03.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-09-20T21:43:03.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:04.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:04 smithi089 ceph-mon[134090]: Deploying daemon osd.4 on smithi089 2023-09-20T21:43:04.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:04 smithi089 ceph-mon[134090]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:04.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:04 smithi022 ceph-mon[97651]: Deploying daemon osd.4 on smithi089 2023-09-20T21:43:04.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:04 smithi022 ceph-mon[97651]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:04.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:04 smithi123 ceph-mon[134433]: Deploying daemon osd.4 on smithi089 2023-09-20T21:43:04.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:04 smithi123 ceph-mon[134433]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:06.423 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:06 smithi089 ceph-mon[134090]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:06.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:06 smithi022 ceph-mon[97651]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:06.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:06 smithi123 ceph-mon[134433]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:07.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:07.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:07.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:07.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:07 smithi089 ceph-mon[134090]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:07 smithi123 ceph-mon[134433]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:07 smithi022 ceph-mon[97651]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:09.275 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.274+0000 7f27daffd700 1 -- 172.21.15.89:0/2939850799 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f27e402d2d0 con 0x7f27d00a1300 2023-09-20T21:43:09.275 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 4 on host 'smithi089' 2023-09-20T21:43:09.277 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 -- 172.21.15.89:0/2939850799 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27d00a1300 msgr2=0x7f27d00a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27d00a1300 0x7f27d00a37c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27e0005950 tx=0x7f27e00058e0 comp rx=0 tx=0).stop 2023-09-20T21:43:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 -- 172.21.15.89:0/2939850799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27e4100e70 msgr2=0x7f27e4196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27e4100e70 0x7f27e4196b70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c9b0 tx=0x7f27d400ccc0 comp rx=0 tx=0).stop 2023-09-20T21:43:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 -- 172.21.15.89:0/2939850799 shutdown_connections 2023-09-20T21:43:09.278 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e4102990 0x7f27e419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27d00a1300 0x7f27d00a37c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27e4100e70 0x7f27e4196b70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 --2- 172.21.15.89:0/2939850799 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27e4101fd0 0x7f27e41970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 -- 172.21.15.89:0/2939850799 >> 172.21.15.89:0/2939850799 conn(0x7f27e40fc3d0 msgr2=0x7f27e40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:43:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 -- 172.21.15.89:0/2939850799 shutdown_connections 2023-09-20T21:43:09.279 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:43:09.277+0000 7f27ebcd1700 1 -- 172.21.15.89:0/2939850799 wait complete. 2023-09-20T21:43:09.740 DEBUG:teuthology.orchestra.run.smithi089:osd.4> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.4.service 2023-09-20T21:43:09.742 INFO:tasks.cephadm:Deploying osd.5 on smithi123 with /dev/vg_nvme/lv_3... 2023-09-20T21:43:09.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-09-20T21:43:09.793 INFO:journalctl@ceph.osd.4.smithi089.stdout:-- Logs begin at Wed 2023-09-20 21:17:56 UTC. -- 2023-09-20T21:43:10.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:10.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:10.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:09 smithi123 ceph-mon[134433]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:10.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:10.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:10.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:10.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:09 smithi089 ceph-mon[134090]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:10.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:10.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:10.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:10.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:09 smithi022 ceph-mon[97651]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:10.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:10.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:10.318 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:43:10.830 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 20 21:43:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-4[147603]: 2023-09-20T21:43:10.533+0000 7f01e4f77640 -1 osd.4 0 log_to_monitors true 2023-09-20T21:43:11.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:10 smithi089 ceph-mon[134090]: from='osd.4 [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-20T21:43:11.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:10 smithi089 ceph-mon[134090]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-20T21:43:11.161 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:10 smithi123 ceph-mon[134433]: from='osd.4 [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-20T21:43:11.162 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:10 smithi123 ceph-mon[134433]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-20T21:43:11.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:10 smithi022 ceph-mon[97651]: from='osd.4 [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-20T21:43:11.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:10 smithi022 ceph-mon[97651]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-09-20T21:43:11.848 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-09-20T21:43:11.849 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-09-20T21:43:11.849 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-09-20T21:43:11.849 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-09-20T21:43:11.849 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131374 s, 798 MB/s 2023-09-20T21:43:11.849 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-09-20T21:43:12.147 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:11 smithi123 ceph-mon[134433]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-20T21:43:12.148 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:11 smithi123 ceph-mon[134433]: osdmap e31: 5 total, 4 up, 5 in 2023-09-20T21:43:12.148 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:12.148 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:11 smithi123 ceph-mon[134433]: from='osd.4 [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:43:12.148 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:11 smithi123 ceph-mon[134433]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:43:12.148 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:11 smithi123 ceph-mon[134433]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:12.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:11 smithi022 ceph-mon[97651]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-20T21:43:12.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:11 smithi022 ceph-mon[97651]: osdmap e31: 5 total, 4 up, 5 in 2023-09-20T21:43:12.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:12.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:11 smithi022 ceph-mon[97651]: from='osd.4 [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:43:12.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:11 smithi022 ceph-mon[97651]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:43:12.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:11 smithi022 ceph-mon[97651]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:12.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2023-09-20T21:43:12.275 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 20 21:43:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-4[147603]: 2023-09-20T21:43:11.946+0000 7f01da5ef700 -1 osd.4 0 waiting for initial osdmap 2023-09-20T21:43:12.275 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 20 21:43:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-4[147603]: 2023-09-20T21:43:11.952+0000 7f01d33de700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:43:12.276 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:11 smithi089 ceph-mon[134090]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-09-20T21:43:12.276 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:11 smithi089 ceph-mon[134090]: osdmap e31: 5 total, 4 up, 5 in 2023-09-20T21:43:12.276 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:12.276 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:11 smithi089 ceph-mon[134090]: from='osd.4 [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:43:12.277 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:11 smithi089 ceph-mon[134090]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:43:12.277 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:11 smithi089 ceph-mon[134090]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:12.815 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:43:13.240 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:43:13.240 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: osdmap e32: 5 total, 4 up, 5 in 2023-09-20T21:43:13.240 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:13.240 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:13.241 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.241 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.241 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:13.241 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:13.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:13.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:13.242 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:43:13.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: osdmap e32: 5 total, 4 up, 5 in 2023-09-20T21:43:13.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:13.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:13.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:13.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:13.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:13.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:13.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:43:13.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: osdmap e32: 5 total, 4 up, 5 in 2023-09-20T21:43:13.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:13.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:13.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:13.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:13.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:13.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:13.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:13.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.992+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/534212593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a00fffe0 msgr2=0x7fa4a0100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:13.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.992+0000 7fa4a4fd0700 1 --2- 172.21.15.123:0/534212593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a00fffe0 0x7fa4a0100460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa494009990 tx=0x7fa494009ca0 comp rx=0 tx=0).stop 2023-09-20T21:43:13.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.993+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/534212593 shutdown_connections 2023-09-20T21:43:13.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.993+0000 7fa4a4fd0700 1 --2- 172.21.15.123:0/534212593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4a01009a0 0x7fa4a0108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:13.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.993+0000 7fa4a4fd0700 1 --2- 172.21.15.123:0/534212593 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a0101380 0x7fa4a01017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:13.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.993+0000 7fa4a4fd0700 1 --2- 172.21.15.123:0/534212593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a00fffe0 0x7fa4a0100460 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:13.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.993+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/534212593 >> 172.21.15.123:0/534212593 conn(0x7fa4a00faa90 msgr2=0x7fa4a00fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:43:13.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.993+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/534212593 shutdown_connections 2023-09-20T21:43:13.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.993+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/534212593 wait complete. 2023-09-20T21:43:13.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.993+0000 7fa4a4fd0700 1 Processor -- start 2023-09-20T21:43:13.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.993+0000 7fa4a4fd0700 1 -- start start 2023-09-20T21:43:13.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa4a4fd0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a00fffe0 0x7fa4a0195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:43:13.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa4a4fd0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4a01009a0 0x7fa4a0195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:43:13.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa4a4fd0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a0101380 0x7fa4a019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:43:13.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa4a4fd0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a019d620 con 0x7fa4a01009a0 2023-09-20T21:43:13.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa4a4fd0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a019d760 con 0x7fa4a00fffe0 2023-09-20T21:43:13.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa4a4fd0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a019d8f0 con 0x7fa4a0101380 2023-09-20T21:43:13.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a0101380 0x7fa4a019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:43:13.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a0101380 0x7fa4a019d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33932/0 (socket says 172.21.15.123:33932) 2023-09-20T21:43:13.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49ed9d700 1 -- 172.21.15.123:0/939368286 learned_addr learned my addr 172.21.15.123:0/939368286 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-20T21:43:13.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49dd9b700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4a01009a0 0x7fa4a0195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:43:13.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49e59c700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a00fffe0 0x7fa4a0195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:43:13.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49ed9d700 1 -- 172.21.15.123:0/939368286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a00fffe0 msgr2=0x7fa4a0195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:13.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49ed9d700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a00fffe0 0x7fa4a0195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:14.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49ed9d700 1 -- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4a01009a0 msgr2=0x7fa4a0195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:14.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49ed9d700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4a01009a0 0x7fa4a0195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:14.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49ed9d700 1 -- 172.21.15.123:0/939368286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4940095f0 con 0x7fa4a0101380 2023-09-20T21:43:14.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49dd9b700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4a01009a0 0x7fa4a0195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:43:14.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49e59c700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a00fffe0 0x7fa4a0195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:43:14.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa49ed9d700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a0101380 0x7fa4a019d050 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa49800b9a0 tx=0x7fa49800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:43:14.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa48f7fe700 1 -- 172.21.15.123:0/939368286 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49800c9b0 con 0x7fa4a0101380 2023-09-20T21:43:14.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa48f7fe700 1 -- 172.21.15.123:0/939368286 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa498014440 con 0x7fa4a0101380 2023-09-20T21:43:14.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/939368286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a019dbd0 con 0x7fa4a0101380 2023-09-20T21:43:14.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa48f7fe700 1 -- 172.21.15.123:0/939368286 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4980075d0 con 0x7fa4a0101380 2023-09-20T21:43:14.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.994+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/939368286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a019e0d0 con 0x7fa4a0101380 2023-09-20T21:43:14.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.995+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/939368286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a010b5e0 con 0x7fa4a0101380 2023-09-20T21:43:14.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.996+0000 7fa48f7fe700 1 -- 172.21.15.123:0/939368286 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa4980145b0 con 0x7fa4a0101380 2023-09-20T21:43:14.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.997+0000 7fa48f7fe700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa4880a1380 0x7fa4880a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:43:14.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.997+0000 7fa48f7fe700 1 -- 172.21.15.123:0/939368286 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fa498006e40 con 0x7fa4a0101380 2023-09-20T21:43:14.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.997+0000 7fa49e59c700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa4880a1380 0x7fa4880a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:43:14.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:13.998+0000 7fa49e59c700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa4880a1380 0x7fa4880a3840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa490005fd0 tx=0x7fa490005dc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:43:14.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:14.001+0000 7fa48f7fe700 1 -- 172.21.15.123:0/939368286 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa498095ed0 con 0x7fa4a0101380 2023-09-20T21:43:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:13 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:43:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:13 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:43:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:13 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi089 2023-09-20T21:43:14.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:13 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi089 to 8584M 2023-09-20T21:43:14.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:13 smithi123 ceph-mon[134433]: osd.4 [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] boot 2023-09-20T21:43:14.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:13 smithi123 ceph-mon[134433]: osdmap e33: 5 total, 5 up, 5 in 2023-09-20T21:43:14.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:13 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:14.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:13 smithi123 ceph-mon[134433]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:14.199+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/939368286 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4a0061220 con 0x7fa4880a1380 2023-09-20T21:43:14.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:13 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:43:14.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:13 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:43:14.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:13 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi089 2023-09-20T21:43:14.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:13 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi089 to 8584M 2023-09-20T21:43:14.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:13 smithi022 ceph-mon[97651]: osd.4 [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] boot 2023-09-20T21:43:14.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:13 smithi022 ceph-mon[97651]: osdmap e33: 5 total, 5 up, 5 in 2023-09-20T21:43:14.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:13 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:14.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:13 smithi022 ceph-mon[97651]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:14.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:13 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:43:14.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:13 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:43:14.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:13 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi089 2023-09-20T21:43:14.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:13 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi089 to 8584M 2023-09-20T21:43:14.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:13 smithi089 ceph-mon[134090]: osd.4 [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] boot 2023-09-20T21:43:14.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:13 smithi089 ceph-mon[134090]: osdmap e33: 5 total, 5 up, 5 in 2023-09-20T21:43:14.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:13 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-09-20T21:43:14.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:13 smithi089 ceph-mon[134090]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-09-20T21:43:15.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:14 smithi022 ceph-mon[97651]: osdmap e34: 5 total, 5 up, 5 in 2023-09-20T21:43:15.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:14 smithi022 ceph-mon[97651]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:43:15.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:43:15.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:43:15.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:15.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:14 smithi123 ceph-mon[134433]: osdmap e34: 5 total, 5 up, 5 in 2023-09-20T21:43:15.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:14 smithi123 ceph-mon[134433]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:43:15.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:43:15.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:43:15.273 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:15.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:14 smithi089 ceph-mon[134090]: osdmap e34: 5 total, 5 up, 5 in 2023-09-20T21:43:15.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:14 smithi089 ceph-mon[134090]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:43:15.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:43:15.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:43:15.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:16.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:15 smithi022 ceph-mon[97651]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:16.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:15 smithi123 ceph-mon[134433]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:16.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:15 smithi089 ceph-mon[134090]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:17.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:16 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/3055592850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d8866df-2568-4a6d-83e7-1be50e327005"}]: dispatch 2023-09-20T21:43:17.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:16 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d8866df-2568-4a6d-83e7-1be50e327005"}]: dispatch 2023-09-20T21:43:17.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:16 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d8866df-2568-4a6d-83e7-1be50e327005"}]': finished 2023-09-20T21:43:17.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:16 smithi022 ceph-mon[97651]: osdmap e35: 6 total, 5 up, 6 in 2023-09-20T21:43:17.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:17.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:16 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/3055592850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d8866df-2568-4a6d-83e7-1be50e327005"}]: dispatch 2023-09-20T21:43:17.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:16 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d8866df-2568-4a6d-83e7-1be50e327005"}]: dispatch 2023-09-20T21:43:17.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:16 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d8866df-2568-4a6d-83e7-1be50e327005"}]': finished 2023-09-20T21:43:17.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:16 smithi123 ceph-mon[134433]: osdmap e35: 6 total, 5 up, 6 in 2023-09-20T21:43:17.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:17.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:16 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/3055592850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d8866df-2568-4a6d-83e7-1be50e327005"}]: dispatch 2023-09-20T21:43:17.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:16 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d8866df-2568-4a6d-83e7-1be50e327005"}]: dispatch 2023-09-20T21:43:17.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:16 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d8866df-2568-4a6d-83e7-1be50e327005"}]': finished 2023-09-20T21:43:17.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:16 smithi089 ceph-mon[134090]: osdmap e35: 6 total, 5 up, 6 in 2023-09-20T21:43:17.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:18.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:17 smithi022 ceph-mon[97651]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:18.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:17 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/397746334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:43:18.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:17 smithi123 ceph-mon[134433]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:18.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:17 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/397746334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:43:18.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:17 smithi089 ceph-mon[134090]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:18.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:17 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/397746334' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:43:20.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:20 smithi089 ceph-mon[134090]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:20.429 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:20 smithi123 ceph-mon[134433]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:20.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:20 smithi022 ceph-mon[97651]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:22.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:22 smithi123 ceph-mon[134433]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:22.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:22 smithi022 ceph-mon[97651]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:22.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:22 smithi089 ceph-mon[134090]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:23.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:23 smithi022 ceph-mon[97651]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-20T21:43:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:23 smithi022 ceph-mon[97651]: Deploying daemon osd.5 on smithi123 2023-09-20T21:43:23.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:23 smithi123 ceph-mon[134433]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:23.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:23 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-20T21:43:23.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:23 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:23.754 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:23 smithi123 ceph-mon[134433]: Deploying daemon osd.5 on smithi123 2023-09-20T21:43:23.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:23 smithi089 ceph-mon[134090]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:23.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:23 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-09-20T21:43:23.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:23 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:23.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:23 smithi089 ceph-mon[134090]: Deploying daemon osd.5 on smithi123 2023-09-20T21:43:26.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:26 smithi123 ceph-mon[134433]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:26.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:26 smithi089 ceph-mon[134090]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:26.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:26 smithi022 ceph-mon[97651]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:28.428 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:28 smithi123 ceph-mon[134433]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:28.428 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:28.428 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:28.429 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:28.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:28 smithi022 ceph-mon[97651]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:28.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:28.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:28.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:28.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:28 smithi089 ceph-mon[134090]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:28.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:28.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:28.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:30.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.211+0000 7fa48f7fe700 1 -- 172.21.15.123:0/939368286 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa4a0061220 con 0x7fa4880a1380 2023-09-20T21:43:30.213 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 5 on host 'smithi123' 2023-09-20T21:43:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.213+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa4880a1380 msgr2=0x7fa4880a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.213+0000 7fa4a4fd0700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa4880a1380 0x7fa4880a3840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa490005fd0 tx=0x7fa490005dc0 comp rx=0 tx=0).stop 2023-09-20T21:43:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.213+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/939368286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a0101380 msgr2=0x7fa4a019d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.213+0000 7fa4a4fd0700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a0101380 0x7fa4a019d050 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa49800b9a0 tx=0x7fa49800bd60 comp rx=0 tx=0).stop 2023-09-20T21:43:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.213+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/939368286 shutdown_connections 2023-09-20T21:43:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.213+0000 7fa4a4fd0700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a0101380 0x7fa4a019d050 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.213+0000 7fa4a4fd0700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa4880a1380 0x7fa4880a3840 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.213+0000 7fa4a4fd0700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4a00fffe0 0x7fa4a0195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.213+0000 7fa4a4fd0700 1 --2- 172.21.15.123:0/939368286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4a01009a0 0x7fa4a0195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.213+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/939368286 >> 172.21.15.123:0/939368286 conn(0x7fa4a00faa90 msgr2=0x7fa4a00fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:43:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.214+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/939368286 shutdown_connections 2023-09-20T21:43:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:43:30.214+0000 7fa4a4fd0700 1 -- 172.21.15.123:0/939368286 wait complete. 2023-09-20T21:43:30.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:30 smithi123 ceph-mon[134433]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:30.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:30.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:30.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:30.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:30.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:30 smithi022 ceph-mon[97651]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:30.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:30.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:30.591 DEBUG:teuthology.orchestra.run.smithi123:osd.5> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.5.service 2023-09-20T21:43:30.593 INFO:tasks.cephadm:Deploying osd.6 on smithi022 with /dev/vg_nvme/lv_2... 2023-09-20T21:43:30.593 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-20T21:43:30.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:30 smithi089 ceph-mon[134090]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:30.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:30.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:30.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:30.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:30.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:30.622 INFO:journalctl@ceph.osd.5.smithi123.stdout:-- Logs begin at Wed 2023-09-20 21:17:48 UTC. -- 2023-09-20T21:43:31.353 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:43:31.444 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:31.444 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:31.444 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:31.444 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 21:43:31 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T21:43:31.162+0000 7f0d53158640 -1 osd.5 0 log_to_monitors true 2023-09-20T21:43:31.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:31.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:31.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:31.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:31.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:31.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:32.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:32 smithi022 ceph-mon[97651]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:32.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:32 smithi022 ceph-mon[97651]: from='osd.5 [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-20T21:43:32.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:32 smithi022 ceph-mon[97651]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-20T21:43:32.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:32 smithi123 ceph-mon[134433]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:32.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:32 smithi123 ceph-mon[134433]: from='osd.5 [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-20T21:43:32.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:32 smithi123 ceph-mon[134433]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-20T21:43:32.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:32 smithi089 ceph-mon[134090]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:32.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:32 smithi089 ceph-mon[134090]: from='osd.5 [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-20T21:43:32.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:32 smithi089 ceph-mon[134090]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-09-20T21:43:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-20T21:43:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-20T21:43:33.484 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2023-09-20T21:43:33.485 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2023-09-20T21:43:33.485 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0403217 s, 260 MB/s 2023-09-20T21:43:33.485 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2023-09-20T21:43:33.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:33 smithi022 ceph-mon[97651]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-20T21:43:33.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:33 smithi022 ceph-mon[97651]: osdmap e36: 6 total, 5 up, 6 in 2023-09-20T21:43:33.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:33.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:33 smithi022 ceph-mon[97651]: from='osd.5 [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:43:33.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:33 smithi022 ceph-mon[97651]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:43:33.593 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 21:43:33 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T21:43:33.180+0000 7f0d487d0700 -1 osd.5 0 waiting for initial osdmap 2023-09-20T21:43:33.593 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 21:43:33 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T21:43:33.187+0000 7f0d42dc2700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:43:33.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:33 smithi123 ceph-mon[134433]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-20T21:43:33.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:33 smithi123 ceph-mon[134433]: osdmap e36: 6 total, 5 up, 6 in 2023-09-20T21:43:33.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:33.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:33 smithi123 ceph-mon[134433]: from='osd.5 [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:43:33.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:33 smithi123 ceph-mon[134433]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:43:33.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:33 smithi089 ceph-mon[134090]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-09-20T21:43:33.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:33 smithi089 ceph-mon[134090]: osdmap e36: 6 total, 5 up, 6 in 2023-09-20T21:43:33.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:33.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:33 smithi089 ceph-mon[134090]: from='osd.5 [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:43:33.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:33 smithi089 ceph-mon[134090]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:43:34.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi022:vg_nvme/lv_2 2023-09-20T21:43:34.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:43:34.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:43:34.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:34.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:43:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: osdmap e37: 6 total, 5 up, 6 in 2023-09-20T21:43:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi123 2023-09-20T21:43:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi123 to 8584M 2023-09-20T21:43:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:34.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:34.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:43:34.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:43:34.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:34.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:43:34.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: osdmap e37: 6 total, 5 up, 6 in 2023-09-20T21:43:34.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:34.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:34.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi123 2023-09-20T21:43:34.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:34.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:34.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi123 to 8584M 2023-09-20T21:43:34.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:34.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:34.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:43:34.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:43:34.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:34.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:43:34.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: osdmap e37: 6 total, 5 up, 6 in 2023-09-20T21:43:34.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:34.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:34.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi123 2023-09-20T21:43:34.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:34.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:43:34.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi123 to 8584M 2023-09-20T21:43:34.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:34.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:34.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:34.771 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:43:35.308 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:35.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:35 smithi022 ceph-mon[97651]: osd.5 [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] boot 2023-09-20T21:43:35.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:35 smithi022 ceph-mon[97651]: osdmap e38: 6 total, 6 up, 6 in 2023-09-20T21:43:35.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:35.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:35.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:35 smithi123 ceph-mon[134433]: osd.5 [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] boot 2023-09-20T21:43:35.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:35 smithi123 ceph-mon[134433]: osdmap e38: 6 total, 6 up, 6 in 2023-09-20T21:43:35.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:35.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:35.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:35 smithi089 ceph-mon[134090]: osd.5 [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] boot 2023-09-20T21:43:35.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:35 smithi089 ceph-mon[134090]: osdmap e38: 6 total, 6 up, 6 in 2023-09-20T21:43:35.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-09-20T21:43:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.149+0000 7f720078e700 1 -- 172.21.15.22:0/4189112853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8101fd0 msgr2=0x7f71f8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.149+0000 7f720078e700 1 --2- 172.21.15.22:0/4189112853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8101fd0 0x7f71f8102450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f71e80098f0 tx=0x7f71e8009c00 comp rx=0 tx=0).stop 2023-09-20T21:43:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.149+0000 7f720078e700 1 -- 172.21.15.22:0/4189112853 shutdown_connections 2023-09-20T21:43:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.149+0000 7f720078e700 1 --2- 172.21.15.22:0/4189112853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f8102990 0x7f71f81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.149+0000 7f720078e700 1 --2- 172.21.15.22:0/4189112853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8100e70 0x7f71f8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.149+0000 7f720078e700 1 --2- 172.21.15.22:0/4189112853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8101fd0 0x7f71f8102450 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.149+0000 7f720078e700 1 -- 172.21.15.22:0/4189112853 >> 172.21.15.22:0/4189112853 conn(0x7f71f80fc3d0 msgr2=0x7f71f80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:43:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.149+0000 7f720078e700 1 -- 172.21.15.22:0/4189112853 shutdown_connections 2023-09-20T21:43:36.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.149+0000 7f720078e700 1 -- 172.21.15.22:0/4189112853 wait complete. 2023-09-20T21:43:36.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.150+0000 7f720078e700 1 Processor -- start 2023-09-20T21:43:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.150+0000 7f720078e700 1 -- start start 2023-09-20T21:43:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f720078e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8100e70 0x7f71f8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:43:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f720078e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f8101fd0 0x7f71f8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:43:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f720078e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8102990 0x7f71f81a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:43:36.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f720078e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81a09e0 con 0x7f71f8100e70 2023-09-20T21:43:36.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f720078e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81a0b20 con 0x7f71f8101fd0 2023-09-20T21:43:36.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f720078e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81a0cb0 con 0x7f71f8102990 2023-09-20T21:43:36.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f71fe52a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8100e70 0x7f71f8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:43:36.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f71fe52a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8100e70 0x7f71f8198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53616/0 (socket says 172.21.15.22:53616) 2023-09-20T21:43:36.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f71fe52a700 1 -- 172.21.15.22:0/2056381576 learned_addr learned my addr 172.21.15.22:0/2056381576 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:43:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f71fdd29700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f8101fd0 0x7f71f8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:43:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f71fed2b700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8102990 0x7f71f81a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:43:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f71fe52a700 1 -- 172.21.15.22:0/2056381576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8102990 msgr2=0x7f71f81a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f71fe52a700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8102990 0x7f71f81a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f71fe52a700 1 -- 172.21.15.22:0/2056381576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f8101fd0 msgr2=0x7f71f8199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:36.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f71fe52a700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f8101fd0 0x7f71f8199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.151+0000 7f71fe52a700 1 -- 172.21.15.22:0/2056381576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e8009520 con 0x7f71f8100e70 2023-09-20T21:43:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.152+0000 7f71fe52a700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8100e70 0x7f71f8198e50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f71f400e9d0 tx=0x7f71f400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:43:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.152+0000 7f71ef7fe700 1 -- 172.21.15.22:0/2056381576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4009790 con 0x7f71f8100e70 2023-09-20T21:43:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.152+0000 7f71ef7fe700 1 -- 172.21.15.22:0/2056381576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71f400cbd0 con 0x7f71f8100e70 2023-09-20T21:43:36.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.152+0000 7f71ef7fe700 1 -- 172.21.15.22:0/2056381576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4018680 con 0x7f71f8100e70 2023-09-20T21:43:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.152+0000 7f720078e700 1 -- 172.21.15.22:0/2056381576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f81a0f90 con 0x7f71f8100e70 2023-09-20T21:43:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.152+0000 7f720078e700 1 -- 172.21.15.22:0/2056381576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f81a12f0 con 0x7f71f8100e70 2023-09-20T21:43:36.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.153+0000 7f720078e700 1 -- 172.21.15.22:0/2056381576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f81934c0 con 0x7f71f8100e70 2023-09-20T21:43:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.155+0000 7f71ef7fe700 1 -- 172.21.15.22:0/2056381576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f71f400cd40 con 0x7f71f8100e70 2023-09-20T21:43:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.157+0000 7f71ef7fe700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71e40a1300 0x7f71e40a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:43:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.157+0000 7f71ef7fe700 1 -- 172.21.15.22:0/2056381576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f71f4014070 con 0x7f71f8100e70 2023-09-20T21:43:36.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.157+0000 7f71fdd29700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71e40a1300 0x7f71e40a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:43:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.157+0000 7f71fdd29700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71e40a1300 0x7f71e40a37c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71e8005e90 tx=0x7f71e8019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:43:36.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.159+0000 7f71ef7fe700 1 -- 172.21.15.22:0/2056381576 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f71f4096880 con 0x7f71f8100e70 2023-09-20T21:43:36.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:36.386+0000 7f720078e700 1 -- 172.21.15.22:0/2056381576 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f71f802d310 con 0x7f71e40a1300 2023-09-20T21:43:36.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:36 smithi022 ceph-mon[97651]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:36.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:36 smithi022 ceph-mon[97651]: osdmap e39: 6 total, 6 up, 6 in 2023-09-20T21:43:36.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:36 smithi123 ceph-mon[134433]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:36.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:36 smithi123 ceph-mon[134433]: osdmap e39: 6 total, 6 up, 6 in 2023-09-20T21:43:36.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:36 smithi089 ceph-mon[134090]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-09-20T21:43:36.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:36 smithi089 ceph-mon[134090]: osdmap e39: 6 total, 6 up, 6 in 2023-09-20T21:43:37.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:37 smithi022 ceph-mon[97651]: osdmap e40: 6 total, 6 up, 6 in 2023-09-20T21:43:37.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:43:37.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:43:37.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:37.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:37 smithi123 ceph-mon[134433]: osdmap e40: 6 total, 6 up, 6 in 2023-09-20T21:43:37.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:43:37.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:43:37.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:37.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:37 smithi089 ceph-mon[134090]: osdmap e40: 6 total, 6 up, 6 in 2023-09-20T21:43:37.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:43:37.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:43:37.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:38.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:38 smithi022 ceph-mon[97651]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:43:38.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:38 smithi022 ceph-mon[97651]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:38.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:38 smithi022 ceph-mon[97651]: osdmap e41: 6 total, 6 up, 6 in 2023-09-20T21:43:38.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:38 smithi123 ceph-mon[134433]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:43:38.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:38 smithi123 ceph-mon[134433]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:38.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:38 smithi123 ceph-mon[134433]: osdmap e41: 6 total, 6 up, 6 in 2023-09-20T21:43:38.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:38 smithi089 ceph-mon[134090]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:43:38.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:38 smithi089 ceph-mon[134090]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:38.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:38 smithi089 ceph-mon[134090]: osdmap e41: 6 total, 6 up, 6 in 2023-09-20T21:43:39.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:39 smithi022 ceph-mon[97651]: osdmap e42: 6 total, 6 up, 6 in 2023-09-20T21:43:39.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:39 smithi123 ceph-mon[134433]: osdmap e42: 6 total, 6 up, 6 in 2023-09-20T21:43:39.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:39 smithi089 ceph-mon[134090]: osdmap e42: 6 total, 6 up, 6 in 2023-09-20T21:43:40.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:40 smithi022 ceph-mon[97651]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:40.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:40 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1325813163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a625e71-db36-4eab-a821-68672c3284e6"}]: dispatch 2023-09-20T21:43:40.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:40 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1325813163' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a625e71-db36-4eab-a821-68672c3284e6"}]': finished 2023-09-20T21:43:40.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:40 smithi022 ceph-mon[97651]: osdmap e43: 7 total, 6 up, 7 in 2023-09-20T21:43:40.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:40 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:43:40.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:40 smithi123 ceph-mon[134433]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:40.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:40 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1325813163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a625e71-db36-4eab-a821-68672c3284e6"}]: dispatch 2023-09-20T21:43:40.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:40 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1325813163' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a625e71-db36-4eab-a821-68672c3284e6"}]': finished 2023-09-20T21:43:40.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:40 smithi123 ceph-mon[134433]: osdmap e43: 7 total, 6 up, 7 in 2023-09-20T21:43:40.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:40 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:43:40.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:40 smithi089 ceph-mon[134090]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:40.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:40 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1325813163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a625e71-db36-4eab-a821-68672c3284e6"}]: dispatch 2023-09-20T21:43:40.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:40 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1325813163' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a625e71-db36-4eab-a821-68672c3284e6"}]': finished 2023-09-20T21:43:40.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:40 smithi089 ceph-mon[134090]: osdmap e43: 7 total, 6 up, 7 in 2023-09-20T21:43:40.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:40 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:43:41.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:41 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4045951816' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:43:41.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:41 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4045951816' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:43:41.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:41 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4045951816' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:43:42.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:42 smithi022 ceph-mon[97651]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2023-09-20T21:43:42.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:42 smithi123 ceph-mon[134433]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2023-09-20T21:43:42.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:42 smithi089 ceph-mon[134090]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2023-09-20T21:43:43.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:43 smithi123 ceph-mon[134433]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-20T21:43:43.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:43 smithi089 ceph-mon[134090]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-20T21:43:43.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:43 smithi022 ceph-mon[97651]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-20T21:43:46.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:46 smithi022 ceph-mon[97651]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-09-20T21:43:46.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:46 smithi123 ceph-mon[134433]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-09-20T21:43:46.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:46 smithi089 ceph-mon[134090]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-09-20T21:43:48.481 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:48 smithi022 ceph-mon[97651]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-09-20T21:43:48.481 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:48 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-20T21:43:48.482 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:48 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:48.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:48 smithi123 ceph-mon[134433]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-09-20T21:43:48.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-20T21:43:48.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:48.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:48 smithi089 ceph-mon[134090]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-09-20T21:43:48.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:48 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-09-20T21:43:48.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:48 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:49.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:49 smithi022 ceph-mon[97651]: Deploying daemon osd.6 on smithi022 2023-09-20T21:43:49.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:49 smithi123 ceph-mon[134433]: Deploying daemon osd.6 on smithi022 2023-09-20T21:43:49.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:49 smithi089 ceph-mon[134090]: Deploying daemon osd.6 on smithi022 2023-09-20T21:43:50.455 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:50 smithi022 ceph-mon[97651]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-20T21:43:50.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:50 smithi123 ceph-mon[134433]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-20T21:43:50.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:50 smithi089 ceph-mon[134090]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-20T21:43:52.425 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:52 smithi022 ceph-mon[97651]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:52 smithi123 ceph-mon[134433]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:52.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:52 smithi089 ceph-mon[134090]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:53.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:53 smithi022 ceph-mon[97651]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:53.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:53.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:53.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:53.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:53 smithi123 ceph-mon[134433]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:53.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:53.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:53.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:53.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:53 smithi089 ceph-mon[134090]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:53.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:53 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:53.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:53 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:53.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:53 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:56.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:56 smithi022 ceph-mon[97651]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:56.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:56 smithi123 ceph-mon[134433]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:56.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:56 smithi089 ceph-mon[134090]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:57.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.297+0000 7f71ef7fe700 1 -- 172.21.15.22:0/2056381576 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f71f802d310 con 0x7f71e40a1300 2023-09-20T21:43:57.299 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 6 on host 'smithi022' 2023-09-20T21:43:57.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 -- 172.21.15.22:0/2056381576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71e40a1300 msgr2=0x7f71e40a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:57.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71e40a1300 0x7f71e40a37c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71e8005e90 tx=0x7f71e8019040 comp rx=0 tx=0).stop 2023-09-20T21:43:57.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 -- 172.21.15.22:0/2056381576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8100e70 msgr2=0x7f71f8198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:43:57.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8100e70 0x7f71f8198e50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f71f400e9d0 tx=0x7f71f400ece0 comp rx=0 tx=0).stop 2023-09-20T21:43:57.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 -- 172.21.15.22:0/2056381576 shutdown_connections 2023-09-20T21:43:57.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f8102990 0x7f71f81a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:57.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71f8100e70 0x7f71f8198e50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:57.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71e40a1300 0x7f71e40a37c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:57.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 --2- 172.21.15.22:0/2056381576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f71f8101fd0 0x7f71f8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:43:57.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 -- 172.21.15.22:0/2056381576 >> 172.21.15.22:0/2056381576 conn(0x7f71f80fc3d0 msgr2=0x7f71f8107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:43:57.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.300+0000 7f720078e700 1 -- 172.21.15.22:0/2056381576 shutdown_connections 2023-09-20T21:43:57.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:43:57.301+0000 7f720078e700 1 -- 172.21.15.22:0/2056381576 wait complete. 2023-09-20T21:43:57.796 DEBUG:teuthology.orchestra.run.smithi022:osd.6> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.6.service 2023-09-20T21:43:57.801 INFO:tasks.cephadm:Deploying osd.7 on smithi089 with /dev/vg_nvme/lv_2... 2023-09-20T21:43:57.801 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-20T21:43:57.833 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.833 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.834 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:57.834 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:57.834 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.834 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:57.834 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.835 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.835 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:57.835 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: from='osd.6 [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-20T21:43:57.835 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:57 smithi089 ceph-mon[134090]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-20T21:43:57.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:57.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:57.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:57.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:57.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: from='osd.6 [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-20T21:43:57.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:57 smithi123 ceph-mon[134433]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-20T21:43:57.860 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.861 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.861 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:43:57.861 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:43:57.861 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.861 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:57.862 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.862 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:43:57.862 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:43:57.862 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: from='osd.6 [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-20T21:43:57.862 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:57 smithi022 ceph-mon[97651]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-09-20T21:43:57.875 INFO:journalctl@ceph.osd.6.smithi022.stdout:-- Logs begin at Wed 2023-09-20 21:22:06 UTC. -- 2023-09-20T21:43:58.363 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T21:43:59.415 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:59 smithi022 ceph-mon[97651]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-20T21:43:59.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:59 smithi022 ceph-mon[97651]: osdmap e44: 7 total, 6 up, 7 in 2023-09-20T21:43:59.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:59 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:43:59.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:59 smithi022 ceph-mon[97651]: from='osd.6 [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:43:59.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:59 smithi022 ceph-mon[97651]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:43:59.416 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:43:59 smithi022 ceph-mon[97651]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:59.417 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 21:43:59 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T21:43:59.285+0000 7f281ccc6700 -1 osd.6 0 waiting for initial osdmap 2023-09-20T21:43:59.417 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 21:43:59 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T21:43:59.296+0000 7f2815ab5700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:43:59.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:59 smithi123 ceph-mon[134433]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-20T21:43:59.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:59 smithi123 ceph-mon[134433]: osdmap e44: 7 total, 6 up, 7 in 2023-09-20T21:43:59.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:59 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:43:59.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:59 smithi123 ceph-mon[134433]: from='osd.6 [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:43:59.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:59 smithi123 ceph-mon[134433]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:43:59.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:43:59 smithi123 ceph-mon[134433]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:59.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:59 smithi089 ceph-mon[134090]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-09-20T21:43:59.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:59 smithi089 ceph-mon[134090]: osdmap e44: 7 total, 6 up, 7 in 2023-09-20T21:43:59.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:59 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:43:59.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:59 smithi089 ceph-mon[134090]: from='osd.6 [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:43:59.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:59 smithi089 ceph-mon[134090]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:43:59.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:43:59 smithi089 ceph-mon[134090]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-09-20T21:43:59.908 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-20T21:43:59.908 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-20T21:43:59.909 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2023-09-20T21:43:59.909 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2023-09-20T21:43:59.910 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0188856 s, 555 MB/s 2023-09-20T21:43:59.910 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2023-09-20T21:44:00.316 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi089:vg_nvme/lv_2 2023-09-20T21:44:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:00 smithi123 ceph-mon[134433]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:44:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:00 smithi123 ceph-mon[134433]: osdmap e45: 7 total, 6 up, 7 in 2023-09-20T21:44:00.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:00 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:44:00.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:00 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:44:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:00 smithi089 ceph-mon[134090]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:44:00.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:00 smithi089 ceph-mon[134090]: osdmap e45: 7 total, 6 up, 7 in 2023-09-20T21:44:00.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:00 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:44:00.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:00 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:44:00.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:00 smithi022 ceph-mon[97651]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:44:00.638 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:00 smithi022 ceph-mon[97651]: osdmap e45: 7 total, 6 up, 7 in 2023-09-20T21:44:00.639 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:00 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:44:00.639 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:00 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:44:00.912 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T21:44:01.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:44:01.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:44:01.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: osd.6 [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] boot 2023-09-20T21:44:01.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: osdmap e46: 7 total, 7 up, 7 in 2023-09-20T21:44:01.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:44:01.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:01.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi022 2023-09-20T21:44:01.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:01.364 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:01.364 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:01.364 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi022 to 5757M 2023-09-20T21:44:01.365 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.365 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:01.365 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:01.365 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:01 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:44:01.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:44:01.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: osd.6 [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] boot 2023-09-20T21:44:01.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: osdmap e46: 7 total, 7 up, 7 in 2023-09-20T21:44:01.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:44:01.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:01.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi022 2023-09-20T21:44:01.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:01.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:01.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:01.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi022 to 5757M 2023-09-20T21:44:01.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:01.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:01.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:01 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:44:01.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:44:01.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: osd.6 [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] boot 2023-09-20T21:44:01.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: osdmap e46: 7 total, 7 up, 7 in 2023-09-20T21:44:01.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-09-20T21:44:01.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:01.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi022 2023-09-20T21:44:01.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:01.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:01.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:01.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi022 to 5757M 2023-09-20T21:44:01.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:01.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:01.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:01.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:01 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:02.142 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 -- 172.21.15.89:0/1547842482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f072c0fffe0 msgr2=0x7f072c100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:02.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 --2- 172.21.15.89:0/1547842482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f072c0fffe0 0x7f072c100460 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0714009990 tx=0x7f0714009ca0 comp rx=0 tx=0).stop 2023-09-20T21:44:02.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 -- 172.21.15.89:0/1547842482 shutdown_connections 2023-09-20T21:44:02.143 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 --2- 172.21.15.89:0/1547842482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f072c1009a0 0x7f072c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:02.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 --2- 172.21.15.89:0/1547842482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c101380 0x7f072c1017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:02.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 --2- 172.21.15.89:0/1547842482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f072c0fffe0 0x7f072c100460 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:02.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 -- 172.21.15.89:0/1547842482 >> 172.21.15.89:0/1547842482 conn(0x7f072c0faa90 msgr2=0x7f072c0fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:44:02.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 -- 172.21.15.89:0/1547842482 shutdown_connections 2023-09-20T21:44:02.144 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 -- 172.21.15.89:0/1547842482 wait complete. 2023-09-20T21:44:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 Processor -- start 2023-09-20T21:44:02.149 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.142+0000 7f07323ea700 1 -- start start 2023-09-20T21:44:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07323ea700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f072c0fffe0 0x7f072c103d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07323ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c1009a0 0x7f072c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07323ea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f072c101380 0x7f072c102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:02.150 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07323ea700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c103090 con 0x7f072c0fffe0 2023-09-20T21:44:02.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07323ea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c103200 con 0x7f072c101380 2023-09-20T21:44:02.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07323ea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c1a0ba0 con 0x7f072c1009a0 2023-09-20T21:44:02.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f0730987700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f072c101380 0x7f072c102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:02.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f0730987700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f072c101380 0x7f072c102910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55206/0 (socket says 172.21.15.89:55206) 2023-09-20T21:44:02.151 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f0730987700 1 -- 172.21.15.89:0/3490936417 learned_addr learned my addr 172.21.15.89:0/3490936417 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-20T21:44:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f072b7fe700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c1009a0 0x7f072c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f0730987700 1 -- 172.21.15.89:0/3490936417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c1009a0 msgr2=0x7f072c1023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f0730987700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c1009a0 0x7f072c1023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f072bfff700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f072c0fffe0 0x7f072c103d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:02.152 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f0730987700 1 -- 172.21.15.89:0/3490936417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f072c0fffe0 msgr2=0x7f072c103d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f0730987700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f072c0fffe0 0x7f072c103d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f0730987700 1 -- 172.21.15.89:0/3490936417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07140095f0 con 0x7f072c101380 2023-09-20T21:44:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f0730987700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f072c101380 0x7f072c102910 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f072400ada0 tx=0x7f072400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:44:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07297fa700 1 -- 172.21.15.89:0/3490936417 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072400f070 con 0x7f072c101380 2023-09-20T21:44:02.153 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07297fa700 1 -- 172.21.15.89:0/3490936417 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0724007b30 con 0x7f072c101380 2023-09-20T21:44:02.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07297fa700 1 -- 172.21.15.89:0/3490936417 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240053e0 con 0x7f072c101380 2023-09-20T21:44:02.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07323ea700 1 -- 172.21.15.89:0/3490936417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c1a0e80 con 0x7f072c101380 2023-09-20T21:44:02.154 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.143+0000 7f07323ea700 1 -- 172.21.15.89:0/3490936417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c1a12a0 con 0x7f072c101380 2023-09-20T21:44:02.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.144+0000 7f07323ea700 1 -- 172.21.15.89:0/3490936417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c10b570 con 0x7f072c101380 2023-09-20T21:44:02.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.145+0000 7f07297fa700 1 -- 172.21.15.89:0/3490936417 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f072400c980 con 0x7f072c101380 2023-09-20T21:44:02.156 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.146+0000 7f07297fa700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07180a1380 0x7f07180a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:02.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.146+0000 7f07297fa700 1 -- 172.21.15.89:0/3490936417 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f07240cb460 con 0x7f072c101380 2023-09-20T21:44:02.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.147+0000 7f072bfff700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07180a1380 0x7f07180a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:02.157 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.147+0000 7f072bfff700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07180a1380 0x7f07180a3840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f072c068680 tx=0x7f071c005cf0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:44:02.158 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.150+0000 7f07297fa700 1 -- 172.21.15.89:0/3490936417 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f07240958d0 con 0x7f072c101380 2023-09-20T21:44:02.346 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:02.345+0000 7f07323ea700 1 -- 172.21.15.89:0/3490936417 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f072c0ff680 con 0x7f07180a1380 2023-09-20T21:44:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:02 smithi123 ceph-mon[134433]: osdmap e47: 7 total, 7 up, 7 in 2023-09-20T21:44:02.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:02 smithi089 ceph-mon[134090]: osdmap e47: 7 total, 7 up, 7 in 2023-09-20T21:44:02.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:02 smithi022 ceph-mon[97651]: osdmap e47: 7 total, 7 up, 7 in 2023-09-20T21:44:03.446 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:03 smithi089 ceph-mon[134090]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:44:03.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:44:03.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:44:03.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:03.447 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:03 smithi089 ceph-mon[134090]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:03.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:03 smithi123 ceph-mon[134433]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:44:03.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:44:03.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:44:03.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:03.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:03 smithi123 ceph-mon[134433]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:03.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:03 smithi022 ceph-mon[97651]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:44:03.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:44:03.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:44:03.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:03.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:03 smithi022 ceph-mon[97651]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:04.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:04 smithi022 ceph-mon[97651]: from='client.? 172.21.15.89:0/2922120135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7674a06e-2830-441e-9178-687e42dfce80"}]: dispatch 2023-09-20T21:44:04.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:04 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7674a06e-2830-441e-9178-687e42dfce80"}]: dispatch 2023-09-20T21:44:04.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:04 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7674a06e-2830-441e-9178-687e42dfce80"}]': finished 2023-09-20T21:44:04.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:04 smithi022 ceph-mon[97651]: osdmap e48: 8 total, 7 up, 8 in 2023-09-20T21:44:04.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:04 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:05.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:04 smithi123 ceph-mon[134433]: from='client.? 172.21.15.89:0/2922120135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7674a06e-2830-441e-9178-687e42dfce80"}]: dispatch 2023-09-20T21:44:05.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:04 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7674a06e-2830-441e-9178-687e42dfce80"}]: dispatch 2023-09-20T21:44:05.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:04 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7674a06e-2830-441e-9178-687e42dfce80"}]': finished 2023-09-20T21:44:05.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:04 smithi123 ceph-mon[134433]: osdmap e48: 8 total, 7 up, 8 in 2023-09-20T21:44:05.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:04 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:05.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:04 smithi089 ceph-mon[134090]: from='client.? 172.21.15.89:0/2922120135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7674a06e-2830-441e-9178-687e42dfce80"}]: dispatch 2023-09-20T21:44:05.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:04 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7674a06e-2830-441e-9178-687e42dfce80"}]: dispatch 2023-09-20T21:44:05.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:04 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7674a06e-2830-441e-9178-687e42dfce80"}]': finished 2023-09-20T21:44:05.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:04 smithi089 ceph-mon[134090]: osdmap e48: 8 total, 7 up, 8 in 2023-09-20T21:44:05.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:04 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:05.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:05 smithi022 ceph-mon[97651]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:05.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:05 smithi022 ceph-mon[97651]: from='client.? 172.21.15.89:0/2418040899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:44:06.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:05 smithi123 ceph-mon[134433]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:06.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:05 smithi123 ceph-mon[134433]: from='client.? 172.21.15.89:0/2418040899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:44:06.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:05 smithi089 ceph-mon[134090]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:06.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:05 smithi089 ceph-mon[134090]: from='client.? 172.21.15.89:0/2418040899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:44:08.471 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:08 smithi089 ceph-mon[134090]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:08.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:08 smithi022 ceph-mon[97651]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:08.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:08 smithi123 ceph-mon[134433]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:10.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:10 smithi022 ceph-mon[97651]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:10.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:10 smithi123 ceph-mon[134433]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:10.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:10 smithi089 ceph-mon[134090]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:11.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-20T21:44:11.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:11.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-20T21:44:11.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:11.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-09-20T21:44:11.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:12.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:12 smithi089 ceph-mon[134090]: Deploying daemon osd.7 on smithi089 2023-09-20T21:44:12.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:12 smithi089 ceph-mon[134090]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:12.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:12 smithi022 ceph-mon[97651]: Deploying daemon osd.7 on smithi089 2023-09-20T21:44:12.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:12 smithi022 ceph-mon[97651]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:12.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:12 smithi123 ceph-mon[134433]: Deploying daemon osd.7 on smithi089 2023-09-20T21:44:12.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:12 smithi123 ceph-mon[134433]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:13.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:13 smithi123 ceph-mon[134433]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:13.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:13 smithi089 ceph-mon[134090]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:13.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:13 smithi022 ceph-mon[97651]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:16.234 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:16 smithi089 ceph-mon[134090]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:16.235 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:16.235 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:16.235 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:16.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:16 smithi022 ceph-mon[97651]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:16.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:16.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:16.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:16 smithi123 ceph-mon[134433]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:16.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:18.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:18 smithi089 ceph-mon[134090]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:18.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:18.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:18.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:18.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:18.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:18.461 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 7 on host 'smithi089' 2023-09-20T21:44:18.462 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.461+0000 7f07297fa700 1 -- 172.21.15.89:0/3490936417 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f072c0ff680 con 0x7f07180a1380 2023-09-20T21:44:18.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 -- 172.21.15.89:0/3490936417 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07180a1380 msgr2=0x7f07180a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:18.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07180a1380 0x7f07180a3840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f072c068680 tx=0x7f071c005cf0 comp rx=0 tx=0).stop 2023-09-20T21:44:18.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 -- 172.21.15.89:0/3490936417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f072c101380 msgr2=0x7f072c102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:18.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f072c101380 0x7f072c102910 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f072400ada0 tx=0x7f072400c330 comp rx=0 tx=0).stop 2023-09-20T21:44:18.464 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 -- 172.21.15.89:0/3490936417 shutdown_connections 2023-09-20T21:44:18.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07180a1380 0x7f07180a3840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:18.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f072c101380 0x7f072c102910 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:18.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f072c0fffe0 0x7f072c103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:18.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 --2- 172.21.15.89:0/3490936417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f072c1009a0 0x7f072c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:18.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 -- 172.21.15.89:0/3490936417 >> 172.21.15.89:0/3490936417 conn(0x7f072c0faa90 msgr2=0x7f072c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:44:18.465 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 -- 172.21.15.89:0/3490936417 shutdown_connections 2023-09-20T21:44:18.466 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:44:18.463+0000 7f07323ea700 1 -- 172.21.15.89:0/3490936417 wait complete. 2023-09-20T21:44:18.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:18 smithi022 ceph-mon[97651]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:18.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:18 smithi123 ceph-mon[134433]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:18.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:18.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:18.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:18.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:18.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:18.901 DEBUG:teuthology.orchestra.run.smithi089:osd.7> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.7.service 2023-09-20T21:44:18.904 INFO:tasks.cephadm:Deploying osd.8 on smithi123 with /dev/vg_nvme/lv_2... 2023-09-20T21:44:18.904 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-09-20T21:44:18.932 INFO:journalctl@ceph.osd.7.smithi089.stdout:-- Logs begin at Wed 2023-09-20 21:17:56 UTC. -- 2023-09-20T21:44:19.467 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:44:19.610 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 21:44:19 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T21:44:19.414+0000 7f8213357640 -1 osd.7 0 log_to_monitors true 2023-09-20T21:44:19.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:19.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:19.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:19.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:19 smithi089 ceph-mon[134090]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:19.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:19 smithi089 ceph-mon[134090]: from='osd.7 [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-20T21:44:19.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:19 smithi089 ceph-mon[134090]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-20T21:44:19.684 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:19.684 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:19.685 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:19.685 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:19 smithi123 ceph-mon[134433]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:19.685 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:19 smithi123 ceph-mon[134433]: from='osd.7 [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-20T21:44:19.686 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:19 smithi123 ceph-mon[134433]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-20T21:44:19.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:19.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:19.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:19.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:19 smithi022 ceph-mon[97651]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:19.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:19 smithi022 ceph-mon[97651]: from='osd.7 [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-20T21:44:19.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:19 smithi022 ceph-mon[97651]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-09-20T21:44:20.507 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:20 smithi123 ceph-mon[134433]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-20T21:44:20.507 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:20 smithi123 ceph-mon[134433]: osdmap e49: 8 total, 7 up, 8 in 2023-09-20T21:44:20.508 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:20.508 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:20 smithi123 ceph-mon[134433]: from='osd.7 [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:44:20.508 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:20 smithi123 ceph-mon[134433]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:44:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:20 smithi022 ceph-mon[97651]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-20T21:44:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:20 smithi022 ceph-mon[97651]: osdmap e49: 8 total, 7 up, 8 in 2023-09-20T21:44:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:20 smithi022 ceph-mon[97651]: from='osd.7 [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:44:20.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:20 smithi022 ceph-mon[97651]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:44:20.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:20 smithi089 ceph-mon[134090]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-09-20T21:44:20.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:20 smithi089 ceph-mon[134090]: osdmap e49: 8 total, 7 up, 8 in 2023-09-20T21:44:20.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:20.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:20 smithi089 ceph-mon[134090]: from='osd.7 [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:44:20.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:20 smithi089 ceph-mon[134090]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:44:20.862 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 21:44:20 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T21:44:20.455+0000 7f82089cf700 -1 osd.7 0 waiting for initial osdmap 2023-09-20T21:44:20.863 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 21:44:20 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T21:44:20.464+0000 7f8201fbf700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:44:20.949 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-09-20T21:44:20.949 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-09-20T21:44:20.949 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-09-20T21:44:20.949 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-09-20T21:44:20.949 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0141509 s, 741 MB/s 2023-09-20T21:44:20.950 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-09-20T21:44:21.309 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2023-09-20T21:44:21.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:21 smithi123 ceph-mon[134433]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:44:21.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:21 smithi123 ceph-mon[134433]: osdmap e50: 8 total, 7 up, 8 in 2023-09-20T21:44:21.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:21.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:21.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:21 smithi123 ceph-mon[134433]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:21.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:21 smithi022 ceph-mon[97651]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:44:21.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:21 smithi022 ceph-mon[97651]: osdmap e50: 8 total, 7 up, 8 in 2023-09-20T21:44:21.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:21.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:21.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:21 smithi022 ceph-mon[97651]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:21.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:21 smithi089 ceph-mon[134090]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:44:21.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:21 smithi089 ceph-mon[134090]: osdmap e50: 8 total, 7 up, 8 in 2023-09-20T21:44:21.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:21.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:21.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:21 smithi089 ceph-mon[134090]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-09-20T21:44:21.863 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:44:22.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:44:22.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:44:22.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:22.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: osd.7 [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] boot 2023-09-20T21:44:22.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: osdmap e51: 8 total, 8 up, 8 in 2023-09-20T21:44:22.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:22.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi089 2023-09-20T21:44:22.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:22.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:22.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:22.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi089 to 5722M 2023-09-20T21:44:22.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:22.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:22.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:44:22.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:44:22.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:22.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: osd.7 [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] boot 2023-09-20T21:44:22.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: osdmap e51: 8 total, 8 up, 8 in 2023-09-20T21:44:22.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:22.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi089 2023-09-20T21:44:22.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:22.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:22.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:22.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi089 to 5722M 2023-09-20T21:44:22.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:22.847 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:22.847 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:44:22.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:44:22.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:22.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: osd.7 [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] boot 2023-09-20T21:44:22.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: osdmap e51: 8 total, 8 up, 8 in 2023-09-20T21:44:22.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-09-20T21:44:22.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi089 2023-09-20T21:44:22.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:22.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:22.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:22.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi089 to 5722M 2023-09-20T21:44:22.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:22.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:22.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:22.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:23.058 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.056+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/4032515552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f0100e70 msgr2=0x7fa6f0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:23.058 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.056+0000 7fa6f7dbb700 1 --2- 172.21.15.123:0/4032515552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f0100e70 0x7fa6f0101290 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009990 tx=0x7fa6e0009ca0 comp rx=0 tx=0).stop 2023-09-20T21:44:23.058 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.056+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/4032515552 shutdown_connections 2023-09-20T21:44:23.058 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.056+0000 7fa6f7dbb700 1 --2- 172.21.15.123:0/4032515552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6f0102990 0x7fa6f01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:23.059 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.056+0000 7fa6f7dbb700 1 --2- 172.21.15.123:0/4032515552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f0100e70 0x7fa6f0101290 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:23.059 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.056+0000 7fa6f7dbb700 1 --2- 172.21.15.123:0/4032515552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f0101fd0 0x7fa6f0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:23.059 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.056+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/4032515552 >> 172.21.15.123:0/4032515552 conn(0x7fa6f00fc3d0 msgr2=0x7fa6f00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:44:23.059 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.056+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/4032515552 shutdown_connections 2023-09-20T21:44:23.059 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.056+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/4032515552 wait complete. 2023-09-20T21:44:23.060 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f7dbb700 1 Processor -- start 2023-09-20T21:44:23.060 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f7dbb700 1 -- start start 2023-09-20T21:44:23.060 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f7dbb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f0100e70 0x7fa6f0196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:23.060 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f7dbb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f0101fd0 0x7fa6f01970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:23.060 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f7dbb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6f0102990 0x7fa6f019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:23.061 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f7dbb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f019e740 con 0x7fa6f0102990 2023-09-20T21:44:23.061 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f7dbb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f019e880 con 0x7fa6f0101fd0 2023-09-20T21:44:23.061 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f7dbb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f019ea10 con 0x7fa6f0100e70 2023-09-20T21:44:23.061 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5b57700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f0100e70 0x7fa6f0196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:23.061 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5b57700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f0100e70 0x7fa6f0196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34052/0 (socket says 172.21.15.123:34052) 2023-09-20T21:44:23.062 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5b57700 1 -- 172.21.15.123:0/1318090511 learned_addr learned my addr 172.21.15.123:0/1318090511 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-20T21:44:23.062 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5356700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f0101fd0 0x7fa6f01970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:23.062 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f6358700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6f0102990 0x7fa6f019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:23.062 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5b57700 1 -- 172.21.15.123:0/1318090511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f0101fd0 msgr2=0x7fa6f01970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:23.062 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5b57700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f0101fd0 0x7fa6f01970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:23.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5b57700 1 -- 172.21.15.123:0/1318090511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6f0102990 msgr2=0x7fa6f019e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:23.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5b57700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6f0102990 0x7fa6f019e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:23.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5b57700 1 -- 172.21.15.123:0/1318090511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e00095f0 con 0x7fa6f0100e70 2023-09-20T21:44:23.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5356700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f0101fd0 0x7fa6f01970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:44:23.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.057+0000 7fa6f5b57700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f0100e70 0x7fa6f0196bb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000f770 tx=0x7fa6e000f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:44:23.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.058+0000 7fa6e6ffd700 1 -- 172.21.15.123:0/1318090511 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e001d070 con 0x7fa6f0100e70 2023-09-20T21:44:23.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.058+0000 7fa6e6ffd700 1 -- 172.21.15.123:0/1318090511 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6e000fd30 con 0x7fa6f0100e70 2023-09-20T21:44:23.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.058+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/1318090511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f019ec90 con 0x7fa6f0100e70 2023-09-20T21:44:23.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.058+0000 7fa6e6ffd700 1 -- 172.21.15.123:0/1318090511 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0005720 con 0x7fa6f0100e70 2023-09-20T21:44:23.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.058+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/1318090511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f019f180 con 0x7fa6f0100e70 2023-09-20T21:44:23.066 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.058+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/1318090511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f0108460 con 0x7fa6f0100e70 2023-09-20T21:44:23.066 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.060+0000 7fa6e6ffd700 1 -- 172.21.15.123:0/1318090511 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa6e0004830 con 0x7fa6f0100e70 2023-09-20T21:44:23.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.060+0000 7fa6e6ffd700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6dc0a1380 0x7fa6dc0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:23.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.060+0000 7fa6e6ffd700 1 -- 172.21.15.123:0/1318090511 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fa6e00cd8f0 con 0x7fa6f0100e70 2023-09-20T21:44:23.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.060+0000 7fa6f5356700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6dc0a1380 0x7fa6dc0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:23.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.061+0000 7fa6f5356700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6dc0a1380 0x7fa6dc0a3840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0099d0 tx=0x7fa6ec008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:44:23.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.064+0000 7fa6e6ffd700 1 -- 172.21.15.123:0/1318090511 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa6e0097b90 con 0x7fa6f0100e70 2023-09-20T21:44:23.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:23.260+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/1318090511 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6f002d230 con 0x7fa6dc0a1380 2023-09-20T21:44:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:23 smithi022 ceph-mon[97651]: osdmap e52: 8 total, 8 up, 8 in 2023-09-20T21:44:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:23 smithi022 ceph-mon[97651]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:44:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:44:23.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:23.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:23 smithi123 ceph-mon[134433]: osdmap e52: 8 total, 8 up, 8 in 2023-09-20T21:44:23.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:23 smithi123 ceph-mon[134433]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:23.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:23 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:44:23.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:23 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:44:23.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:23 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:23.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:23 smithi089 ceph-mon[134090]: osdmap e52: 8 total, 8 up, 8 in 2023-09-20T21:44:23.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:23 smithi089 ceph-mon[134090]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:23.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:23 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:44:23.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:23 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:44:23.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:23 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:24.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:24 smithi123 ceph-mon[134433]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:44:24.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:24 smithi123 ceph-mon[134433]: osdmap e53: 8 total, 8 up, 8 in 2023-09-20T21:44:24.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:24 smithi022 ceph-mon[97651]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:44:24.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:24 smithi022 ceph-mon[97651]: osdmap e53: 8 total, 8 up, 8 in 2023-09-20T21:44:24.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:24 smithi089 ceph-mon[134090]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:44:24.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:24 smithi089 ceph-mon[134090]: osdmap e53: 8 total, 8 up, 8 in 2023-09-20T21:44:25.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:25 smithi123 ceph-mon[134433]: osdmap e54: 8 total, 8 up, 8 in 2023-09-20T21:44:25.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:25 smithi123 ceph-mon[134433]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:25.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:25 smithi089 ceph-mon[134090]: osdmap e54: 8 total, 8 up, 8 in 2023-09-20T21:44:25.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:25 smithi089 ceph-mon[134090]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:25.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:25 smithi022 ceph-mon[97651]: osdmap e54: 8 total, 8 up, 8 in 2023-09-20T21:44:25.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:25 smithi022 ceph-mon[97651]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:26.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:26 smithi123 ceph-mon[134433]: osdmap e55: 8 total, 8 up, 8 in 2023-09-20T21:44:26.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:26 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/527587230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf42a69e-43d0-47b2-bd13-9378e765e7d1"}]: dispatch 2023-09-20T21:44:26.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:26 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf42a69e-43d0-47b2-bd13-9378e765e7d1"}]: dispatch 2023-09-20T21:44:26.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:26 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf42a69e-43d0-47b2-bd13-9378e765e7d1"}]': finished 2023-09-20T21:44:26.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:26 smithi123 ceph-mon[134433]: osdmap e56: 9 total, 8 up, 9 in 2023-09-20T21:44:26.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:26 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:26.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:26 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/3637466604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:44:26.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:26 smithi089 ceph-mon[134090]: osdmap e55: 8 total, 8 up, 8 in 2023-09-20T21:44:26.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:26 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/527587230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf42a69e-43d0-47b2-bd13-9378e765e7d1"}]: dispatch 2023-09-20T21:44:26.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:26 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf42a69e-43d0-47b2-bd13-9378e765e7d1"}]: dispatch 2023-09-20T21:44:26.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:26 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf42a69e-43d0-47b2-bd13-9378e765e7d1"}]': finished 2023-09-20T21:44:26.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:26 smithi089 ceph-mon[134090]: osdmap e56: 9 total, 8 up, 9 in 2023-09-20T21:44:26.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:26 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:26.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:26 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/3637466604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:44:26.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:26 smithi022 ceph-mon[97651]: osdmap e55: 8 total, 8 up, 8 in 2023-09-20T21:44:26.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:26 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/527587230' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf42a69e-43d0-47b2-bd13-9378e765e7d1"}]: dispatch 2023-09-20T21:44:26.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:26 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf42a69e-43d0-47b2-bd13-9378e765e7d1"}]: dispatch 2023-09-20T21:44:26.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:26 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf42a69e-43d0-47b2-bd13-9378e765e7d1"}]': finished 2023-09-20T21:44:26.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:26 smithi022 ceph-mon[97651]: osdmap e56: 9 total, 8 up, 9 in 2023-09-20T21:44:26.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:26 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:26.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:26 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/3637466604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:44:27.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:27 smithi123 ceph-mon[134433]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:27.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:27 smithi089 ceph-mon[134090]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:27.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:27 smithi022 ceph-mon[97651]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:30.424 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:30 smithi123 ceph-mon[134433]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:30.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:30 smithi022 ceph-mon[97651]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:30.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:30 smithi089 ceph-mon[134090]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:32.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:32 smithi123 ceph-mon[134433]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:32.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:32 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-20T21:44:32.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:32 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:32.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:32 smithi022 ceph-mon[97651]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:32 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-20T21:44:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:32 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:32.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:32 smithi089 ceph-mon[134090]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:32.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:32 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-09-20T21:44:32.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:32 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:33.399 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:33 smithi123 ceph-mon[134433]: Deploying daemon osd.8 on smithi123 2023-09-20T21:44:33.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:33 smithi022 ceph-mon[97651]: Deploying daemon osd.8 on smithi123 2023-09-20T21:44:33.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:33 smithi089 ceph-mon[134090]: Deploying daemon osd.8 on smithi123 2023-09-20T21:44:34.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:34 smithi123 ceph-mon[134433]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:34.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:34 smithi022 ceph-mon[97651]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:34.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:34 smithi089 ceph-mon[134090]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:36.400 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:36 smithi123 ceph-mon[134433]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:36.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:36 smithi022 ceph-mon[97651]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:36.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:36 smithi089 ceph-mon[134090]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:37.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:37.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:37.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:37.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:37 smithi123 ceph-mon[134433]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:37.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:37.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:37.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:37.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:37 smithi022 ceph-mon[97651]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:37.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:37.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:37.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:37.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:37 smithi089 ceph-mon[134090]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:39.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.123+0000 7fa6e6ffd700 1 -- 172.21.15.123:0/1318090511 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa6f002d230 con 0x7fa6dc0a1380 2023-09-20T21:44:39.125 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 8 on host 'smithi123' 2023-09-20T21:44:39.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/1318090511 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6dc0a1380 msgr2=0x7fa6dc0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:39.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6dc0a1380 0x7fa6dc0a3840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0099d0 tx=0x7fa6ec008040 comp rx=0 tx=0).stop 2023-09-20T21:44:39.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/1318090511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f0100e70 msgr2=0x7fa6f0196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:39.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f0100e70 0x7fa6f0196bb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000f770 tx=0x7fa6e000f850 comp rx=0 tx=0).stop 2023-09-20T21:44:39.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/1318090511 shutdown_connections 2023-09-20T21:44:39.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6f0102990 0x7fa6f019e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:39.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6f0100e70 0x7fa6f0196bb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:39.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6dc0a1380 0x7fa6dc0a3840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:39.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 --2- 172.21.15.123:0/1318090511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa6f0101fd0 0x7fa6f01970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:39.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/1318090511 >> 172.21.15.123:0/1318090511 conn(0x7fa6f00fc3d0 msgr2=0x7fa6f01055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:44:39.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/1318090511 shutdown_connections 2023-09-20T21:44:39.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:44:39.126+0000 7fa6f7dbb700 1 -- 172.21.15.123:0/1318090511 wait complete. 2023-09-20T21:44:39.553 DEBUG:teuthology.orchestra.run.smithi123:osd.8> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.8.service 2023-09-20T21:44:39.555 INFO:tasks.cephadm:Deploying osd.9 on smithi022 with /dev/vg_nvme/lv_1... 2023-09-20T21:44:39.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-20T21:44:39.587 INFO:journalctl@ceph.osd.8.smithi123.stdout:-- Logs begin at Wed 2023-09-20 21:17:48 UTC. -- 2023-09-20T21:44:40.172 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.173 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.173 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:40.173 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:40.174 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.174 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:39 smithi123 ceph-mon[134433]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:40.175 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.175 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.175 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:40.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:40.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:40.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:39 smithi022 ceph-mon[97651]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:40.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:40.296 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:44:40.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:40.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:40.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:39 smithi089 ceph-mon[134090]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:40.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:40.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:44:40.466 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 21:44:40 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T21:44:40.384+0000 7f1d873f5640 -1 osd.8 0 log_to_monitors true 2023-09-20T21:44:41.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:40 smithi123 ceph-mon[134433]: from='osd.8 [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-20T21:44:41.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:40 smithi123 ceph-mon[134433]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-20T21:44:41.142 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:40 smithi022 ceph-mon[97651]: from='osd.8 [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-20T21:44:41.142 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:40 smithi022 ceph-mon[97651]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-20T21:44:41.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:40 smithi089 ceph-mon[134090]: from='osd.8 [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-20T21:44:41.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:40 smithi089 ceph-mon[134090]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-09-20T21:44:42.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:41 smithi123 ceph-mon[134433]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-20T21:44:42.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:41 smithi123 ceph-mon[134433]: osdmap e57: 9 total, 8 up, 9 in 2023-09-20T21:44:42.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:41 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:42.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:41 smithi123 ceph-mon[134433]: from='osd.8 [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:44:42.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:41 smithi123 ceph-mon[134433]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:44:42.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:41 smithi123 ceph-mon[134433]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:42.095 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 21:44:41 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T21:44:41.932+0000 7f1d7ca6d700 -1 osd.8 0 waiting for initial osdmap 2023-09-20T21:44:42.095 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 21:44:41 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T21:44:41.942+0000 7f1d7485a700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:44:42.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:41 smithi022 ceph-mon[97651]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-20T21:44:42.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:41 smithi022 ceph-mon[97651]: osdmap e57: 9 total, 8 up, 9 in 2023-09-20T21:44:42.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:41 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:42.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:41 smithi022 ceph-mon[97651]: from='osd.8 [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:44:42.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:41 smithi022 ceph-mon[97651]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:44:42.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:41 smithi022 ceph-mon[97651]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:42.312 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-20T21:44:42.313 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-20T21:44:42.313 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2023-09-20T21:44:42.313 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2023-09-20T21:44:42.313 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0326866 s, 321 MB/s 2023-09-20T21:44:42.313 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2023-09-20T21:44:42.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:41 smithi089 ceph-mon[134090]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-09-20T21:44:42.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:41 smithi089 ceph-mon[134090]: osdmap e57: 9 total, 8 up, 9 in 2023-09-20T21:44:42.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:41 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:42.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:41 smithi089 ceph-mon[134090]: from='osd.8 [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:44:42.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:41 smithi089 ceph-mon[134090]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:44:42.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:41 smithi089 ceph-mon[134090]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-09-20T21:44:42.872 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi022:vg_nvme/lv_1 2023-09-20T21:44:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:44:43.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: osdmap e58: 9 total, 8 up, 9 in 2023-09-20T21:44:43.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:43.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:43.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi123 2023-09-20T21:44:43.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:43.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:43.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:43.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:43.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:43.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: osd.8 [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] boot 2023-09-20T21:44:43.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: osdmap e59: 9 total, 9 up, 9 in 2023-09-20T21:44:43.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:43.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:44:43.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: osdmap e58: 9 total, 8 up, 9 in 2023-09-20T21:44:43.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:43.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:43.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi123 2023-09-20T21:44:43.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:43.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:43.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:43.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:43.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:43.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: osd.8 [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] boot 2023-09-20T21:44:43.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: osdmap e59: 9 total, 9 up, 9 in 2023-09-20T21:44:43.347 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:43.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:44:43.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: osdmap e58: 9 total, 8 up, 9 in 2023-09-20T21:44:43.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:43.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:43.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi123 2023-09-20T21:44:43.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:43.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:43.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:44:43.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:43.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:44:43.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:44:43.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: osd.8 [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] boot 2023-09-20T21:44:43.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: osdmap e59: 9 total, 9 up, 9 in 2023-09-20T21:44:43.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-09-20T21:44:43.610 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:44:44.214 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:43 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:44:44.214 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:43 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:44:44.214 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:43 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi123 to 5722M 2023-09-20T21:44:44.214 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:43 smithi022 ceph-mon[97651]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:44.214 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:43 smithi022 ceph-mon[97651]: osdmap e60: 9 total, 9 up, 9 in 2023-09-20T21:44:44.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:43 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:44:44.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:43 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:44:44.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:43 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi123 to 5722M 2023-09-20T21:44:44.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:43 smithi123 ceph-mon[134433]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:44.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:43 smithi123 ceph-mon[134433]: osdmap e60: 9 total, 9 up, 9 in 2023-09-20T21:44:44.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:43 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:44:44.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:43 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:44:44.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:43 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi123 to 5722M 2023-09-20T21:44:44.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:43 smithi089 ceph-mon[134090]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:44.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:43 smithi089 ceph-mon[134090]: osdmap e60: 9 total, 9 up, 9 in 2023-09-20T21:44:45.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.165+0000 7f049de86700 1 -- 172.21.15.22:0/2860999271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498101fd0 msgr2=0x7f0498102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:45.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.165+0000 7f049de86700 1 --2- 172.21.15.22:0/2860999271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498101fd0 0x7f0498102450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f048c009990 tx=0x7f048c009ca0 comp rx=0 tx=0).stop 2023-09-20T21:44:45.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.165+0000 7f049de86700 1 -- 172.21.15.22:0/2860999271 shutdown_connections 2023-09-20T21:44:45.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.165+0000 7f049de86700 1 --2- 172.21.15.22:0/2860999271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102990 0x7f04981092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.165+0000 7f049de86700 1 --2- 172.21.15.22:0/2860999271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0498100e70 0x7f0498101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.165+0000 7f049de86700 1 --2- 172.21.15.22:0/2860999271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498101fd0 0x7f0498102450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.165+0000 7f049de86700 1 -- 172.21.15.22:0/2860999271 >> 172.21.15.22:0/2860999271 conn(0x7f04980fc3d0 msgr2=0x7f04980fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:44:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.165+0000 7f049de86700 1 -- 172.21.15.22:0/2860999271 shutdown_connections 2023-09-20T21:44:45.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.165+0000 7f049de86700 1 -- 172.21.15.22:0/2860999271 wait complete. 2023-09-20T21:44:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.166+0000 7f049de86700 1 Processor -- start 2023-09-20T21:44:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.166+0000 7f049de86700 1 -- start start 2023-09-20T21:44:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.166+0000 7f049de86700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498100e70 0x7f0498074b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:45.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.166+0000 7f049de86700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0498101fd0 0x7f04980731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.166+0000 7f049de86700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102990 0x7f0498073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.166+0000 7f049de86700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049819a970 con 0x7f0498101fd0 2023-09-20T21:44:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.166+0000 7f049de86700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498073c50 con 0x7f0498102990 2023-09-20T21:44:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.166+0000 7f049de86700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498073e10 con 0x7f0498100e70 2023-09-20T21:44:45.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f0496ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0498101fd0 0x7f04980731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f04977fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498100e70 0x7f0498074b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f0496ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0498101fd0 0x7f04980731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57984/0 (socket says 172.21.15.22:57984) 2023-09-20T21:44:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f0496ffd700 1 -- 172.21.15.22:0/923586010 learned_addr learned my addr 172.21.15.22:0/923586010 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:44:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f0497fff700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102990 0x7f0498073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:45.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f0496ffd700 1 -- 172.21.15.22:0/923586010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498100e70 msgr2=0x7f0498074b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f0496ffd700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498100e70 0x7f0498074b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f0496ffd700 1 -- 172.21.15.22:0/923586010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102990 msgr2=0x7f0498073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:44:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f0496ffd700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102990 0x7f0498073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:44:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f0496ffd700 1 -- 172.21.15.22:0/923586010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0488009520 con 0x7f0498101fd0 2023-09-20T21:44:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f04977fe700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498100e70 0x7f0498074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:44:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.167+0000 7f0496ffd700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0498101fd0 0x7f04980731d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f048c000c00 tx=0x7f048c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:44:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.168+0000 7f0494ff9700 1 -- 172.21.15.22:0/923586010 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c01d070 con 0x7f0498101fd0 2023-09-20T21:44:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.168+0000 7f049de86700 1 -- 172.21.15.22:0/923586010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c0095f0 con 0x7f0498101fd0 2023-09-20T21:44:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.168+0000 7f0494ff9700 1 -- 172.21.15.22:0/923586010 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f048c022470 con 0x7f0498101fd0 2023-09-20T21:44:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.168+0000 7f0494ff9700 1 -- 172.21.15.22:0/923586010 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c017650 con 0x7f0498101fd0 2023-09-20T21:44:45.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.168+0000 7f049de86700 1 -- 172.21.15.22:0/923586010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04981a5560 con 0x7f0498101fd0 2023-09-20T21:44:45.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.169+0000 7f049de86700 1 -- 172.21.15.22:0/923586010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0498108460 con 0x7f0498101fd0 2023-09-20T21:44:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.171+0000 7f0494ff9700 1 -- 172.21.15.22:0/923586010 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f048c0225e0 con 0x7f0498101fd0 2023-09-20T21:44:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.173+0000 7f0494ff9700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f04800a1380 0x7f04800a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:44:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.173+0000 7f0494ff9700 1 -- 172.21.15.22:0/923586010 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f048c0cd5c0 con 0x7f0498101fd0 2023-09-20T21:44:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.173+0000 7f04977fe700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f04800a1380 0x7f04800a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:44:45.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.174+0000 7f04977fe700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f04800a1380 0x7f04800a3840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0488009fd0 tx=0x7f0488009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:44:45.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.177+0000 7f0494ff9700 1 -- 172.21.15.22:0/923586010 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f048c097610 con 0x7f0498101fd0 2023-09-20T21:44:45.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:44:45.397+0000 7f049de86700 1 -- 172.21.15.22:0/923586010 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f049802d140 con 0x7f04800a1380 2023-09-20T21:44:45.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:45 smithi123 ceph-mon[134433]: osdmap e61: 9 total, 9 up, 9 in 2023-09-20T21:44:45.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:45 smithi123 ceph-mon[134433]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:45.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:45 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:44:45.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:45 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:44:45.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:45 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:45.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:45 smithi089 ceph-mon[134090]: osdmap e61: 9 total, 9 up, 9 in 2023-09-20T21:44:45.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:45 smithi089 ceph-mon[134090]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:45.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:45 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:44:45.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:45 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:44:45.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:45 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:45.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:45 smithi022 ceph-mon[97651]: osdmap e61: 9 total, 9 up, 9 in 2023-09-20T21:44:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:45 smithi022 ceph-mon[97651]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:44:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:44:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:46.821 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:46 smithi022 ceph-mon[97651]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:44:46.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:46 smithi123 ceph-mon[134433]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:44:46.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:46 smithi089 ceph-mon[134090]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:44:47.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:47 smithi123 ceph-mon[134433]: osdmap e62: 9 total, 9 up, 9 in 2023-09-20T21:44:47.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:47 smithi123 ceph-mon[134433]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:47.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:47 smithi089 ceph-mon[134090]: osdmap e62: 9 total, 9 up, 9 in 2023-09-20T21:44:47.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:47 smithi089 ceph-mon[134090]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:47.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:47 smithi022 ceph-mon[97651]: osdmap e62: 9 total, 9 up, 9 in 2023-09-20T21:44:47.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:47 smithi022 ceph-mon[97651]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:48.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:48 smithi123 ceph-mon[134433]: osdmap e63: 9 total, 9 up, 9 in 2023-09-20T21:44:48.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:48 smithi089 ceph-mon[134090]: osdmap e63: 9 total, 9 up, 9 in 2023-09-20T21:44:48.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:48 smithi022 ceph-mon[97651]: osdmap e63: 9 total, 9 up, 9 in 2023-09-20T21:44:49.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:49 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1917152534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06"}]: dispatch 2023-09-20T21:44:49.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:49 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1917152534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06"}]': finished 2023-09-20T21:44:49.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:49 smithi123 ceph-mon[134433]: osdmap e64: 10 total, 9 up, 10 in 2023-09-20T21:44:49.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:44:49.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:49 smithi123 ceph-mon[134433]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-09-20T21:44:49.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:49 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4005398671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:44:49.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:49 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1917152534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06"}]: dispatch 2023-09-20T21:44:49.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:49 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1917152534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06"}]': finished 2023-09-20T21:44:49.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:49 smithi089 ceph-mon[134090]: osdmap e64: 10 total, 9 up, 10 in 2023-09-20T21:44:49.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:44:49.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:49 smithi089 ceph-mon[134090]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-09-20T21:44:49.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:49 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4005398671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:44:49.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:49 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1917152534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06"}]: dispatch 2023-09-20T21:44:49.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:49 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1917152534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06"}]': finished 2023-09-20T21:44:49.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:49 smithi022 ceph-mon[97651]: osdmap e64: 10 total, 9 up, 10 in 2023-09-20T21:44:49.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:44:49.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:49 smithi022 ceph-mon[97651]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2023-09-20T21:44:49.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:49 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4005398671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:44:52.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:52 smithi022 ceph-mon[97651]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-20T21:44:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:52 smithi123 ceph-mon[134433]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-20T21:44:52.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:52 smithi089 ceph-mon[134090]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-09-20T21:44:53.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:53 smithi123 ceph-mon[134433]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-09-20T21:44:53.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:53 smithi089 ceph-mon[134090]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-09-20T21:44:53.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:53 smithi022 ceph-mon[97651]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-09-20T21:44:56.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:56 smithi022 ceph-mon[97651]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-20T21:44:56.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:56 smithi123 ceph-mon[134433]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-20T21:44:56.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:56 smithi089 ceph-mon[134090]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-20T21:44:57.469 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-20T21:44:57.469 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:57 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:57.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-20T21:44:57.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:57 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:57.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-09-20T21:44:57.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:57 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:44:58.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:58 smithi022 ceph-mon[97651]: Deploying daemon osd.9 on smithi022 2023-09-20T21:44:58.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:44:58 smithi022 ceph-mon[97651]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:58 smithi123 ceph-mon[134433]: Deploying daemon osd.9 on smithi022 2023-09-20T21:44:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:44:58 smithi123 ceph-mon[134433]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:44:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:58 smithi089 ceph-mon[134090]: Deploying daemon osd.9 on smithi022 2023-09-20T21:44:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:44:58 smithi089 ceph-mon[134090]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:00.280 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:00 smithi022 ceph-mon[97651]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:00 smithi123 ceph-mon[134433]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:00 smithi089 ceph-mon[134090]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:02.480 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:02 smithi022 ceph-mon[97651]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:02 smithi123 ceph-mon[134433]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:02.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:02 smithi089 ceph-mon[134090]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:04.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:04.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:04.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:04.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:03 smithi123 ceph-mon[134433]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:04.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:04.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:04.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:04.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:03 smithi089 ceph-mon[134090]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:04.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:04.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:04.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:04.158 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:03 smithi022 ceph-mon[97651]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:06.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:06 smithi022 ceph-mon[97651]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:06.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:06 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:06.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:06 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:06.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:06 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:06.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:06 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:06.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:06 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:06.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.421+0000 7f0494ff9700 1 -- 172.21.15.22:0/923586010 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f049802d140 con 0x7f04800a1380 2023-09-20T21:45:06.422 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 9 on host 'smithi022' 2023-09-20T21:45:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.424+0000 7f049de86700 1 -- 172.21.15.22:0/923586010 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f04800a1380 msgr2=0x7f04800a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f04800a1380 0x7f04800a3840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0488009fd0 tx=0x7f0488009260 comp rx=0 tx=0).stop 2023-09-20T21:45:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 -- 172.21.15.22:0/923586010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0498101fd0 msgr2=0x7f04980731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0498101fd0 0x7f04980731d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f048c000c00 tx=0x7f048c004730 comp rx=0 tx=0).stop 2023-09-20T21:45:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 -- 172.21.15.22:0/923586010 shutdown_connections 2023-09-20T21:45:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f04800a1380 0x7f04800a3840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0498102990 0x7f0498073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0498100e70 0x7f0498074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 --2- 172.21.15.22:0/923586010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0498101fd0 0x7f04980731d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 -- 172.21.15.22:0/923586010 >> 172.21.15.22:0/923586010 conn(0x7f04980fc3d0 msgr2=0x7f04981055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 -- 172.21.15.22:0/923586010 shutdown_connections 2023-09-20T21:45:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:06.425+0000 7f049de86700 1 -- 172.21.15.22:0/923586010 wait complete. 2023-09-20T21:45:06.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:06 smithi123 ceph-mon[134433]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:06.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:06 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:06.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:06 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:06.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:06 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:06.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:06 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:06.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:06 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:06.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:06 smithi089 ceph-mon[134090]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:06.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:06 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:06.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:06 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:06.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:06 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:06.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:06 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:06.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:06 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:06.858 DEBUG:teuthology.orchestra.run.smithi022:osd.9> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.9.service 2023-09-20T21:45:06.862 INFO:tasks.cephadm:Deploying osd.10 on smithi089 with /dev/vg_nvme/lv_1... 2023-09-20T21:45:06.863 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-20T21:45:06.938 INFO:journalctl@ceph.osd.9.smithi022.stdout:-- Logs begin at Wed 2023-09-20 21:22:06 UTC. -- 2023-09-20T21:45:07.429 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T21:45:07.649 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:07.650 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:07.650 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:07.651 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:07 smithi089 ceph-mon[134090]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-20T21:45:07.651 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:07 smithi089 ceph-mon[134090]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:07.702 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:07.703 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:07.703 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:07.703 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:07 smithi022 ceph-mon[97651]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-20T21:45:07.703 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:07 smithi022 ceph-mon[97651]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:07 smithi123 ceph-mon[134433]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-09-20T21:45:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:07 smithi123 ceph-mon[134433]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:08 smithi022 ceph-mon[97651]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-20T21:45:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:08 smithi022 ceph-mon[97651]: osdmap e65: 10 total, 9 up, 10 in 2023-09-20T21:45:08.417 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:08.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:08 smithi022 ceph-mon[97651]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:45:08.418 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 21:45:08 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T21:45:08.402+0000 7f642136b700 -1 osd.9 0 waiting for initial osdmap 2023-09-20T21:45:08.418 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 21:45:08 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T21:45:08.416+0000 7f6419959700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:45:08.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:08 smithi123 ceph-mon[134433]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-20T21:45:08.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:08 smithi123 ceph-mon[134433]: osdmap e65: 10 total, 9 up, 10 in 2023-09-20T21:45:08.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:08.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:08 smithi123 ceph-mon[134433]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:45:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:08 smithi089 ceph-mon[134090]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-09-20T21:45:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:08 smithi089 ceph-mon[134090]: osdmap e65: 10 total, 9 up, 10 in 2023-09-20T21:45:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:08.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:08 smithi089 ceph-mon[134090]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2023-09-20T21:45:09.035 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-20T21:45:09.036 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-20T21:45:09.036 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2023-09-20T21:45:09.036 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2023-09-20T21:45:09.036 INFO:teuthology.orchestra.run.smithi089.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111776 s, 938 MB/s 2023-09-20T21:45:09.036 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2023-09-20T21:45:09.472 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi089:vg_nvme/lv_1 2023-09-20T21:45:09.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:09 smithi022 ceph-mon[97651]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:45:09.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:09 smithi022 ceph-mon[97651]: osdmap e66: 10 total, 9 up, 10 in 2023-09-20T21:45:09.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:09.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:09.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:09 smithi022 ceph-mon[97651]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:09.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:09 smithi123 ceph-mon[134433]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:45:09.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:09 smithi123 ceph-mon[134433]: osdmap e66: 10 total, 9 up, 10 in 2023-09-20T21:45:09.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:09.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:09.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:09 smithi123 ceph-mon[134433]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:09.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:09 smithi089 ceph-mon[134090]: from='osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2023-09-20T21:45:09.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:09 smithi089 ceph-mon[134090]: osdmap e66: 10 total, 9 up, 10 in 2023-09-20T21:45:09.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:09.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:09.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:09 smithi089 ceph-mon[134090]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-09-20T21:45:10.026 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T21:45:10.561 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:10 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:45:10.562 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:10 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:45:10.562 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:10 smithi089 ceph-mon[134090]: osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] boot 2023-09-20T21:45:10.562 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:10 smithi089 ceph-mon[134090]: osdmap e67: 10 total, 10 up, 10 in 2023-09-20T21:45:10.562 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:10.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:10 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:45:10.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:10 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:45:10.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:10 smithi022 ceph-mon[97651]: osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] boot 2023-09-20T21:45:10.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:10 smithi022 ceph-mon[97651]: osdmap e67: 10 total, 10 up, 10 in 2023-09-20T21:45:10.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:10.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:10 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:45:10.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:10 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:45:10.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:10 smithi123 ceph-mon[134433]: osd.9 [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] boot 2023-09-20T21:45:10.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:10 smithi123 ceph-mon[134433]: osdmap e67: 10 total, 10 up, 10 in 2023-09-20T21:45:10.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-09-20T21:45:11.198 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.198+0000 7f95bceab700 1 -- 172.21.15.89:0/3801837038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95b8100f70 msgr2=0x7f95b81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.198+0000 7f95bceab700 1 --2- 172.21.15.89:0/3801837038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95b8100f70 0x7f95b81013d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95a80099c0 tx=0x7f95a8009cd0 comp rx=0 tx=0).stop 2023-09-20T21:45:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.198+0000 7f95bceab700 1 -- 172.21.15.89:0/3801837038 shutdown_connections 2023-09-20T21:45:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.198+0000 7f95bceab700 1 --2- 172.21.15.89:0/3801837038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95b8101910 0x7f95b81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.198+0000 7f95bceab700 1 --2- 172.21.15.89:0/3801837038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95b80ffd70 0x7f95b8100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:11.199 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.198+0000 7f95bceab700 1 --2- 172.21.15.89:0/3801837038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95b8100f70 0x7f95b81013d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:11.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.198+0000 7f95bceab700 1 -- 172.21.15.89:0/3801837038 >> 172.21.15.89:0/3801837038 conn(0x7f95b80fb2f0 msgr2=0x7f95b80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:11.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.198+0000 7f95bceab700 1 -- 172.21.15.89:0/3801837038 shutdown_connections 2023-09-20T21:45:11.200 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.198+0000 7f95bceab700 1 -- 172.21.15.89:0/3801837038 wait complete. 2023-09-20T21:45:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95bceab700 1 Processor -- start 2023-09-20T21:45:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95bceab700 1 -- start start 2023-09-20T21:45:11.202 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95bceab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95b80ffd70 0x7f95b8196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95bceab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95b8100f70 0x7f95b81970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95bceab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95b8101910 0x7f95b819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95bceab700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b819e740 con 0x7f95b8101910 2023-09-20T21:45:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95bceab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b819e880 con 0x7f95b80ffd70 2023-09-20T21:45:11.203 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95bceab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b819ea10 con 0x7f95b8100f70 2023-09-20T21:45:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95b80ffd70 0x7f95b8196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95b80ffd70 0x7f95b8196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55346/0 (socket says 172.21.15.89:55346) 2023-09-20T21:45:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95b659c700 1 -- 172.21.15.89:0/726611131 learned_addr learned my addr 172.21.15.89:0/726611131 (peer_addr_for_me v2:172.21.15.89:0/0) 2023-09-20T21:45:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95b5d9b700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95b8100f70 0x7f95b81970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:11.204 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95b659c700 1 -- 172.21.15.89:0/726611131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95b8100f70 msgr2=0x7f95b81970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95b659c700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95b8100f70 0x7f95b81970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95b659c700 1 -- 172.21.15.89:0/726611131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95b8101910 msgr2=0x7f95b819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95b659c700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95b8101910 0x7f95b819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95b659c700 1 -- 172.21.15.89:0/726611131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80095f0 con 0x7f95b80ffd70 2023-09-20T21:45:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95b659c700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95b80ffd70 0x7f95b8196bb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00b500 tx=0x7f95ac00b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:45:11.205 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95a77fe700 1 -- 172.21.15.89:0/726611131 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac011830 con 0x7f95b80ffd70 2023-09-20T21:45:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95a77fe700 1 -- 172.21.15.89:0/726611131 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95ac011e50 con 0x7f95b80ffd70 2023-09-20T21:45:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95a77fe700 1 -- 172.21.15.89:0/726611131 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00f2e0 con 0x7f95b80ffd70 2023-09-20T21:45:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.199+0000 7f95bceab700 1 -- 172.21.15.89:0/726611131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b819ecf0 con 0x7f95b80ffd70 2023-09-20T21:45:11.206 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.200+0000 7f95bceab700 1 -- 172.21.15.89:0/726611131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b819f1a0 con 0x7f95b80ffd70 2023-09-20T21:45:11.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.200+0000 7f95bceab700 1 -- 172.21.15.89:0/726611131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b810b610 con 0x7f95b80ffd70 2023-09-20T21:45:11.207 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.202+0000 7f95a77fe700 1 -- 172.21.15.89:0/726611131 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f95ac010580 con 0x7f95b80ffd70 2023-09-20T21:45:11.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.203+0000 7f95a77fe700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95a00a1300 0x7f95a00a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:11.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.203+0000 7f95a77fe700 1 -- 172.21.15.89:0/726611131 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f95ac0cc520 con 0x7f95b80ffd70 2023-09-20T21:45:11.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.203+0000 7f95b5d9b700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95a00a1300 0x7f95a00a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:11.208 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.204+0000 7f95b5d9b700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95a00a1300 0x7f95a00a37c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95a80098e0 tx=0x7f95a800b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:45:11.209 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.207+0000 7f95a77fe700 1 -- 172.21.15.89:0/726611131 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f95ac096350 con 0x7f95b80ffd70 2023-09-20T21:45:11.403 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:11.402+0000 7f95bceab700 1 -- 172.21.15.89:0/726611131 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f95b8069030 con 0x7f95a00a1300 2023-09-20T21:45:11.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: osdmap e68: 10 total, 10 up, 10 in 2023-09-20T21:45:11.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi022 2023-09-20T21:45:11.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi022 to 4318M 2023-09-20T21:45:11.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:11.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:11.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:11.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:45:11.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:45:11.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:11.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: osdmap e68: 10 total, 10 up, 10 in 2023-09-20T21:45:11.904 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi022 2023-09-20T21:45:11.905 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.905 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.905 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.905 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.906 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.906 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.906 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi022 to 4318M 2023-09-20T21:45:11.906 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.907 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:11.907 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:11.907 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.907 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:11.907 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:45:11.908 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:45:11.908 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:11.909 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: osdmap e68: 10 total, 10 up, 10 in 2023-09-20T21:45:11.909 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi022 2023-09-20T21:45:11.909 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.909 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.909 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:11.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi022 to 4318M 2023-09-20T21:45:11.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.911 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:11.911 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:11.911 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:11.911 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:11.911 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:45:11.912 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:45:11.912 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:12.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:12 smithi123 ceph-mon[134433]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:45:12.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:12 smithi089 ceph-mon[134090]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:45:12.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:12 smithi022 ceph-mon[97651]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:45:13.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:13 smithi123 ceph-mon[134433]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:13.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:13 smithi089 ceph-mon[134090]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:13.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:13 smithi022 ceph-mon[97651]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:14.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:14 smithi123 ceph-mon[134433]: from='client.? 172.21.15.89:0/1930996809' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d118a3d-5e31-4f50-aabd-72d9125546ee"}]: dispatch 2023-09-20T21:45:14.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:14 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d118a3d-5e31-4f50-aabd-72d9125546ee"}]: dispatch 2023-09-20T21:45:14.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:14 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d118a3d-5e31-4f50-aabd-72d9125546ee"}]': finished 2023-09-20T21:45:14.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:14 smithi123 ceph-mon[134433]: osdmap e69: 11 total, 10 up, 11 in 2023-09-20T21:45:14.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:14 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:14.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:14 smithi123 ceph-mon[134433]: from='client.? 172.21.15.89:0/3927906890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:45:14.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:14 smithi089 ceph-mon[134090]: from='client.? 172.21.15.89:0/1930996809' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d118a3d-5e31-4f50-aabd-72d9125546ee"}]: dispatch 2023-09-20T21:45:14.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:14 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d118a3d-5e31-4f50-aabd-72d9125546ee"}]: dispatch 2023-09-20T21:45:14.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:14 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d118a3d-5e31-4f50-aabd-72d9125546ee"}]': finished 2023-09-20T21:45:14.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:14 smithi089 ceph-mon[134090]: osdmap e69: 11 total, 10 up, 11 in 2023-09-20T21:45:14.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:14 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:14.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:14 smithi089 ceph-mon[134090]: from='client.? 172.21.15.89:0/3927906890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:45:14.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:14 smithi022 ceph-mon[97651]: from='client.? 172.21.15.89:0/1930996809' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d118a3d-5e31-4f50-aabd-72d9125546ee"}]: dispatch 2023-09-20T21:45:14.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:14 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d118a3d-5e31-4f50-aabd-72d9125546ee"}]: dispatch 2023-09-20T21:45:14.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:14 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d118a3d-5e31-4f50-aabd-72d9125546ee"}]': finished 2023-09-20T21:45:14.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:14 smithi022 ceph-mon[97651]: osdmap e69: 11 total, 10 up, 11 in 2023-09-20T21:45:14.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:14 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:14.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:14 smithi022 ceph-mon[97651]: from='client.? 172.21.15.89:0/3927906890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:45:15.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:15 smithi123 ceph-mon[134433]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:15.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:15 smithi089 ceph-mon[134090]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:15.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:15 smithi022 ceph-mon[97651]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:18.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:18 smithi089 ceph-mon[134090]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:18.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:18 smithi022 ceph-mon[97651]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:18.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:18 smithi123 ceph-mon[134433]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:20.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:20 smithi089 ceph-mon[134090]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:20.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:20 smithi022 ceph-mon[97651]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:20.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:20 smithi123 ceph-mon[134433]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:21.467 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-20T21:45:21.468 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:21.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-20T21:45:21.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:21.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-09-20T21:45:21.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:22.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:22 smithi089 ceph-mon[134090]: Deploying daemon osd.10 on smithi089 2023-09-20T21:45:22.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:22 smithi089 ceph-mon[134090]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:22.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:22 smithi022 ceph-mon[97651]: Deploying daemon osd.10 on smithi089 2023-09-20T21:45:22.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:22 smithi022 ceph-mon[97651]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:22.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:22 smithi123 ceph-mon[134433]: Deploying daemon osd.10 on smithi089 2023-09-20T21:45:22.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:22 smithi123 ceph-mon[134433]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:23.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:23 smithi123 ceph-mon[134433]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:23.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:23 smithi089 ceph-mon[134090]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:23.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:23 smithi022 ceph-mon[97651]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:26.196 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:25 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:26.196 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:25 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:26.197 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:25 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:26.197 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:25 smithi089 ceph-mon[134090]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:26.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:26.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:26.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:26.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:25 smithi022 ceph-mon[97651]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:26.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:25 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:26.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:25 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:26.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:25 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:26.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:25 smithi123 ceph-mon[134433]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:27.922 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.923+0000 7f95a77fe700 1 -- 172.21.15.89:0/726611131 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f95b8069030 con 0x7f95a00a1300 2023-09-20T21:45:27.923 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 10 on host 'smithi089' 2023-09-20T21:45:27.924 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 -- 172.21.15.89:0/726611131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95a00a1300 msgr2=0x7f95a00a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:27.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95a00a1300 0x7f95a00a37c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95a80098e0 tx=0x7f95a800b540 comp rx=0 tx=0).stop 2023-09-20T21:45:27.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 -- 172.21.15.89:0/726611131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95b80ffd70 msgr2=0x7f95b8196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:27.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95b80ffd70 0x7f95b8196bb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00b500 tx=0x7f95ac00b8c0 comp rx=0 tx=0).stop 2023-09-20T21:45:27.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 -- 172.21.15.89:0/726611131 shutdown_connections 2023-09-20T21:45:27.925 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95b8101910 0x7f95b819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95a00a1300 0x7f95a00a37c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95b80ffd70 0x7f95b8196bb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 --2- 172.21.15.89:0/726611131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95b8100f70 0x7f95b81970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:27.926 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 -- 172.21.15.89:0/726611131 >> 172.21.15.89:0/726611131 conn(0x7f95b80fb2f0 msgr2=0x7f95b8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:27.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 -- 172.21.15.89:0/726611131 shutdown_connections 2023-09-20T21:45:27.927 INFO:teuthology.orchestra.run.smithi089.stderr:2023-09-20T21:45:27.925+0000 7f95bceab700 1 -- 172.21.15.89:0/726611131 wait complete. 2023-09-20T21:45:28.305 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:28 smithi089 ceph-mon[134090]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:28.306 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.306 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.306 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:28.306 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:28.306 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.307 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.307 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:28.307 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:28 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.308 DEBUG:teuthology.orchestra.run.smithi089:osd.10> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.10.service 2023-09-20T21:45:28.310 INFO:tasks.cephadm:Deploying osd.11 on smithi123 with /dev/vg_nvme/lv_1... 2023-09-20T21:45:28.311 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-09-20T21:45:28.339 INFO:journalctl@ceph.osd.10.smithi089.stdout:-- Logs begin at Wed 2023-09-20 21:17:56 UTC. -- 2023-09-20T21:45:28.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:28 smithi022 ceph-mon[97651]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:28.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:28.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:28.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:28.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:28 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:28 smithi123 ceph-mon[134433]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:28.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:28.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:28.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:28.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:28 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:28.879 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:45:29.225 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 21:45:28 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T21:45:28.898+0000 7f32d5f79640 -1 osd.10 0 log_to_monitors true 2023-09-20T21:45:29.226 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:29 smithi089 ceph-mon[134090]: from='osd.10 [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-20T21:45:29.226 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:29 smithi089 ceph-mon[134090]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-20T21:45:29.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:29 smithi123 ceph-mon[134433]: from='osd.10 [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-20T21:45:29.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:29 smithi123 ceph-mon[134433]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-20T21:45:29.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:29 smithi022 ceph-mon[97651]: from='osd.10 [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-20T21:45:29.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:29 smithi022 ceph-mon[97651]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-09-20T21:45:30.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:30 smithi123 ceph-mon[134433]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:30.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:30 smithi123 ceph-mon[134433]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-20T21:45:30.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:30 smithi123 ceph-mon[134433]: osdmap e70: 11 total, 10 up, 11 in 2023-09-20T21:45:30.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:30.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:30 smithi123 ceph-mon[134433]: from='osd.10 [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:45:30.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:30 smithi123 ceph-mon[134433]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:45:30.468 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-09-20T21:45:30.468 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-09-20T21:45:30.469 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2023-09-20T21:45:30.469 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2023-09-20T21:45:30.469 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111058 s, 944 MB/s 2023-09-20T21:45:30.469 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 2023-09-20T21:45:30.469 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2023-09-20T21:45:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:30 smithi022 ceph-mon[97651]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:30 smithi022 ceph-mon[97651]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-20T21:45:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:30 smithi022 ceph-mon[97651]: osdmap e70: 11 total, 10 up, 11 in 2023-09-20T21:45:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:30.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:30 smithi022 ceph-mon[97651]: from='osd.10 [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:45:30.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:30 smithi022 ceph-mon[97651]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:45:30.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:30 smithi089 ceph-mon[134090]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:30.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:30 smithi089 ceph-mon[134090]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-09-20T21:45:30.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:30 smithi089 ceph-mon[134090]: osdmap e70: 11 total, 10 up, 11 in 2023-09-20T21:45:30.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:30.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:30 smithi089 ceph-mon[134090]: from='osd.10 [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:45:30.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:30 smithi089 ceph-mon[134090]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2023-09-20T21:45:30.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 21:45:30 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T21:45:30.151+0000 7f32cbdf2700 -1 osd.10 0 waiting for initial osdmap 2023-09-20T21:45:30.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 21:45:30 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T21:45:30.166+0000 7f32c6be5700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:45:30.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2023-09-20T21:45:31.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:31 smithi089 ceph-mon[134090]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:45:31.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:31 smithi089 ceph-mon[134090]: osdmap e71: 11 total, 10 up, 11 in 2023-09-20T21:45:31.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:31.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:31.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:31.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:31.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.388 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:45:31.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:31 smithi022 ceph-mon[97651]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:45:31.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:31 smithi022 ceph-mon[97651]: osdmap e71: 11 total, 10 up, 11 in 2023-09-20T21:45:31.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:31.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:31.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:31.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:31.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:31 smithi123 ceph-mon[134433]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2023-09-20T21:45:31.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:31 smithi123 ceph-mon[134433]: osdmap e71: 11 total, 10 up, 11 in 2023-09-20T21:45:31.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:31.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:32.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:45:32.372 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:45:32.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:32.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi089 2023-09-20T21:45:32.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi089 to 4292M 2023-09-20T21:45:32.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: osd.10 [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] boot 2023-09-20T21:45:32.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: osdmap e72: 11 total, 11 up, 11 in 2023-09-20T21:45:32.374 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:32.374 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:32.374 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:32.374 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:32.374 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:32.375 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:32 smithi123 ceph-mon[134433]: osdmap e73: 11 total, 11 up, 11 in 2023-09-20T21:45:32.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:45:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:45:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi089 2023-09-20T21:45:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi089 to 4292M 2023-09-20T21:45:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: osd.10 [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] boot 2023-09-20T21:45:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: osdmap e72: 11 total, 11 up, 11 in 2023-09-20T21:45:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:32.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:32.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:32.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:32.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:32 smithi022 ceph-mon[97651]: osdmap e73: 11 total, 11 up, 11 in 2023-09-20T21:45:32.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:45:32.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:45:32.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-09-20T21:45:32.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi089 2023-09-20T21:45:32.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi089 to 4292M 2023-09-20T21:45:32.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: osd.10 [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] boot 2023-09-20T21:45:32.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: osdmap e72: 11 total, 11 up, 11 in 2023-09-20T21:45:32.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-09-20T21:45:32.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:32.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:32.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:32.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:32.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:32 smithi089 ceph-mon[134090]: osdmap e73: 11 total, 11 up, 11 in 2023-09-20T21:45:32.690 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.685+0000 7ff4eda47700 1 -- 172.21.15.123:0/2131124771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e8100010 msgr2=0x7ff4e8100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:32.691 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.685+0000 7ff4eda47700 1 --2- 172.21.15.123:0/2131124771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e8100010 0x7ff4e8100490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009990 tx=0x7ff4d8009ca0 comp rx=0 tx=0).stop 2023-09-20T21:45:32.691 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.685+0000 7ff4eda47700 1 -- 172.21.15.123:0/2131124771 shutdown_connections 2023-09-20T21:45:32.691 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.685+0000 7ff4eda47700 1 --2- 172.21.15.123:0/2131124771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e81009d0 0x7ff4e8108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:32.691 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.685+0000 7ff4eda47700 1 --2- 172.21.15.123:0/2131124771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e8101440 0x7ff4e8101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:32.691 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.685+0000 7ff4eda47700 1 --2- 172.21.15.123:0/2131124771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e8100010 0x7ff4e8100490 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:32.692 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.685+0000 7ff4eda47700 1 -- 172.21.15.123:0/2131124771 >> 172.21.15.123:0/2131124771 conn(0x7ff4e80faac0 msgr2=0x7ff4e80fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:32.692 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.685+0000 7ff4eda47700 1 -- 172.21.15.123:0/2131124771 shutdown_connections 2023-09-20T21:45:32.692 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.685+0000 7ff4eda47700 1 -- 172.21.15.123:0/2131124771 wait complete. 2023-09-20T21:45:32.692 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.685+0000 7ff4eda47700 1 Processor -- start 2023-09-20T21:45:32.692 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4eda47700 1 -- start start 2023-09-20T21:45:32.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4eda47700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e8100010 0x7ff4e80ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:32.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4eda47700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81009d0 0x7ff4e819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:32.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4eda47700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8101440 0x7ff4e819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:32.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4eda47700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e819e610 con 0x7ff4e81009d0 2023-09-20T21:45:32.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4eda47700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e819e780 con 0x7ff4e8101440 2023-09-20T21:45:32.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4eda47700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e819e940 con 0x7ff4e8100010 2023-09-20T21:45:32.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e8100010 0x7ff4e80ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:32.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e8100010 0x7ff4e80ffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34200/0 (socket says 172.21.15.123:34200) 2023-09-20T21:45:32.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e6ffd700 1 -- 172.21.15.123:0/1542023748 learned_addr learned my addr 172.21.15.123:0/1542023748 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-20T21:45:32.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e67fc700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81009d0 0x7ff4e819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:32.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e77fe700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8101440 0x7ff4e819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:32.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e6ffd700 1 -- 172.21.15.123:0/1542023748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8101440 msgr2=0x7ff4e819de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:32.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e6ffd700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8101440 0x7ff4e819de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:32.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e6ffd700 1 -- 172.21.15.123:0/1542023748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81009d0 msgr2=0x7ff4e819b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:32.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e6ffd700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81009d0 0x7ff4e819b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:32.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e6ffd700 1 -- 172.21.15.123:0/1542023748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d80095f0 con 0x7ff4e8100010 2023-09-20T21:45:32.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.686+0000 7ff4e6ffd700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e8100010 0x7ff4e80ffa80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c920 tx=0x7ff4dc00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:45:32.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.687+0000 7ff4eca45700 1 -- 172.21.15.123:0/1542023748 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc004a30 con 0x7ff4e8100010 2023-09-20T21:45:32.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.687+0000 7ff4eda47700 1 -- 172.21.15.123:0/1542023748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e819ec20 con 0x7ff4e8100010 2023-09-20T21:45:32.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.687+0000 7ff4eca45700 1 -- 172.21.15.123:0/1542023748 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4dc0054e0 con 0x7ff4e8100010 2023-09-20T21:45:32.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.687+0000 7ff4eca45700 1 -- 172.21.15.123:0/1542023748 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc007c10 con 0x7ff4e8100010 2023-09-20T21:45:32.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.687+0000 7ff4eda47700 1 -- 172.21.15.123:0/1542023748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e819f170 con 0x7ff4e8100010 2023-09-20T21:45:32.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.688+0000 7ff4eda47700 1 -- 172.21.15.123:0/1542023748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e810b5c0 con 0x7ff4e8100010 2023-09-20T21:45:32.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.689+0000 7ff4eca45700 1 -- 172.21.15.123:0/1542023748 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ff4dc005000 con 0x7ff4e8100010 2023-09-20T21:45:32.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.690+0000 7ff4eca45700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4d00a1300 0x7ff4d00a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:32.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.690+0000 7ff4eca45700 1 -- 172.21.15.123:0/1542023748 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7ff4dc0cc540 con 0x7ff4e8100010 2023-09-20T21:45:32.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.690+0000 7ff4e67fc700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4d00a1300 0x7ff4d00a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:32.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.691+0000 7ff4e67fc700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4d00a1300 0x7ff4d00a37c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80fffe0 tx=0x7ff4d8005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:45:32.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.694+0000 7ff4eca45700 1 -- 172.21.15.123:0/1542023748 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff4dc0961a0 con 0x7ff4e8100010 2023-09-20T21:45:32.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:32.890+0000 7ff4eda47700 1 -- 172.21.15.123:0/1542023748 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4e8061220 con 0x7ff4d00a1300 2023-09-20T21:45:33.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:33 smithi123 ceph-mon[134433]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:45:33.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:45:33.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:45:33.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:33.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:33 smithi123 ceph-mon[134433]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:33.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:33 smithi123 ceph-mon[134433]: osdmap e74: 11 total, 11 up, 11 in 2023-09-20T21:45:33.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:33 smithi089 ceph-mon[134090]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:45:33.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:45:33.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:45:33.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:33.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:33 smithi089 ceph-mon[134090]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:33.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:33 smithi089 ceph-mon[134090]: osdmap e74: 11 total, 11 up, 11 in 2023-09-20T21:45:33.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:33 smithi022 ceph-mon[97651]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:45:33.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-09-20T21:45:33.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-09-20T21:45:33.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:33.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:33 smithi022 ceph-mon[97651]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:34.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:33 smithi022 ceph-mon[97651]: osdmap e74: 11 total, 11 up, 11 in 2023-09-20T21:45:35.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:35 smithi022 ceph-mon[97651]: osdmap e75: 11 total, 11 up, 11 in 2023-09-20T21:45:35.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:35 smithi123 ceph-mon[134433]: osdmap e75: 11 total, 11 up, 11 in 2023-09-20T21:45:35.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:35 smithi089 ceph-mon[134090]: osdmap e75: 11 total, 11 up, 11 in 2023-09-20T21:45:36.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:36 smithi022 ceph-mon[97651]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:36.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:36 smithi022 ceph-mon[97651]: osdmap e76: 11 total, 11 up, 11 in 2023-09-20T21:45:36.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:36 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/2731066412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41242a75-3273-4217-ad0d-c5da81c6b369"}]: dispatch 2023-09-20T21:45:36.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:36 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41242a75-3273-4217-ad0d-c5da81c6b369"}]: dispatch 2023-09-20T21:45:36.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:36 smithi022 ceph-mon[97651]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41242a75-3273-4217-ad0d-c5da81c6b369"}]': finished 2023-09-20T21:45:36.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:36 smithi022 ceph-mon[97651]: osdmap e77: 12 total, 11 up, 12 in 2023-09-20T21:45:36.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:36 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:36.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:36 smithi022 ceph-mon[97651]: from='client.? 172.21.15.123:0/3946228981' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:45:36.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:36 smithi123 ceph-mon[134433]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:36.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:36 smithi123 ceph-mon[134433]: osdmap e76: 11 total, 11 up, 11 in 2023-09-20T21:45:36.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:36 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/2731066412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41242a75-3273-4217-ad0d-c5da81c6b369"}]: dispatch 2023-09-20T21:45:36.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:36 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41242a75-3273-4217-ad0d-c5da81c6b369"}]: dispatch 2023-09-20T21:45:36.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:36 smithi123 ceph-mon[134433]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41242a75-3273-4217-ad0d-c5da81c6b369"}]': finished 2023-09-20T21:45:36.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:36 smithi123 ceph-mon[134433]: osdmap e77: 12 total, 11 up, 12 in 2023-09-20T21:45:36.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:36 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:36.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:36 smithi123 ceph-mon[134433]: from='client.? 172.21.15.123:0/3946228981' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:45:36.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:36 smithi089 ceph-mon[134090]: pgmap v229: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:36.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:36 smithi089 ceph-mon[134090]: osdmap e76: 11 total, 11 up, 11 in 2023-09-20T21:45:36.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:36 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/2731066412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41242a75-3273-4217-ad0d-c5da81c6b369"}]: dispatch 2023-09-20T21:45:36.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:36 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41242a75-3273-4217-ad0d-c5da81c6b369"}]: dispatch 2023-09-20T21:45:36.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:36 smithi089 ceph-mon[134090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41242a75-3273-4217-ad0d-c5da81c6b369"}]': finished 2023-09-20T21:45:36.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:36 smithi089 ceph-mon[134090]: osdmap e77: 12 total, 11 up, 12 in 2023-09-20T21:45:36.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:36 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:36.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:36 smithi089 ceph-mon[134090]: from='client.? 172.21.15.123:0/3946228981' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-09-20T21:45:38.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:38 smithi022 ceph-mon[97651]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:38.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:38 smithi123 ceph-mon[134433]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:38.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:38 smithi089 ceph-mon[134090]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:40.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:40 smithi123 ceph-mon[134433]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-09-20T21:45:40.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:40 smithi022 ceph-mon[97651]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-09-20T21:45:40.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:40 smithi089 ceph-mon[134090]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-09-20T21:45:42.485 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:42 smithi123 ceph-mon[134433]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-09-20T21:45:42.486 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-20T21:45:42.486 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:42.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:42 smithi022 ceph-mon[97651]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-09-20T21:45:42.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-20T21:45:42.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:42.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:42 smithi089 ceph-mon[134090]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-09-20T21:45:42.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-09-20T21:45:42.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:43.494 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:43 smithi123 ceph-mon[134433]: Deploying daemon osd.11 on smithi123 2023-09-20T21:45:43.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:43 smithi022 ceph-mon[97651]: Deploying daemon osd.11 on smithi123 2023-09-20T21:45:43.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:43 smithi089 ceph-mon[134090]: Deploying daemon osd.11 on smithi123 2023-09-20T21:45:44.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:44 smithi123 ceph-mon[134433]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-20T21:45:44.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:44 smithi022 ceph-mon[97651]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-20T21:45:44.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:44 smithi089 ceph-mon[134090]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-09-20T21:45:46.443 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:46 smithi123 ceph-mon[134433]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-20T21:45:46.443 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:46 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:46.444 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:46 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:46.444 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:46 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:46.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:46 smithi022 ceph-mon[97651]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-20T21:45:46.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:46 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:46.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:46 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:46.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:46 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:46.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:46 smithi089 ceph-mon[134090]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-09-20T21:45:46.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:46 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:46.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:46 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:46.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:46 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:48.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:48 smithi123 ceph-mon[134433]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-20T21:45:48.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:48 smithi022 ceph-mon[97651]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-20T21:45:48.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:48 smithi089 ceph-mon[134090]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-09-20T21:45:48.803 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 11 on host 'smithi123' 2023-09-20T21:45:48.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.802+0000 7ff4eca45700 1 -- 172.21.15.123:0/1542023748 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff4e8061220 con 0x7ff4d00a1300 2023-09-20T21:45:48.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 -- 172.21.15.123:0/1542023748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4d00a1300 msgr2=0x7ff4d00a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:48.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4d00a1300 0x7ff4d00a37c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80fffe0 tx=0x7ff4d8005fd0 comp rx=0 tx=0).stop 2023-09-20T21:45:48.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 -- 172.21.15.123:0/1542023748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e8100010 msgr2=0x7ff4e80ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:48.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e8100010 0x7ff4e80ffa80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c920 tx=0x7ff4dc00cc30 comp rx=0 tx=0).stop 2023-09-20T21:45:48.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 -- 172.21.15.123:0/1542023748 shutdown_connections 2023-09-20T21:45:48.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff4d00a1300 0x7ff4d00a37c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:48.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff4e8101440 0x7ff4e819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:48.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e8100010 0x7ff4e80ffa80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:48.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 --2- 172.21.15.123:0/1542023748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4e81009d0 0x7ff4e819b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:48.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 -- 172.21.15.123:0/1542023748 >> 172.21.15.123:0/1542023748 conn(0x7ff4e80faac0 msgr2=0x7ff4e80fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:48.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 -- 172.21.15.123:0/1542023748 shutdown_connections 2023-09-20T21:45:48.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:45:48.805+0000 7ff4eda47700 1 -- 172.21.15.123:0/1542023748 wait complete. 2023-09-20T21:45:49.279 DEBUG:teuthology.orchestra.run.smithi123:osd.11> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.11.service 2023-09-20T21:45:49.281 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-09-20T21:45:49.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd stat -f json 2023-09-20T21:45:49.310 INFO:journalctl@ceph.osd.11.smithi123.stdout:-- Logs begin at Wed 2023-09-20 21:17:48 UTC. -- 2023-09-20T21:45:49.726 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.726 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.726 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:49.727 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:49.727 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.727 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.727 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:49.727 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.728 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:49 smithi123 ceph-mon[134433]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-20T21:45:49.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:49.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:49.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:49.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:49 smithi022 ceph-mon[97651]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-20T21:45:49.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:49.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:49.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:45:49.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:49.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:49 smithi089 ceph-mon[134090]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-09-20T21:45:50.028 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:45:50.344 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 21:45:49 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T21:45:49.985+0000 7f4859588640 -1 osd.11 0 log_to_monitors true 2023-09-20T21:45:50.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:50 smithi123 ceph-mon[134433]: from='osd.11 [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-20T21:45:50.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:50 smithi123 ceph-mon[134433]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-20T21:45:50.921 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:50 smithi022 ceph-mon[97651]: from='osd.11 [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-20T21:45:50.922 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:50 smithi022 ceph-mon[97651]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-20T21:45:51.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:50 smithi089 ceph-mon[134090]: from='osd.11 [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-20T21:45:51.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:50 smithi089 ceph-mon[134090]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-09-20T21:45:51.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.540+0000 7f95edb14700 1 -- 172.21.15.22:0/3876759126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95e8101fd0 msgr2=0x7f95e8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:51.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.540+0000 7f95edb14700 1 --2- 172.21.15.22:0/3876759126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95e8101fd0 0x7f95e8102450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009990 tx=0x7f95dc009ca0 comp rx=0 tx=0).stop 2023-09-20T21:45:51.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.540+0000 7f95edb14700 1 -- 172.21.15.22:0/3876759126 shutdown_connections 2023-09-20T21:45:51.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.540+0000 7f95edb14700 1 --2- 172.21.15.22:0/3876759126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95e8102990 0x7f95e81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:51.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.540+0000 7f95edb14700 1 --2- 172.21.15.22:0/3876759126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8100e70 0x7f95e8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:51.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.540+0000 7f95edb14700 1 --2- 172.21.15.22:0/3876759126 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95e8101fd0 0x7f95e8102450 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:51.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.540+0000 7f95edb14700 1 -- 172.21.15.22:0/3876759126 >> 172.21.15.22:0/3876759126 conn(0x7f95e80fc3d0 msgr2=0x7f95e80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:51.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.540+0000 7f95edb14700 1 -- 172.21.15.22:0/3876759126 shutdown_connections 2023-09-20T21:45:51.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.540+0000 7f95edb14700 1 -- 172.21.15.22:0/3876759126 wait complete. 2023-09-20T21:45:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.541+0000 7f95edb14700 1 Processor -- start 2023-09-20T21:45:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95edb14700 1 -- start start 2023-09-20T21:45:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95edb14700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8100e70 0x7f95e8198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95edb14700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95e8101fd0 0x7f95e8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:51.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95edb14700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95e8102990 0x7f95e81a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:51.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95edb14700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e81a0890 con 0x7f95e8102990 2023-09-20T21:45:51.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95edb14700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e81a09d0 con 0x7f95e8101fd0 2023-09-20T21:45:51.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95edb14700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e81a0b90 con 0x7f95e8100e70 2023-09-20T21:45:51.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95e77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8100e70 0x7f95e8198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95e8102990 0x7f95e81a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95e77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8100e70 0x7f95e8198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:34474/0 (socket says 172.21.15.22:34474) 2023-09-20T21:45:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95e77fe700 1 -- 172.21.15.22:0/3696218545 learned_addr learned my addr 172.21.15.22:0/3696218545 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:45:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.542+0000 7f95e6ffd700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95e8101fd0 0x7f95e8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95e77fe700 1 -- 172.21.15.22:0/3696218545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95e8101fd0 msgr2=0x7f95e8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:51.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95e77fe700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95e8101fd0 0x7f95e8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:51.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95e77fe700 1 -- 172.21.15.22:0/3696218545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95e8102990 msgr2=0x7f95e81a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:51.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95e77fe700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95e8102990 0x7f95e81a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:51.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95e77fe700 1 -- 172.21.15.22:0/3696218545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0095f0 con 0x7f95e8100e70 2023-09-20T21:45:51.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95e7fff700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95e8102990 0x7f95e81a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:45:51.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95e77fe700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8100e70 0x7f95e8198d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f95d800d950 tx=0x7f95d800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:45:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95e4ff9700 1 -- 172.21.15.22:0/3696218545 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d800f890 con 0x7f95e8100e70 2023-09-20T21:45:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95e4ff9700 1 -- 172.21.15.22:0/3696218545 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95d8010460 con 0x7f95e8100e70 2023-09-20T21:45:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95edb14700 1 -- 172.21.15.22:0/3696218545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e81a0e70 con 0x7f95e8100e70 2023-09-20T21:45:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95e4ff9700 1 -- 172.21.15.22:0/3696218545 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d800e610 con 0x7f95e8100e70 2023-09-20T21:45:51.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.543+0000 7f95edb14700 1 -- 172.21.15.22:0/3696218545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e81a1290 con 0x7f95e8100e70 2023-09-20T21:45:51.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.544+0000 7f95edb14700 1 -- 172.21.15.22:0/3696218545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e8193470 con 0x7f95e8100e70 2023-09-20T21:45:51.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.546+0000 7f95e4ff9700 1 -- 172.21.15.22:0/3696218545 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f95d80108d0 con 0x7f95e8100e70 2023-09-20T21:45:51.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.547+0000 7f95e4ff9700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95d00a1380 0x7f95d00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:51.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.548+0000 7f95e4ff9700 1 -- 172.21.15.22:0/3696218545 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f95d80cc3e0 con 0x7f95e8100e70 2023-09-20T21:45:51.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.548+0000 7f95e6ffd700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95d00a1380 0x7f95d00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:51.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.548+0000 7f95e6ffd700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95d00a1380 0x7f95d00a3840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00b5c0 tx=0x7f95dc005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:45:51.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.550+0000 7f95e4ff9700 1 -- 172.21.15.22:0/3696218545 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f95d8095fe0 con 0x7f95e8100e70 2023-09-20T21:45:51.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.762+0000 7f95edb14700 1 -- 172.21.15.22:0/3696218545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f95e810c680 con 0x7f95e8100e70 2023-09-20T21:45:51.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.762+0000 7f95e4ff9700 1 -- 172.21.15.22:0/3696218545 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7f95d8099600 con 0x7f95e8100e70 2023-09-20T21:45:51.766 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:45:51.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 -- 172.21.15.22:0/3696218545 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95d00a1380 msgr2=0x7f95d00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:51.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95d00a1380 0x7f95d00a3840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00b5c0 tx=0x7f95dc005960 comp rx=0 tx=0).stop 2023-09-20T21:45:51.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 -- 172.21.15.22:0/3696218545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8100e70 msgr2=0x7f95e8198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:51.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8100e70 0x7f95e8198d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f95d800d950 tx=0x7f95d800dc60 comp rx=0 tx=0).stop 2023-09-20T21:45:51.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 -- 172.21.15.22:0/3696218545 shutdown_connections 2023-09-20T21:45:51.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95e8102990 0x7f95e81a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:51.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e8100e70 0x7f95e8198d40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:51.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95d00a1380 0x7f95d00a3840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:51.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 --2- 172.21.15.22:0/3696218545 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95e8101fd0 0x7f95e8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:51.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 -- 172.21.15.22:0/3696218545 >> 172.21.15.22:0/3696218545 conn(0x7f95e80fc3d0 msgr2=0x7f95e81079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:51.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 -- 172.21.15.22:0/3696218545 shutdown_connections 2023-09-20T21:45:51.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:51.765+0000 7f95edb14700 1 -- 172.21.15.22:0/3696218545 wait complete. 2023-09-20T21:45:52.093 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 21:45:51 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T21:45:51.775+0000 7f484ec00700 -1 osd.11 0 waiting for initial osdmap 2023-09-20T21:45:52.093 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 21:45:51 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T21:45:51.787+0000 7f48491f2700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-20T21:45:52.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:51 smithi123 ceph-mon[134433]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-20T21:45:52.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:51 smithi123 ceph-mon[134433]: osdmap e78: 12 total, 11 up, 12 in 2023-09-20T21:45:52.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:51 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:52.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:51 smithi123 ceph-mon[134433]: from='osd.11 [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:45:52.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:51 smithi123 ceph-mon[134433]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:45:52.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:51 smithi123 ceph-mon[134433]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:52.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:51 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3696218545' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-20T21:45:52.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:51 smithi089 ceph-mon[134090]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-20T21:45:52.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:51 smithi089 ceph-mon[134090]: osdmap e78: 12 total, 11 up, 12 in 2023-09-20T21:45:52.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:51 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:52.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:51 smithi089 ceph-mon[134090]: from='osd.11 [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:45:52.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:51 smithi089 ceph-mon[134090]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:45:52.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:51 smithi089 ceph-mon[134090]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:52.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:51 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3696218545' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-20T21:45:52.203 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:51 smithi022 ceph-mon[97651]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-09-20T21:45:52.203 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:51 smithi022 ceph-mon[97651]: osdmap e78: 12 total, 11 up, 12 in 2023-09-20T21:45:52.203 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:51 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:52.203 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:51 smithi022 ceph-mon[97651]: from='osd.11 [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:45:52.203 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:51 smithi022 ceph-mon[97651]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2023-09-20T21:45:52.204 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:51 smithi022 ceph-mon[97651]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:52.204 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:51 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3696218545' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-20T21:45:52.205 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":78,"num_osds":12,"num_up_osds":11,"osd_up_since":1695246331,"num_in_osds":12,"osd_in_since":1695246335,"num_remapped_pgs":0} 2023-09-20T21:45:53.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:45:53.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: osdmap e79: 12 total, 11 up, 12 in 2023-09-20T21:45:53.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:53.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:53.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: Detected new or changed devices on smithi123 2023-09-20T21:45:53.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi123 to 4292M 2023-09-20T21:45:53.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:53.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:53.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:52 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:45:53.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: osdmap e79: 12 total, 11 up, 12 in 2023-09-20T21:45:53.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:53.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:53.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: Detected new or changed devices on smithi123 2023-09-20T21:45:53.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi123 to 4292M 2023-09-20T21:45:53.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:53.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:53.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:52 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd stat -f json 2023-09-20T21:45:53.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2023-09-20T21:45:53.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: osdmap e79: 12 total, 11 up, 12 in 2023-09-20T21:45:53.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:53.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:53.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: Detected new or changed devices on smithi123 2023-09-20T21:45:53.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:45:53.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi123 to 4292M 2023-09-20T21:45:53.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:45:53.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:45:53.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:52 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:45:53.956 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:45:54.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:53 smithi123 ceph-mon[134433]: purged_snaps scrub starts 2023-09-20T21:45:54.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:53 smithi123 ceph-mon[134433]: purged_snaps scrub ok 2023-09-20T21:45:54.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:54.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:53 smithi123 ceph-mon[134433]: osd.11 [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] boot 2023-09-20T21:45:54.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:53 smithi123 ceph-mon[134433]: osdmap e80: 12 total, 12 up, 12 in 2023-09-20T21:45:54.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:53 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:54.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:53 smithi123 ceph-mon[134433]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:54.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:53 smithi089 ceph-mon[134090]: purged_snaps scrub starts 2023-09-20T21:45:54.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:53 smithi089 ceph-mon[134090]: purged_snaps scrub ok 2023-09-20T21:45:54.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:53 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:54.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:53 smithi089 ceph-mon[134090]: osd.11 [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] boot 2023-09-20T21:45:54.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:53 smithi089 ceph-mon[134090]: osdmap e80: 12 total, 12 up, 12 in 2023-09-20T21:45:54.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:53 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:54.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:53 smithi089 ceph-mon[134090]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:54.210 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:53 smithi022 ceph-mon[97651]: purged_snaps scrub starts 2023-09-20T21:45:54.210 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:53 smithi022 ceph-mon[97651]: purged_snaps scrub ok 2023-09-20T21:45:54.210 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:54.211 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:53 smithi022 ceph-mon[97651]: osd.11 [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] boot 2023-09-20T21:45:54.211 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:53 smithi022 ceph-mon[97651]: osdmap e80: 12 total, 12 up, 12 in 2023-09-20T21:45:54.211 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:53 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-09-20T21:45:54.211 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:53 smithi022 ceph-mon[97651]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail 2023-09-20T21:45:55.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:54 smithi123 ceph-mon[134433]: osdmap e81: 12 total, 12 up, 12 in 2023-09-20T21:45:55.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:54 smithi089 ceph-mon[134090]: osdmap e81: 12 total, 12 up, 12 in 2023-09-20T21:45:55.211 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:54 smithi022 ceph-mon[97651]: osdmap e81: 12 total, 12 up, 12 in 2023-09-20T21:45:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.465+0000 7faf5011e700 1 -- 172.21.15.22:0/2637685221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf48101fd0 msgr2=0x7faf48102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.465+0000 7faf5011e700 1 --2- 172.21.15.22:0/2637685221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf48101fd0 0x7faf48102450 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf38009990 tx=0x7faf38009ca0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.465+0000 7faf5011e700 1 -- 172.21.15.22:0/2637685221 shutdown_connections 2023-09-20T21:45:55.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.465+0000 7faf5011e700 1 --2- 172.21.15.22:0/2637685221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf48102990 0x7faf481092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.465+0000 7faf5011e700 1 --2- 172.21.15.22:0/2637685221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48100e70 0x7faf48101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.465+0000 7faf5011e700 1 --2- 172.21.15.22:0/2637685221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf48101fd0 0x7faf48102450 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.465+0000 7faf5011e700 1 -- 172.21.15.22:0/2637685221 >> 172.21.15.22:0/2637685221 conn(0x7faf480fc3d0 msgr2=0x7faf480fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:55.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.465+0000 7faf5011e700 1 -- 172.21.15.22:0/2637685221 shutdown_connections 2023-09-20T21:45:55.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.465+0000 7faf5011e700 1 -- 172.21.15.22:0/2637685221 wait complete. 2023-09-20T21:45:55.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.466+0000 7faf5011e700 1 Processor -- start 2023-09-20T21:45:55.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf5011e700 1 -- start start 2023-09-20T21:45:55.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf5011e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf48100e70 0x7faf48074b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:55.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf5011e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101fd0 0x7faf480731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:55.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf5011e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf48102990 0x7faf48073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:55.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf5011e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf48073e90 con 0x7faf48102990 2023-09-20T21:45:55.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf5011e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf48074000 con 0x7faf48100e70 2023-09-20T21:45:55.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf5011e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf481a0be0 con 0x7faf48101fd0 2023-09-20T21:45:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf4deba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf48100e70 0x7faf48074b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf4e6bb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf48102990 0x7faf48073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf4deba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf48100e70 0x7faf48074b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60974/0 (socket says 172.21.15.22:60974) 2023-09-20T21:45:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf4deba700 1 -- 172.21.15.22:0/2663655267 learned_addr learned my addr 172.21.15.22:0/2663655267 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:45:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.467+0000 7faf4d6b9700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101fd0 0x7faf480731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:55.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf4e6bb700 1 -- 172.21.15.22:0/2663655267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101fd0 msgr2=0x7faf480731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:55.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf4e6bb700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101fd0 0x7faf480731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf4e6bb700 1 -- 172.21.15.22:0/2663655267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf48100e70 msgr2=0x7faf48074b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:55.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf4e6bb700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf48100e70 0x7faf48074b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf4e6bb700 1 -- 172.21.15.22:0/2663655267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf40009c40 con 0x7faf48102990 2023-09-20T21:45:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf4deba700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf48100e70 0x7faf48074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:45:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf4e6bb700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf48102990 0x7faf48073710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf4000efb0 tx=0x7faf4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:45:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf3effd700 1 -- 172.21.15.22:0/2663655267 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40003a00 con 0x7faf48102990 2023-09-20T21:45:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf5011e700 1 -- 172.21.15.22:0/2663655267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf380095f0 con 0x7faf48102990 2023-09-20T21:45:55.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf3effd700 1 -- 172.21.15.22:0/2663655267 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf4000f040 con 0x7faf48102990 2023-09-20T21:45:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf3effd700 1 -- 172.21.15.22:0/2663655267 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40004540 con 0x7faf48102990 2023-09-20T21:45:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.468+0000 7faf5011e700 1 -- 172.21.15.22:0/2663655267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf481a1220 con 0x7faf48102990 2023-09-20T21:45:55.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.469+0000 7faf5011e700 1 -- 172.21.15.22:0/2663655267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4810c6d0 con 0x7faf48102990 2023-09-20T21:45:55.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.471+0000 7faf3effd700 1 -- 172.21.15.22:0/2663655267 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7faf40003bf0 con 0x7faf48102990 2023-09-20T21:45:55.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.473+0000 7faf3effd700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faf340a1380 0x7faf340a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:55.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.473+0000 7faf3effd700 1 -- 172.21.15.22:0/2663655267 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faf400cc370 con 0x7faf48102990 2023-09-20T21:45:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.473+0000 7faf4deba700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faf340a1380 0x7faf340a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.474+0000 7faf4deba700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faf340a1380 0x7faf340a3840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf44011550 tx=0x7faf44009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:45:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.477+0000 7faf3effd700 1 -- 172.21.15.22:0/2663655267 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7faf40095ed0 con 0x7faf48102990 2023-09-20T21:45:55.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.683+0000 7faf5011e700 1 -- 172.21.15.22:0/2663655267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7faf4810c8c0 con 0x7faf48102990 2023-09-20T21:45:55.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.683+0000 7faf3effd700 1 -- 172.21.15.22:0/2663655267 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7faf400994f0 con 0x7faf48102990 2023-09-20T21:45:55.684 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:45:55.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 -- 172.21.15.22:0/2663655267 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faf340a1380 msgr2=0x7faf340a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:55.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faf340a1380 0x7faf340a3840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf44011550 tx=0x7faf44009260 comp rx=0 tx=0).stop 2023-09-20T21:45:55.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 -- 172.21.15.22:0/2663655267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf48102990 msgr2=0x7faf48073710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:55.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf48102990 0x7faf48073710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf4000efb0 tx=0x7faf4000c3a0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 -- 172.21.15.22:0/2663655267 shutdown_connections 2023-09-20T21:45:55.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf48102990 0x7faf48073710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faf340a1380 0x7faf340a3840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faf48100e70 0x7faf48074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 --2- 172.21.15.22:0/2663655267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf48101fd0 0x7faf480731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:55.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 -- 172.21.15.22:0/2663655267 >> 172.21.15.22:0/2663655267 conn(0x7faf480fc3d0 msgr2=0x7faf480fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:55.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 -- 172.21.15.22:0/2663655267 shutdown_connections 2023-09-20T21:45:55.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:55.686+0000 7faf5011e700 1 -- 172.21.15.22:0/2663655267 wait complete. 2023-09-20T21:45:55.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:55 smithi022 ceph-mon[97651]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:45:55.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:55 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2663655267' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-20T21:45:56.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:55 smithi123 ceph-mon[134433]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:45:56.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:55 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2663655267' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-20T21:45:56.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:55 smithi089 ceph-mon[134090]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:45:56.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:55 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2663655267' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-09-20T21:45:56.134 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1695246352,"num_in_osds":12,"osd_in_since":1695246335,"num_remapped_pgs":0} 2023-09-20T21:45:56.134 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd dump --format=json 2023-09-20T21:45:56.908 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:45:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.413+0000 7f1e5eed4700 1 -- 172.21.15.22:0/830540247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e58073b30 msgr2=0x7f1e5810d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.413+0000 7f1e5eed4700 1 --2- 172.21.15.22:0/830540247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e58073b30 0x7f1e5810d100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009990 tx=0x7f1e44009ca0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.414+0000 7f1e5eed4700 1 -- 172.21.15.22:0/830540247 shutdown_connections 2023-09-20T21:45:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.414+0000 7f1e5eed4700 1 --2- 172.21.15.22:0/830540247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5810d640 0x7f1e5810fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.414+0000 7f1e5eed4700 1 --2- 172.21.15.22:0/830540247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e58073210 0x7f1e580735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.414+0000 7f1e5eed4700 1 --2- 172.21.15.22:0/830540247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e58073b30 0x7f1e5810d100 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.414+0000 7f1e5eed4700 1 -- 172.21.15.22:0/830540247 >> 172.21.15.22:0/830540247 conn(0x7f1e580fc8d0 msgr2=0x7f1e580fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.414+0000 7f1e5eed4700 1 -- 172.21.15.22:0/830540247 shutdown_connections 2023-09-20T21:45:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.414+0000 7f1e5eed4700 1 -- 172.21.15.22:0/830540247 wait complete. 2023-09-20T21:45:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.415+0000 7f1e5eed4700 1 Processor -- start 2023-09-20T21:45:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.415+0000 7f1e5eed4700 1 -- start start 2023-09-20T21:45:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.415+0000 7f1e5eed4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e58073210 0x7f1e58199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:58.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.415+0000 7f1e5eed4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e58073b30 0x7f1e58199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:58.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.415+0000 7f1e5eed4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5810d640 0x7f1e5819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:58.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.415+0000 7f1e5eed4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5819e100 con 0x7f1e58073210 2023-09-20T21:45:58.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.415+0000 7f1e5eed4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5819e270 con 0x7f1e5810d640 2023-09-20T21:45:58.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.415+0000 7f1e5eed4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5819e430 con 0x7f1e58073b30 2023-09-20T21:45:58.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e5cc70700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e58073210 0x7f1e58199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e5cc70700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e58073210 0x7f1e58199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54980/0 (socket says 172.21.15.22:54980) 2023-09-20T21:45:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e5cc70700 1 -- 172.21.15.22:0/1626090181 learned_addr learned my addr 172.21.15.22:0/1626090181 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:45:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e57fff700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e58073b30 0x7f1e58199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e5d471700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5810d640 0x7f1e5819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e5cc70700 1 -- 172.21.15.22:0/1626090181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e58073b30 msgr2=0x7f1e58199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e5cc70700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e58073b30 0x7f1e58199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e5cc70700 1 -- 172.21.15.22:0/1626090181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5810d640 msgr2=0x7f1e5819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e5cc70700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5810d640 0x7f1e5819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e5cc70700 1 -- 172.21.15.22:0/1626090181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440095f0 con 0x7f1e58073210 2023-09-20T21:45:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.416+0000 7f1e5cc70700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e58073210 0x7f1e58199140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00d980 tx=0x7f1e4c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:45:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.417+0000 7f1e55ffb700 1 -- 172.21.15.22:0/1626090181 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c004220 con 0x7f1e58073210 2023-09-20T21:45:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.417+0000 7f1e5eed4700 1 -- 172.21.15.22:0/1626090181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5819e710 con 0x7f1e58073210 2023-09-20T21:45:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.417+0000 7f1e55ffb700 1 -- 172.21.15.22:0/1626090181 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e4c007640 con 0x7f1e58073210 2023-09-20T21:45:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.417+0000 7f1e55ffb700 1 -- 172.21.15.22:0/1626090181 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c005690 con 0x7f1e58073210 2023-09-20T21:45:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.417+0000 7f1e5eed4700 1 -- 172.21.15.22:0/1626090181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e581a5f70 con 0x7f1e58073210 2023-09-20T21:45:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.418+0000 7f1e5eed4700 1 -- 172.21.15.22:0/1626090181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5804eae0 con 0x7f1e58073210 2023-09-20T21:45:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.420+0000 7f1e55ffb700 1 -- 172.21.15.22:0/1626090181 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1e4c004750 con 0x7f1e58073210 2023-09-20T21:45:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.422+0000 7f1e55ffb700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e480a1300 0x7f1e480a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:45:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.422+0000 7f1e55ffb700 1 -- 172.21.15.22:0/1626090181 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1e4c0cca80 con 0x7f1e58073210 2023-09-20T21:45:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.422+0000 7f1e57fff700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e480a1300 0x7f1e480a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:45:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.423+0000 7f1e57fff700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e480a1300 0x7f1e480a37c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e5819a760 tx=0x7f1e44005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:45:58.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.425+0000 7f1e55ffb700 1 -- 172.21.15.22:0/1626090181 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1e4c096550 con 0x7f1e58073210 2023-09-20T21:45:58.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:58 smithi022 ceph-mon[97651]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:45:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:58 smithi123 ceph-mon[134433]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:45:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:58 smithi089 ceph-mon[134090]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:45:58.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.635+0000 7f1e5eed4700 1 -- 172.21.15.22:0/1626090181 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e5819a4a0 con 0x7f1e58073210 2023-09-20T21:45:58.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.637+0000 7f1e55ffb700 1 -- 172.21.15.22:0/1626090181 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18375 (secure 0 0 0) 0x7f1e4c099b70 con 0x7f1e58073210 2023-09-20T21:45:58.637 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:45:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":81,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:45:53.805187+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:45:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 -- 172.21.15.22:0/1626090181 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e480a1300 msgr2=0x7f1e480a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e480a1300 0x7f1e480a37c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1e5819a760 tx=0x7f1e44005fd0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 -- 172.21.15.22:0/1626090181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e58073210 msgr2=0x7f1e58199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:45:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e58073210 0x7f1e58199140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00d980 tx=0x7f1e4c00dc90 comp rx=0 tx=0).stop 2023-09-20T21:45:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 -- 172.21.15.22:0/1626090181 shutdown_connections 2023-09-20T21:45:58.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e480a1300 0x7f1e480a37c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e5810d640 0x7f1e5819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e58073210 0x7f1e58199140 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 --2- 172.21.15.22:0/1626090181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e58073b30 0x7f1e58199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:45:58.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 -- 172.21.15.22:0/1626090181 >> 172.21.15.22:0/1626090181 conn(0x7f1e580fc8d0 msgr2=0x7f1e5810f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:45:58.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 -- 172.21.15.22:0/1626090181 shutdown_connections 2023-09-20T21:45:58.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:45:58.639+0000 7f1e5eed4700 1 -- 172.21.15.22:0/1626090181 wait complete. 2023-09-20T21:45:59.184 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-20T21:45:59.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd pool get .mgr pg_num 2023-09-20T21:45:59.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:45:59 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1626090181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:45:59.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:45:59 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1626090181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:45:59.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:45:59 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1626090181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:45:59.917 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:46:00.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:00 smithi022 ceph-mon[97651]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:00 smithi123 ceph-mon[134433]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:00 smithi089 ceph-mon[134090]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:01.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.493+0000 7fccfbe98700 1 -- 172.21.15.22:0/2938115513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccf4106140 msgr2=0x7fccf410dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:01.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.493+0000 7fccfbe98700 1 --2- 172.21.15.22:0/2938115513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccf4106140 0x7fccf410dd10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b0a0 tx=0x7fccec00b3b0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.493+0000 7fccfbe98700 1 -- 172.21.15.22:0/2938115513 shutdown_connections 2023-09-20T21:46:01.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.493+0000 7fccfbe98700 1 --2- 172.21.15.22:0/2938115513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccf4106140 0x7fccf410dd10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.493+0000 7fccfbe98700 1 --2- 172.21.15.22:0/2938115513 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf4101060 0x7fccf4101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.493+0000 7fccfbe98700 1 --2- 172.21.15.22:0/2938115513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf4101a10 0x7fccf4105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.493+0000 7fccfbe98700 1 -- 172.21.15.22:0/2938115513 >> 172.21.15.22:0/2938115513 conn(0x7fccf40fc8d0 msgr2=0x7fccf40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:01.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.493+0000 7fccfbe98700 1 -- 172.21.15.22:0/2938115513 shutdown_connections 2023-09-20T21:46:01.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.494+0000 7fccfbe98700 1 -- 172.21.15.22:0/2938115513 wait complete. 2023-09-20T21:46:01.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.495+0000 7fccfbe98700 1 Processor -- start 2023-09-20T21:46:01.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.495+0000 7fccfbe98700 1 -- start start 2023-09-20T21:46:01.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccfbe98700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf4101060 0x7fccf40730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:01.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccfbe98700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf4101a10 0x7fccf4073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:01.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccfbe98700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccf4106140 0x7fccf4073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:01.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccfbe98700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf41a5440 con 0x7fccf4106140 2023-09-20T21:46:01.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccfbe98700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf41a55b0 con 0x7fccf4101060 2023-09-20T21:46:01.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccfbe98700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf41a5770 con 0x7fccf4101a10 2023-09-20T21:46:01.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccf9c34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf4101060 0x7fccf40730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:01.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccf9c34700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf4101060 0x7fccf40730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:59984/0 (socket says 172.21.15.22:59984) 2023-09-20T21:46:01.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccf9c34700 1 -- 172.21.15.22:0/1769889554 learned_addr learned my addr 172.21.15.22:0/1769889554 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:01.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccfa435700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccf4106140 0x7fccf4073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:01.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.496+0000 7fccf9433700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf4101a10 0x7fccf4073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:01.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fccf9c34700 1 -- 172.21.15.22:0/1769889554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf4101a10 msgr2=0x7fccf4073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:01.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fccf9c34700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf4101a10 0x7fccf4073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fccf9c34700 1 -- 172.21.15.22:0/1769889554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccf4106140 msgr2=0x7fccf4073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:01.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fccf9c34700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccf4106140 0x7fccf4073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fccf9c34700 1 -- 172.21.15.22:0/1769889554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec009d00 con 0x7fccf4101060 2023-09-20T21:46:01.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fccf9c34700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf4101060 0x7fccf40730e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fccf000e9a0 tx=0x7fccf000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:01.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fcceaffd700 1 -- 172.21.15.22:0/1769889554 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0009790 con 0x7fccf4101060 2023-09-20T21:46:01.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fcceaffd700 1 -- 172.21.15.22:0/1769889554 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccf0010430 con 0x7fccf4101060 2023-09-20T21:46:01.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fccfbe98700 1 -- 172.21.15.22:0/1769889554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf41a5a50 con 0x7fccf4101060 2023-09-20T21:46:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fcceaffd700 1 -- 172.21.15.22:0/1769889554 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0018680 con 0x7fccf4101060 2023-09-20T21:46:01.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.497+0000 7fccfbe98700 1 -- 172.21.15.22:0/1769889554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf41a5fa0 con 0x7fccf4101060 2023-09-20T21:46:01.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.498+0000 7fccfbe98700 1 -- 172.21.15.22:0/1769889554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf404eae0 con 0x7fccf4101060 2023-09-20T21:46:01.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.500+0000 7fcceaffd700 1 -- 172.21.15.22:0/1769889554 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fccf00105a0 con 0x7fccf4101060 2023-09-20T21:46:01.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.502+0000 7fcceaffd700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcce00a1380 0x7fcce00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:01.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.502+0000 7fcceaffd700 1 -- 172.21.15.22:0/1769889554 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fccf0014070 con 0x7fccf4101060 2023-09-20T21:46:01.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.502+0000 7fccf9433700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcce00a1380 0x7fcce00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:01.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.503+0000 7fccf9433700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcce00a1380 0x7fcce00a3840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fccf40748d0 tx=0x7fcce4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:01.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.504+0000 7fcceaffd700 1 -- 172.21.15.22:0/1769889554 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fccf00966a0 con 0x7fccf4101060 2023-09-20T21:46:01.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.715+0000 7fccfbe98700 1 -- 172.21.15.22:0/1769889554 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fccf4074610 con 0x7fccf4101060 2023-09-20T21:46:01.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.716+0000 7fcceaffd700 1 -- 172.21.15.22:0/1769889554 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fccf0099cc0 con 0x7fccf4101060 2023-09-20T21:46:01.717 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:46:01.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.718+0000 7fccfbe98700 1 -- 172.21.15.22:0/1769889554 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcce00a1380 msgr2=0x7fcce00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:01.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.718+0000 7fccfbe98700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcce00a1380 0x7fcce00a3840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fccf40748d0 tx=0x7fcce4009260 comp rx=0 tx=0).stop 2023-09-20T21:46:01.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.718+0000 7fccfbe98700 1 -- 172.21.15.22:0/1769889554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf4101060 msgr2=0x7fccf40730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:01.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.718+0000 7fccfbe98700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf4101060 0x7fccf40730e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fccf000e9a0 tx=0x7fccf000ecb0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.719+0000 7fccfbe98700 1 -- 172.21.15.22:0/1769889554 shutdown_connections 2023-09-20T21:46:01.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.719+0000 7fccfbe98700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fccf4106140 0x7fccf4073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.719+0000 7fccfbe98700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcce00a1380 0x7fcce00a3840 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.719+0000 7fccfbe98700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fccf4101060 0x7fccf40730e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.719+0000 7fccfbe98700 1 --2- 172.21.15.22:0/1769889554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccf4101a10 0x7fccf4073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:01.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.719+0000 7fccfbe98700 1 -- 172.21.15.22:0/1769889554 >> 172.21.15.22:0/1769889554 conn(0x7fccf40fc8d0 msgr2=0x7fccf40fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:01.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.719+0000 7fccfbe98700 1 -- 172.21.15.22:0/1769889554 shutdown_connections 2023-09-20T21:46:01.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:01.719+0000 7fccfbe98700 1 -- 172.21.15.22:0/1769889554 wait complete. 2023-09-20T21:46:02.196 INFO:tasks.cephadm:Adding mds.a on smithi022 2023-09-20T21:46:02.197 INFO:tasks.cephadm:Adding mds.d on smithi022 2023-09-20T21:46:02.197 INFO:tasks.cephadm:Adding mds.g on smithi022 2023-09-20T21:46:02.197 INFO:tasks.cephadm:Adding mds.j on smithi022 2023-09-20T21:46:02.197 INFO:tasks.cephadm:Adding mds.b on smithi089 2023-09-20T21:46:02.198 INFO:tasks.cephadm:Adding mds.e on smithi089 2023-09-20T21:46:02.198 INFO:tasks.cephadm:Adding mds.h on smithi089 2023-09-20T21:46:02.198 INFO:tasks.cephadm:Adding mds.k on smithi089 2023-09-20T21:46:02.199 INFO:tasks.cephadm:Adding mds.c on smithi123 2023-09-20T21:46:02.199 INFO:tasks.cephadm:Adding mds.f on smithi123 2023-09-20T21:46:02.199 INFO:tasks.cephadm:Adding mds.i on smithi123 2023-09-20T21:46:02.199 INFO:tasks.cephadm:Adding mds.l on smithi123 2023-09-20T21:46:02.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph orch apply mds all '12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l' 2023-09-20T21:46:02.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:02 smithi022 ceph-mon[97651]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:02 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1769889554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:02 smithi123 ceph-mon[134433]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:02 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1769889554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:02.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:02 smithi089 ceph-mon[134090]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:02.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:02 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1769889554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:02.771 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T21:46:03.801 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:03 smithi123 ceph-mon[134433]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:03.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:03 smithi089 ceph-mon[134090]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 -- 172.21.15.123:0/2767157181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e700ff4f0 msgr2=0x7f1e700ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 --2- 172.21.15.123:0/2767157181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e700ff4f0 0x7f1e700ff910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009990 tx=0x7f1e60009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:03.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 -- 172.21.15.123:0/2767157181 shutdown_connections 2023-09-20T21:46:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 --2- 172.21.15.123:0/2767157181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e70101090 0x7f1e70107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 --2- 172.21.15.123:0/2767157181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e700ff4f0 0x7f1e700ff910 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 --2- 172.21.15.123:0/2767157181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e701006f0 0x7f1e70100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 -- 172.21.15.123:0/2767157181 >> 172.21.15.123:0/2767157181 conn(0x7f1e700faa90 msgr2=0x7f1e700fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 -- 172.21.15.123:0/2767157181 shutdown_connections 2023-09-20T21:46:03.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 -- 172.21.15.123:0/2767157181 wait complete. 2023-09-20T21:46:03.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 Processor -- start 2023-09-20T21:46:03.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.978+0000 7f1e779ba700 1 -- start start 2023-09-20T21:46:03.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e779ba700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e700ff4f0 0x7f1e70196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:03.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e779ba700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e701006f0 0x7f1e70196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e779ba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e70101090 0x7f1e7019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e779ba700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7019e620 con 0x7f1e701006f0 2023-09-20T21:46:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e779ba700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7019e760 con 0x7f1e700ff4f0 2023-09-20T21:46:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e779ba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7019e8f0 con 0x7f1e70101090 2023-09-20T21:46:03.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75f57700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e70101090 0x7f1e7019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:03.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75f57700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e70101090 0x7f1e7019e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34326/0 (socket says 172.21.15.123:34326) 2023-09-20T21:46:03.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75f57700 1 -- 172.21.15.123:0/3035272864 learned_addr learned my addr 172.21.15.123:0/3035272864 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-09-20T21:46:03.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75756700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e700ff4f0 0x7f1e70196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:03.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75f57700 1 -- 172.21.15.123:0/3035272864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e700ff4f0 msgr2=0x7f1e70196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75f57700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e700ff4f0 0x7f1e70196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75f57700 1 -- 172.21.15.123:0/3035272864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e701006f0 msgr2=0x7f1e70196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75f57700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e701006f0 0x7f1e70196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75f57700 1 -- 172.21.15.123:0/3035272864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e600095f0 con 0x7f1e70101090 2023-09-20T21:46:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75756700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e700ff4f0 0x7f1e70196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:46:03.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e75f57700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e70101090 0x7f1e7019e050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1e68007f10 tx=0x7f1e6800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:03.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e667fc700 1 -- 172.21.15.123:0/3035272864 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68017070 con 0x7f1e70101090 2023-09-20T21:46:03.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e667fc700 1 -- 172.21.15.123:0/3035272864 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e6800f040 con 0x7f1e70101090 2023-09-20T21:46:03.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e779ba700 1 -- 172.21.15.123:0/3035272864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7019ebd0 con 0x7f1e70101090 2023-09-20T21:46:03.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e667fc700 1 -- 172.21.15.123:0/3035272864 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68005740 con 0x7f1e70101090 2023-09-20T21:46:03.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.979+0000 7f1e779ba700 1 -- 172.21.15.123:0/3035272864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7019f080 con 0x7f1e70101090 2023-09-20T21:46:03.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.980+0000 7f1e779ba700 1 -- 172.21.15.123:0/3035272864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e70106a40 con 0x7f1e70101090 2023-09-20T21:46:03.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.981+0000 7f1e667fc700 1 -- 172.21.15.123:0/3035272864 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1e680058a0 con 0x7f1e70101090 2023-09-20T21:46:03.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.982+0000 7f1e667fc700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e5c0a1300 0x7f1e5c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:03.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.982+0000 7f1e667fc700 1 -- 172.21.15.123:0/3035272864 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1e680cc020 con 0x7f1e70101090 2023-09-20T21:46:03.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.982+0000 7f1e75756700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e5c0a1300 0x7f1e5c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:03.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.983+0000 7f1e75756700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e5c0a1300 0x7f1e5c0a37c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006010 tx=0x7f1e6000b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:03.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:03.987+0000 7f1e667fc700 1 -- 172.21.15.123:0/3035272864 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1e68095b90 con 0x7f1e70101090 2023-09-20T21:46:03.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:03 smithi022 ceph-mon[97651]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:04.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.183+0000 7f1e779ba700 1 -- 172.21.15.123:0/3035272864 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e7002d330 con 0x7f1e5c0a1300 2023-09-20T21:46:04.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.192+0000 7f1e667fc700 1 -- 172.21.15.123:0/3035272864 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f1e7002d330 con 0x7f1e5c0a1300 2023-09-20T21:46:04.192 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mds.all update... 2023-09-20T21:46:04.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 -- 172.21.15.123:0/3035272864 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e5c0a1300 msgr2=0x7f1e5c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:04.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e5c0a1300 0x7f1e5c0a37c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006010 tx=0x7f1e6000b540 comp rx=0 tx=0).stop 2023-09-20T21:46:04.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 -- 172.21.15.123:0/3035272864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e70101090 msgr2=0x7f1e7019e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:04.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e70101090 0x7f1e7019e050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1e68007f10 tx=0x7f1e6800d3b0 comp rx=0 tx=0).stop 2023-09-20T21:46:04.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 -- 172.21.15.123:0/3035272864 shutdown_connections 2023-09-20T21:46:04.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e70101090 0x7f1e7019e050 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:04.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1e5c0a1300 0x7f1e5c0a37c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:04.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e700ff4f0 0x7f1e70196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:04.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 --2- 172.21.15.123:0/3035272864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e701006f0 0x7f1e70196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:04.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 -- 172.21.15.123:0/3035272864 >> 172.21.15.123:0/3035272864 conn(0x7f1e700faa90 msgr2=0x7f1e70103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:04.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 -- 172.21.15.123:0/3035272864 shutdown_connections 2023-09-20T21:46:04.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-09-20T21:46:04.194+0000 7f1e779ba700 1 -- 172.21.15.123:0/3035272864 wait complete. 2023-09-20T21:46:04.624 DEBUG:teuthology.orchestra.run.smithi022:mds.a> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.a.service 2023-09-20T21:46:04.630 DEBUG:teuthology.orchestra.run.smithi022:mds.d> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.d.service 2023-09-20T21:46:04.675 DEBUG:teuthology.orchestra.run.smithi022:mds.g> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.g.service 2023-09-20T21:46:04.712 INFO:journalctl@ceph.mds.a.smithi022.stdout:-- Logs begin at Wed 2023-09-20 21:22:06 UTC. -- 2023-09-20T21:46:04.716 DEBUG:teuthology.orchestra.run.smithi022:mds.j> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.j.service 2023-09-20T21:46:04.752 INFO:journalctl@ceph.mds.d.smithi022.stdout:-- Logs begin at Wed 2023-09-20 21:22:06 UTC. -- 2023-09-20T21:46:04.756 DEBUG:teuthology.orchestra.run.smithi089:mds.b> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.b.service 2023-09-20T21:46:04.758 DEBUG:teuthology.orchestra.run.smithi089:mds.e> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.e.service 2023-09-20T21:46:04.789 INFO:journalctl@ceph.mds.b.smithi089.stdout:-- Logs begin at Wed 2023-09-20 21:17:56 UTC. -- 2023-09-20T21:46:04.790 DEBUG:teuthology.orchestra.run.smithi089:mds.h> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.h.service 2023-09-20T21:46:04.798 INFO:journalctl@ceph.mds.g.smithi022.stdout:-- Logs begin at Wed 2023-09-20 21:22:06 UTC. -- 2023-09-20T21:46:04.815 INFO:journalctl@ceph.mds.e.smithi089.stdout:-- Logs begin at Wed 2023-09-20 21:17:56 UTC. -- 2023-09-20T21:46:04.816 DEBUG:teuthology.orchestra.run.smithi089:mds.k> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.k.service 2023-09-20T21:46:04.828 INFO:journalctl@ceph.mds.j.smithi022.stdout:-- Logs begin at Wed 2023-09-20 21:22:06 UTC. -- 2023-09-20T21:46:04.842 INFO:journalctl@ceph.mds.h.smithi089.stdout:-- Logs begin at Wed 2023-09-20 21:17:56 UTC. -- 2023-09-20T21:46:04.843 DEBUG:teuthology.orchestra.run.smithi123:mds.c> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.c.service 2023-09-20T21:46:04.845 DEBUG:teuthology.orchestra.run.smithi123:mds.f> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.f.service 2023-09-20T21:46:04.868 INFO:journalctl@ceph.mds.k.smithi089.stdout:-- Logs begin at Wed 2023-09-20 21:17:56 UTC. -- 2023-09-20T21:46:04.874 INFO:journalctl@ceph.mds.c.smithi123.stdout:-- Logs begin at Wed 2023-09-20 21:17:48 UTC. -- 2023-09-20T21:46:04.875 DEBUG:teuthology.orchestra.run.smithi123:mds.i> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.i.service 2023-09-20T21:46:04.900 INFO:journalctl@ceph.mds.f.smithi123.stdout:-- Logs begin at Wed 2023-09-20 21:17:48 UTC. -- 2023-09-20T21:46:04.901 DEBUG:teuthology.orchestra.run.smithi123:mds.l> sudo journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.l.service 2023-09-20T21:46:04.928 INFO:journalctl@ceph.mds.i.smithi123.stdout:-- Logs begin at Wed 2023-09-20 21:17:48 UTC. -- 2023-09-20T21:46:04.929 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-09-20T21:46:04.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:46:04.957 INFO:journalctl@ceph.mds.l.smithi123.stdout:-- Logs begin at Wed 2023-09-20 21:17:48 UTC. -- 2023-09-20T21:46:05.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.148+0000 7ff017dff700 1 -- 172.21.15.22:0/1384185484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff010101fd0 msgr2=0x7ff010102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:05.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.148+0000 7ff017dff700 1 --2- 172.21.15.22:0/1384185484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff010101fd0 0x7ff010102430 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff000009920 tx=0x7ff000009c30 comp rx=0 tx=0).stop 2023-09-20T21:46:05.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.148+0000 7ff017dff700 1 -- 172.21.15.22:0/1384185484 shutdown_connections 2023-09-20T21:46:05.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.148+0000 7ff017dff700 1 --2- 172.21.15.22:0/1384185484 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff010102970 0x7ff010109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.148+0000 7ff017dff700 1 --2- 172.21.15.22:0/1384185484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff010100de0 0x7ff010101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.148+0000 7ff017dff700 1 --2- 172.21.15.22:0/1384185484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff010101fd0 0x7ff010102430 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.148+0000 7ff017dff700 1 -- 172.21.15.22:0/1384185484 >> 172.21.15.22:0/1384185484 conn(0x7ff0100fc310 msgr2=0x7ff0100fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.148+0000 7ff017dff700 1 -- 172.21.15.22:0/1384185484 shutdown_connections 2023-09-20T21:46:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.148+0000 7ff017dff700 1 -- 172.21.15.22:0/1384185484 wait complete. 2023-09-20T21:46:05.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.149+0000 7ff017dff700 1 Processor -- start 2023-09-20T21:46:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.149+0000 7ff017dff700 1 -- start start 2023-09-20T21:46:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.149+0000 7ff017dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff010100de0 0x7ff010198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.149+0000 7ff017dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff010101fd0 0x7ff010199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.149+0000 7ff017dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff010102970 0x7ff0101a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:05.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.149+0000 7ff017dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0101a0890 con 0x7ff010101fd0 2023-09-20T21:46:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.149+0000 7ff017dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0101a0a00 con 0x7ff010100de0 2023-09-20T21:46:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.149+0000 7ff017dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0101a0bc0 con 0x7ff010102970 2023-09-20T21:46:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff01639c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff010102970 0x7ff0101a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff01539a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff010101fd0 0x7ff010199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff015b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff010100de0 0x7ff010198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:05.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff01639c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff010102970 0x7ff0101a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:51458/0 (socket says 172.21.15.22:51458) 2023-09-20T21:46:05.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff01639c700 1 -- 172.21.15.22:0/3897811909 learned_addr learned my addr 172.21.15.22:0/3897811909 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:05.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff015b9b700 1 -- 172.21.15.22:0/3897811909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff010102970 msgr2=0x7ff0101a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:05.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff015b9b700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff010102970 0x7ff0101a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff015b9b700 1 -- 172.21.15.22:0/3897811909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff010101fd0 msgr2=0x7ff010199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:05.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff015b9b700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff010101fd0 0x7ff010199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff015b9b700 1 -- 172.21.15.22:0/3897811909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff000009520 con 0x7ff010100de0 2023-09-20T21:46:05.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff015b9b700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff010100de0 0x7ff010198d50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00d740 tx=0x7ff00c00da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:05.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff006ffd700 1 -- 172.21.15.22:0/3897811909 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c004440 con 0x7ff010100de0 2023-09-20T21:46:05.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff006ffd700 1 -- 172.21.15.22:0/3897811909 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff00c003b40 con 0x7ff010100de0 2023-09-20T21:46:05.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.150+0000 7ff006ffd700 1 -- 172.21.15.22:0/3897811909 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c00fc90 con 0x7ff010100de0 2023-09-20T21:46:05.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.151+0000 7ff017dff700 1 -- 172.21.15.22:0/3897811909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0101a0ea0 con 0x7ff010100de0 2023-09-20T21:46:05.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.151+0000 7ff017dff700 1 -- 172.21.15.22:0/3897811909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0101a1270 con 0x7ff010100de0 2023-09-20T21:46:05.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.152+0000 7ff017dff700 1 -- 172.21.15.22:0/3897811909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff01010c640 con 0x7ff010100de0 2023-09-20T21:46:05.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.154+0000 7ff006ffd700 1 -- 172.21.15.22:0/3897811909 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ff00c003cb0 con 0x7ff010100de0 2023-09-20T21:46:05.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.155+0000 7ff006ffd700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7feffc0a1350 0x7feffc0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:05.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.155+0000 7ff006ffd700 1 -- 172.21.15.22:0/3897811909 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff00c0cc840 con 0x7ff010100de0 2023-09-20T21:46:05.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.155+0000 7ff01539a700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7feffc0a1350 0x7feffc0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:05.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.156+0000 7ff01539a700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7feffc0a1350 0x7feffc0a3810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff00000fa40 tx=0x7ff000019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:05.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.157+0000 7ff006ffd700 1 -- 172.21.15.22:0/3897811909 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff00c096310 con 0x7ff010100de0 2023-09-20T21:46:05.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:05 smithi022 ceph-mon[97651]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:46:05.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:05 smithi022 ceph-mon[97651]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-09-20T21:46:05.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:05.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:46:05.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:05.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:46:05.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:05.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:05.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:05.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:05 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:05.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.369+0000 7ff017dff700 1 -- 172.21.15.22:0/3897811909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff01010c820 con 0x7ff010100de0 2023-09-20T21:46:05.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.371+0000 7ff006ffd700 1 -- 172.21.15.22:0/3897811909 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18375 (secure 0 0 0) 0x7ff00c099930 con 0x7ff010100de0 2023-09-20T21:46:05.372 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:05.372 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":81,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:45:53.805187+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:46:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.373+0000 7ff017dff700 1 -- 172.21.15.22:0/3897811909 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7feffc0a1350 msgr2=0x7feffc0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.373+0000 7ff017dff700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7feffc0a1350 0x7feffc0a3810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff00000fa40 tx=0x7ff000019040 comp rx=0 tx=0).stop 2023-09-20T21:46:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.373+0000 7ff017dff700 1 -- 172.21.15.22:0/3897811909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff010100de0 msgr2=0x7ff010198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:05.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.373+0000 7ff017dff700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff010100de0 0x7ff010198d50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00d740 tx=0x7ff00c00da50 comp rx=0 tx=0).stop 2023-09-20T21:46:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.373+0000 7ff017dff700 1 -- 172.21.15.22:0/3897811909 shutdown_connections 2023-09-20T21:46:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.373+0000 7ff017dff700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff010102970 0x7ff0101a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.373+0000 7ff017dff700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7feffc0a1350 0x7feffc0a3810 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.373+0000 7ff017dff700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff010100de0 0x7ff010198d50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.373+0000 7ff017dff700 1 --2- 172.21.15.22:0/3897811909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff010101fd0 0x7ff010199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.373+0000 7ff017dff700 1 -- 172.21.15.22:0/3897811909 >> 172.21.15.22:0/3897811909 conn(0x7ff0100fc310 msgr2=0x7ff0100fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:05.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.374+0000 7ff017dff700 1 -- 172.21.15.22:0/3897811909 shutdown_connections 2023-09-20T21:46:05.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.374+0000 7ff017dff700 1 -- 172.21.15.22:0/3897811909 wait complete. 2023-09-20T21:46:05.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-20T21:46:05.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:46:05.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:05 smithi123 ceph-mon[134433]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:46:05.566 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:05 smithi123 ceph-mon[134433]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-09-20T21:46:05.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:05.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:46:05.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:05.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:46:05.567 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:05.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:05.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:05.568 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:05 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:05.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.602+0000 7f2d3edff700 1 -- 172.21.15.22:0/2924655556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d38108cb0 msgr2=0x7f2d3810b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:05.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.602+0000 7f2d3edff700 1 --2- 172.21.15.22:0/2924655556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d38108cb0 0x7f2d3810b140 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000b0a0 tx=0x7f2d3000b3b0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.603+0000 7f2d3edff700 1 -- 172.21.15.22:0/2924655556 shutdown_connections 2023-09-20T21:46:05.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.603+0000 7f2d3edff700 1 --2- 172.21.15.22:0/2924655556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d38108cb0 0x7f2d3810b140 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.603+0000 7f2d3edff700 1 --2- 172.21.15.22:0/2924655556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d38074e20 0x7f2d38073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.603+0000 7f2d3edff700 1 --2- 172.21.15.22:0/2924655556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d38073840 0x7f2d38073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.603+0000 7f2d3edff700 1 -- 172.21.15.22:0/2924655556 >> 172.21.15.22:0/2924655556 conn(0x7f2d380fc360 msgr2=0x7f2d380fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:05.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.603+0000 7f2d3edff700 1 -- 172.21.15.22:0/2924655556 shutdown_connections 2023-09-20T21:46:05.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.603+0000 7f2d3edff700 1 -- 172.21.15.22:0/2924655556 wait complete. 2023-09-20T21:46:05.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3edff700 1 Processor -- start 2023-09-20T21:46:05.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3edff700 1 -- start start 2023-09-20T21:46:05.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3edff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d38073840 0x7f2d38198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:05.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3edff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d38074e20 0x7f2d381992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:05.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3edff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d38108cb0 0x7f2d381a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:05.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3edff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381a08f0 con 0x7f2d38073840 2023-09-20T21:46:05.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3edff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381a0a30 con 0x7f2d38074e20 2023-09-20T21:46:05.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3edff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381a0bc0 con 0x7f2d38108cb0 2023-09-20T21:46:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3cb9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d38073840 0x7f2d38198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3d39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d38108cb0 0x7f2d381a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3cb9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d38073840 0x7f2d38198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55052/0 (socket says 172.21.15.22:55052) 2023-09-20T21:46:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3d39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d38108cb0 0x7f2d381a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:51464/0 (socket says 172.21.15.22:51464) 2023-09-20T21:46:05.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.604+0000 7f2d3cb9b700 1 -- 172.21.15.22:0/3460001800 learned_addr learned my addr 172.21.15.22:0/3460001800 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d37fff700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d38074e20 0x7f2d381992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d3d39c700 1 -- 172.21.15.22:0/3460001800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d38074e20 msgr2=0x7f2d381992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d3d39c700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d38074e20 0x7f2d381992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d3d39c700 1 -- 172.21.15.22:0/3460001800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d38073840 msgr2=0x7f2d38198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:05.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d3d39c700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d38073840 0x7f2d38198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d3d39c700 1 -- 172.21.15.22:0/3460001800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c009520 con 0x7f2d38108cb0 2023-09-20T21:46:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d3d39c700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d38108cb0 0x7f2d381a0320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009fd0 tx=0x7f2d3000bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d35ffb700 1 -- 172.21.15.22:0/3460001800 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000e040 con 0x7f2d38108cb0 2023-09-20T21:46:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d35ffb700 1 -- 172.21.15.22:0/3460001800 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d300094c0 con 0x7f2d38108cb0 2023-09-20T21:46:05.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d3edff700 1 -- 172.21.15.22:0/3460001800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d30009d00 con 0x7f2d38108cb0 2023-09-20T21:46:05.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.605+0000 7f2d35ffb700 1 -- 172.21.15.22:0/3460001800 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300052a0 con 0x7f2d38108cb0 2023-09-20T21:46:05.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.606+0000 7f2d3edff700 1 -- 172.21.15.22:0/3460001800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d381a1130 con 0x7f2d38108cb0 2023-09-20T21:46:05.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.607+0000 7f2d3edff700 1 -- 172.21.15.22:0/3460001800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d38107360 con 0x7f2d38108cb0 2023-09-20T21:46:05.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:05 smithi089 ceph-mon[134090]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:46:05.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:05 smithi089 ceph-mon[134090]: Saving service mds.all spec with placement smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-09-20T21:46:05.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:05.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:46:05.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:05.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:46:05.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:05.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:05.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:05.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:05 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:05.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.609+0000 7f2d35ffb700 1 -- 172.21.15.22:0/3460001800 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2d3001d470 con 0x7f2d38108cb0 2023-09-20T21:46:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.610+0000 7f2d35ffb700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d280a1300 0x7f2d280a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.610+0000 7f2d35ffb700 1 -- 172.21.15.22:0/3460001800 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2d300ce180 con 0x7f2d38108cb0 2023-09-20T21:46:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.610+0000 7f2d3cb9b700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d280a1300 0x7f2d280a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.611+0000 7f2d3cb9b700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d280a1300 0x7f2d280a37c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009ef0 tx=0x7f2d2c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:05.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.612+0000 7f2d35ffb700 1 -- 172.21.15.22:0/3460001800 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2d30097ce0 con 0x7f2d38108cb0 2023-09-20T21:46:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.824+0000 7f2d3edff700 1 -- 172.21.15.22:0/3460001800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d3810e9c0 con 0x7f2d38108cb0 2023-09-20T21:46:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.825+0000 7f2d35ffb700 1 -- 172.21.15.22:0/3460001800 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d3009b300 con 0x7f2d38108cb0 2023-09-20T21:46:05.826 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:46:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.827+0000 7f2d3edff700 1 -- 172.21.15.22:0/3460001800 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d280a1300 msgr2=0x7f2d280a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.827+0000 7f2d3edff700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d280a1300 0x7f2d280a37c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009ef0 tx=0x7f2d2c009260 comp rx=0 tx=0).stop 2023-09-20T21:46:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.828+0000 7f2d3edff700 1 -- 172.21.15.22:0/3460001800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d38108cb0 msgr2=0x7f2d381a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.828+0000 7f2d3edff700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d38108cb0 0x7f2d381a0320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009fd0 tx=0x7f2d3000bce0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.828+0000 7f2d3edff700 1 -- 172.21.15.22:0/3460001800 shutdown_connections 2023-09-20T21:46:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.828+0000 7f2d3edff700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d38108cb0 0x7f2d381a0320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.828+0000 7f2d3edff700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d38073840 0x7f2d38198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.828+0000 7f2d3edff700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d280a1300 0x7f2d280a37c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.828+0000 7f2d3edff700 1 --2- 172.21.15.22:0/3460001800 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d38074e20 0x7f2d381992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.828+0000 7f2d3edff700 1 -- 172.21.15.22:0/3460001800 >> 172.21.15.22:0/3460001800 conn(0x7f2d380fc360 msgr2=0x7f2d380fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.828+0000 7f2d3edff700 1 -- 172.21.15.22:0/3460001800 shutdown_connections 2023-09-20T21:46:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:05.828+0000 7f2d3edff700 1 -- 172.21.15.22:0/3460001800 wait complete. 2023-09-20T21:46:05.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:06.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.051+0000 7f52405ff700 1 -- 172.21.15.22:0/3114730401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5238074e20 msgr2=0x7f5238073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:06.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.051+0000 7f52405ff700 1 --2- 172.21.15.22:0/3114730401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5238074e20 0x7f5238073270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5234009990 tx=0x7f5234009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.051+0000 7f52405ff700 1 -- 172.21.15.22:0/3114730401 shutdown_connections 2023-09-20T21:46:06.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.051+0000 7f52405ff700 1 --2- 172.21.15.22:0/3114730401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5238108cb0 0x7f523810b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.051+0000 7f52405ff700 1 --2- 172.21.15.22:0/3114730401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5238074e20 0x7f5238073270 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.051+0000 7f52405ff700 1 --2- 172.21.15.22:0/3114730401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5238073840 0x7f5238073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.051+0000 7f52405ff700 1 -- 172.21.15.22:0/3114730401 >> 172.21.15.22:0/3114730401 conn(0x7f52380fc320 msgr2=0x7f52380fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:06.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.052+0000 7f52405ff700 1 -- 172.21.15.22:0/3114730401 shutdown_connections 2023-09-20T21:46:06.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.052+0000 7f52405ff700 1 -- 172.21.15.22:0/3114730401 wait complete. 2023-09-20T21:46:06.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.052+0000 7f52405ff700 1 Processor -- start 2023-09-20T21:46:06.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f52405ff700 1 -- start start 2023-09-20T21:46:06.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f52405ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5238073840 0x7f5238198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:06.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f52405ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5238074e20 0x7f5238199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:06.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f52405ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5238108cb0 0x7f52381a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:06.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f52405ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381a0840 con 0x7f5238108cb0 2023-09-20T21:46:06.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f52405ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381a09b0 con 0x7f5238073840 2023-09-20T21:46:06.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f52405ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381a0b70 con 0x7f5238074e20 2023-09-20T21:46:06.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f523db9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5238074e20 0x7f5238199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:06.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f523eb9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5238108cb0 0x7f52381a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:06.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f523db9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5238074e20 0x7f5238199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:46142/0 (socket says 172.21.15.22:46142) 2023-09-20T21:46:06.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f523db9a700 1 -- 172.21.15.22:0/4147766049 learned_addr learned my addr 172.21.15.22:0/4147766049 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:06.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.053+0000 7f523e39b700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5238073840 0x7f5238198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:06.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f523db9a700 1 -- 172.21.15.22:0/4147766049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5238073840 msgr2=0x7f5238198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:06.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f523db9a700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5238073840 0x7f5238198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f523db9a700 1 -- 172.21.15.22:0/4147766049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5238108cb0 msgr2=0x7f52381a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:06.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f523db9a700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5238108cb0 0x7f52381a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f523db9a700 1 -- 172.21.15.22:0/4147766049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52340095f0 con 0x7f5238074e20 2023-09-20T21:46:06.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f523db9a700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5238074e20 0x7f5238199290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f522800e970 tx=0x7f522800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:06.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f522f7fe700 1 -- 172.21.15.22:0/4147766049 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228009790 con 0x7f5238074e20 2023-09-20T21:46:06.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f522f7fe700 1 -- 172.21.15.22:0/4147766049 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f522800c7e0 con 0x7f5238074e20 2023-09-20T21:46:06.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f52405ff700 1 -- 172.21.15.22:0/4147766049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52381a0e50 con 0x7f5238074e20 2023-09-20T21:46:06.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f522f7fe700 1 -- 172.21.15.22:0/4147766049 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228003c70 con 0x7f5238074e20 2023-09-20T21:46:06.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.054+0000 7f52405ff700 1 -- 172.21.15.22:0/4147766049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52381a1270 con 0x7f5238074e20 2023-09-20T21:46:06.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.055+0000 7f52405ff700 1 -- 172.21.15.22:0/4147766049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5238107360 con 0x7f5238074e20 2023-09-20T21:46:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.057+0000 7f522f7fe700 1 -- 172.21.15.22:0/4147766049 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f522800cc50 con 0x7f5238074e20 2023-09-20T21:46:06.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.058+0000 7f522f7fe700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f52240a1350 0x7f52240a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:06.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.059+0000 7f522f7fe700 1 -- 172.21.15.22:0/4147766049 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5228014070 con 0x7f5238074e20 2023-09-20T21:46:06.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.059+0000 7f523e39b700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f52240a1350 0x7f52240a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:06.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.059+0000 7f523e39b700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f52240a1350 0x7f52240a3810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f523400b2e0 tx=0x7f5234005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:06.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.060+0000 7f522f7fe700 1 -- 172.21.15.22:0/4147766049 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f52280964b0 con 0x7f5238074e20 2023-09-20T21:46:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.315+0000 7f52405ff700 1 -- 172.21.15.22:0/4147766049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f523802d170 con 0x7f5238074e20 2023-09-20T21:46:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.316+0000 7f522f7fe700 1 -- 172.21.15.22:0/4147766049 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f5228099ad0 con 0x7f5238074e20 2023-09-20T21:46:06.318 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:06.318 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-09-20T21:46:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 -- 172.21.15.22:0/4147766049 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f52240a1350 msgr2=0x7f52240a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f52240a1350 0x7f52240a3810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f523400b2e0 tx=0x7f5234005fb0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 -- 172.21.15.22:0/4147766049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5238074e20 msgr2=0x7f5238199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5238074e20 0x7f5238199290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f522800e970 tx=0x7f522800ec80 comp rx=0 tx=0).stop 2023-09-20T21:46:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 -- 172.21.15.22:0/4147766049 shutdown_connections 2023-09-20T21:46:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5238108cb0 0x7f52381a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f52240a1350 0x7f52240a3810 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5238073840 0x7f5238198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 --2- 172.21.15.22:0/4147766049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5238074e20 0x7f5238199290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 -- 172.21.15.22:0/4147766049 >> 172.21.15.22:0/4147766049 conn(0x7f52380fc320 msgr2=0x7f52380fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 -- 172.21.15.22:0/4147766049 shutdown_connections 2023-09-20T21:46:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:06.318+0000 7f52405ff700 1 -- 172.21.15.22:0/4147766049 wait complete. 2023-09-20T21:46:06.322 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 1 2023-09-20T21:46:06.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:06 smithi022 ceph-mon[97651]: Deploying daemon mds.l on smithi123 2023-09-20T21:46:06.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:06 smithi022 ceph-mon[97651]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:06.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:06 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3897811909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:06.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:06 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3460001800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:06.542 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:06 smithi123 ceph-mon[134433]: Deploying daemon mds.l on smithi123 2023-09-20T21:46:06.543 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:06 smithi123 ceph-mon[134433]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:06.543 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:06 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3897811909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:06.543 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:06 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3460001800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:06.544 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 20 21:46:06 smithi123 systemd[1]: Starting Ceph mds.l for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:06.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:06 smithi089 ceph-mon[134090]: Deploying daemon mds.l on smithi123 2023-09-20T21:46:06.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:06 smithi089 ceph-mon[134090]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:06.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:06 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3897811909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:06.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:06 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3460001800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:06.829 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 20 21:46:06 smithi123 podman[171561]: 2023-09-20T21:46:06.829 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 20 21:46:06 smithi123 bash[171561]: 74b0c9fb763cf5e79102d7dc5348a1e87e98d6c5bfdf11a84edfa5879f6ac81d 2023-09-20T21:46:06.830 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 20 21:46:06 smithi123 ceph-mds[171601]: -- 172.21.15.123:0/3174820956 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55573dac31e0 con 0x55573da99000 2023-09-20T21:46:06.830 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 20 21:46:06 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-l[171577]: starting mds.l at 2023-09-20T21:46:07.093 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 20 21:46:06 smithi123 systemd[1]: Started Ceph mds.l for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:07 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4147766049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:07 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.123:6832/2368905204,v1:172.21.15.123:6833/2368905204] up:boot 2023-09-20T21:46:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:07 smithi022 ceph-mon[97651]: fsmap 1 up:standby 2023-09-20T21:46:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-20T21:46:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:07.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:07.518 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:07 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4147766049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:07.518 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:07 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.123:6832/2368905204,v1:172.21.15.123:6833/2368905204] up:boot 2023-09-20T21:46:07.518 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:07 smithi089 ceph-mon[134090]: fsmap 1 up:standby 2023-09-20T21:46:07.519 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-20T21:46:07.519 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:07.519 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:07.519 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:07.519 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:07.520 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:07.520 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:07.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:07 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4147766049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:07.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:07 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.123:6832/2368905204,v1:172.21.15.123:6833/2368905204] up:boot 2023-09-20T21:46:07.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:07 smithi123 ceph-mon[134433]: fsmap 1 up:standby 2023-09-20T21:46:07.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-09-20T21:46:07.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:07.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:07.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:07.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:07.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:07.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:08.338 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:08.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:08 smithi022 ceph-mon[97651]: Deploying daemon mds.k on smithi089 2023-09-20T21:46:08.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:08 smithi022 ceph-mon[97651]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:08.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.553+0000 7f229cbff700 1 -- 172.21.15.22:0/3561380777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2298100de0 msgr2=0x7f2298101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:08.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.553+0000 7f229cbff700 1 --2- 172.21.15.22:0/3561380777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2298100de0 0x7f2298101200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f22880099c0 tx=0x7f2288009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:08.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.553+0000 7f229cbff700 1 -- 172.21.15.22:0/3561380777 shutdown_connections 2023-09-20T21:46:08.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.553+0000 7f229cbff700 1 --2- 172.21.15.22:0/3561380777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2298102970 0x7f2298109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:08.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.553+0000 7f229cbff700 1 --2- 172.21.15.22:0/3561380777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2298100de0 0x7f2298101200 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:08.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.553+0000 7f229cbff700 1 --2- 172.21.15.22:0/3561380777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2298101fd0 0x7f2298102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:08.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.553+0000 7f229cbff700 1 -- 172.21.15.22:0/3561380777 >> 172.21.15.22:0/3561380777 conn(0x7f22980fc310 msgr2=0x7f22980fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:08.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.553+0000 7f229cbff700 1 -- 172.21.15.22:0/3561380777 shutdown_connections 2023-09-20T21:46:08.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.553+0000 7f229cbff700 1 -- 172.21.15.22:0/3561380777 wait complete. 2023-09-20T21:46:08.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.554+0000 7f229cbff700 1 Processor -- start 2023-09-20T21:46:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.554+0000 7f229cbff700 1 -- start start 2023-09-20T21:46:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229cbff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2298100de0 0x7f2298074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229cbff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2298101fd0 0x7f22980731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:08.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229cbff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2298102970 0x7f2298073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:08.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229cbff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2298073eb0 con 0x7f2298101fd0 2023-09-20T21:46:08.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229cbff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2298074020 con 0x7f2298100de0 2023-09-20T21:46:08.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229cbff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22981a0b30 con 0x7f2298102970 2023-09-20T21:46:08.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2298100de0 0x7f2298074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:08.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f2295d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2298101fd0 0x7f22980731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:08.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2298100de0 0x7f2298074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:54416/0 (socket says 172.21.15.22:54416) 2023-09-20T21:46:08.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229659c700 1 -- 172.21.15.22:0/882226850 learned_addr learned my addr 172.21.15.22:0/882226850 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:08.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f2296d9d700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2298102970 0x7f2298073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:08.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229659c700 1 -- 172.21.15.22:0/882226850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2298102970 msgr2=0x7f2298073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:08.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229659c700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2298102970 0x7f2298073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:08.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229659c700 1 -- 172.21.15.22:0/882226850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2298101fd0 msgr2=0x7f22980731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:08.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229659c700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2298101fd0 0x7f22980731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:08.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.555+0000 7f229659c700 1 -- 172.21.15.22:0/882226850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880095f0 con 0x7f2298100de0 2023-09-20T21:46:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.556+0000 7f229659c700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2298100de0 0x7f2298074b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2288009fd0 tx=0x7f228800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.556+0000 7f22877fe700 1 -- 172.21.15.22:0/882226850 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228801d070 con 0x7f2298100de0 2023-09-20T21:46:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.556+0000 7f22877fe700 1 -- 172.21.15.22:0/882226850 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2288005050 con 0x7f2298100de0 2023-09-20T21:46:08.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.556+0000 7f229cbff700 1 -- 172.21.15.22:0/882226850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22981a0db0 con 0x7f2298100de0 2023-09-20T21:46:08.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.556+0000 7f22877fe700 1 -- 172.21.15.22:0/882226850 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288004a70 con 0x7f2298100de0 2023-09-20T21:46:08.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.556+0000 7f229cbff700 1 -- 172.21.15.22:0/882226850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22981a1180 con 0x7f2298100de0 2023-09-20T21:46:08.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.557+0000 7f229cbff700 1 -- 172.21.15.22:0/882226850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229810c640 con 0x7f2298100de0 2023-09-20T21:46:08.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.559+0000 7f22877fe700 1 -- 172.21.15.22:0/882226850 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2288005450 con 0x7f2298100de0 2023-09-20T21:46:08.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.560+0000 7f22877fe700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22800a13d0 0x7f22800a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.561+0000 7f22877fe700 1 -- 172.21.15.22:0/882226850 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f22880cd4e0 con 0x7f2298100de0 2023-09-20T21:46:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.561+0000 7f2295d9b700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22800a13d0 0x7f22800a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.561+0000 7f2295d9b700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22800a13d0 0x7f22800a3890 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f228c009c60 tx=0x7f228c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.563+0000 7f22877fe700 1 -- 172.21.15.22:0/882226850 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2288097040 con 0x7f2298100de0 2023-09-20T21:46:08.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:08 smithi123 ceph-mon[134433]: Deploying daemon mds.k on smithi089 2023-09-20T21:46:08.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:08 smithi123 ceph-mon[134433]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:08.611 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 21:46:08 smithi089 systemd[1]: Starting Ceph mds.k for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:08.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:08 smithi089 ceph-mon[134090]: Deploying daemon mds.k on smithi089 2023-09-20T21:46:08.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:08 smithi089 ceph-mon[134090]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:08.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.820+0000 7f229cbff700 1 -- 172.21.15.22:0/882226850 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2298107d00 con 0x7f2298100de0 2023-09-20T21:46:08.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.821+0000 7f22877fe700 1 -- 172.21.15.22:0/882226850 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f228809a660 con 0x7f2298100de0 2023-09-20T21:46:08.822 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:08.822 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-09-20T21:46:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.823+0000 7f229cbff700 1 -- 172.21.15.22:0/882226850 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22800a13d0 msgr2=0x7f22800a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.823+0000 7f229cbff700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22800a13d0 0x7f22800a3890 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f228c009c60 tx=0x7f228c009260 comp rx=0 tx=0).stop 2023-09-20T21:46:08.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.823+0000 7f229cbff700 1 -- 172.21.15.22:0/882226850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2298100de0 msgr2=0x7f2298074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:08.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.823+0000 7f229cbff700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2298100de0 0x7f2298074b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2288009fd0 tx=0x7f228800f690 comp rx=0 tx=0).stop 2023-09-20T21:46:08.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.823+0000 7f229cbff700 1 -- 172.21.15.22:0/882226850 shutdown_connections 2023-09-20T21:46:08.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.823+0000 7f229cbff700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2298102970 0x7f2298073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:08.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.823+0000 7f229cbff700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22800a13d0 0x7f22800a3890 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:08.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.823+0000 7f229cbff700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2298100de0 0x7f2298074b90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:08.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.823+0000 7f229cbff700 1 --2- 172.21.15.22:0/882226850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2298101fd0 0x7f22980731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:08.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.823+0000 7f229cbff700 1 -- 172.21.15.22:0/882226850 >> 172.21.15.22:0/882226850 conn(0x7f22980fc310 msgr2=0x7f22980fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:08.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.824+0000 7f229cbff700 1 -- 172.21.15.22:0/882226850 shutdown_connections 2023-09-20T21:46:08.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:08.824+0000 7f229cbff700 1 -- 172.21.15.22:0/882226850 wait complete. 2023-09-20T21:46:08.827 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 2 2023-09-20T21:46:08.917 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 21:46:08 smithi089 podman[170376]: 2023-09-20T21:46:09.200 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 21:46:08 smithi089 bash[170376]: 511d7fb0e69978f82421ccee8156576d484e42025562a77fc32e50af419759d8 2023-09-20T21:46:09.201 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 21:46:08 smithi089 ceph-mds[170416]: -- 172.21.15.89:0/1244384561 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e42a111e0 con 0x560e429e7000 2023-09-20T21:46:09.201 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 21:46:08 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-k[170392]: starting mds.k at 2023-09-20T21:46:09.201 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 21:46:08 smithi089 systemd[1]: Started Ceph mds.k for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:09.202 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:09 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/882226850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:09.202 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:09.202 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:09.202 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:09.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:09 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/882226850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:09.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:09.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:09.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:09.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:09.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:09.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:09.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:09 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/882226850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:09.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:09.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:09.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:09.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:09.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:09.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:09.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:09.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:09.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:10.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:10 smithi022 ceph-mon[97651]: Deploying daemon mds.j on smithi022 2023-09-20T21:46:10.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:10 smithi022 ceph-mon[97651]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:10.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:10 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.89:6832/3516975495,v1:172.21.15.89:6833/3516975495] up:boot 2023-09-20T21:46:10.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:10 smithi022 ceph-mon[97651]: fsmap 2 up:standby 2023-09-20T21:46:10.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-20T21:46:10.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:10 smithi123 ceph-mon[134433]: Deploying daemon mds.j on smithi022 2023-09-20T21:46:10.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:10 smithi123 ceph-mon[134433]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:10.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:10 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.89:6832/3516975495,v1:172.21.15.89:6833/3516975495] up:boot 2023-09-20T21:46:10.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:10 smithi123 ceph-mon[134433]: fsmap 2 up:standby 2023-09-20T21:46:10.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-20T21:46:10.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:10 smithi089 ceph-mon[134090]: Deploying daemon mds.j on smithi022 2023-09-20T21:46:10.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:10 smithi089 ceph-mon[134090]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:10.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:10 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.89:6832/3516975495,v1:172.21.15.89:6833/3516975495] up:boot 2023-09-20T21:46:10.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:10 smithi089 ceph-mon[134090]: fsmap 2 up:standby 2023-09-20T21:46:10.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-20T21:46:10.843 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:11.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.080+0000 7f1d663ff700 1 -- 172.21.15.22:0/863382904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60074e20 msgr2=0x7f1d60073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:11.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.080+0000 7f1d663ff700 1 --2- 172.21.15.22:0/863382904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60074e20 0x7f1d60073270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009990 tx=0x7f1d50009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:11.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.080+0000 7f1d663ff700 1 -- 172.21.15.22:0/863382904 shutdown_connections 2023-09-20T21:46:11.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.080+0000 7f1d663ff700 1 --2- 172.21.15.22:0/863382904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d6010d3b0 0x7f1d6010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:11.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.080+0000 7f1d663ff700 1 --2- 172.21.15.22:0/863382904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60074e20 0x7f1d60073270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:11.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.080+0000 7f1d663ff700 1 --2- 172.21.15.22:0/863382904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60073840 0x7f1d60073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:11.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.080+0000 7f1d663ff700 1 -- 172.21.15.22:0/863382904 >> 172.21.15.22:0/863382904 conn(0x7f1d600fc310 msgr2=0x7f1d600fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:11.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.080+0000 7f1d663ff700 1 -- 172.21.15.22:0/863382904 shutdown_connections 2023-09-20T21:46:11.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.080+0000 7f1d663ff700 1 -- 172.21.15.22:0/863382904 wait complete. 2023-09-20T21:46:11.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.082+0000 7f1d663ff700 1 Processor -- start 2023-09-20T21:46:11.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.082+0000 7f1d663ff700 1 -- start start 2023-09-20T21:46:11.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.082+0000 7f1d663ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60073840 0x7f1d6019d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:11.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.082+0000 7f1d663ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60074e20 0x7f1d6019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:11.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.082+0000 7f1d663ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6010d3b0 0x7f1d601a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:11.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.082+0000 7f1d663ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601a4c50 con 0x7f1d60074e20 2023-09-20T21:46:11.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.082+0000 7f1d663ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601a4d90 con 0x7f1d60073840 2023-09-20T21:46:11.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.082+0000 7f1d663ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601a4f20 con 0x7f1d6010d3b0 2023-09-20T21:46:11.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60073840 0x7f1d6019d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:11.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60073840 0x7f1d6019d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:54436/0 (socket says 172.21.15.22:54436) 2023-09-20T21:46:11.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5ffff700 1 -- 172.21.15.22:0/2191557777 learned_addr learned my addr 172.21.15.22:0/2191557777 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:11.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5f7fe700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60074e20 0x7f1d6019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:11.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d6499c700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6010d3b0 0x7f1d601a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:11.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5ffff700 1 -- 172.21.15.22:0/2191557777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6010d3b0 msgr2=0x7f1d601a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:11.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5ffff700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6010d3b0 0x7f1d601a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:11.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5ffff700 1 -- 172.21.15.22:0/2191557777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60074e20 msgr2=0x7f1d6019d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:11.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5ffff700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60074e20 0x7f1d6019d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:11.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5ffff700 1 -- 172.21.15.22:0/2191557777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d500095f0 con 0x7f1d60073840 2023-09-20T21:46:11.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5ffff700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60073840 0x7f1d6019d0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d50004730 tx=0x7f1d50004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:11.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.083+0000 7f1d5d7fa700 1 -- 172.21.15.22:0/2191557777 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5001d070 con 0x7f1d60073840 2023-09-20T21:46:11.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.084+0000 7f1d663ff700 1 -- 172.21.15.22:0/2191557777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d601a51a0 con 0x7f1d60073840 2023-09-20T21:46:11.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.084+0000 7f1d5d7fa700 1 -- 172.21.15.22:0/2191557777 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d50004c90 con 0x7f1d60073840 2023-09-20T21:46:11.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.084+0000 7f1d5d7fa700 1 -- 172.21.15.22:0/2191557777 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50005570 con 0x7f1d60073840 2023-09-20T21:46:11.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.084+0000 7f1d663ff700 1 -- 172.21.15.22:0/2191557777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d601a55c0 con 0x7f1d60073840 2023-09-20T21:46:11.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.085+0000 7f1d663ff700 1 -- 172.21.15.22:0/2191557777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d60102f00 con 0x7f1d60073840 2023-09-20T21:46:11.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.087+0000 7f1d5d7fa700 1 -- 172.21.15.22:0/2191557777 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1d5000fc70 con 0x7f1d60073840 2023-09-20T21:46:11.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.089+0000 7f1d5d7fa700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a13d0 0x7f1d480a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:11.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.090+0000 7f1d5d7fa700 1 -- 172.21.15.22:0/2191557777 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1d500cdfb0 con 0x7f1d60073840 2023-09-20T21:46:11.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.090+0000 7f1d5f7fe700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a13d0 0x7f1d480a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:11.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.090+0000 7f1d5f7fe700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a13d0 0x7f1d480a3890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d60074b40 tx=0x7f1d5400b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:11.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.090+0000 7f1d5d7fa700 1 -- 172.21.15.22:0/2191557777 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1d50097b10 con 0x7f1d60073840 2023-09-20T21:46:11.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.363+0000 7f1d663ff700 1 -- 172.21.15.22:0/2191557777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d60112c20 con 0x7f1d60073840 2023-09-20T21:46:11.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.363+0000 7f1d5d7fa700 1 -- 172.21.15.22:0/2191557777 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f1d5009b130 con 0x7f1d60073840 2023-09-20T21:46:11.364 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:11.364 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-20T21:46:11.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 -- 172.21.15.22:0/2191557777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a13d0 msgr2=0x7f1d480a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a13d0 0x7f1d480a3890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d60074b40 tx=0x7f1d5400b410 comp rx=0 tx=0).stop 2023-09-20T21:46:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 -- 172.21.15.22:0/2191557777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60073840 msgr2=0x7f1d6019d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60073840 0x7f1d6019d0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d50004730 tx=0x7f1d50004760 comp rx=0 tx=0).stop 2023-09-20T21:46:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 -- 172.21.15.22:0/2191557777 shutdown_connections 2023-09-20T21:46:11.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6010d3b0 0x7f1d601a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a13d0 0x7f1d480a3890 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60073840 0x7f1d6019d0d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 --2- 172.21.15.22:0/2191557777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60074e20 0x7f1d6019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 -- 172.21.15.22:0/2191557777 >> 172.21.15.22:0/2191557777 conn(0x7f1d600fc310 msgr2=0x7f1d600fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:11.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 -- 172.21.15.22:0/2191557777 shutdown_connections 2023-09-20T21:46:11.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:11.366+0000 7f1d663ff700 1 -- 172.21.15.22:0/2191557777 wait complete. 2023-09-20T21:46:11.369 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 3 2023-09-20T21:46:11.747 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 21:46:11 smithi022 systemd[1]: Starting Ceph mds.j for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:12.030 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 21:46:11 smithi022 podman[147568]: 2023-09-20T21:46:12.497 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 21:46:12 smithi022 bash[147568]: 80c202ec73c80678bdd883f33543d5114f514a0bde7180c368e7a8566a851932 2023-09-20T21:46:12.498 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 21:46:12 smithi022 ceph-mds[147608]: -- 172.21.15.22:0/1460673803 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fed07351e0 con 0x55fed070a800 2023-09-20T21:46:12.498 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 21:46:12 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-j[147584]: starting mds.j at 2023-09-20T21:46:12.498 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 21:46:12 smithi022 systemd[1]: Started Ceph mds.j for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:12.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:12 smithi022 ceph-mon[97651]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:12.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:12 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2191557777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:12.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:12.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:12.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:12.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:12.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:12.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:12.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:12 smithi123 ceph-mon[134433]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:12.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:12 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2191557777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:12.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:12.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:12.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:12.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:12.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:12.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:12.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:12 smithi089 ceph-mon[134090]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:12.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:12 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2191557777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:12.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:12.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:12.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:12.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:12.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:12.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:13.388 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:13.423 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:13 smithi022 ceph-mon[97651]: Deploying daemon mds.d on smithi022 2023-09-20T21:46:13.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:13 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] up:boot 2023-09-20T21:46:13.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:13 smithi022 ceph-mon[97651]: fsmap 3 up:standby 2023-09-20T21:46:13.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:13 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-20T21:46:13.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:13 smithi123 ceph-mon[134433]: Deploying daemon mds.d on smithi022 2023-09-20T21:46:13.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:13 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] up:boot 2023-09-20T21:46:13.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:13 smithi123 ceph-mon[134433]: fsmap 3 up:standby 2023-09-20T21:46:13.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:13 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-20T21:46:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.603+0000 7f58479ff700 1 -- 172.21.15.22:0/1299269142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5840102980 msgr2=0x7f5840109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.603+0000 7f58479ff700 1 --2- 172.21.15.22:0/1299269142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5840102980 0x7f5840109230 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f583800b0d0 tx=0x7f583800b3e0 comp rx=0 tx=0).stop 2023-09-20T21:46:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.603+0000 7f58479ff700 1 -- 172.21.15.22:0/1299269142 shutdown_connections 2023-09-20T21:46:13.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.603+0000 7f58479ff700 1 --2- 172.21.15.22:0/1299269142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5840102980 0x7f5840109230 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.603+0000 7f58479ff700 1 --2- 172.21.15.22:0/1299269142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5840100df0 0x7f5840101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.603+0000 7f58479ff700 1 --2- 172.21.15.22:0/1299269142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5840101fe0 0x7f5840102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.603+0000 7f58479ff700 1 -- 172.21.15.22:0/1299269142 >> 172.21.15.22:0/1299269142 conn(0x7f58400fc320 msgr2=0x7f58400fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.603+0000 7f58479ff700 1 -- 172.21.15.22:0/1299269142 shutdown_connections 2023-09-20T21:46:13.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.603+0000 7f58479ff700 1 -- 172.21.15.22:0/1299269142 wait complete. 2023-09-20T21:46:13.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.604+0000 7f58479ff700 1 Processor -- start 2023-09-20T21:46:13.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.604+0000 7f58479ff700 1 -- start start 2023-09-20T21:46:13.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f58479ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5840100df0 0x7f5840196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:13.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f58479ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5840101fe0 0x7f58401970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:13.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f58479ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5840102980 0x7f584019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:13.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f58479ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584019e720 con 0x7f5840101fe0 2023-09-20T21:46:13.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f58479ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584019e860 con 0x7f5840100df0 2023-09-20T21:46:13.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f58479ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584019e9f0 con 0x7f5840102980 2023-09-20T21:46:13.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f5844f9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5840101fe0 0x7f58401970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:13.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f5844f9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5840101fe0 0x7f58401970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51960/0 (socket says 172.21.15.22:51960) 2023-09-20T21:46:13.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f5844f9a700 1 -- 172.21.15.22:0/108717327 learned_addr learned my addr 172.21.15.22:0/108717327 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:13.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f584579b700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5840100df0 0x7f5840196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:13.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f5845f9c700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5840102980 0x7f584019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:13.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f5844f9a700 1 -- 172.21.15.22:0/108717327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5840102980 msgr2=0x7f584019e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:13.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f5844f9a700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5840102980 0x7f584019e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:13.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f5844f9a700 1 -- 172.21.15.22:0/108717327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5840100df0 msgr2=0x7f5840196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:13.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f5844f9a700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5840100df0 0x7f5840196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:13.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.605+0000 7f5844f9a700 1 -- 172.21.15.22:0/108717327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5830009520 con 0x7f5840101fe0 2023-09-20T21:46:13.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.606+0000 7f5844f9a700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5840101fe0 0x7f58401970e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f583000eb30 tx=0x7f583000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:13.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.606+0000 7f58367fc700 1 -- 172.21.15.22:0/108717327 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830009880 con 0x7f5840101fe0 2023-09-20T21:46:13.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.606+0000 7f58367fc700 1 -- 172.21.15.22:0/108717327 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58300044e0 con 0x7f5840101fe0 2023-09-20T21:46:13.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.606+0000 7f58367fc700 1 -- 172.21.15.22:0/108717327 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830003d80 con 0x7f5840101fe0 2023-09-20T21:46:13.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.606+0000 7f58479ff700 1 -- 172.21.15.22:0/108717327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5838009d00 con 0x7f5840101fe0 2023-09-20T21:46:13.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.606+0000 7f58479ff700 1 -- 172.21.15.22:0/108717327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584019f000 con 0x7f5840101fe0 2023-09-20T21:46:13.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.607+0000 7f58479ff700 1 -- 172.21.15.22:0/108717327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584010c660 con 0x7f5840101fe0 2023-09-20T21:46:13.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:13 smithi089 ceph-mon[134090]: Deploying daemon mds.d on smithi022 2023-09-20T21:46:13.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:13 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] up:boot 2023-09-20T21:46:13.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:13 smithi089 ceph-mon[134090]: fsmap 3 up:standby 2023-09-20T21:46:13.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:13 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-20T21:46:13.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.609+0000 7f58367fc700 1 -- 172.21.15.22:0/108717327 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5830003f00 con 0x7f5840101fe0 2023-09-20T21:46:13.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.610+0000 7f58367fc700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f582c0a13d0 0x7f582c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:13.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.611+0000 7f58367fc700 1 -- 172.21.15.22:0/108717327 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5830014070 con 0x7f5840101fe0 2023-09-20T21:46:13.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.611+0000 7f584579b700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f582c0a13d0 0x7f582c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:13.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.611+0000 7f584579b700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f582c0a13d0 0x7f582c0a3890 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5840101e40 tx=0x7f583c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:13.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.613+0000 7f58367fc700 1 -- 172.21.15.22:0/108717327 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5830096900 con 0x7f5840101fe0 2023-09-20T21:46:13.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.876+0000 7f58479ff700 1 -- 172.21.15.22:0/108717327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f584010c8d0 con 0x7f5840101fe0 2023-09-20T21:46:13.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.876+0000 7f58367fc700 1 -- 172.21.15.22:0/108717327 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2904 (secure 0 0 0) 0x7f5830099f20 con 0x7f5840101fe0 2023-09-20T21:46:13.877 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:13.878 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2023-09-20T21:46:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 -- 172.21.15.22:0/108717327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f582c0a13d0 msgr2=0x7f582c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f582c0a13d0 0x7f582c0a3890 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5840101e40 tx=0x7f583c009260 comp rx=0 tx=0).stop 2023-09-20T21:46:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 -- 172.21.15.22:0/108717327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5840101fe0 msgr2=0x7f58401970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5840101fe0 0x7f58401970e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f583000eb30 tx=0x7f583000ee40 comp rx=0 tx=0).stop 2023-09-20T21:46:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 -- 172.21.15.22:0/108717327 shutdown_connections 2023-09-20T21:46:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5840102980 0x7f584019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f582c0a13d0 0x7f582c0a3890 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5840100df0 0x7f5840196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 --2- 172.21.15.22:0/108717327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5840101fe0 0x7f58401970e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:13.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 -- 172.21.15.22:0/108717327 >> 172.21.15.22:0/108717327 conn(0x7f58400fc320 msgr2=0x7f58400fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:13.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 -- 172.21.15.22:0/108717327 shutdown_connections 2023-09-20T21:46:13.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:13.879+0000 7f58479ff700 1 -- 172.21.15.22:0/108717327 wait complete. 2023-09-20T21:46:13.883 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 4 2023-09-20T21:46:14.472 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:14 smithi022 ceph-mon[97651]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:14.472 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:14 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/108717327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:14.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:14 smithi123 ceph-mon[134433]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:14.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:14 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/108717327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:14.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:14 smithi089 ceph-mon[134090]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:14.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:14 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/108717327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:14.747 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 21:46:14 smithi022 systemd[1]: Starting Ceph mds.d for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:15.204 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 21:46:14 smithi022 podman[147955]: 2023-09-20T21:46:15.497 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 21:46:15 smithi022 bash[147955]: e72068a2f4fd240199aaccbdd2dfaafe7a6d1472f8a3ccde5c9beff01bef0ef2 2023-09-20T21:46:15.498 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 21:46:15 smithi022 ceph-mds[147996]: -- 172.21.15.22:0/1114430441 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5604342391e0 con 0x56043420e800 2023-09-20T21:46:15.498 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 21:46:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-d[147971]: starting mds.d at 2023-09-20T21:46:15.498 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 21:46:15 smithi022 systemd[1]: Started Ceph mds.d for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:15.900 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.113+0000 7f81453ff700 1 -- 172.21.15.22:0/3625429417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101fd0 msgr2=0x7f8140102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:16.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.113+0000 7f81453ff700 1 --2- 172.21.15.22:0/3625429417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101fd0 0x7f8140102430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81340099c0 tx=0x7f8134009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.114+0000 7f81453ff700 1 -- 172.21.15.22:0/3625429417 shutdown_connections 2023-09-20T21:46:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.114+0000 7f81453ff700 1 --2- 172.21.15.22:0/3625429417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8140102970 0x7f8140109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.114+0000 7f81453ff700 1 --2- 172.21.15.22:0/3625429417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8140100de0 0x7f8140101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.114+0000 7f81453ff700 1 --2- 172.21.15.22:0/3625429417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101fd0 0x7f8140102430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:16.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.114+0000 7f81453ff700 1 -- 172.21.15.22:0/3625429417 >> 172.21.15.22:0/3625429417 conn(0x7f81400fc310 msgr2=0x7f81400fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:16.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.114+0000 7f81453ff700 1 -- 172.21.15.22:0/3625429417 shutdown_connections 2023-09-20T21:46:16.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.114+0000 7f81453ff700 1 -- 172.21.15.22:0/3625429417 wait complete. 2023-09-20T21:46:16.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.115+0000 7f81453ff700 1 Processor -- start 2023-09-20T21:46:16.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.115+0000 7f81453ff700 1 -- start start 2023-09-20T21:46:16.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f81453ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8140100de0 0x7f8140196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f81453ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101fd0 0x7f8140197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f81453ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8140102970 0x7f814019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f81453ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814019e6b0 con 0x7f8140101fd0 2023-09-20T21:46:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f81453ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814019e7f0 con 0x7f8140100de0 2023-09-20T21:46:16.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f81453ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814019e980 con 0x7f8140102970 2023-09-20T21:46:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8140100de0 0x7f8140196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101fd0 0x7f8140197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8140100de0 0x7f8140196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:47078/0 (socket says 172.21.15.22:47078) 2023-09-20T21:46:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813effd700 1 -- 172.21.15.22:0/1514438077 learned_addr learned my addr 172.21.15.22:0/1514438077 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:16.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813f7fe700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8140102970 0x7f814019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813e7fc700 1 -- 172.21.15.22:0/1514438077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8140102970 msgr2=0x7f814019e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813e7fc700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8140102970 0x7f814019e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813e7fc700 1 -- 172.21.15.22:0/1514438077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8140100de0 msgr2=0x7f8140196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813e7fc700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8140100de0 0x7f8140196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:16.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813e7fc700 1 -- 172.21.15.22:0/1514438077 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81340095f0 con 0x7f8140101fd0 2023-09-20T21:46:16.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.116+0000 7f813effd700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8140100de0 0x7f8140196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:16.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.117+0000 7f813e7fc700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101fd0 0x7f8140197070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f81340098e0 tx=0x7f813400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:16.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.117+0000 7f8127fff700 1 -- 172.21.15.22:0/1514438077 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813401d070 con 0x7f8140101fd0 2023-09-20T21:46:16.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.117+0000 7f8127fff700 1 -- 172.21.15.22:0/1514438077 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8134005020 con 0x7f8140101fd0 2023-09-20T21:46:16.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.117+0000 7f8127fff700 1 -- 172.21.15.22:0/1514438077 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134004a70 con 0x7f8140101fd0 2023-09-20T21:46:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.117+0000 7f81453ff700 1 -- 172.21.15.22:0/1514438077 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814019ec00 con 0x7f8140101fd0 2023-09-20T21:46:16.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.117+0000 7f81453ff700 1 -- 172.21.15.22:0/1514438077 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814019efc0 con 0x7f8140101fd0 2023-09-20T21:46:16.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.118+0000 7f81453ff700 1 -- 172.21.15.22:0/1514438077 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8140108370 con 0x7f8140101fd0 2023-09-20T21:46:16.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.120+0000 7f8127fff700 1 -- 172.21.15.22:0/1514438077 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f8134005400 con 0x7f8140101fd0 2023-09-20T21:46:16.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.121+0000 7f8127fff700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f81280a13d0 0x7f81280a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:16.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.121+0000 7f8127fff700 1 -- 172.21.15.22:0/1514438077 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f81340cd1f0 con 0x7f8140101fd0 2023-09-20T21:46:16.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.122+0000 7f813effd700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f81280a13d0 0x7f81280a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:16.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.122+0000 7f813effd700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f81280a13d0 0x7f81280a3890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8130005950 tx=0x7f8130009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:16.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.125+0000 7f8127fff700 1 -- 172.21.15.22:0/1514438077 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8134096d50 con 0x7f8140101fd0 2023-09-20T21:46:16.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:16 smithi022 ceph-mon[97651]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:16.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:16.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:16.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:16.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:16.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:16.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:16.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.384+0000 7f81453ff700 1 -- 172.21.15.22:0/1514438077 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f814010c640 con 0x7f8140101fd0 2023-09-20T21:46:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.385+0000 7f8127fff700 1 -- 172.21.15.22:0/1514438077 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3695 (secure 0 0 0) 0x7f813409a370 con 0x7f8140101fd0 2023-09-20T21:46:16.386 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:16.386 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[]} 2023-09-20T21:46:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.387+0000 7f81453ff700 1 -- 172.21.15.22:0/1514438077 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f81280a13d0 msgr2=0x7f81280a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.387+0000 7f81453ff700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f81280a13d0 0x7f81280a3890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8130005950 tx=0x7f8130009260 comp rx=0 tx=0).stop 2023-09-20T21:46:16.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.387+0000 7f81453ff700 1 -- 172.21.15.22:0/1514438077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101fd0 msgr2=0x7f8140197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.387+0000 7f81453ff700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101fd0 0x7f8140197070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f81340098e0 tx=0x7f813400f690 comp rx=0 tx=0).stop 2023-09-20T21:46:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.387+0000 7f81453ff700 1 -- 172.21.15.22:0/1514438077 shutdown_connections 2023-09-20T21:46:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.387+0000 7f81453ff700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8140102970 0x7f814019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.387+0000 7f81453ff700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f81280a13d0 0x7f81280a3890 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:16.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.387+0000 7f81453ff700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8140100de0 0x7f8140196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.387+0000 7f81453ff700 1 --2- 172.21.15.22:0/1514438077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8140101fd0 0x7f8140197070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.387+0000 7f81453ff700 1 -- 172.21.15.22:0/1514438077 >> 172.21.15.22:0/1514438077 conn(0x7f81400fc310 msgr2=0x7f81401054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.388+0000 7f81453ff700 1 -- 172.21.15.22:0/1514438077 shutdown_connections 2023-09-20T21:46:16.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:16.388+0000 7f81453ff700 1 -- 172.21.15.22:0/1514438077 wait complete. 2023-09-20T21:46:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 5 2023-09-20T21:46:16.584 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:16 smithi089 ceph-mon[134090]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:16.585 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:16.585 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:16.585 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:16.585 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:16.586 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:16.586 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:16 smithi123 ceph-mon[134433]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:16.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:16.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:16.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:16.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:17.191 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 21:46:16 smithi089 systemd[1]: Starting Ceph mds.e for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:17.482 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 21:46:17 smithi089 podman[170716]: 2023-09-20T21:46:17.483 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:17 smithi089 ceph-mon[134090]: Deploying daemon mds.e on smithi089 2023-09-20T21:46:17.483 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:17 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.22:6836/4039103472,v1:172.21.15.22:6837/4039103472] up:boot 2023-09-20T21:46:17.483 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:17 smithi089 ceph-mon[134090]: fsmap 4 up:standby 2023-09-20T21:46:17.483 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-20T21:46:17.483 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:17 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1514438077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:17 smithi022 ceph-mon[97651]: Deploying daemon mds.e on smithi089 2023-09-20T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:17 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.22:6836/4039103472,v1:172.21.15.22:6837/4039103472] up:boot 2023-09-20T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:17 smithi022 ceph-mon[97651]: fsmap 4 up:standby 2023-09-20T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-20T21:46:17.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:17 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1514438077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:17.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:17 smithi123 ceph-mon[134433]: Deploying daemon mds.e on smithi089 2023-09-20T21:46:17.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:17 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.22:6836/4039103472,v1:172.21.15.22:6837/4039103472] up:boot 2023-09-20T21:46:17.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:17 smithi123 ceph-mon[134433]: fsmap 4 up:standby 2023-09-20T21:46:17.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-20T21:46:17.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:17 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1514438077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:17.861 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 21:46:17 smithi089 ceph-mds[170756]: -- 172.21.15.89:0/78286115 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d1ea4fb1e0 con 0x55d1ea4d0800 2023-09-20T21:46:17.861 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 21:46:17 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-e[170732]: starting mds.e at 2023-09-20T21:46:17.861 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 21:46:17 smithi089 bash[170716]: 37415591aa5915aaea81af089f9153042f86e4631383da9b63ed2d92b780578c 2023-09-20T21:46:17.862 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 21:46:17 smithi089 systemd[1]: Started Ceph mds.e for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:18.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:18.481 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:18 smithi022 ceph-mon[97651]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:18.481 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:18.482 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:18.482 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:18.482 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:18.482 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:18.483 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:18.533 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:18 smithi123 ceph-mon[134433]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:18.534 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:18.534 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:18.534 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:18.534 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:18.534 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:18.535 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:18.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:18 smithi089 ceph-mon[134090]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:18.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:18.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:18.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:18.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:18.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:18.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.626+0000 7f0a751ff700 1 -- 172.21.15.22:0/2878155384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a70101fe0 msgr2=0x7f0a70102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:18.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.626+0000 7f0a751ff700 1 --2- 172.21.15.22:0/2878155384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a70101fe0 0x7f0a70102440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a640099c0 tx=0x7f0a64009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.626+0000 7f0a751ff700 1 -- 172.21.15.22:0/2878155384 shutdown_connections 2023-09-20T21:46:18.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.626+0000 7f0a751ff700 1 --2- 172.21.15.22:0/2878155384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a70102980 0x7f0a70109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.626+0000 7f0a751ff700 1 --2- 172.21.15.22:0/2878155384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a70100df0 0x7f0a70101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.626+0000 7f0a751ff700 1 --2- 172.21.15.22:0/2878155384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a70101fe0 0x7f0a70102440 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.626+0000 7f0a751ff700 1 -- 172.21.15.22:0/2878155384 >> 172.21.15.22:0/2878155384 conn(0x7f0a700fc320 msgr2=0x7f0a700fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.626+0000 7f0a751ff700 1 -- 172.21.15.22:0/2878155384 shutdown_connections 2023-09-20T21:46:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.626+0000 7f0a751ff700 1 -- 172.21.15.22:0/2878155384 wait complete. 2023-09-20T21:46:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.627+0000 7f0a751ff700 1 Processor -- start 2023-09-20T21:46:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.627+0000 7f0a751ff700 1 -- start start 2023-09-20T21:46:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.627+0000 7f0a751ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a70100df0 0x7f0a70198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a751ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a70101fe0 0x7f0a701991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a751ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a70102980 0x7f0a701a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a751ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a701a0820 con 0x7f0a70101fe0 2023-09-20T21:46:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a751ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a701a0990 con 0x7f0a70100df0 2023-09-20T21:46:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a751ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a701a0b50 con 0x7f0a70102980 2023-09-20T21:46:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a70100df0 0x7f0a70198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a70102980 0x7f0a701a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a70101fe0 0x7f0a701991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:18.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a70100df0 0x7f0a70198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:47098/0 (socket says 172.21.15.22:47098) 2023-09-20T21:46:18.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6ed9d700 1 -- 172.21.15.22:0/4220776731 learned_addr learned my addr 172.21.15.22:0/4220776731 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:18.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6f59e700 1 -- 172.21.15.22:0/4220776731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a70100df0 msgr2=0x7f0a70198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:18.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6f59e700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a70100df0 0x7f0a70198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6f59e700 1 -- 172.21.15.22:0/4220776731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a70101fe0 msgr2=0x7f0a701991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6f59e700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a70101fe0 0x7f0a701991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6f59e700 1 -- 172.21.15.22:0/4220776731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a640095f0 con 0x7f0a70102980 2023-09-20T21:46:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.628+0000 7f0a6f59e700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a70102980 0x7f0a701a0250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800efc0 tx=0x7f0a6800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.629+0000 7f0a57fff700 1 -- 172.21.15.22:0/4220776731 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68018070 con 0x7f0a70102980 2023-09-20T21:46:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.629+0000 7f0a57fff700 1 -- 172.21.15.22:0/4220776731 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6800f040 con 0x7f0a70102980 2023-09-20T21:46:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.629+0000 7f0a57fff700 1 -- 172.21.15.22:0/4220776731 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6800cd40 con 0x7f0a70102980 2023-09-20T21:46:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.629+0000 7f0a751ff700 1 -- 172.21.15.22:0/4220776731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a701a0dd0 con 0x7f0a70102980 2023-09-20T21:46:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.629+0000 7f0a751ff700 1 -- 172.21.15.22:0/4220776731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a701a1200 con 0x7f0a70102980 2023-09-20T21:46:18.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.630+0000 7f0a751ff700 1 -- 172.21.15.22:0/4220776731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a70193440 con 0x7f0a70102980 2023-09-20T21:46:18.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.632+0000 7f0a57fff700 1 -- 172.21.15.22:0/4220776731 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f0a68003a00 con 0x7f0a70102980 2023-09-20T21:46:18.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.633+0000 7f0a57fff700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0a580a1350 0x7f0a580a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:18.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.633+0000 7f0a57fff700 1 -- 172.21.15.22:0/4220776731 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0a680cc520 con 0x7f0a70102980 2023-09-20T21:46:18.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.633+0000 7f0a6ed9d700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0a580a1350 0x7f0a580a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.634+0000 7f0a6ed9d700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0a580a1350 0x7f0a580a3810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a70199b80 tx=0x7f0a60009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.635+0000 7f0a57fff700 1 -- 172.21.15.22:0/4220776731 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0a68096080 con 0x7f0a70102980 2023-09-20T21:46:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.890+0000 7f0a751ff700 1 -- 172.21.15.22:0/4220776731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a7010c8a0 con 0x7f0a70102980 2023-09-20T21:46:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.891+0000 7f0a57fff700 1 -- 172.21.15.22:0/4220776731 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f0a680996a0 con 0x7f0a70102980 2023-09-20T21:46:18.891 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:18.892 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-09-20T21:46:18.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.893+0000 7f0a751ff700 1 -- 172.21.15.22:0/4220776731 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0a580a1350 msgr2=0x7f0a580a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:18.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.893+0000 7f0a751ff700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0a580a1350 0x7f0a580a3810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a70199b80 tx=0x7f0a60009260 comp rx=0 tx=0).stop 2023-09-20T21:46:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.893+0000 7f0a751ff700 1 -- 172.21.15.22:0/4220776731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a70102980 msgr2=0x7f0a701a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.893+0000 7f0a751ff700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a70102980 0x7f0a701a0250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800efc0 tx=0x7f0a6800c3a0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.894+0000 7f0a751ff700 1 -- 172.21.15.22:0/4220776731 shutdown_connections 2023-09-20T21:46:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.894+0000 7f0a751ff700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a70102980 0x7f0a701a0250 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.894+0000 7f0a751ff700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0a580a1350 0x7f0a580a3810 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.894+0000 7f0a751ff700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a70100df0 0x7f0a70198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.894+0000 7f0a751ff700 1 --2- 172.21.15.22:0/4220776731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a70101fe0 0x7f0a701991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.894+0000 7f0a751ff700 1 -- 172.21.15.22:0/4220776731 >> 172.21.15.22:0/4220776731 conn(0x7f0a700fc320 msgr2=0x7f0a70107890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.894+0000 7f0a751ff700 1 -- 172.21.15.22:0/4220776731 shutdown_connections 2023-09-20T21:46:18.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:18.894+0000 7f0a751ff700 1 -- 172.21.15.22:0/4220776731 wait complete. 2023-09-20T21:46:18.897 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 6 2023-09-20T21:46:19.076 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 21:46:19 smithi123 systemd[1]: Starting Ceph mds.f for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:19.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:19 smithi123 ceph-mon[134433]: Deploying daemon mds.f on smithi123 2023-09-20T21:46:19.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:19 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.89:6834/794452586,v1:172.21.15.89:6835/794452586] up:boot 2023-09-20T21:46:19.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:19 smithi123 ceph-mon[134433]: fsmap 5 up:standby 2023-09-20T21:46:19.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-20T21:46:19.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:19 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4220776731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:19.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:19 smithi022 ceph-mon[97651]: Deploying daemon mds.f on smithi123 2023-09-20T21:46:19.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:19 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.89:6834/794452586,v1:172.21.15.89:6835/794452586] up:boot 2023-09-20T21:46:19.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:19 smithi022 ceph-mon[97651]: fsmap 5 up:standby 2023-09-20T21:46:19.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-20T21:46:19.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:19 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4220776731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:19.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:19 smithi089 ceph-mon[134090]: Deploying daemon mds.f on smithi123 2023-09-20T21:46:19.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:19 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.89:6834/794452586,v1:172.21.15.89:6835/794452586] up:boot 2023-09-20T21:46:19.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:19 smithi089 ceph-mon[134090]: fsmap 5 up:standby 2023-09-20T21:46:19.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-20T21:46:19.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:19 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4220776731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:19.661 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 21:46:19 smithi123 podman[171898]: 2023-09-20T21:46:19.662 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 21:46:19 smithi123 ceph-mds[171938]: -- 172.21.15.123:0/3036366703 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555e6e8031e0 con 0x555e6e7d8800 2023-09-20T21:46:19.662 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 21:46:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-f[171914]: starting mds.f at 2023-09-20T21:46:19.662 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 21:46:19 smithi123 bash[171898]: c640e4b2ac2c1311512834f24b20ba77c1ab10c8dde3ab192cc2e22112ed39e4 2023-09-20T21:46:20.093 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 21:46:19 smithi123 systemd[1]: Started Ceph mds.f for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:20.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:20 smithi022 ceph-mon[97651]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:20.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:20.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:20.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:20.528 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:20 smithi123 ceph-mon[134433]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:20.528 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:20.528 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:20.528 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:20.529 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:20.529 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:20.529 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:20.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:20 smithi089 ceph-mon[134090]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:20.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:20.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:20.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:21.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.127+0000 7fe237fff700 1 -- 172.21.15.22:0/532521949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe230101fd0 msgr2=0x7fe230102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:21.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.127+0000 7fe237fff700 1 --2- 172.21.15.22:0/532521949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe230101fd0 0x7fe230102430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe220009990 tx=0x7fe220009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:21.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.127+0000 7fe237fff700 1 -- 172.21.15.22:0/532521949 shutdown_connections 2023-09-20T21:46:21.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.127+0000 7fe237fff700 1 --2- 172.21.15.22:0/532521949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230102970 0x7fe230109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.127+0000 7fe237fff700 1 --2- 172.21.15.22:0/532521949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe230100de0 0x7fe230101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.127+0000 7fe237fff700 1 --2- 172.21.15.22:0/532521949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe230101fd0 0x7fe230102430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.127+0000 7fe237fff700 1 -- 172.21.15.22:0/532521949 >> 172.21.15.22:0/532521949 conn(0x7fe2300fc370 msgr2=0x7fe2300fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.127+0000 7fe237fff700 1 -- 172.21.15.22:0/532521949 shutdown_connections 2023-09-20T21:46:21.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.127+0000 7fe237fff700 1 -- 172.21.15.22:0/532521949 wait complete. 2023-09-20T21:46:21.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.129+0000 7fe237fff700 1 Processor -- start 2023-09-20T21:46:21.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.129+0000 7fe237fff700 1 -- start start 2023-09-20T21:46:21.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.129+0000 7fe237fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe230100de0 0x7fe230074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.129+0000 7fe237fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe230101fd0 0x7fe2300731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.129+0000 7fe237fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230102970 0x7fe230073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.129+0000 7fe237fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230073eb0 con 0x7fe230102970 2023-09-20T21:46:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe237fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230074020 con 0x7fe230100de0 2023-09-20T21:46:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe237fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2301a0b30 con 0x7fe230101fd0 2023-09-20T21:46:21.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe23559a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe230101fd0 0x7fe2300731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe235d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe230100de0 0x7fe230074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe235d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe230100de0 0x7fe230074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:47122/0 (socket says 172.21.15.22:47122) 2023-09-20T21:46:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe235d9b700 1 -- 172.21.15.22:0/727699341 learned_addr learned my addr 172.21.15.22:0/727699341 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe23659c700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230102970 0x7fe230073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe23559a700 1 -- 172.21.15.22:0/727699341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe230100de0 msgr2=0x7fe230074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe23559a700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe230100de0 0x7fe230074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe23559a700 1 -- 172.21.15.22:0/727699341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230102970 msgr2=0x7fe230073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe23559a700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230102970 0x7fe230073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe23559a700 1 -- 172.21.15.22:0/727699341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2200095f0 con 0x7fe230101fd0 2023-09-20T21:46:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.130+0000 7fe23559a700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe230101fd0 0x7fe2300731d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe2200098e0 tx=0x7fe220004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.131+0000 7fe226ffd700 1 -- 172.21.15.22:0/727699341 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22001d070 con 0x7fe230101fd0 2023-09-20T21:46:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.131+0000 7fe226ffd700 1 -- 172.21.15.22:0/727699341 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2200054f0 con 0x7fe230101fd0 2023-09-20T21:46:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.131+0000 7fe237fff700 1 -- 172.21.15.22:0/727699341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2301a0db0 con 0x7fe230101fd0 2023-09-20T21:46:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.131+0000 7fe226ffd700 1 -- 172.21.15.22:0/727699341 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220004b90 con 0x7fe230101fd0 2023-09-20T21:46:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.131+0000 7fe237fff700 1 -- 172.21.15.22:0/727699341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2301a11d0 con 0x7fe230101fd0 2023-09-20T21:46:21.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.132+0000 7fe237fff700 1 -- 172.21.15.22:0/727699341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2301083c0 con 0x7fe230101fd0 2023-09-20T21:46:21.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.134+0000 7fe226ffd700 1 -- 172.21.15.22:0/727699341 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe22000fb90 con 0x7fe230101fd0 2023-09-20T21:46:21.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.135+0000 7fe226ffd700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe21c0a13d0 0x7fe21c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:21.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.135+0000 7fe226ffd700 1 -- 172.21.15.22:0/727699341 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2200cd8b0 con 0x7fe230101fd0 2023-09-20T21:46:21.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.135+0000 7fe235d9b700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe21c0a13d0 0x7fe21c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:21.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.136+0000 7fe235d9b700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe21c0a13d0 0x7fe21c0a3890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe22c005950 tx=0x7fe22c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.139+0000 7fe226ffd700 1 -- 172.21.15.22:0/727699341 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe220097410 con 0x7fe230101fd0 2023-09-20T21:46:21.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.394+0000 7fe237fff700 1 -- 172.21.15.22:0/727699341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe23010c640 con 0x7fe230101fd0 2023-09-20T21:46:21.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.395+0000 7fe226ffd700 1 -- 172.21.15.22:0/727699341 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5274 (secure 0 0 0) 0x7fe22009aa30 con 0x7fe230101fd0 2023-09-20T21:46:21.396 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:21.396 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-09-20T21:46:21.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 -- 172.21.15.22:0/727699341 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe21c0a13d0 msgr2=0x7fe21c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:21.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe21c0a13d0 0x7fe21c0a3890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe22c005950 tx=0x7fe22c009260 comp rx=0 tx=0).stop 2023-09-20T21:46:21.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 -- 172.21.15.22:0/727699341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe230101fd0 msgr2=0x7fe2300731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:21.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe230101fd0 0x7fe2300731d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe2200098e0 tx=0x7fe220004730 comp rx=0 tx=0).stop 2023-09-20T21:46:21.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 -- 172.21.15.22:0/727699341 shutdown_connections 2023-09-20T21:46:21.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe230102970 0x7fe230073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:21.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe21c0a13d0 0x7fe21c0a3890 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:21.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe230100de0 0x7fe230074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:21.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 --2- 172.21.15.22:0/727699341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe230101fd0 0x7fe2300731d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:21.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 -- 172.21.15.22:0/727699341 >> 172.21.15.22:0/727699341 conn(0x7fe2300fc370 msgr2=0x7fe230105500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:21.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 -- 172.21.15.22:0/727699341 shutdown_connections 2023-09-20T21:46:21.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:21.397+0000 7fe237fff700 1 -- 172.21.15.22:0/727699341 wait complete. 2023-09-20T21:46:21.401 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-09-20T21:46:21.489 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:21 smithi123 ceph-mon[134433]: Deploying daemon mds.c on smithi123 2023-09-20T21:46:21.490 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:21 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] up:boot 2023-09-20T21:46:21.490 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:21 smithi123 ceph-mon[134433]: fsmap 6 up:standby 2023-09-20T21:46:21.490 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-20T21:46:21.491 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 21:46:21 smithi123 systemd[1]: Starting Ceph mds.c for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:21.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:21 smithi022 ceph-mon[97651]: Deploying daemon mds.c on smithi123 2023-09-20T21:46:21.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:21 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] up:boot 2023-09-20T21:46:21.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:21 smithi022 ceph-mon[97651]: fsmap 6 up:standby 2023-09-20T21:46:21.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-20T21:46:21.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:21 smithi089 ceph-mon[134090]: Deploying daemon mds.c on smithi123 2023-09-20T21:46:21.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:21 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] up:boot 2023-09-20T21:46:21.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:21 smithi089 ceph-mon[134090]: fsmap 6 up:standby 2023-09-20T21:46:21.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-20T21:46:21.756 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 21:46:21 smithi123 podman[172236]: 2023-09-20T21:46:22.093 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 21:46:21 smithi123 ceph-mds[172276]: -- 172.21.15.123:0/1894770619 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5588b980f1e0 con 0x5588b97e4800 2023-09-20T21:46:22.093 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 21:46:21 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-c[172252]: starting mds.c at 2023-09-20T21:46:22.094 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 21:46:21 smithi123 bash[172236]: b4bce249e0a935cfb40848a4dac7c8fadd50448533d5733f592184f67fc602dc 2023-09-20T21:46:22.094 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 21:46:21 smithi123 systemd[1]: Started Ceph mds.c for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:22.507 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:22 smithi089 ceph-mon[134090]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:22.507 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:22 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/727699341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:22.507 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:22.507 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:22.508 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:22.508 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:22.508 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:22.508 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:22 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:22.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:22 smithi123 ceph-mon[134433]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:22.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:22 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/727699341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:22.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:22.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:22.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:22.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:22.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:22.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:22 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:22.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:22 smithi022 ceph-mon[97651]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:22.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:22 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/727699341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:22.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:22.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:22.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:22.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:22.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:22.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:22 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:23.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:23.466 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:23 smithi089 ceph-mon[134090]: Deploying daemon mds.h on smithi089 2023-09-20T21:46:23.467 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:23 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.123:6836/1168160038,v1:172.21.15.123:6837/1168160038] up:boot 2023-09-20T21:46:23.467 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:23 smithi089 ceph-mon[134090]: fsmap 7 up:standby 2023-09-20T21:46:23.468 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:23 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-20T21:46:23.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:23 smithi123 ceph-mon[134433]: Deploying daemon mds.h on smithi089 2023-09-20T21:46:23.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:23 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.123:6836/1168160038,v1:172.21.15.123:6837/1168160038] up:boot 2023-09-20T21:46:23.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:23 smithi123 ceph-mon[134433]: fsmap 7 up:standby 2023-09-20T21:46:23.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:23 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-20T21:46:23.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.634+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1938326388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8101fd0 msgr2=0x7f7bd8102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:23.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.634+0000 7f7bdc9ff700 1 --2- 172.21.15.22:0/1938326388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8101fd0 0x7f7bd8102430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0099c0 tx=0x7f7bcc009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:23.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.634+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1938326388 shutdown_connections 2023-09-20T21:46:23.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.634+0000 7f7bdc9ff700 1 --2- 172.21.15.22:0/1938326388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd8102970 0x7f7bd8109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:23.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.634+0000 7f7bdc9ff700 1 --2- 172.21.15.22:0/1938326388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bd8100de0 0x7f7bd8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:23.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.634+0000 7f7bdc9ff700 1 --2- 172.21.15.22:0/1938326388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8101fd0 0x7f7bd8102430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:23.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.634+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1938326388 >> 172.21.15.22:0/1938326388 conn(0x7f7bd80fc310 msgr2=0x7f7bd80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:23.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.634+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1938326388 shutdown_connections 2023-09-20T21:46:23.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.634+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1938326388 wait complete. 2023-09-20T21:46:23.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.635+0000 7f7bdc9ff700 1 Processor -- start 2023-09-20T21:46:23.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.635+0000 7f7bdc9ff700 1 -- start start 2023-09-20T21:46:23.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.635+0000 7f7bdc9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8100de0 0x7f7bd8196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:23.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bdc9ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bd8101fd0 0x7f7bd8196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:23.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bdc9ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd8102970 0x7f7bd819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:23.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bdc9ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd819e630 con 0x7f7bd8101fd0 2023-09-20T21:46:23.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bdc9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd819e770 con 0x7f7bd8100de0 2023-09-20T21:46:23.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bdc9ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd819e900 con 0x7f7bd8102970 2023-09-20T21:46:23.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bd5d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bd8101fd0 0x7f7bd8196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:23.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bd5d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bd8101fd0 0x7f7bd8196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:56156/0 (socket says 172.21.15.22:56156) 2023-09-20T21:46:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bd5d9b700 1 -- 172.21.15.22:0/1873322923 learned_addr learned my addr 172.21.15.22:0/1873322923 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bd6d9d700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd8102970 0x7f7bd819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bd659c700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8100de0 0x7f7bd8196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:23.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bd5d9b700 1 -- 172.21.15.22:0/1873322923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd8102970 msgr2=0x7f7bd819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bd5d9b700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd8102970 0x7f7bd819e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bd5d9b700 1 -- 172.21.15.22:0/1873322923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8100de0 msgr2=0x7f7bd8196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bd5d9b700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8100de0 0x7f7bd8196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.636+0000 7f7bd5d9b700 1 -- 172.21.15.22:0/1873322923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0095f0 con 0x7f7bd8101fd0 2023-09-20T21:46:23.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.637+0000 7f7bd5d9b700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bd8101fd0 0x7f7bd8196ff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009990 tx=0x7f7bcc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:23.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.637+0000 7f7bc77fe700 1 -- 172.21.15.22:0/1873322923 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc01d070 con 0x7f7bd8101fd0 2023-09-20T21:46:23.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.637+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1873322923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd819eb80 con 0x7f7bd8101fd0 2023-09-20T21:46:23.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.637+0000 7f7bc77fe700 1 -- 172.21.15.22:0/1873322923 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bcc005030 con 0x7f7bd8101fd0 2023-09-20T21:46:23.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.637+0000 7f7bc77fe700 1 -- 172.21.15.22:0/1873322923 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc004b30 con 0x7f7bd8101fd0 2023-09-20T21:46:23.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.637+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1873322923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd819f000 con 0x7f7bd8101fd0 2023-09-20T21:46:23.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.638+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1873322923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd8108320 con 0x7f7bd8101fd0 2023-09-20T21:46:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.640+0000 7f7bc77fe700 1 -- 172.21.15.22:0/1873322923 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f7bcc004cd0 con 0x7f7bd8101fd0 2023-09-20T21:46:23.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.641+0000 7f7bc77fe700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7bc00a1350 0x7f7bc00a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.642+0000 7f7bc77fe700 1 -- 172.21.15.22:0/1873322923 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7bcc0cd630 con 0x7f7bd8101fd0 2023-09-20T21:46:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.642+0000 7f7bd659c700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7bc00a1350 0x7f7bc00a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.642+0000 7f7bd659c700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7bc00a1350 0x7f7bc00a3810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8101e30 tx=0x7f7bc8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.646+0000 7f7bc77fe700 1 -- 172.21.15.22:0/1873322923 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7bcc097190 con 0x7f7bd8101fd0 2023-09-20T21:46:23.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:23 smithi022 ceph-mon[97651]: Deploying daemon mds.h on smithi089 2023-09-20T21:46:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:23 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.123:6836/1168160038,v1:172.21.15.123:6837/1168160038] up:boot 2023-09-20T21:46:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:23 smithi022 ceph-mon[97651]: fsmap 7 up:standby 2023-09-20T21:46:23.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:23 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-20T21:46:23.767 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 21:46:23 smithi089 systemd[1]: Starting Ceph mds.h for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:23.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.903+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1873322923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bd802d110 con 0x7f7bd8101fd0 2023-09-20T21:46:23.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.904+0000 7f7bc77fe700 1 -- 172.21.15.22:0/1873322923 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6068 (secure 0 0 0) 0x7f7bcc09a7b0 con 0x7f7bd8101fd0 2023-09-20T21:46:23.905 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:23.905 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-09-20T21:46:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1873322923 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7bc00a1350 msgr2=0x7f7bc00a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7bc00a1350 0x7f7bc00a3810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8101e30 tx=0x7f7bc8009260 comp rx=0 tx=0).stop 2023-09-20T21:46:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1873322923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bd8101fd0 msgr2=0x7f7bd8196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:23.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bd8101fd0 0x7f7bd8196ff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009990 tx=0x7f7bcc00f690 comp rx=0 tx=0).stop 2023-09-20T21:46:23.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1873322923 shutdown_connections 2023-09-20T21:46:23.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bd8102970 0x7f7bd819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:23.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7bc00a1350 0x7f7bc00a3810 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:23.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bd8100de0 0x7f7bd8196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:23.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 --2- 172.21.15.22:0/1873322923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7bd8101fd0 0x7f7bd8196ff0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:23.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1873322923 >> 172.21.15.22:0/1873322923 conn(0x7f7bd80fc310 msgr2=0x7f7bd8105460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:23.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1873322923 shutdown_connections 2023-09-20T21:46:23.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:23.906+0000 7f7bdc9ff700 1 -- 172.21.15.22:0/1873322923 wait complete. 2023-09-20T21:46:23.910 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2023-09-20T21:46:24.064 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 21:46:23 smithi089 podman[171058]: 2023-09-20T21:46:24.360 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 21:46:24 smithi089 ceph-mds[171099]: -- 172.21.15.89:0/595512479 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d3edb2d1e0 con 0x55d3edb02800 2023-09-20T21:46:24.361 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 21:46:24 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-h[171074]: starting mds.h at 2023-09-20T21:46:24.361 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 21:46:24 smithi089 bash[171058]: 2ecc198b1095414666aa9d541ab8c45e8043c89e081c50a45144af24998c770c 2023-09-20T21:46:24.361 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 21:46:24 smithi089 systemd[1]: Started Ceph mds.h for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:24.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:24 smithi089 ceph-mon[134090]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:24.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:24 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1873322923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:24.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:24.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:24.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:24.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:24.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:24.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:24.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:24 smithi123 ceph-mon[134433]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:24.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:24 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1873322923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:24.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:24.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:24.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:24.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:24.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:24.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:24.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:24 smithi022 ceph-mon[97651]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:24.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:24 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1873322923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:24.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:24.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:24.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:24.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:24.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:24.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:25.482 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:25 smithi123 ceph-mon[134433]: Deploying daemon mds.i on smithi123 2023-09-20T21:46:25.482 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:25 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] up:boot 2023-09-20T21:46:25.483 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:25 smithi123 ceph-mon[134433]: fsmap 8 up:standby 2023-09-20T21:46:25.483 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:25 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-20T21:46:25.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:25 smithi089 ceph-mon[134090]: Deploying daemon mds.i on smithi123 2023-09-20T21:46:25.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:25 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] up:boot 2023-09-20T21:46:25.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:25 smithi089 ceph-mon[134090]: fsmap 8 up:standby 2023-09-20T21:46:25.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:25 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-20T21:46:25.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:25 smithi022 ceph-mon[97651]: Deploying daemon mds.i on smithi123 2023-09-20T21:46:25.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:25 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] up:boot 2023-09-20T21:46:25.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:25 smithi022 ceph-mon[97651]: fsmap 8 up:standby 2023-09-20T21:46:25.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-20T21:46:25.843 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 21:46:25 smithi123 systemd[1]: Starting Ceph mds.i for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:25.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.141+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3640887308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f8100de0 msgr2=0x7fa5f8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.141+0000 7fa5ff3ff700 1 --2- 172.21.15.22:0/3640887308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f8100de0 0x7fa5f8101200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40099c0 tx=0x7fa5f4009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.141+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3640887308 shutdown_connections 2023-09-20T21:46:26.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.141+0000 7fa5ff3ff700 1 --2- 172.21.15.22:0/3640887308 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5f8102970 0x7fa5f8109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.141+0000 7fa5ff3ff700 1 --2- 172.21.15.22:0/3640887308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f8100de0 0x7fa5f8101200 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.141+0000 7fa5ff3ff700 1 --2- 172.21.15.22:0/3640887308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5f8101fd0 0x7fa5f8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.141+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3640887308 >> 172.21.15.22:0/3640887308 conn(0x7fa5f80fc310 msgr2=0x7fa5f80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:26.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.141+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3640887308 shutdown_connections 2023-09-20T21:46:26.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.141+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3640887308 wait complete. 2023-09-20T21:46:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.142+0000 7fa5ff3ff700 1 Processor -- start 2023-09-20T21:46:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.142+0000 7fa5ff3ff700 1 -- start start 2023-09-20T21:46:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5ff3ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5f8100de0 0x7fa5f8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:26.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5ff3ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f8101fd0 0x7fa5f8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:26.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5ff3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5f8102970 0x7fa5f81a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:26.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5ff3ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f81a0830 con 0x7fa5f8101fd0 2023-09-20T21:46:26.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5ff3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f81a09a0 con 0x7fa5f8102970 2023-09-20T21:46:26.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5ff3ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f81a0b60 con 0x7fa5f8100de0 2023-09-20T21:46:26.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5fd19b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5f8100de0 0x7fa5f8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5fd19b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5f8100de0 0x7fa5f8198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:39998/0 (socket says 172.21.15.22:39998) 2023-09-20T21:46:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5fd19b700 1 -- 172.21.15.22:0/3469445080 learned_addr learned my addr 172.21.15.22:0/3469445080 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5fd99c700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5f8102970 0x7fa5f81a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.143+0000 7fa5fc99a700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f8101fd0 0x7fa5f8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:26.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5fd19b700 1 -- 172.21.15.22:0/3469445080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5f8102970 msgr2=0x7fa5f81a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:26.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5fd19b700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5f8102970 0x7fa5f81a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5fd19b700 1 -- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f8101fd0 msgr2=0x7fa5f8199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:26.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5fd19b700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f8101fd0 0x7fa5f8199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5fd19b700 1 -- 172.21.15.22:0/3469445080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f40095f0 con 0x7fa5f8100de0 2023-09-20T21:46:26.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5fc99a700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f8101fd0 0x7fa5f8199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:46:26.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5fd19b700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5f8100de0 0x7fa5f8198cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009990 tx=0x7fa5f400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:26.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5ee7fc700 1 -- 172.21.15.22:0/3469445080 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f401d070 con 0x7fa5f8100de0 2023-09-20T21:46:26.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5ee7fc700 1 -- 172.21.15.22:0/3469445080 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5f40054f0 con 0x7fa5f8100de0 2023-09-20T21:46:26.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5ee7fc700 1 -- 172.21.15.22:0/3469445080 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4004bd0 con 0x7fa5f8100de0 2023-09-20T21:46:26.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3469445080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f81a0de0 con 0x7fa5f8100de0 2023-09-20T21:46:26.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.144+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3469445080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f81a11b0 con 0x7fa5f8100de0 2023-09-20T21:46:26.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.145+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3469445080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f8193430 con 0x7fa5f8100de0 2023-09-20T21:46:26.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.147+0000 7fa5ee7fc700 1 -- 172.21.15.22:0/3469445080 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa5f400fb80 con 0x7fa5f8100de0 2023-09-20T21:46:26.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.149+0000 7fa5ee7fc700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa5e40a13d0 0x7fa5e40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:26.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.149+0000 7fa5ee7fc700 1 -- 172.21.15.22:0/3469445080 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5f40cd680 con 0x7fa5f8100de0 2023-09-20T21:46:26.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.149+0000 7fa5fc99a700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa5e40a13d0 0x7fa5e40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:26.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.150+0000 7fa5fc99a700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa5e40a13d0 0x7fa5e40a3890 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8005950 tx=0x7fa5e80058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:26.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.151+0000 7fa5ee7fc700 1 -- 172.21.15.22:0/3469445080 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa5f40971e0 con 0x7fa5f8100de0 2023-09-20T21:46:26.305 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:26 smithi123 ceph-mon[134433]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:26.306 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 21:46:26 smithi123 podman[172576]: 2023-09-20T21:46:26.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.415+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3469445080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5f810c640 con 0x7fa5f8100de0 2023-09-20T21:46:26.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.416+0000 7fa5ee7fc700 1 -- 172.21.15.22:0/3469445080 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6859 (secure 0 0 0) 0x7fa5f409a800 con 0x7fa5f8100de0 2023-09-20T21:46:26.417 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:26.417 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-20T21:46:26.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.419+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa5e40a13d0 msgr2=0x7fa5e40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:26.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.419+0000 7fa5ff3ff700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa5e40a13d0 0x7fa5e40a3890 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8005950 tx=0x7fa5e80058e0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.419+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3469445080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5f8100de0 msgr2=0x7fa5f8198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:26.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.419+0000 7fa5ff3ff700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5f8100de0 0x7fa5f8198cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009990 tx=0x7fa5f400f690 comp rx=0 tx=0).stop 2023-09-20T21:46:26.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.419+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3469445080 shutdown_connections 2023-09-20T21:46:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.419+0000 7fa5ff3ff700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa5e40a13d0 0x7fa5e40a3890 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.419+0000 7fa5ff3ff700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5f8102970 0x7fa5f81a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.419+0000 7fa5ff3ff700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5f8100de0 0x7fa5f8198cf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.419+0000 7fa5ff3ff700 1 --2- 172.21.15.22:0/3469445080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f8101fd0 0x7fa5f8199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:26.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.419+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3469445080 >> 172.21.15.22:0/3469445080 conn(0x7fa5f80fc310 msgr2=0x7fa5f81078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.420+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3469445080 shutdown_connections 2023-09-20T21:46:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:26.420+0000 7fa5ff3ff700 1 -- 172.21.15.22:0/3469445080 wait complete. 2023-09-20T21:46:26.422 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 9 2023-09-20T21:46:26.593 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 21:46:26 smithi123 ceph-mds[172616]: -- 172.21.15.123:0/1016958648 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56396f6951e0 con 0x56396f66ac00 2023-09-20T21:46:26.593 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 21:46:26 smithi123 bash[172576]: 5c07339c5af72d67cad0ae5fd33f3bd865dd998428befb977aa3017ca755d232 2023-09-20T21:46:26.594 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 21:46:26 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-i[172592]: starting mds.i at 2023-09-20T21:46:26.594 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 21:46:26 smithi123 systemd[1]: Started Ceph mds.i for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:26.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:26 smithi089 ceph-mon[134090]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:26.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:26 smithi022 ceph-mon[97651]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:27.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:27.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:27.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:27.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:27.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:27.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:27.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: Deploying daemon mds.g on smithi022 2023-09-20T21:46:27.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3469445080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:27.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:27.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.123:6838/2368757728,v1:172.21.15.123:6839/2368757728] up:boot 2023-09-20T21:46:27.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: fsmap 9 up:standby 2023-09-20T21:46:27.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:27 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-20T21:46:27.630 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:27.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:27.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:27.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:27.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:27.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:27.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: Deploying daemon mds.g on smithi022 2023-09-20T21:46:27.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3469445080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:27.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:27.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.123:6838/2368757728,v1:172.21.15.123:6839/2368757728] up:boot 2023-09-20T21:46:27.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: fsmap 9 up:standby 2023-09-20T21:46:27.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:27 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-20T21:46:27.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:27.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:27.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:27.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:27.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:27.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:27.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: Deploying daemon mds.g on smithi022 2023-09-20T21:46:27.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3469445080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:27.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:27.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.123:6838/2368757728,v1:172.21.15.123:6839/2368757728] up:boot 2023-09-20T21:46:27.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: fsmap 9 up:standby 2023-09-20T21:46:27.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:27 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-20T21:46:28.441 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:28.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.693+0000 7f75f53ff700 1 -- 172.21.15.22:0/3084695180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75f0101840 msgr2=0x7f75f0103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:28.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.693+0000 7f75f53ff700 1 --2- 172.21.15.22:0/3084695180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75f0101840 0x7f75f0103c30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009990 tx=0x7f75e0009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.693+0000 7f75f53ff700 1 -- 172.21.15.22:0/3084695180 shutdown_connections 2023-09-20T21:46:28.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.693+0000 7f75f53ff700 1 --2- 172.21.15.22:0/3084695180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75f0106aa0 0x7f75f0108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.693+0000 7f75f53ff700 1 --2- 172.21.15.22:0/3084695180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75f0101840 0x7f75f0103c30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.693+0000 7f75f53ff700 1 --2- 172.21.15.22:0/3084695180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75f0104170 0x7f75f0106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.693+0000 7f75f53ff700 1 -- 172.21.15.22:0/3084695180 >> 172.21.15.22:0/3084695180 conn(0x7f75f00fb210 msgr2=0x7f75f00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:28.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.693+0000 7f75f53ff700 1 -- 172.21.15.22:0/3084695180 shutdown_connections 2023-09-20T21:46:28.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.693+0000 7f75f53ff700 1 -- 172.21.15.22:0/3084695180 wait complete. 2023-09-20T21:46:28.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.695+0000 7f75f53ff700 1 Processor -- start 2023-09-20T21:46:28.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.695+0000 7f75f53ff700 1 -- start start 2023-09-20T21:46:28.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75f53ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75f0101840 0x7f75f0196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:28.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75f53ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75f0104170 0x7f75f0196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:28.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75f53ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75f0106aa0 0x7f75f019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:28.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75f53ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f019e600 con 0x7f75f0104170 2023-09-20T21:46:28.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75f53ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f019e740 con 0x7f75f0106aa0 2023-09-20T21:46:28.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75f53ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f019e8d0 con 0x7f75f0101840 2023-09-20T21:46:28.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75ee7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75f0104170 0x7f75f0196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:28.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75ee7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75f0104170 0x7f75f0196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49286/0 (socket says 172.21.15.22:49286) 2023-09-20T21:46:28.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75ee7fc700 1 -- 172.21.15.22:0/1817104834 learned_addr learned my addr 172.21.15.22:0/1817104834 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:28.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75eeffd700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75f0101840 0x7f75f0196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:28.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75ef7fe700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75f0106aa0 0x7f75f019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:28.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75ee7fc700 1 -- 172.21.15.22:0/1817104834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75f0101840 msgr2=0x7f75f0196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:28.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.696+0000 7f75ee7fc700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75f0101840 0x7f75f0196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.697+0000 7f75ee7fc700 1 -- 172.21.15.22:0/1817104834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75f0106aa0 msgr2=0x7f75f019e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:28.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.697+0000 7f75ee7fc700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75f0106aa0 0x7f75f019e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.697+0000 7f75ee7fc700 1 -- 172.21.15.22:0/1817104834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00095f0 con 0x7f75f0104170 2023-09-20T21:46:28.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.697+0000 7f75ee7fc700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75f0104170 0x7f75f0196fc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75e400e9a0 tx=0x7f75e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:28.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.697+0000 7f75d7fff700 1 -- 172.21.15.22:0/1817104834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4009790 con 0x7f75f0104170 2023-09-20T21:46:28.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.697+0000 7f75d7fff700 1 -- 172.21.15.22:0/1817104834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e4010430 con 0x7f75f0104170 2023-09-20T21:46:28.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.697+0000 7f75f53ff700 1 -- 172.21.15.22:0/1817104834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f019ebb0 con 0x7f75f0104170 2023-09-20T21:46:28.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.697+0000 7f75d7fff700 1 -- 172.21.15.22:0/1817104834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4018680 con 0x7f75f0104170 2023-09-20T21:46:28.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.697+0000 7f75f53ff700 1 -- 172.21.15.22:0/1817104834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f019f0c0 con 0x7f75f0104170 2023-09-20T21:46:28.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.699+0000 7f75f53ff700 1 -- 172.21.15.22:0/1817104834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f00fcd90 con 0x7f75f0104170 2023-09-20T21:46:28.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.702+0000 7f75d7fff700 1 -- 172.21.15.22:0/1817104834 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f75e40105a0 con 0x7f75f0104170 2023-09-20T21:46:28.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.704+0000 7f75d7fff700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75d80a1350 0x7f75d80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:28.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.704+0000 7f75d7fff700 1 -- 172.21.15.22:0/1817104834 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f75e4014070 con 0x7f75f0104170 2023-09-20T21:46:28.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.705+0000 7f75eeffd700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75d80a1350 0x7f75d80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:28.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.705+0000 7f75eeffd700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75d80a1350 0x7f75d80a3810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f75e000b2e0 tx=0x7f75e0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:28.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.709+0000 7f75d7fff700 1 -- 172.21.15.22:0/1817104834 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f75e40964e0 con 0x7f75f0104170 2023-09-20T21:46:28.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.972+0000 7f75f53ff700 1 -- 172.21.15.22:0/1817104834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75f010c5f0 con 0x7f75f0104170 2023-09-20T21:46:28.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.972+0000 7f75d7fff700 1 -- 172.21.15.22:0/1817104834 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7655 (secure 0 0 0) 0x7f75e4099b00 con 0x7f75f0104170 2023-09-20T21:46:28.973 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:28.973 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-20T21:46:28.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 -- 172.21.15.22:0/1817104834 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75d80a1350 msgr2=0x7f75d80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:28.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75d80a1350 0x7f75d80a3810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f75e000b2e0 tx=0x7f75e0005fb0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 -- 172.21.15.22:0/1817104834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75f0104170 msgr2=0x7f75f0196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:28.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75f0104170 0x7f75f0196fc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75e400e9a0 tx=0x7f75e400ecb0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 -- 172.21.15.22:0/1817104834 shutdown_connections 2023-09-20T21:46:28.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75d80a1350 0x7f75d80a3810 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f75f0106aa0 0x7f75f019e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75f0101840 0x7f75f0196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 --2- 172.21.15.22:0/1817104834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75f0104170 0x7f75f0196fc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:28.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 -- 172.21.15.22:0/1817104834 >> 172.21.15.22:0/1817104834 conn(0x7f75f00fb210 msgr2=0x7f75f01024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:28.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 -- 172.21.15.22:0/1817104834 shutdown_connections 2023-09-20T21:46:28.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:28.975+0000 7f75f53ff700 1 -- 172.21.15.22:0/1817104834 wait complete. 2023-09-20T21:46:28.978 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 10 2023-09-20T21:46:29.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:29 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1817104834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:29.248 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 20 21:46:28 smithi022 systemd[1]: Starting Ceph mds.g for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:29.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:29 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1817104834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:29.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:29 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1817104834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:29.609 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 20 21:46:29 smithi022 podman[148573]: 2023-09-20T21:46:29.609 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 20 21:46:29 smithi022 ceph-mds[148613]: -- 172.21.15.22:0/2069490855 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556860ab11e0 con 0x556860a86c00 2023-09-20T21:46:29.609 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 20 21:46:29 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-g[148589]: starting mds.g at 2023-09-20T21:46:29.997 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 20 21:46:29 smithi022 bash[148573]: af018144da6c96ef2f7bc4edc786a4afe02866ec48bc1c5f99646a86e0fcdfe4 2023-09-20T21:46:29.997 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 20 21:46:29 smithi022 systemd[1]: Started Ceph mds.g for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:30.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:30 smithi123 ceph-mon[134433]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:30.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:30.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:30.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:30.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:30.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:30.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:30 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:30.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:30 smithi089 ceph-mon[134090]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:30.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:30.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:30.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:30.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:30.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:30.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:30 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:30.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:30 smithi022 ceph-mon[97651]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:30.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:30.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:30 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:30.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:31.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.209+0000 7f130d3ff700 1 -- 172.21.15.22:0/645900199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1308108ca0 msgr2=0x7f130810b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:31.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.209+0000 7f130d3ff700 1 --2- 172.21.15.22:0/645900199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1308108ca0 0x7f130810b130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f130000b0a0 tx=0x7f130000b3b0 comp rx=0 tx=0).stop 2023-09-20T21:46:31.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.209+0000 7f130d3ff700 1 -- 172.21.15.22:0/645900199 shutdown_connections 2023-09-20T21:46:31.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.209+0000 7f130d3ff700 1 --2- 172.21.15.22:0/645900199 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1308108ca0 0x7f130810b130 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:31.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.209+0000 7f130d3ff700 1 --2- 172.21.15.22:0/645900199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1308074e20 0x7f1308073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:31.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.209+0000 7f130d3ff700 1 --2- 172.21.15.22:0/645900199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1308073840 0x7f1308073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:31.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.209+0000 7f130d3ff700 1 -- 172.21.15.22:0/645900199 >> 172.21.15.22:0/645900199 conn(0x7f13080fc310 msgr2=0x7f13080fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:31.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.210+0000 7f130d3ff700 1 -- 172.21.15.22:0/645900199 shutdown_connections 2023-09-20T21:46:31.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.210+0000 7f130d3ff700 1 -- 172.21.15.22:0/645900199 wait complete. 2023-09-20T21:46:31.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.210+0000 7f130d3ff700 1 Processor -- start 2023-09-20T21:46:31.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f130d3ff700 1 -- start start 2023-09-20T21:46:31.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f130d3ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1308073840 0x7f1308198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:31.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f130d3ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1308074e20 0x7f1308199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:31.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f130d3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1308108ca0 0x7f13081a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:31.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f130d3ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13081a0900 con 0x7f1308074e20 2023-09-20T21:46:31.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f130d3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13081a0a70 con 0x7f1308108ca0 2023-09-20T21:46:31.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f130d3ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13081a0c30 con 0x7f1308073840 2023-09-20T21:46:31.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f13067fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1308074e20 0x7f1308199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:31.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f1306ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1308073840 0x7f1308198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:31.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f13067fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1308074e20 0x7f1308199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49314/0 (socket says 172.21.15.22:49314) 2023-09-20T21:46:31.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f13067fc700 1 -- 172.21.15.22:0/318604738 learned_addr learned my addr 172.21.15.22:0/318604738 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:31.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.211+0000 7f13077fe700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1308108ca0 0x7f13081a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:31.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f1306ffd700 1 -- 172.21.15.22:0/318604738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1308108ca0 msgr2=0x7f13081a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:31.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f1306ffd700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1308108ca0 0x7f13081a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:31.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f1306ffd700 1 -- 172.21.15.22:0/318604738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1308074e20 msgr2=0x7f1308199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:31.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f1306ffd700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1308074e20 0x7f1308199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:31.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f1306ffd700 1 -- 172.21.15.22:0/318604738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1300009d00 con 0x7f1308073840 2023-09-20T21:46:31.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f1306ffd700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1308073840 0x7f1308198dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f12f800e9f0 tx=0x7f12f800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:31.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f12effff700 1 -- 172.21.15.22:0/318604738 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f800c9c0 con 0x7f1308073840 2023-09-20T21:46:31.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f12effff700 1 -- 172.21.15.22:0/318604738 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12f8010430 con 0x7f1308073840 2023-09-20T21:46:31.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f130d3ff700 1 -- 172.21.15.22:0/318604738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13081a0f10 con 0x7f1308073840 2023-09-20T21:46:31.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f12effff700 1 -- 172.21.15.22:0/318604738 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8018680 con 0x7f1308073840 2023-09-20T21:46:31.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.212+0000 7f130d3ff700 1 -- 172.21.15.22:0/318604738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13081a1340 con 0x7f1308073840 2023-09-20T21:46:31.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.214+0000 7f130d3ff700 1 -- 172.21.15.22:0/318604738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1308107350 con 0x7f1308073840 2023-09-20T21:46:31.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.215+0000 7f12effff700 1 -- 172.21.15.22:0/318604738 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f12f8010860 con 0x7f1308073840 2023-09-20T21:46:31.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.217+0000 7f12effff700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12f00a13d0 0x7f12f00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:31.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.217+0000 7f12effff700 1 -- 172.21.15.22:0/318604738 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f12f8014070 con 0x7f1308073840 2023-09-20T21:46:31.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.217+0000 7f13067fc700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12f00a13d0 0x7f12f00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:31.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.217+0000 7f13067fc700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12f00a13d0 0x7f12f00a3890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f12fc005950 tx=0x7f12fc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:31.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.221+0000 7f12effff700 1 -- 172.21.15.22:0/318604738 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f12f8051e10 con 0x7f1308073840 2023-09-20T21:46:31.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:31 smithi123 ceph-mon[134433]: Deploying daemon mds.a on smithi022 2023-09-20T21:46:31.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:31 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.22:6838/3781158526,v1:172.21.15.22:6839/3781158526] up:boot 2023-09-20T21:46:31.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:31 smithi123 ceph-mon[134433]: fsmap 10 up:standby 2023-09-20T21:46:31.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-20T21:46:31.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:31 smithi089 ceph-mon[134090]: Deploying daemon mds.a on smithi022 2023-09-20T21:46:31.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:31 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.22:6838/3781158526,v1:172.21.15.22:6839/3781158526] up:boot 2023-09-20T21:46:31.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:31 smithi089 ceph-mon[134090]: fsmap 10 up:standby 2023-09-20T21:46:31.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-20T21:46:31.386 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:31 smithi022 ceph-mon[97651]: Deploying daemon mds.a on smithi022 2023-09-20T21:46:31.386 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:31 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.22:6838/3781158526,v1:172.21.15.22:6839/3781158526] up:boot 2023-09-20T21:46:31.386 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:31 smithi022 ceph-mon[97651]: fsmap 10 up:standby 2023-09-20T21:46:31.387 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-09-20T21:46:31.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.496+0000 7f130d3ff700 1 -- 172.21.15.22:0/318604738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f130810e820 con 0x7f1308073840 2023-09-20T21:46:31.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.498+0000 7f12effff700 1 -- 172.21.15.22:0/318604738 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8447 (secure 0 0 0) 0x7f130810e820 con 0x7f1308073840 2023-09-20T21:46:31.498 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:31.499 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-09-20T21:46:31.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.500+0000 7f130d3ff700 1 -- 172.21.15.22:0/318604738 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12f00a13d0 msgr2=0x7f12f00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:31.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.500+0000 7f130d3ff700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12f00a13d0 0x7f12f00a3890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f12fc005950 tx=0x7f12fc009260 comp rx=0 tx=0).stop 2023-09-20T21:46:31.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.500+0000 7f130d3ff700 1 -- 172.21.15.22:0/318604738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1308073840 msgr2=0x7f1308198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:31.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.500+0000 7f130d3ff700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1308073840 0x7f1308198dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f12f800e9f0 tx=0x7f12f800ed00 comp rx=0 tx=0).stop 2023-09-20T21:46:31.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.501+0000 7f130d3ff700 1 -- 172.21.15.22:0/318604738 shutdown_connections 2023-09-20T21:46:31.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.501+0000 7f130d3ff700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12f00a13d0 0x7f12f00a3890 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:31.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.501+0000 7f130d3ff700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1308108ca0 0x7f13081a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:31.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.501+0000 7f130d3ff700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1308073840 0x7f1308198dc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:31.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.501+0000 7f130d3ff700 1 --2- 172.21.15.22:0/318604738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1308074e20 0x7f1308199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:31.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.501+0000 7f130d3ff700 1 -- 172.21.15.22:0/318604738 >> 172.21.15.22:0/318604738 conn(0x7f13080fc310 msgr2=0x7f13080fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:31.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.501+0000 7f130d3ff700 1 -- 172.21.15.22:0/318604738 shutdown_connections 2023-09-20T21:46:31.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:31.501+0000 7f130d3ff700 1 -- 172.21.15.22:0/318604738 wait complete. 2023-09-20T21:46:31.504 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 11 2023-09-20T21:46:32.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:32 smithi123 ceph-mon[134433]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:32.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:32 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/318604738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:32.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:32 smithi089 ceph-mon[134090]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:32.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:32 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/318604738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:32 smithi022 ceph-mon[97651]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:32.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:32 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/318604738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:32.499 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 21:46:32 smithi022 systemd[1]: Starting Ceph mds.a for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:32.818 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 21:46:32 smithi022 podman[148962]: 2023-09-20T21:46:33.248 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 21:46:32 smithi022 bash[148962]: e77b6e976816b4bc198f13f4fd467c2ad7ccbde716d2665c2f07ac8fe627c1ee 2023-09-20T21:46:33.248 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 21:46:32 smithi022 ceph-mds[149002]: -- 172.21.15.22:0/1803572711 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e71caa91e0 con 0x55e71ca7e800 2023-09-20T21:46:33.248 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 21:46:32 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-a[148978]: starting mds.a at 2023-09-20T21:46:33.248 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 21:46:32 smithi022 systemd[1]: Started Ceph mds.a for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:33.522 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:33.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.737+0000 7f2ba8bff700 1 -- 172.21.15.22:0/1798138565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba4100de0 msgr2=0x7f2ba4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:33.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.737+0000 7f2ba8bff700 1 --2- 172.21.15.22:0/1798138565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba4100de0 0x7f2ba4101200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2b940099c0 tx=0x7f2b94009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.738+0000 7f2ba8bff700 1 -- 172.21.15.22:0/1798138565 shutdown_connections 2023-09-20T21:46:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.738+0000 7f2ba8bff700 1 --2- 172.21.15.22:0/1798138565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba4102970 0x7f2ba4109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.738+0000 7f2ba8bff700 1 --2- 172.21.15.22:0/1798138565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba4100de0 0x7f2ba4101200 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.738+0000 7f2ba8bff700 1 --2- 172.21.15.22:0/1798138565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ba4101fd0 0x7f2ba4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.738+0000 7f2ba8bff700 1 -- 172.21.15.22:0/1798138565 >> 172.21.15.22:0/1798138565 conn(0x7f2ba40fc330 msgr2=0x7f2ba40fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:33.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.738+0000 7f2ba8bff700 1 -- 172.21.15.22:0/1798138565 shutdown_connections 2023-09-20T21:46:33.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.738+0000 7f2ba8bff700 1 -- 172.21.15.22:0/1798138565 wait complete. 2023-09-20T21:46:33.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba8bff700 1 Processor -- start 2023-09-20T21:46:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba8bff700 1 -- start start 2023-09-20T21:46:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba8bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ba4100de0 0x7f2ba4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba8bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba4101fd0 0x7f2ba4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba8bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba4102970 0x7f2ba41a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba8bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba41a0830 con 0x7f2ba4101fd0 2023-09-20T21:46:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba8bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba41a09a0 con 0x7f2ba4100de0 2023-09-20T21:46:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba8bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba41a0b60 con 0x7f2ba4102970 2023-09-20T21:46:33.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ba4100de0 0x7f2ba4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:33.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba4101fd0 0x7f2ba4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:33.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ba4100de0 0x7f2ba4198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:51536/0 (socket says 172.21.15.22:51536) 2023-09-20T21:46:33.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.739+0000 7f2ba259c700 1 -- 172.21.15.22:0/2104776839 learned_addr learned my addr 172.21.15.22:0/2104776839 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:33.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2ba2d9d700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba4102970 0x7f2ba41a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:33.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2ba1d9b700 1 -- 172.21.15.22:0/2104776839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba4102970 msgr2=0x7f2ba41a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:33.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2ba1d9b700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba4102970 0x7f2ba41a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:33.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2ba1d9b700 1 -- 172.21.15.22:0/2104776839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ba4100de0 msgr2=0x7f2ba4198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:33.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2ba1d9b700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ba4100de0 0x7f2ba4198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:33.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2ba1d9b700 1 -- 172.21.15.22:0/2104776839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b940095f0 con 0x7f2ba4101fd0 2023-09-20T21:46:33.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2ba259c700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ba4100de0 0x7f2ba4198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:33.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2ba1d9b700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba4101fd0 0x7f2ba4199230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800d750 tx=0x7f2b9800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:33.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2b937fe700 1 -- 172.21.15.22:0/2104776839 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98004020 con 0x7f2ba4101fd0 2023-09-20T21:46:33.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2b937fe700 1 -- 172.21.15.22:0/2104776839 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b98004830 con 0x7f2ba4101fd0 2023-09-20T21:46:33.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2ba8bff700 1 -- 172.21.15.22:0/2104776839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba41a0e40 con 0x7f2ba4101fd0 2023-09-20T21:46:33.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.740+0000 7f2b937fe700 1 -- 172.21.15.22:0/2104776839 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b980054c0 con 0x7f2ba4101fd0 2023-09-20T21:46:33.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.741+0000 7f2ba8bff700 1 -- 172.21.15.22:0/2104776839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba41a1260 con 0x7f2ba4101fd0 2023-09-20T21:46:33.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.741+0000 7f2ba8bff700 1 -- 172.21.15.22:0/2104776839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba4193430 con 0x7f2ba4101fd0 2023-09-20T21:46:33.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.743+0000 7f2b937fe700 1 -- 172.21.15.22:0/2104776839 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2b980049a0 con 0x7f2ba4101fd0 2023-09-20T21:46:33.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.746+0000 7f2b937fe700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2b8c0a1350 0x7f2b8c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:33.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.746+0000 7f2b937fe700 1 -- 172.21.15.22:0/2104776839 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2b980cc180 con 0x7f2ba4101fd0 2023-09-20T21:46:33.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.746+0000 7f2ba259c700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2b8c0a1350 0x7f2b8c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:33.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.747+0000 7f2ba259c700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2b8c0a1350 0x7f2b8c0a3810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2b94004800 tx=0x7f2b9401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:33.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:33.750+0000 7f2b937fe700 1 -- 172.21.15.22:0/2104776839 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2b98095c50 con 0x7f2ba4101fd0 2023-09-20T21:46:34.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.009+0000 7f2ba8bff700 1 -- 172.21.15.22:0/2104776839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ba410c640 con 0x7f2ba4101fd0 2023-09-20T21:46:34.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.009+0000 7f2b937fe700 1 -- 172.21.15.22:0/2104776839 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9239 (secure 0 0 0) 0x7f2b98099270 con 0x7f2ba4101fd0 2023-09-20T21:46:34.011 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:34.012 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2023-09-20T21:46:34.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 -- 172.21.15.22:0/2104776839 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2b8c0a1350 msgr2=0x7f2b8c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:34.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2b8c0a1350 0x7f2b8c0a3810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2b94004800 tx=0x7f2b9401a040 comp rx=0 tx=0).stop 2023-09-20T21:46:34.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 -- 172.21.15.22:0/2104776839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba4101fd0 msgr2=0x7f2ba4199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:34.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba4101fd0 0x7f2ba4199230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800d750 tx=0x7f2b9800da60 comp rx=0 tx=0).stop 2023-09-20T21:46:34.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 -- 172.21.15.22:0/2104776839 shutdown_connections 2023-09-20T21:46:34.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba4102970 0x7f2ba41a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:34.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2b8c0a1350 0x7f2b8c0a3810 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:34.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ba4100de0 0x7f2ba4198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:34.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 --2- 172.21.15.22:0/2104776839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba4101fd0 0x7f2ba4199230 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:34.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 -- 172.21.15.22:0/2104776839 >> 172.21.15.22:0/2104776839 conn(0x7f2ba40fc330 msgr2=0x7f2ba41078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:34.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 -- 172.21.15.22:0/2104776839 shutdown_connections 2023-09-20T21:46:34.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:34.012+0000 7f2ba8bff700 1 -- 172.21.15.22:0/2104776839 wait complete. 2023-09-20T21:46:34.017 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 12 2023-09-20T21:46:34.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:34.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:34.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:34.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:34.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:34.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:34.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: Deploying daemon mds.b on smithi089 2023-09-20T21:46:34.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:34.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] up:boot 2023-09-20T21:46:34.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: fsmap 11 up:standby 2023-09-20T21:46:34.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:33 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-20T21:46:34.334 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:34.334 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:34.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:34.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:34.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:34.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:34.335 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: Deploying daemon mds.b on smithi089 2023-09-20T21:46:34.336 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:34.336 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] up:boot 2023-09-20T21:46:34.336 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: fsmap 11 up:standby 2023-09-20T21:46:34.336 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:33 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-20T21:46:34.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:34.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:34.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:34.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-09-20T21:46:34.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-09-20T21:46:34.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:34.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: Deploying daemon mds.b on smithi089 2023-09-20T21:46:34.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:34.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] up:boot 2023-09-20T21:46:34.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: fsmap 11 up:standby 2023-09-20T21:46:34.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:33 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-20T21:46:34.957 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 21:46:34 smithi089 systemd[1]: Starting Ceph mds.b for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T21:46:35.246 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 21:46:34 smithi089 podman[171400]: 2023-09-20T21:46:35.246 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:34 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2104776839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:35.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:34 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2104776839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:35.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:34 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2104776839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:35.610 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 21:46:35 smithi089 ceph-mds[171440]: -- 172.21.15.89:0/1179623946 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5628b1d931e0 con 0x5628b1d68c00 2023-09-20T21:46:35.611 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 21:46:35 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b[171416]: starting mds.b at 2023-09-20T21:46:35.611 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 21:46:35 smithi089 bash[171400]: 52eb3a32f9683f802ebc19fb587de9b9b488ddbeac3dcf918a041c4e7a217e7a 2023-09-20T21:46:35.611 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 21:46:35 smithi089 systemd[1]: Started Ceph mds.b for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T21:46:36.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:36.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.244+0000 7f9d0abff700 1 -- 172.21.15.22:0/1097321250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d04108cc0 msgr2=0x7f9d0410b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:36.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.244+0000 7f9d0abff700 1 --2- 172.21.15.22:0/1097321250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d04108cc0 0x7f9d0410b130 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00b0a0 tx=0x7f9cfc00b3b0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.244+0000 7f9d0abff700 1 -- 172.21.15.22:0/1097321250 shutdown_connections 2023-09-20T21:46:36.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.244+0000 7f9d0abff700 1 --2- 172.21.15.22:0/1097321250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d04108cc0 0x7f9d0410b130 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.244+0000 7f9d0abff700 1 --2- 172.21.15.22:0/1097321250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d04074dc0 0x7f9d04073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.244+0000 7f9d0abff700 1 --2- 172.21.15.22:0/1097321250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d040737e0 0x7f9d04073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.244+0000 7f9d0abff700 1 -- 172.21.15.22:0/1097321250 >> 172.21.15.22:0/1097321250 conn(0x7f9d040fc320 msgr2=0x7f9d040fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:36.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.244+0000 7f9d0abff700 1 -- 172.21.15.22:0/1097321250 shutdown_connections 2023-09-20T21:46:36.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.244+0000 7f9d0abff700 1 -- 172.21.15.22:0/1097321250 wait complete. 2023-09-20T21:46:36.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.245+0000 7f9d0abff700 1 Processor -- start 2023-09-20T21:46:36.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.245+0000 7f9d0abff700 1 -- start start 2023-09-20T21:46:36.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.245+0000 7f9d0abff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d040737e0 0x7f9d04198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:36.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.245+0000 7f9d0abff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d04074dc0 0x7f9d041992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:36.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.245+0000 7f9d0abff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d04108cc0 0x7f9d041a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:36.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.245+0000 7f9d0abff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041a08f0 con 0x7f9d04108cc0 2023-09-20T21:46:36.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.245+0000 7f9d0abff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041a0a30 con 0x7f9d040737e0 2023-09-20T21:46:36.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.245+0000 7f9d0abff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041a0bc0 con 0x7f9d04074dc0 2023-09-20T21:46:36.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0899b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d040737e0 0x7f9d04198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:36.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0899b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d040737e0 0x7f9d04198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60776/0 (socket says 172.21.15.22:60776) 2023-09-20T21:46:36.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0899b700 1 -- 172.21.15.22:0/170149628 learned_addr learned my addr 172.21.15.22:0/170149628 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:36.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0919c700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d04108cc0 0x7f9d041a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:36.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d03fff700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d04074dc0 0x7f9d041992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:36.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0899b700 1 -- 172.21.15.22:0/170149628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d04074dc0 msgr2=0x7f9d041992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:36.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0899b700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d04074dc0 0x7f9d041992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0899b700 1 -- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d04108cc0 msgr2=0x7f9d041a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0899b700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d04108cc0 0x7f9d041a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0899b700 1 -- 172.21.15.22:0/170149628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc009d00 con 0x7f9d040737e0 2023-09-20T21:46:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d03fff700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d04074dc0 0x7f9d041992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0919c700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d04108cc0 0x7f9d041a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:36.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.246+0000 7f9d0899b700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d040737e0 0x7f9d04198d70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800e9a0 tx=0x7f9cf800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.247+0000 7f9d01ffb700 1 -- 172.21.15.22:0/170149628 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8009790 con 0x7f9d040737e0 2023-09-20T21:46:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.247+0000 7f9d01ffb700 1 -- 172.21.15.22:0/170149628 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cf8010430 con 0x7f9d040737e0 2023-09-20T21:46:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.247+0000 7f9d0abff700 1 -- 172.21.15.22:0/170149628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d041a0ea0 con 0x7f9d040737e0 2023-09-20T21:46:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.247+0000 7f9d01ffb700 1 -- 172.21.15.22:0/170149628 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8018680 con 0x7f9d040737e0 2023-09-20T21:46:36.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.247+0000 7f9d0abff700 1 -- 172.21.15.22:0/170149628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d041a12c0 con 0x7f9d040737e0 2023-09-20T21:46:36.255 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:35 smithi022 ceph-mon[97651]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:36.255 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.255 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.255 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:46:36.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.248+0000 7f9d0abff700 1 -- 172.21.15.22:0/170149628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d041073c0 con 0x7f9d040737e0 2023-09-20T21:46:36.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.250+0000 7f9d01ffb700 1 -- 172.21.15.22:0/170149628 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f9cf80105a0 con 0x7f9d040737e0 2023-09-20T21:46:36.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.251+0000 7f9d01ffb700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9cf40a1350 0x7f9cf40a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:36.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.251+0000 7f9d01ffb700 1 -- 172.21.15.22:0/170149628 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9cf8014070 con 0x7f9d040737e0 2023-09-20T21:46:36.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.251+0000 7f9d03fff700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9cf40a1350 0x7f9cf40a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:36.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.252+0000 7f9d03fff700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9cf40a1350 0x7f9cf40a3810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0011490 tx=0x7f9cf0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:36.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.253+0000 7f9d01ffb700 1 -- 172.21.15.22:0/170149628 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9cf80963e0 con 0x7f9d040737e0 2023-09-20T21:46:36.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:35 smithi123 ceph-mon[134433]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:36.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:46:36.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:35 smithi089 ceph-mon[134090]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:36.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:36.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:46:36.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.523+0000 7f9d0abff700 1 -- 172.21.15.22:0/170149628 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d0402d260 con 0x7f9d040737e0 2023-09-20T21:46:36.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.524+0000 7f9d01ffb700 1 -- 172.21.15.22:0/170149628 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10031 (secure 0 0 0) 0x7f9cf8099a00 con 0x7f9d040737e0 2023-09-20T21:46:36.525 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:36.525 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-09-20T21:46:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 -- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9cf40a1350 msgr2=0x7f9cf40a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9cf40a1350 0x7f9cf40a3810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0011490 tx=0x7f9cf0009260 comp rx=0 tx=0).stop 2023-09-20T21:46:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 -- 172.21.15.22:0/170149628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d040737e0 msgr2=0x7f9d04198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d040737e0 0x7f9d04198d70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800e9a0 tx=0x7f9cf800ecb0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 -- 172.21.15.22:0/170149628 shutdown_connections 2023-09-20T21:46:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d04108cc0 0x7f9d041a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9cf40a1350 0x7f9cf40a3810 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d040737e0 0x7f9d04198d70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 --2- 172.21.15.22:0/170149628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d04074dc0 0x7f9d041992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 -- 172.21.15.22:0/170149628 >> 172.21.15.22:0/170149628 conn(0x7f9d040fc320 msgr2=0x7f9d040fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:36.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 -- 172.21.15.22:0/170149628 shutdown_connections 2023-09-20T21:46:36.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.527+0000 7f9d0abff700 1 -- 172.21.15.22:0/170149628 wait complete. 2023-09-20T21:46:36.531 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 13 2023-09-20T21:46:36.547 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:46:36.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.764+0000 7fd7d81ff700 1 -- 172.21.15.22:0/2117162171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7d00737e0 msgr2=0x7fd7d0073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:36.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.764+0000 7fd7d81ff700 1 --2- 172.21.15.22:0/2117162171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7d00737e0 0x7fd7d0073c60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009990 tx=0x7fd7c0009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.764+0000 7fd7d81ff700 1 -- 172.21.15.22:0/2117162171 shutdown_connections 2023-09-20T21:46:36.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.764+0000 7fd7d81ff700 1 --2- 172.21.15.22:0/2117162171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7d0108cc0 0x7fd7d010b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.764+0000 7fd7d81ff700 1 --2- 172.21.15.22:0/2117162171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7d0074dc0 0x7fd7d0073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.764+0000 7fd7d81ff700 1 --2- 172.21.15.22:0/2117162171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7d00737e0 0x7fd7d0073c60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.764+0000 7fd7d81ff700 1 -- 172.21.15.22:0/2117162171 >> 172.21.15.22:0/2117162171 conn(0x7fd7d00fc320 msgr2=0x7fd7d00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:36.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.764+0000 7fd7d81ff700 1 -- 172.21.15.22:0/2117162171 shutdown_connections 2023-09-20T21:46:36.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.764+0000 7fd7d81ff700 1 -- 172.21.15.22:0/2117162171 wait complete. 2023-09-20T21:46:36.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.765+0000 7fd7d81ff700 1 Processor -- start 2023-09-20T21:46:36.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.765+0000 7fd7d81ff700 1 -- start start 2023-09-20T21:46:36.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d81ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7d00737e0 0x7fd7d0198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:36.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d81ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7d0074dc0 0x7fd7d0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:36.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d81ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7d0108cc0 0x7fd7d01a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:36.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d81ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01a0830 con 0x7fd7d0108cc0 2023-09-20T21:46:36.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d81ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01a09a0 con 0x7fd7d00737e0 2023-09-20T21:46:36.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d81ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01a0b60 con 0x7fd7d0074dc0 2023-09-20T21:46:36.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d5f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7d00737e0 0x7fd7d0198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:36.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d5f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7d00737e0 0x7fd7d0198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60804/0 (socket says 172.21.15.22:60804) 2023-09-20T21:46:36.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d5f9b700 1 -- 172.21.15.22:0/146788625 learned_addr learned my addr 172.21.15.22:0/146788625 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:36.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d579a700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7d0074dc0 0x7fd7d0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:36.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d679c700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7d0108cc0 0x7fd7d01a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:36.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.766+0000 7fd7d5f9b700 1 -- 172.21.15.22:0/146788625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7d0074dc0 msgr2=0x7fd7d0199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:36.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7d5f9b700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7d0074dc0 0x7fd7d0199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7d5f9b700 1 -- 172.21.15.22:0/146788625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7d0108cc0 msgr2=0x7fd7d01a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:36.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7d5f9b700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7d0108cc0 0x7fd7d01a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:36.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7d5f9b700 1 -- 172.21.15.22:0/146788625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00095f0 con 0x7fd7d00737e0 2023-09-20T21:46:36.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7d579a700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7d0074dc0 0x7fd7d0199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:36.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7d5f9b700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7d00737e0 0x7fd7d0198cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00e9a0 tx=0x7fd7cc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:36.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7c6ffd700 1 -- 172.21.15.22:0/146788625 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc009790 con 0x7fd7d00737e0 2023-09-20T21:46:36.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7c6ffd700 1 -- 172.21.15.22:0/146788625 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7cc010430 con 0x7fd7d00737e0 2023-09-20T21:46:36.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7d81ff700 1 -- 172.21.15.22:0/146788625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d01a0e40 con 0x7fd7d00737e0 2023-09-20T21:46:36.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7c6ffd700 1 -- 172.21.15.22:0/146788625 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc018680 con 0x7fd7d00737e0 2023-09-20T21:46:36.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.767+0000 7fd7d81ff700 1 -- 172.21.15.22:0/146788625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d01a1270 con 0x7fd7d00737e0 2023-09-20T21:46:36.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.769+0000 7fd7d81ff700 1 -- 172.21.15.22:0/146788625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d01073c0 con 0x7fd7d00737e0 2023-09-20T21:46:36.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.770+0000 7fd7c6ffd700 1 -- 172.21.15.22:0/146788625 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fd7cc0105a0 con 0x7fd7d00737e0 2023-09-20T21:46:36.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.772+0000 7fd7c6ffd700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd7bc0a13d0 0x7fd7bc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:36.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.772+0000 7fd7c6ffd700 1 -- 172.21.15.22:0/146788625 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd7cc014070 con 0x7fd7d00737e0 2023-09-20T21:46:36.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.772+0000 7fd7d579a700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd7bc0a13d0 0x7fd7bc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:36.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.772+0000 7fd7d579a700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd7bc0a13d0 0x7fd7bc0a3890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd7d019a1c0 tx=0x7fd7c0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:36.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.775+0000 7fd7c6ffd700 1 -- 172.21.15.22:0/146788625 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd7cc051e10 con 0x7fd7d00737e0 2023-09-20T21:46:36.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.996+0000 7fd7d81ff700 1 -- 172.21.15.22:0/146788625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7d002d0d0 con 0x7fd7d00737e0 2023-09-20T21:46:36.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.997+0000 7fd7c6ffd700 1 -- 172.21.15.22:0/146788625 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18375 (secure 0 0 0) 0x7fd7cc096430 con 0x7fd7d00737e0 2023-09-20T21:46:36.998 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:36.998 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":81,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:45:53.805187+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:46:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 -- 172.21.15.22:0/146788625 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd7bc0a13d0 msgr2=0x7fd7bc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd7bc0a13d0 0x7fd7bc0a3890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd7d019a1c0 tx=0x7fd7c0005fb0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 -- 172.21.15.22:0/146788625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7d00737e0 msgr2=0x7fd7d0198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7d00737e0 0x7fd7d0198cc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00e9a0 tx=0x7fd7cc00ecb0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 -- 172.21.15.22:0/146788625 shutdown_connections 2023-09-20T21:46:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7d0108cc0 0x7fd7d01a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd7bc0a13d0 0x7fd7bc0a3890 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd7d00737e0 0x7fd7d0198cc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 --2- 172.21.15.22:0/146788625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7d0074dc0 0x7fd7d0199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 -- 172.21.15.22:0/146788625 >> 172.21.15.22:0/146788625 conn(0x7fd7d00fc320 msgr2=0x7fd7d00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 -- 172.21.15.22:0/146788625 shutdown_connections 2023-09-20T21:46:37.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:36.999+0000 7fd7d81ff700 1 -- 172.21.15.22:0/146788625 wait complete. 2023-09-20T21:46:37.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-20T21:46:37.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:46:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.242+0000 7fb133bff700 1 -- 172.21.15.22:0/1889247980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb12c101fe0 msgr2=0x7fb12c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.242+0000 7fb133bff700 1 --2- 172.21.15.22:0/1889247980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb12c101fe0 0x7fb12c102440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009990 tx=0x7fb11c009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.242+0000 7fb133bff700 1 -- 172.21.15.22:0/1889247980 shutdown_connections 2023-09-20T21:46:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.242+0000 7fb133bff700 1 --2- 172.21.15.22:0/1889247980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb12c102980 0x7fb12c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.242+0000 7fb133bff700 1 --2- 172.21.15.22:0/1889247980 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb12c100df0 0x7fb12c101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.242+0000 7fb133bff700 1 --2- 172.21.15.22:0/1889247980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb12c101fe0 0x7fb12c102440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.242+0000 7fb133bff700 1 -- 172.21.15.22:0/1889247980 >> 172.21.15.22:0/1889247980 conn(0x7fb12c0fc360 msgr2=0x7fb12c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:37.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.243+0000 7fb133bff700 1 -- 172.21.15.22:0/1889247980 shutdown_connections 2023-09-20T21:46:37.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.243+0000 7fb133bff700 1 -- 172.21.15.22:0/1889247980 wait complete. 2023-09-20T21:46:37.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb133bff700 1 Processor -- start 2023-09-20T21:46:37.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb133bff700 1 -- start start 2023-09-20T21:46:37.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb133bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb12c100df0 0x7fb12c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:37.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb133bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb12c101fe0 0x7fb12c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:37.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb133bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb12c102980 0x7fb12c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:37.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb133bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c1a0870 con 0x7fb12c101fe0 2023-09-20T21:46:37.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb133bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c1a09e0 con 0x7fb12c102980 2023-09-20T21:46:37.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb133bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c1a0ba0 con 0x7fb12c100df0 2023-09-20T21:46:37.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb13119a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb12c101fe0 0x7fb12c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:37.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb13119a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb12c101fe0 0x7fb12c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41892/0 (socket says 172.21.15.22:41892) 2023-09-20T21:46:37.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb13199b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb12c100df0 0x7fb12c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:37.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb13119a700 1 -- 172.21.15.22:0/2411634987 learned_addr learned my addr 172.21.15.22:0/2411634987 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:37.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.244+0000 7fb13219c700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb12c102980 0x7fb12c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:37.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.245+0000 7fb13119a700 1 -- 172.21.15.22:0/2411634987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb12c100df0 msgr2=0x7fb12c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.245+0000 7fb13119a700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb12c100df0 0x7fb12c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.245+0000 7fb13119a700 1 -- 172.21.15.22:0/2411634987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb12c102980 msgr2=0x7fb12c1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.245+0000 7fb13119a700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb12c102980 0x7fb12c1a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.245+0000 7fb13119a700 1 -- 172.21.15.22:0/2411634987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb128009520 con 0x7fb12c101fe0 2023-09-20T21:46:37.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.245+0000 7fb13119a700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb12c101fe0 0x7fb12c199270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00b680 tx=0x7fb11c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:37.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.245+0000 7fb122ffd700 1 -- 172.21.15.22:0/2411634987 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c01d070 con 0x7fb12c101fe0 2023-09-20T21:46:37.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.245+0000 7fb133bff700 1 -- 172.21.15.22:0/2411634987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb124009c40 con 0x7fb12c101fe0 2023-09-20T21:46:37.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.245+0000 7fb122ffd700 1 -- 172.21.15.22:0/2411634987 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb11c022470 con 0x7fb12c101fe0 2023-09-20T21:46:37.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.245+0000 7fb122ffd700 1 -- 172.21.15.22:0/2411634987 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c017650 con 0x7fb12c101fe0 2023-09-20T21:46:37.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.246+0000 7fb133bff700 1 -- 172.21.15.22:0/2411634987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c0095f0 con 0x7fb12c101fe0 2023-09-20T21:46:37.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.246+0000 7fb133bff700 1 -- 172.21.15.22:0/2411634987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12c10c650 con 0x7fb12c101fe0 2023-09-20T21:46:37.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:36 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] up:boot 2023-09-20T21:46:37.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:36 smithi022 ceph-mon[97651]: fsmap 12 up:standby 2023-09-20T21:46:37.254 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:36 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-20T21:46:37.254 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:36 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/170149628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:37.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.249+0000 7fb122ffd700 1 -- 172.21.15.22:0/2411634987 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fb11c0225e0 con 0x7fb12c101fe0 2023-09-20T21:46:37.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.251+0000 7fb122ffd700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1180a1300 0x7fb1180a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:37.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.251+0000 7fb122ffd700 1 -- 172.21.15.22:0/2411634987 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb11c0ce3b0 con 0x7fb12c101fe0 2023-09-20T21:46:37.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.251+0000 7fb13199b700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1180a1300 0x7fb1180a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.252+0000 7fb13199b700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1180a1300 0x7fb1180a37c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb12c19a200 tx=0x7fb128009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:37.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.254+0000 7fb122ffd700 1 -- 172.21.15.22:0/2411634987 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb11c097f10 con 0x7fb12c101fe0 2023-09-20T21:46:37.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:36 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] up:boot 2023-09-20T21:46:37.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:36 smithi123 ceph-mon[134433]: fsmap 12 up:standby 2023-09-20T21:46:37.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:36 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-20T21:46:37.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:36 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/170149628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:37.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:36 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] up:boot 2023-09-20T21:46:37.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:36 smithi089 ceph-mon[134090]: fsmap 12 up:standby 2023-09-20T21:46:37.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:36 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-20T21:46:37.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:36 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/170149628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:37.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.474+0000 7fb133bff700 1 -- 172.21.15.22:0/2411634987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb12c1a1110 con 0x7fb12c101fe0 2023-09-20T21:46:37.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.474+0000 7fb122ffd700 1 -- 172.21.15.22:0/2411634987 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb11c09b530 con 0x7fb12c101fe0 2023-09-20T21:46:37.478 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:46:37.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 -- 172.21.15.22:0/2411634987 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1180a1300 msgr2=0x7fb1180a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1180a1300 0x7fb1180a37c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb12c19a200 tx=0x7fb128009260 comp rx=0 tx=0).stop 2023-09-20T21:46:37.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 -- 172.21.15.22:0/2411634987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb12c101fe0 msgr2=0x7fb12c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb12c101fe0 0x7fb12c199270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00b680 tx=0x7fb11c005740 comp rx=0 tx=0).stop 2023-09-20T21:46:37.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 -- 172.21.15.22:0/2411634987 shutdown_connections 2023-09-20T21:46:37.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1180a1300 0x7fb1180a37c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb12c102980 0x7fb12c1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb12c100df0 0x7fb12c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 --2- 172.21.15.22:0/2411634987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb12c101fe0 0x7fb12c199270 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 -- 172.21.15.22:0/2411634987 >> 172.21.15.22:0/2411634987 conn(0x7fb12c0fc360 msgr2=0x7fb12c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:37.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 -- 172.21.15.22:0/2411634987 shutdown_connections 2023-09-20T21:46:37.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.477+0000 7fb133bff700 1 -- 172.21.15.22:0/2411634987 wait complete. 2023-09-20T21:46:37.496 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-20T21:46:37.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.710+0000 7f1436dff700 1 -- 172.21.15.22:0/4103483429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430100de0 msgr2=0x7f1430101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.710+0000 7f1436dff700 1 --2- 172.21.15.22:0/4103483429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430100de0 0x7f1430101200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f14240099c0 tx=0x7f1424009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.710+0000 7f1436dff700 1 -- 172.21.15.22:0/4103483429 shutdown_connections 2023-09-20T21:46:37.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.710+0000 7f1436dff700 1 --2- 172.21.15.22:0/4103483429 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1430102970 0x7f1430109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.710+0000 7f1436dff700 1 --2- 172.21.15.22:0/4103483429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430100de0 0x7f1430101200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.710+0000 7f1436dff700 1 --2- 172.21.15.22:0/4103483429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430101fd0 0x7f1430102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.710+0000 7f1436dff700 1 -- 172.21.15.22:0/4103483429 >> 172.21.15.22:0/4103483429 conn(0x7f14300fc310 msgr2=0x7f14300fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.710+0000 7f1436dff700 1 -- 172.21.15.22:0/4103483429 shutdown_connections 2023-09-20T21:46:37.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.710+0000 7f1436dff700 1 -- 172.21.15.22:0/4103483429 wait complete. 2023-09-20T21:46:37.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.711+0000 7f1436dff700 1 Processor -- start 2023-09-20T21:46:37.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.711+0000 7f1436dff700 1 -- start start 2023-09-20T21:46:37.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1436dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430100de0 0x7f1430196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:37.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1436dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1430101fd0 0x7f1430197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:37.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1436dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430102970 0x7f143019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:37.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1436dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143019e650 con 0x7f1430100de0 2023-09-20T21:46:37.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1436dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143019e790 con 0x7f1430101fd0 2023-09-20T21:46:37.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1436dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143019e920 con 0x7f1430102970 2023-09-20T21:46:37.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f142ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1430101fd0 0x7f1430197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:37.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1434b9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430100de0 0x7f1430196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:37.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f142ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1430101fd0 0x7f1430197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60846/0 (socket says 172.21.15.22:60846) 2023-09-20T21:46:37.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f142ffff700 1 -- 172.21.15.22:0/4204542823 learned_addr learned my addr 172.21.15.22:0/4204542823 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:37.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f143539c700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430102970 0x7f143019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:37.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1434b9b700 1 -- 172.21.15.22:0/4204542823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430102970 msgr2=0x7f143019e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1434b9b700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430102970 0x7f143019e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1434b9b700 1 -- 172.21.15.22:0/4204542823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1430101fd0 msgr2=0x7f1430197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1434b9b700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1430101fd0 0x7f1430197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f1434b9b700 1 -- 172.21.15.22:0/4204542823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14240095f0 con 0x7f1430100de0 2023-09-20T21:46:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.712+0000 7f143539c700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430102970 0x7f143019e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:37.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.713+0000 7f1434b9b700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430100de0 0x7f1430196ad0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f14240058e0 tx=0x7f142400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.713+0000 7f142dffb700 1 -- 172.21.15.22:0/4204542823 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142401d070 con 0x7f1430100de0 2023-09-20T21:46:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.713+0000 7f142dffb700 1 -- 172.21.15.22:0/4204542823 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1424005020 con 0x7f1430100de0 2023-09-20T21:46:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.713+0000 7f1436dff700 1 -- 172.21.15.22:0/4204542823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143019eba0 con 0x7f1430100de0 2023-09-20T21:46:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.713+0000 7f142dffb700 1 -- 172.21.15.22:0/4204542823 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424004a90 con 0x7f1430100de0 2023-09-20T21:46:37.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.713+0000 7f1436dff700 1 -- 172.21.15.22:0/4204542823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143019efc0 con 0x7f1430100de0 2023-09-20T21:46:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.714+0000 7f1436dff700 1 -- 172.21.15.22:0/4204542823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143010c640 con 0x7f1430100de0 2023-09-20T21:46:37.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.716+0000 7f142dffb700 1 -- 172.21.15.22:0/4204542823 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1424005400 con 0x7f1430100de0 2023-09-20T21:46:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.717+0000 7f142dffb700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14200a13d0 0x7f14200a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.717+0000 7f142dffb700 1 -- 172.21.15.22:0/4204542823 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f14240cd380 con 0x7f1430100de0 2023-09-20T21:46:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.717+0000 7f142ffff700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14200a13d0 0x7f14200a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:37.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.718+0000 7f142ffff700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14200a13d0 0x7f14200a3890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f141c009d20 tx=0x7f141c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:37.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.721+0000 7f142dffb700 1 -- 172.21.15.22:0/4204542823 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1424096ee0 con 0x7f1430100de0 2023-09-20T21:46:37.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.943+0000 7f1436dff700 1 -- 172.21.15.22:0/4204542823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f143010c830 con 0x7f1430100de0 2023-09-20T21:46:37.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.943+0000 7f142dffb700 1 -- 172.21.15.22:0/4204542823 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1424005190 con 0x7f1430100de0 2023-09-20T21:46:37.944 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:37.944 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2023-09-20T21:46:37.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T21:46:37.944 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2023-09-20T21:46:37.945 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2023-09-20T21:46:37.945 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:46:37.945 INFO:teuthology.orchestra.run.smithi022.stdout:] 2023-09-20T21:46:37.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.945+0000 7f1436dff700 1 -- 172.21.15.22:0/4204542823 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14200a13d0 msgr2=0x7f14200a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.945+0000 7f1436dff700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14200a13d0 0x7f14200a3890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f141c009d20 tx=0x7f141c009260 comp rx=0 tx=0).stop 2023-09-20T21:46:37.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.945+0000 7f1436dff700 1 -- 172.21.15.22:0/4204542823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430100de0 msgr2=0x7f1430196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:37.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.945+0000 7f1436dff700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430100de0 0x7f1430196ad0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f14240058e0 tx=0x7f142400f690 comp rx=0 tx=0).stop 2023-09-20T21:46:37.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.945+0000 7f1436dff700 1 -- 172.21.15.22:0/4204542823 shutdown_connections 2023-09-20T21:46:37.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.945+0000 7f1436dff700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430102970 0x7f143019e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.945+0000 7f1436dff700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1430100de0 0x7f1430196ad0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.945+0000 7f1436dff700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14200a13d0 0x7f14200a3890 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.945+0000 7f1436dff700 1 --2- 172.21.15.22:0/4204542823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1430101fd0 0x7f1430197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:37.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.945+0000 7f1436dff700 1 -- 172.21.15.22:0/4204542823 >> 172.21.15.22:0/4204542823 conn(0x7f14300fc310 msgr2=0x7f14300fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:37.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.946+0000 7f1436dff700 1 -- 172.21.15.22:0/4204542823 shutdown_connections 2023-09-20T21:46:37.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:37.946+0000 7f1436dff700 1 -- 172.21.15.22:0/4204542823 wait complete. 2023-09-20T21:46:37.965 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-20T21:46:37.965 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-20T21:46:38.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.186+0000 7f5a19dff700 1 -- 172.21.15.22:0/2631861661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a14100e40 msgr2=0x7f5a14101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.186+0000 7f5a19dff700 1 --2- 172.21.15.22:0/2631861661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a14100e40 0x7f5a14101260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009990 tx=0x7f5a04009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.186+0000 7f5a19dff700 1 -- 172.21.15.22:0/2631861661 shutdown_connections 2023-09-20T21:46:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.186+0000 7f5a19dff700 1 --2- 172.21.15.22:0/2631861661 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a14102950 0x7f5a14109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.186+0000 7f5a19dff700 1 --2- 172.21.15.22:0/2631861661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a14100e40 0x7f5a14101260 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.186+0000 7f5a19dff700 1 --2- 172.21.15.22:0/2631861661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a14101f90 0x7f5a14102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:38.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.186+0000 7f5a19dff700 1 -- 172.21.15.22:0/2631861661 >> 172.21.15.22:0/2631861661 conn(0x7f5a140fc3b0 msgr2=0x7f5a140fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:38.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.186+0000 7f5a19dff700 1 -- 172.21.15.22:0/2631861661 shutdown_connections 2023-09-20T21:46:38.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.186+0000 7f5a19dff700 1 -- 172.21.15.22:0/2631861661 wait complete. 2023-09-20T21:46:38.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.187+0000 7f5a19dff700 1 Processor -- start 2023-09-20T21:46:38.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a19dff700 1 -- start start 2023-09-20T21:46:38.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a19dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a14100e40 0x7f5a14196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:38.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a19dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a14101f90 0x7f5a14197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:38.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a19dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a14102950 0x7f5a1419e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:38.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a19dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1419e670 con 0x7f5a14100e40 2023-09-20T21:46:38.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a19dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1419e7b0 con 0x7f5a14101f90 2023-09-20T21:46:38.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a19dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1419e940 con 0x7f5a14102950 2023-09-20T21:46:38.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a137fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a14100e40 0x7f5a14196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:38.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a13fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a14102950 0x7f5a1419e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:38.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a12ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a14101f90 0x7f5a14197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:38.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a13fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a14102950 0x7f5a1419e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:39068/0 (socket says 172.21.15.22:39068) 2023-09-20T21:46:38.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a13fff700 1 -- 172.21.15.22:0/93250316 learned_addr learned my addr 172.21.15.22:0/93250316 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:38.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.188+0000 7f5a137fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a14100e40 0x7f5a14196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41932/0 (socket says 172.21.15.22:41932) 2023-09-20T21:46:38.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a13fff700 1 -- 172.21.15.22:0/93250316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a14101f90 msgr2=0x7f5a14197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a13fff700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a14101f90 0x7f5a14197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a13fff700 1 -- 172.21.15.22:0/93250316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a14100e40 msgr2=0x7f5a14196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a13fff700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a14100e40 0x7f5a14196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a13fff700 1 -- 172.21.15.22:0/93250316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a08009520 con 0x7f5a14102950 2023-09-20T21:46:38.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a12ffd700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a14101f90 0x7f5a14197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:38.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a137fe700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a14100e40 0x7f5a14196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:38.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a13fff700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a14102950 0x7f5a1419e0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00ef30 tx=0x7f5a0c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:38.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a10ff9700 1 -- 172.21.15.22:0/93250316 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c018070 con 0x7f5a14102950 2023-09-20T21:46:38.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a10ff9700 1 -- 172.21.15.22:0/93250316 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a0c00f040 con 0x7f5a14102950 2023-09-20T21:46:38.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a10ff9700 1 -- 172.21.15.22:0/93250316 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00cd00 con 0x7f5a14102950 2023-09-20T21:46:38.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.189+0000 7f5a19dff700 1 -- 172.21.15.22:0/93250316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a040095f0 con 0x7f5a14102950 2023-09-20T21:46:38.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.190+0000 7f5a19dff700 1 -- 172.21.15.22:0/93250316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1419eef0 con 0x7f5a14102950 2023-09-20T21:46:38.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.191+0000 7f5a19dff700 1 -- 172.21.15.22:0/93250316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a14108470 con 0x7f5a14102950 2023-09-20T21:46:38.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.193+0000 7f5a10ff9700 1 -- 172.21.15.22:0/93250316 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5a0c003a00 con 0x7f5a14102950 2023-09-20T21:46:38.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.194+0000 7f5a10ff9700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f59fc0a1380 0x7f59fc0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:38.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.194+0000 7f5a10ff9700 1 -- 172.21.15.22:0/93250316 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5a0c0cc3f0 con 0x7f5a14102950 2023-09-20T21:46:38.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.195+0000 7f5a137fe700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f59fc0a1380 0x7f59fc0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:38.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.195+0000 7f5a137fe700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f59fc0a1380 0x7f59fc0a3840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400b2e0 tx=0x7f5a04005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:38.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.197+0000 7f5a10ff9700 1 -- 172.21.15.22:0/93250316 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5a0c095f50 con 0x7f5a14102950 2023-09-20T21:46:38.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:37 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/146788625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:38.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:37 smithi022 ceph-mon[97651]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:38.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:38.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:37 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2411634987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:38.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:37 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4204542823' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-20T21:46:38.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:37 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/146788625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:38.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:37 smithi123 ceph-mon[134433]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:38.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:38.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:37 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2411634987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:38.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:37 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4204542823' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-20T21:46:38.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:37 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/146788625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:38.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:37 smithi089 ceph-mon[134090]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:38.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:38.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:37 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2411634987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:38.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:37 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4204542823' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-09-20T21:46:38.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:38.442+0000 7f5a19dff700 1 -- 172.21.15.22:0/93250316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5a1402d1f0 con 0x7f5a14102950 2023-09-20T21:46:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.419+0000 7f5a10ff9700 1 -- 172.21.15.22:0/93250316 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5a0c099570 con 0x7f5a14102950 2023-09-20T21:46:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.424+0000 7f5a19dff700 1 -- 172.21.15.22:0/93250316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f59fc0a1380 msgr2=0x7f59fc0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.424+0000 7f5a19dff700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f59fc0a1380 0x7f59fc0a3840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400b2e0 tx=0x7f5a04005fb0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.424+0000 7f5a19dff700 1 -- 172.21.15.22:0/93250316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a14102950 msgr2=0x7f5a1419e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.424+0000 7f5a19dff700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a14102950 0x7f5a1419e0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00ef30 tx=0x7f5a0c00c3a0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.425+0000 7f5a19dff700 1 -- 172.21.15.22:0/93250316 shutdown_connections 2023-09-20T21:46:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.425+0000 7f5a19dff700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a14102950 0x7f5a1419e0a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.425+0000 7f5a19dff700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a14100e40 0x7f5a14196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.425+0000 7f5a19dff700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f59fc0a1380 0x7f59fc0a3840 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.425+0000 7f5a19dff700 1 --2- 172.21.15.22:0/93250316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5a14101f90 0x7f5a14197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.425+0000 7f5a19dff700 1 -- 172.21.15.22:0/93250316 >> 172.21.15.22:0/93250316 conn(0x7f5a140fc3b0 msgr2=0x7f5a141055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:39.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.426+0000 7f5a19dff700 1 -- 172.21.15.22:0/93250316 shutdown_connections 2023-09-20T21:46:39.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.426+0000 7f5a19dff700 1 -- 172.21.15.22:0/93250316 wait complete. 2023-09-20T21:46:39.429 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2023-09-20T21:46:39.448 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-09-20T21:46:39.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.683+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/3781903253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc810d3c0 msgr2=0x7f2fc810f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:39.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.683+0000 7f2fcd1ff700 1 --2- 172.21.15.22:0/3781903253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc810d3c0 0x7f2fc810f7b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000b0a0 tx=0x7f2fc000b3b0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.683+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/3781903253 shutdown_connections 2023-09-20T21:46:39.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.683+0000 7f2fcd1ff700 1 --2- 172.21.15.22:0/3781903253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc810d3c0 0x7f2fc810f7b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.683+0000 7f2fcd1ff700 1 --2- 172.21.15.22:0/3781903253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc8074e20 0x7f2fc8073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.683+0000 7f2fcd1ff700 1 --2- 172.21.15.22:0/3781903253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fc8073840 0x7f2fc8073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.683+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/3781903253 >> 172.21.15.22:0/3781903253 conn(0x7f2fc80fc360 msgr2=0x7f2fc80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.683+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/3781903253 shutdown_connections 2023-09-20T21:46:39.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.683+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/3781903253 wait complete. 2023-09-20T21:46:39.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.684+0000 7f2fcd1ff700 1 Processor -- start 2023-09-20T21:46:39.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.684+0000 7f2fcd1ff700 1 -- start start 2023-09-20T21:46:39.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fcd1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc8073840 0x7f2fc819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:39.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fcd1ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc8074e20 0x7f2fc819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:39.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fcd1ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fc810d3c0 0x7f2fc81a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:39.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fcd1ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc81a4d90 con 0x7f2fc8074e20 2023-09-20T21:46:39.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fcd1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc81a4ed0 con 0x7f2fc8073840 2023-09-20T21:46:39.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fcd1ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc81a5060 con 0x7f2fc810d3c0 2023-09-20T21:46:39.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fc6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc8073840 0x7f2fc819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:39.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fc6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc8073840 0x7f2fc819d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60870/0 (socket says 172.21.15.22:60870) 2023-09-20T21:46:39.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fc6d9d700 1 -- 172.21.15.22:0/1416113917 learned_addr learned my addr 172.21.15.22:0/1416113917 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fc759e700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fc810d3c0 0x7f2fc81a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fc659c700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc8074e20 0x7f2fc819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fc6d9d700 1 -- 172.21.15.22:0/1416113917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fc810d3c0 msgr2=0x7f2fc81a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.685+0000 7f2fc6d9d700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fc810d3c0 0x7f2fc81a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.686+0000 7f2fc6d9d700 1 -- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc8074e20 msgr2=0x7f2fc819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.686+0000 7f2fc6d9d700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc8074e20 0x7f2fc819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.686+0000 7f2fc6d9d700 1 -- 172.21.15.22:0/1416113917 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc0009d00 con 0x7f2fc8073840 2023-09-20T21:46:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.686+0000 7f2fc659c700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc8074e20 0x7f2fc819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:46:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.686+0000 7f2fc6d9d700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc8073840 0x7f2fc819d210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800e9a0 tx=0x7f2fb800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:39.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.686+0000 7f2faffff700 1 -- 172.21.15.22:0/1416113917 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8009790 con 0x7f2fc8073840 2023-09-20T21:46:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.686+0000 7f2faffff700 1 -- 172.21.15.22:0/1416113917 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fb800cbd0 con 0x7f2fc8073840 2023-09-20T21:46:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.686+0000 7f2faffff700 1 -- 172.21.15.22:0/1416113917 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8018680 con 0x7f2fc8073840 2023-09-20T21:46:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.686+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/1416113917 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc81a5340 con 0x7f2fc8073840 2023-09-20T21:46:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.686+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/1416113917 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc81a5800 con 0x7f2fc8073840 2023-09-20T21:46:39.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.687+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/1416113917 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc8102f10 con 0x7f2fc8073840 2023-09-20T21:46:39.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.689+0000 7f2faffff700 1 -- 172.21.15.22:0/1416113917 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2fb800cd40 con 0x7f2fc8073840 2023-09-20T21:46:39.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.691+0000 7f2faffff700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2fb00a1380 0x7f2fb00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.692+0000 7f2faffff700 1 -- 172.21.15.22:0/1416113917 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f2fb8014070 con 0x7f2fc8073840 2023-09-20T21:46:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.692+0000 7f2fc659c700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2fb00a1380 0x7f2fb00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:39.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.693+0000 7f2fc659c700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2fb00a1380 0x7f2fb00a3840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8074b40 tx=0x7f2fbc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:39.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.695+0000 7f2faffff700 1 -- 172.21.15.22:0/1416113917 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2fb8051e10 con 0x7f2fc8073840 2023-09-20T21:46:39.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/93250316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-20T21:46:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-20T21:46:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:39.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:46:39.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/93250316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-20T21:46:39.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-20T21:46:39.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:39.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:39.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:46:39.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/93250316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-20T21:46:39.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-09-20T21:46:39.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:39.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:39.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:46:39.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:39.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:39.912+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/1416113917 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f2fc8112c30 con 0x7f2fc8073840 2023-09-20T21:46:40.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.495+0000 7f2faffff700 1 -- 172.21.15.22:0/1416113917 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7f2fb8096550 con 0x7f2fc8073840 2023-09-20T21:46:40.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.511+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2fb00a1380 msgr2=0x7f2fb00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.511+0000 7f2fcd1ff700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2fb00a1380 0x7f2fb00a3840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8074b40 tx=0x7f2fbc009260 comp rx=0 tx=0).stop 2023-09-20T21:46:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.511+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/1416113917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc8073840 msgr2=0x7f2fc819d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:40.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.511+0000 7f2fcd1ff700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc8073840 0x7f2fc819d210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800e9a0 tx=0x7f2fb800ecb0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.512+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/1416113917 shutdown_connections 2023-09-20T21:46:40.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.512+0000 7f2fcd1ff700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fc810d3c0 0x7f2fc81a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.512+0000 7f2fcd1ff700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2fb00a1380 0x7f2fb00a3840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.512+0000 7f2fcd1ff700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2fc8073840 0x7f2fc819d210 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.512+0000 7f2fcd1ff700 1 --2- 172.21.15.22:0/1416113917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fc8074e20 0x7f2fc819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.512+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/1416113917 >> 172.21.15.22:0/1416113917 conn(0x7f2fc80fc360 msgr2=0x7f2fc80fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.513+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/1416113917 shutdown_connections 2023-09-20T21:46:40.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.513+0000 7f2fcd1ff700 1 -- 172.21.15.22:0/1416113917 wait complete. 2023-09-20T21:46:40.516 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2023-09-20T21:46:40.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-20T21:46:40.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:40 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-20T21:46:40.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:40 smithi022 ceph-mon[97651]: osdmap e82: 12 total, 12 up, 12 in 2023-09-20T21:46:40.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:40 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:46:40.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.750+0000 7f1d655ff700 1 -- 172.21.15.22:0/1284612142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60100df0 msgr2=0x7f1d60101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:40.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.750+0000 7f1d655ff700 1 --2- 172.21.15.22:0/1284612142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60100df0 0x7f1d60101210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009990 tx=0x7f1d50009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.750+0000 7f1d655ff700 1 -- 172.21.15.22:0/1284612142 shutdown_connections 2023-09-20T21:46:40.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.750+0000 7f1d655ff700 1 --2- 172.21.15.22:0/1284612142 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60102980 0x7f1d60109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.750+0000 7f1d655ff700 1 --2- 172.21.15.22:0/1284612142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60100df0 0x7f1d60101210 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.750+0000 7f1d655ff700 1 --2- 172.21.15.22:0/1284612142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60101fe0 0x7f1d60102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.750+0000 7f1d655ff700 1 -- 172.21.15.22:0/1284612142 >> 172.21.15.22:0/1284612142 conn(0x7f1d600fc360 msgr2=0x7f1d600fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:40.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.750+0000 7f1d655ff700 1 -- 172.21.15.22:0/1284612142 shutdown_connections 2023-09-20T21:46:40.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.750+0000 7f1d655ff700 1 -- 172.21.15.22:0/1284612142 wait complete. 2023-09-20T21:46:40.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.751+0000 7f1d655ff700 1 Processor -- start 2023-09-20T21:46:40.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.751+0000 7f1d655ff700 1 -- start start 2023-09-20T21:46:40.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d655ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60100df0 0x7f1d60198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:40.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d655ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60101fe0 0x7f1d60199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:40.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d655ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60102980 0x7f1d601a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:40.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d655ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601a0840 con 0x7f1d60101fe0 2023-09-20T21:46:40.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d655ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601a09b0 con 0x7f1d60102980 2023-09-20T21:46:40.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d655ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601a0b70 con 0x7f1d60100df0 2023-09-20T21:46:40.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60100df0 0x7f1d60198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:40.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d5effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60100df0 0x7f1d60198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:39122/0 (socket says 172.21.15.22:39122) 2023-09-20T21:46:40.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d5e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60101fe0 0x7f1d60199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:40.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d5effd700 1 -- 172.21.15.22:0/1874940627 learned_addr learned my addr 172.21.15.22:0/1874940627 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:40.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d5f7fe700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60102980 0x7f1d601a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:40.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d5effd700 1 -- 172.21.15.22:0/1874940627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60102980 msgr2=0x7f1d601a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:40.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d5effd700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60102980 0x7f1d601a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d5effd700 1 -- 172.21.15.22:0/1874940627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60101fe0 msgr2=0x7f1d60199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:40.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d5effd700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60101fe0 0x7f1d60199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:40.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.752+0000 7f1d5effd700 1 -- 172.21.15.22:0/1874940627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d500095f0 con 0x7f1d60100df0 2023-09-20T21:46:40.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.753+0000 7f1d5effd700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60100df0 0x7f1d60198d00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d50000c00 tx=0x7f1d50005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:40.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.753+0000 7f1d47fff700 1 -- 172.21.15.22:0/1874940627 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5001d070 con 0x7f1d60100df0 2023-09-20T21:46:40.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.753+0000 7f1d47fff700 1 -- 172.21.15.22:0/1874940627 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d50022470 con 0x7f1d60100df0 2023-09-20T21:46:40.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.753+0000 7f1d655ff700 1 -- 172.21.15.22:0/1874940627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d601a0df0 con 0x7f1d60100df0 2023-09-20T21:46:40.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.753+0000 7f1d47fff700 1 -- 172.21.15.22:0/1874940627 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50017650 con 0x7f1d60100df0 2023-09-20T21:46:40.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.753+0000 7f1d655ff700 1 -- 172.21.15.22:0/1874940627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d601a11c0 con 0x7f1d60100df0 2023-09-20T21:46:40.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.754+0000 7f1d655ff700 1 -- 172.21.15.22:0/1874940627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d601933f0 con 0x7f1d60100df0 2023-09-20T21:46:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.756+0000 7f1d47fff700 1 -- 172.21.15.22:0/1874940627 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1d500225e0 con 0x7f1d60100df0 2023-09-20T21:46:40.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.757+0000 7f1d47fff700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a1300 0x7f1d480a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:40.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.758+0000 7f1d47fff700 1 -- 172.21.15.22:0/1874940627 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f1d500cdd80 con 0x7f1d60100df0 2023-09-20T21:46:40.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.758+0000 7f1d5e7fc700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a1300 0x7f1d480a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:40.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.758+0000 7f1d5e7fc700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a1300 0x7f1d480a37c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009c60 tx=0x7f1d54009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:40.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:40.759+0000 7f1d47fff700 1 -- 172.21.15.22:0/1874940627 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1d500975f0 con 0x7f1d60100df0 2023-09-20T21:46:40.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:40 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-20T21:46:40.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:40 smithi123 ceph-mon[134433]: osdmap e82: 12 total, 12 up, 12 in 2023-09-20T21:46:40.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:40 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:46:40.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:40 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-09-20T21:46:40.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:40 smithi089 ceph-mon[134090]: osdmap e82: 12 total, 12 up, 12 in 2023-09-20T21:46:40.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:40 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:46:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.041+0000 7f1d655ff700 1 -- 172.21.15.22:0/1874940627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1d6002cef0 con 0x7f1d60100df0 2023-09-20T21:46:41.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.517+0000 7f1d47fff700 1 -- 172.21.15.22:0/1874940627 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1d5009ac10 con 0x7f1d60100df0 2023-09-20T21:46:41.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.522+0000 7f1d655ff700 1 -- 172.21.15.22:0/1874940627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a1300 msgr2=0x7f1d480a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:41.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.522+0000 7f1d655ff700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a1300 0x7f1d480a37c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009c60 tx=0x7f1d54009260 comp rx=0 tx=0).stop 2023-09-20T21:46:41.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.522+0000 7f1d655ff700 1 -- 172.21.15.22:0/1874940627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60100df0 msgr2=0x7f1d60198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:41.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.522+0000 7f1d655ff700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60100df0 0x7f1d60198d00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d50000c00 tx=0x7f1d50005720 comp rx=0 tx=0).stop 2023-09-20T21:46:41.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.523+0000 7f1d655ff700 1 -- 172.21.15.22:0/1874940627 shutdown_connections 2023-09-20T21:46:41.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.523+0000 7f1d655ff700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d480a1300 0x7f1d480a37c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:41.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.523+0000 7f1d655ff700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d60102980 0x7f1d601a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:41.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.523+0000 7f1d655ff700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60100df0 0x7f1d60198d00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:41.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.523+0000 7f1d655ff700 1 --2- 172.21.15.22:0/1874940627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d60101fe0 0x7f1d60199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:41.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.523+0000 7f1d655ff700 1 -- 172.21.15.22:0/1874940627 >> 172.21.15.22:0/1874940627 conn(0x7f1d600fc360 msgr2=0x7f1d60107920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:41.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.523+0000 7f1d655ff700 1 -- 172.21.15.22:0/1874940627 shutdown_connections 2023-09-20T21:46:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.523+0000 7f1d655ff700 1 -- 172.21.15.22:0/1874940627 wait complete. 2023-09-20T21:46:41.526 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 2 and data pool 3 2023-09-20T21:46:41.546 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-09-20T21:46:41.546 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-09-20T21:46:41.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:41 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1416113917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-20T21:46:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:41 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-20T21:46:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:41 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-20T21:46:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:41 smithi022 ceph-mon[97651]: osdmap e83: 12 total, 12 up, 12 in 2023-09-20T21:46:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:41 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1874940627' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-20T21:46:41.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:41 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-20T21:46:41.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:41 smithi022 ceph-mon[97651]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-20T21:46:41.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:41 smithi022 ceph-mon[97651]: pgmap v270: 97 pgs: 15 creating+peering, 4 active+clean, 78 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.766+0000 7f98a03ff700 1 -- 172.21.15.22:0/2751109381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9898074e20 msgr2=0x7f9898073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:41.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.766+0000 7f98a03ff700 1 --2- 172.21.15.22:0/2751109381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9898074e20 0x7f9898073270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9894009990 tx=0x7f9894009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:41.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.766+0000 7f98a03ff700 1 -- 172.21.15.22:0/2751109381 shutdown_connections 2023-09-20T21:46:41.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.766+0000 7f98a03ff700 1 --2- 172.21.15.22:0/2751109381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9898108c90 0x7f989810b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:41.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.766+0000 7f98a03ff700 1 --2- 172.21.15.22:0/2751109381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9898074e20 0x7f9898073270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:41.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.766+0000 7f98a03ff700 1 --2- 172.21.15.22:0/2751109381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9898073840 0x7f9898073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:41.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.766+0000 7f98a03ff700 1 -- 172.21.15.22:0/2751109381 >> 172.21.15.22:0/2751109381 conn(0x7f98980fc320 msgr2=0x7f98980fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:41.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.766+0000 7f98a03ff700 1 -- 172.21.15.22:0/2751109381 shutdown_connections 2023-09-20T21:46:41.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.766+0000 7f98a03ff700 1 -- 172.21.15.22:0/2751109381 wait complete. 2023-09-20T21:46:41.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.767+0000 7f98a03ff700 1 Processor -- start 2023-09-20T21:46:41.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.767+0000 7f98a03ff700 1 -- start start 2023-09-20T21:46:41.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f98a03ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9898073840 0x7f9898198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:41.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f98a03ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9898074e20 0x7f9898199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:41.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f98a03ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9898108c90 0x7f98981a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:41.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f98a03ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98981a0840 con 0x7f9898074e20 2023-09-20T21:46:41.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f98a03ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98981a09b0 con 0x7f9898108c90 2023-09-20T21:46:41.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f98a03ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98981a0b70 con 0x7f9898073840 2023-09-20T21:46:41.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989d99a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9898074e20 0x7f9898199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:41.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989e99c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9898108c90 0x7f98981a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:41.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989d99a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9898074e20 0x7f9898199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41986/0 (socket says 172.21.15.22:41986) 2023-09-20T21:46:41.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989d99a700 1 -- 172.21.15.22:0/4203629356 learned_addr learned my addr 172.21.15.22:0/4203629356 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:41.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989e19b700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9898073840 0x7f9898198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:41.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989e99c700 1 -- 172.21.15.22:0/4203629356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9898073840 msgr2=0x7f9898198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989e99c700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9898073840 0x7f9898198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989e99c700 1 -- 172.21.15.22:0/4203629356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9898074e20 msgr2=0x7f9898199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989e99c700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9898074e20 0x7f9898199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989e99c700 1 -- 172.21.15.22:0/4203629356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98940095f0 con 0x7f9898108c90 2023-09-20T21:46:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.768+0000 7f989e19b700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9898073840 0x7f9898198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:46:41.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.769+0000 7f989e99c700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9898108c90 0x7f98981a0270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9890007f10 tx=0x7f989000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:41.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.769+0000 7f988f7fe700 1 -- 172.21.15.22:0/4203629356 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989000ddd0 con 0x7f9898108c90 2023-09-20T21:46:41.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.769+0000 7f988f7fe700 1 -- 172.21.15.22:0/4203629356 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f989000f040 con 0x7f9898108c90 2023-09-20T21:46:41.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.769+0000 7f988f7fe700 1 -- 172.21.15.22:0/4203629356 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890005a90 con 0x7f9898108c90 2023-09-20T21:46:41.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.769+0000 7f98a03ff700 1 -- 172.21.15.22:0/4203629356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98981a0e50 con 0x7f9898108c90 2023-09-20T21:46:41.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.769+0000 7f98a03ff700 1 -- 172.21.15.22:0/4203629356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98981a1220 con 0x7f9898108c90 2023-09-20T21:46:41.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.770+0000 7f98a03ff700 1 -- 172.21.15.22:0/4203629356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9898107340 con 0x7f9898108c90 2023-09-20T21:46:41.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.772+0000 7f988f7fe700 1 -- 172.21.15.22:0/4203629356 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f9890004ad0 con 0x7f9898108c90 2023-09-20T21:46:41.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.773+0000 7f988f7fe700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f98840a1380 0x7f98840a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:41.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.773+0000 7f988f7fe700 1 -- 172.21.15.22:0/4203629356 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f98900cc530 con 0x7f9898108c90 2023-09-20T21:46:41.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.774+0000 7f989e19b700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f98840a1380 0x7f98840a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:41.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.774+0000 7f989e19b700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f98840a1380 0x7f98840a3840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f989400b2e0 tx=0x7f9894005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:41.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:41.775+0000 7f988f7fe700 1 -- 172.21.15.22:0/4203629356 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9890095cb0 con 0x7f9898108c90 2023-09-20T21:46:41.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:41 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1416113917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-20T21:46:41.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:41 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-20T21:46:41.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:41 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-20T21:46:41.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:41 smithi123 ceph-mon[134433]: osdmap e83: 12 total, 12 up, 12 in 2023-09-20T21:46:41.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:41 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1874940627' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-20T21:46:41.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:41 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-20T21:46:41.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:41 smithi123 ceph-mon[134433]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-20T21:46:41.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:41 smithi123 ceph-mon[134433]: pgmap v270: 97 pgs: 15 creating+peering, 4 active+clean, 78 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:41.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:41 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1416113917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-20T21:46:41.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:41 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-09-20T21:46:41.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:41 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-09-20T21:46:41.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:41 smithi089 ceph-mon[134090]: osdmap e83: 12 total, 12 up, 12 in 2023-09-20T21:46:41.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:41 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1874940627' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-20T21:46:41.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:41 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-09-20T21:46:41.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:41 smithi089 ceph-mon[134090]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-09-20T21:46:41.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:41 smithi089 ceph-mon[134090]: pgmap v270: 97 pgs: 15 creating+peering, 4 active+clean, 78 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T21:46:42.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.007+0000 7f98a03ff700 1 -- 172.21.15.22:0/4203629356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f989810e810 con 0x7f9898108c90 2023-09-20T21:46:42.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.502+0000 7f988f7fe700 1 -- 172.21.15.22:0/4203629356 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7f98900992d0 con 0x7f9898108c90 2023-09-20T21:46:42.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 -- 172.21.15.22:0/4203629356 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f98840a1380 msgr2=0x7f98840a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:42.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f98840a1380 0x7f98840a3840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f989400b2e0 tx=0x7f9894005fb0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 -- 172.21.15.22:0/4203629356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9898108c90 msgr2=0x7f98981a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:42.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9898108c90 0x7f98981a0270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9890007f10 tx=0x7f989000d3b0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 -- 172.21.15.22:0/4203629356 shutdown_connections 2023-09-20T21:46:42.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f98840a1380 0x7f98840a3840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9898108c90 0x7f98981a0270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9898073840 0x7f9898198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 --2- 172.21.15.22:0/4203629356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9898074e20 0x7f9898199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 -- 172.21.15.22:0/4203629356 >> 172.21.15.22:0/4203629356 conn(0x7f98980fc320 msgr2=0x7f98980fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:42.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 -- 172.21.15.22:0/4203629356 shutdown_connections 2023-09-20T21:46:42.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.505+0000 7f98a03ff700 1 -- 172.21.15.22:0/4203629356 wait complete. 2023-09-20T21:46:42.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-09-20T21:46:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: osdmap e84: 12 total, 12 up, 12 in 2023-09-20T21:46:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-20T21:46:42.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-20T21:46:42.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-20T21:46:42.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-20T21:46:42.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4203629356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-20T21:46:42.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-20T21:46:42.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.751+0000 7f443c1ff700 1 -- 172.21.15.22:0/2408835480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44341018f0 msgr2=0x7f4434103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:42.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.751+0000 7f443c1ff700 1 --2- 172.21.15.22:0/2408835480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44341018f0 0x7f4434103ce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4430009990 tx=0x7f4430009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.752+0000 7f443c1ff700 1 -- 172.21.15.22:0/2408835480 shutdown_connections 2023-09-20T21:46:42.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.752+0000 7f443c1ff700 1 --2- 172.21.15.22:0/2408835480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434106b50 0x7f4434108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.752+0000 7f443c1ff700 1 --2- 172.21.15.22:0/2408835480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44341018f0 0x7f4434103ce0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.752+0000 7f443c1ff700 1 --2- 172.21.15.22:0/2408835480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4434104220 0x7f4434106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.752+0000 7f443c1ff700 1 -- 172.21.15.22:0/2408835480 >> 172.21.15.22:0/2408835480 conn(0x7f44340fb240 msgr2=0x7f44340fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:42.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.752+0000 7f443c1ff700 1 -- 172.21.15.22:0/2408835480 shutdown_connections 2023-09-20T21:46:42.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.752+0000 7f443c1ff700 1 -- 172.21.15.22:0/2408835480 wait complete. 2023-09-20T21:46:42.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.753+0000 7f443c1ff700 1 Processor -- start 2023-09-20T21:46:42.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.753+0000 7f443c1ff700 1 -- start start 2023-09-20T21:46:42.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f443c1ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44341018f0 0x7f4434198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:42.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f443c1ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434104220 0x7f4434199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:42.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f443c1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4434106b50 0x7f44341a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:42.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f443c1ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341a0750 con 0x7f44341018f0 2023-09-20T21:46:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f443c1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341a0890 con 0x7f4434106b50 2023-09-20T21:46:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f443c1ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44341a0a20 con 0x7f4434104220 2023-09-20T21:46:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f4439f9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44341018f0 0x7f4434198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f443979a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434104220 0x7f4434199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:42.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f4439f9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44341018f0 0x7f4434198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41998/0 (socket says 172.21.15.22:41998) 2023-09-20T21:46:42.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f4439f9b700 1 -- 172.21.15.22:0/1532039645 learned_addr learned my addr 172.21.15.22:0/1532039645 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:42.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.754+0000 7f443a79c700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4434106b50 0x7f44341a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:42.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f443979a700 1 -- 172.21.15.22:0/1532039645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4434106b50 msgr2=0x7f44341a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:42.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f443979a700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4434106b50 0x7f44341a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f443979a700 1 -- 172.21.15.22:0/1532039645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44341018f0 msgr2=0x7f4434198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:42.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f443979a700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44341018f0 0x7f4434198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:42.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f443979a700 1 -- 172.21.15.22:0/1532039645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300095f0 con 0x7f4434104220 2023-09-20T21:46:42.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f443979a700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434104220 0x7f4434199110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f442400e9f0 tx=0x7f442400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:42.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f442affd700 1 -- 172.21.15.22:0/1532039645 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400c9c0 con 0x7f4434104220 2023-09-20T21:46:42.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f442affd700 1 -- 172.21.15.22:0/1532039645 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4424010430 con 0x7f4434104220 2023-09-20T21:46:42.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f442affd700 1 -- 172.21.15.22:0/1532039645 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424018680 con 0x7f4434104220 2023-09-20T21:46:42.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f443c1ff700 1 -- 172.21.15.22:0/1532039645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44341a0d00 con 0x7f4434104220 2023-09-20T21:46:42.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.755+0000 7f443c1ff700 1 -- 172.21.15.22:0/1532039645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44341a11c0 con 0x7f4434104220 2023-09-20T21:46:42.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.757+0000 7f443c1ff700 1 -- 172.21.15.22:0/1532039645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44341086d0 con 0x7f4434104220 2023-09-20T21:46:42.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.758+0000 7f442affd700 1 -- 172.21.15.22:0/1532039645 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f4424010860 con 0x7f4434104220 2023-09-20T21:46:42.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.760+0000 7f442affd700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44200a1420 0x7f44200a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:42.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.760+0000 7f442affd700 1 -- 172.21.15.22:0/1532039645 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f4424014070 con 0x7f4434104220 2023-09-20T21:46:42.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.760+0000 7f4439f9b700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44200a1420 0x7f44200a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:42.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.761+0000 7f4439f9b700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44200a1420 0x7f44200a38e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f443000b2e0 tx=0x7f4430005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:42.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.764+0000 7f442affd700 1 -- 172.21.15.22:0/1532039645 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4424051e10 con 0x7f4434104220 2023-09-20T21:46:42.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: osdmap e84: 12 total, 12 up, 12 in 2023-09-20T21:46:42.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-20T21:46:42.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-20T21:46:42.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-20T21:46:42.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-20T21:46:42.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4203629356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-20T21:46:42.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-20T21:46:42.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: osdmap e84: 12 total, 12 up, 12 in 2023-09-20T21:46:42.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-20T21:46:42.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-09-20T21:46:42.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-09-20T21:46:42.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-09-20T21:46:42.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4203629356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-20T21:46:42.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-09-20T21:46:42.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:42.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:42.982+0000 7f443c1ff700 1 -- 172.21.15.22:0/1532039645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f443410c600 con 0x7f4434104220 2023-09-20T21:46:43.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-20T21:46:43.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: osdmap e85: 12 total, 12 up, 12 in 2023-09-20T21:46:43.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] up:active 2023-09-20T21:46:43.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:46:43.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:46:43.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1532039645' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-20T21:46:43.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-20T21:46:43.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: pgmap v273: 97 pgs: 27 creating+peering, 38 active+clean, 32 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2023-09-20T21:46:43.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:43.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:43.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:43.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:46:43.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:43 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:43.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-20T21:46:43.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: osdmap e85: 12 total, 12 up, 12 in 2023-09-20T21:46:43.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] up:active 2023-09-20T21:46:43.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:46:43.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:46:43.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1532039645' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-20T21:46:43.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-20T21:46:43.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: pgmap v273: 97 pgs: 27 creating+peering, 38 active+clean, 32 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2023-09-20T21:46:43.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:43.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:43.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:43.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:46:43.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:43 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:43.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-09-20T21:46:43.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: osdmap e85: 12 total, 12 up, 12 in 2023-09-20T21:46:43.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] up:active 2023-09-20T21:46:43.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:46:43.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:46:43.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1532039645' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-20T21:46:43.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-09-20T21:46:43.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: pgmap v273: 97 pgs: 27 creating+peering, 38 active+clean, 32 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2023-09-20T21:46:43.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:43.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:44.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:46:44.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:46:44.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:46:44.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.558+0000 7f442affd700 1 -- 172.21.15.22:0/1532039645 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7f4424096370 con 0x7f4434104220 2023-09-20T21:46:44.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.566+0000 7f443c1ff700 1 -- 172.21.15.22:0/1532039645 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44200a1420 msgr2=0x7f44200a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:44.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.566+0000 7f443c1ff700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44200a1420 0x7f44200a38e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f443000b2e0 tx=0x7f4430005960 comp rx=0 tx=0).stop 2023-09-20T21:46:44.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.566+0000 7f443c1ff700 1 -- 172.21.15.22:0/1532039645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434104220 msgr2=0x7f4434199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:44.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.566+0000 7f443c1ff700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434104220 0x7f4434199110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f442400e9f0 tx=0x7f442400ed00 comp rx=0 tx=0).stop 2023-09-20T21:46:44.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.568+0000 7f443c1ff700 1 -- 172.21.15.22:0/1532039645 shutdown_connections 2023-09-20T21:46:44.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.568+0000 7f443c1ff700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44200a1420 0x7f44200a38e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:44.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.568+0000 7f443c1ff700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4434106b50 0x7f44341a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:44.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.568+0000 7f443c1ff700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44341018f0 0x7f4434198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:44.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.568+0000 7f443c1ff700 1 --2- 172.21.15.22:0/1532039645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434104220 0x7f4434199110 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:44.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.568+0000 7f443c1ff700 1 -- 172.21.15.22:0/1532039645 >> 172.21.15.22:0/1532039645 conn(0x7f44340fb240 msgr2=0x7f44340ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:44.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.568+0000 7f443c1ff700 1 -- 172.21.15.22:0/1532039645 shutdown_connections 2023-09-20T21:46:44.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.568+0000 7f443c1ff700 1 -- 172.21.15.22:0/1532039645 wait complete. 2023-09-20T21:46:44.570 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data_ec' created 2023-09-20T21:46:44.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-09-20T21:46:44.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.800+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1649550745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa758101850 msgr2=0x7fa758103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:44.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.800+0000 7fa75e5ff700 1 --2- 172.21.15.22:0/1649550745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa758101850 0x7fa758103c40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa748009990 tx=0x7fa748009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:44.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.800+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1649550745 shutdown_connections 2023-09-20T21:46:44.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.800+0000 7fa75e5ff700 1 --2- 172.21.15.22:0/1649550745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa758106ab0 0x7fa758108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:44.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.800+0000 7fa75e5ff700 1 --2- 172.21.15.22:0/1649550745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa758101850 0x7fa758103c40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:44.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.800+0000 7fa75e5ff700 1 --2- 172.21.15.22:0/1649550745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa758104180 0x7fa758106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:44.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.800+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1649550745 >> 172.21.15.22:0/1649550745 conn(0x7fa7580fb240 msgr2=0x7fa7580fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:44.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.800+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1649550745 shutdown_connections 2023-09-20T21:46:44.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.800+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1649550745 wait complete. 2023-09-20T21:46:44.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.801+0000 7fa75e5ff700 1 Processor -- start 2023-09-20T21:46:44.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa75e5ff700 1 -- start start 2023-09-20T21:46:44.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa75e5ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa758101850 0x7fa7581011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:44.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa75e5ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa758104180 0x7fa7580ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:44.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa75e5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa758106ab0 0x7fa7580ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:44.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa75e5ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7581004e0 con 0x7fa758101850 2023-09-20T21:46:44.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa75e5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa758100650 con 0x7fa758106ab0 2023-09-20T21:46:44.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa75e5ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7581a0b20 con 0x7fa758104180 2023-09-20T21:46:44.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa757fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa758101850 0x7fa7581011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:44.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa75cb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa758106ab0 0x7fa7580ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:44.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa757fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa758101850 0x7fa7581011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42026/0 (socket says 172.21.15.22:42026) 2023-09-20T21:46:44.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa757fff700 1 -- 172.21.15.22:0/1233316159 learned_addr learned my addr 172.21.15.22:0/1233316159 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:44.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.802+0000 7fa7577fe700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa758104180 0x7fa7580ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:44.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa75cb9c700 1 -- 172.21.15.22:0/1233316159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa758104180 msgr2=0x7fa7580ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:44.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa75cb9c700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa758104180 0x7fa7580ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:44.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa75cb9c700 1 -- 172.21.15.22:0/1233316159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa758101850 msgr2=0x7fa7581011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:44.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa75cb9c700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa758101850 0x7fa7581011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:44.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa75cb9c700 1 -- 172.21.15.22:0/1233316159 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7480095f0 con 0x7fa758106ab0 2023-09-20T21:46:44.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa75cb9c700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa758106ab0 0x7fa7580ffd40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa750007f10 tx=0x7fa75000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:44.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa7557fa700 1 -- 172.21.15.22:0/1233316159 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa750004db0 con 0x7fa758106ab0 2023-09-20T21:46:44.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa7557fa700 1 -- 172.21.15.22:0/1233316159 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75000f040 con 0x7fa758106ab0 2023-09-20T21:46:44.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1233316159 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7581a0e00 con 0x7fa758106ab0 2023-09-20T21:46:44.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa7557fa700 1 -- 172.21.15.22:0/1233316159 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa750005990 con 0x7fa758106ab0 2023-09-20T21:46:44.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.803+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1233316159 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7581a1280 con 0x7fa758106ab0 2023-09-20T21:46:44.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.804+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1233316159 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7580fcdc0 con 0x7fa758106ab0 2023-09-20T21:46:44.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.806+0000 7fa7557fa700 1 -- 172.21.15.22:0/1233316159 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa75000db20 con 0x7fa758106ab0 2023-09-20T21:46:44.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.808+0000 7fa7557fa700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7400a1300 0x7fa7400a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:44.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.808+0000 7fa7557fa700 1 -- 172.21.15.22:0/1233316159 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fa7500cce90 con 0x7fa758106ab0 2023-09-20T21:46:44.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.808+0000 7fa757fff700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7400a1300 0x7fa7400a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:44.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.809+0000 7fa757fff700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7400a1300 0x7fa7400a37c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa748006010 tx=0x7fa748005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:44.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:44.810+0000 7fa7557fa700 1 -- 172.21.15.22:0/1233316159 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa750096330 con 0x7fa758106ab0 2023-09-20T21:46:44.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:44 smithi123 ceph-mon[134433]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-20T21:46:44.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:44 smithi123 ceph-mon[134433]: Cluster is now healthy 2023-09-20T21:46:44.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:44 smithi123 ceph-mon[134433]: osdmap e86: 12 total, 12 up, 12 in 2023-09-20T21:46:44.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:44 smithi089 ceph-mon[134090]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-20T21:46:44.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:44 smithi089 ceph-mon[134090]: Cluster is now healthy 2023-09-20T21:46:44.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:44 smithi089 ceph-mon[134090]: osdmap e86: 12 total, 12 up, 12 in 2023-09-20T21:46:44.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:44 smithi022 ceph-mon[97651]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-09-20T21:46:44.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:44 smithi022 ceph-mon[97651]: Cluster is now healthy 2023-09-20T21:46:44.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:44 smithi022 ceph-mon[97651]: osdmap e86: 12 total, 12 up, 12 in 2023-09-20T21:46:45.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.020+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1233316159 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fa75802d260 con 0x7fa758106ab0 2023-09-20T21:46:45.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.566+0000 7fa7557fa700 1 -- 172.21.15.22:0/1233316159 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa750023020 con 0x7fa758106ab0 2023-09-20T21:46:45.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.580+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1233316159 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7400a1300 msgr2=0x7fa7400a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.580+0000 7fa75e5ff700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7400a1300 0x7fa7400a37c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa748006010 tx=0x7fa748005960 comp rx=0 tx=0).stop 2023-09-20T21:46:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.580+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1233316159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa758106ab0 msgr2=0x7fa7580ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:45.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.580+0000 7fa75e5ff700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa758106ab0 0x7fa7580ffd40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa750007f10 tx=0x7fa75000d3b0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.589+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1233316159 shutdown_connections 2023-09-20T21:46:45.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.589+0000 7fa75e5ff700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7400a1300 0x7fa7400a37c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.589+0000 7fa75e5ff700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa758106ab0 0x7fa7580ffd40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.589+0000 7fa75e5ff700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa758101850 0x7fa7581011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.589+0000 7fa75e5ff700 1 --2- 172.21.15.22:0/1233316159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa758104180 0x7fa7580ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.590+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1233316159 >> 172.21.15.22:0/1233316159 conn(0x7fa7580fb240 msgr2=0x7fa7581024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:45.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.590+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1233316159 shutdown_connections 2023-09-20T21:46:45.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.590+0000 7fa75e5ff700 1 -- 172.21.15.22:0/1233316159 wait complete. 2023-09-20T21:46:45.592 INFO:teuthology.orchestra.run.smithi022.stderr:set pool 4 allow_ec_overwrites to true 2023-09-20T21:46:45.612 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-09-20T21:46:45.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.820+0000 7f06013ff700 1 -- 172.21.15.22:0/2010815982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05fc100df0 msgr2=0x7f05fc101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:45.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.820+0000 7f06013ff700 1 --2- 172.21.15.22:0/2010815982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05fc100df0 0x7f05fc101210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0099c0 tx=0x7f05ec009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.820+0000 7f06013ff700 1 -- 172.21.15.22:0/2010815982 shutdown_connections 2023-09-20T21:46:45.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.820+0000 7f06013ff700 1 --2- 172.21.15.22:0/2010815982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05fc102980 0x7f05fc109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.820+0000 7f06013ff700 1 --2- 172.21.15.22:0/2010815982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05fc100df0 0x7f05fc101210 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.820+0000 7f06013ff700 1 --2- 172.21.15.22:0/2010815982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05fc101fe0 0x7f05fc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.820+0000 7f06013ff700 1 -- 172.21.15.22:0/2010815982 >> 172.21.15.22:0/2010815982 conn(0x7f05fc0fc360 msgr2=0x7f05fc0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:45.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.820+0000 7f06013ff700 1 -- 172.21.15.22:0/2010815982 shutdown_connections 2023-09-20T21:46:45.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.820+0000 7f06013ff700 1 -- 172.21.15.22:0/2010815982 wait complete. 2023-09-20T21:46:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.821+0000 7f06013ff700 1 Processor -- start 2023-09-20T21:46:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f06013ff700 1 -- start start 2023-09-20T21:46:45.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f06013ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05fc100df0 0x7f05fc198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f06013ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05fc101fe0 0x7f05fc199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f06013ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05fc102980 0x7f05fc1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f06013ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a0870 con 0x7f05fc101fe0 2023-09-20T21:46:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f06013ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a09e0 con 0x7f05fc102980 2023-09-20T21:46:45.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f06013ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a0ba0 con 0x7f05fc100df0 2023-09-20T21:46:45.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f05faffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05fc100df0 0x7f05fc198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:45.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f05faffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05fc100df0 0x7f05fc198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:39200/0 (socket says 172.21.15.22:39200) 2023-09-20T21:46:45.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f05faffd700 1 -- 172.21.15.22:0/1379486782 learned_addr learned my addr 172.21.15.22:0/1379486782 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:45.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f05fa7fc700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05fc101fe0 0x7f05fc199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:45.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.822+0000 7f05fb7fe700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05fc102980 0x7f05fc1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:45.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f05faffd700 1 -- 172.21.15.22:0/1379486782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05fc102980 msgr2=0x7f05fc1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:45.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f05faffd700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05fc102980 0x7f05fc1a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f05faffd700 1 -- 172.21.15.22:0/1379486782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05fc101fe0 msgr2=0x7f05fc199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:45.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f05faffd700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05fc101fe0 0x7f05fc199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:45.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f05faffd700 1 -- 172.21.15.22:0/1379486782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0095f0 con 0x7f05fc100df0 2023-09-20T21:46:45.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f05faffd700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05fc100df0 0x7f05fc198d30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009fd0 tx=0x7f05ec00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:45.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f05e3fff700 1 -- 172.21.15.22:0/1379486782 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec01d070 con 0x7f05fc100df0 2023-09-20T21:46:45.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f06013ff700 1 -- 172.21.15.22:0/1379486782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc1a0e20 con 0x7f05fc100df0 2023-09-20T21:46:45.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f05e3fff700 1 -- 172.21.15.22:0/1379486782 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05ec005050 con 0x7f05fc100df0 2023-09-20T21:46:45.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f05e3fff700 1 -- 172.21.15.22:0/1379486782 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec004a70 con 0x7f05fc100df0 2023-09-20T21:46:45.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.823+0000 7f06013ff700 1 -- 172.21.15.22:0/1379486782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc1a11e0 con 0x7f05fc100df0 2023-09-20T21:46:45.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.824+0000 7f06013ff700 1 -- 172.21.15.22:0/1379486782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc10c650 con 0x7f05fc100df0 2023-09-20T21:46:45.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.827+0000 7f05e3fff700 1 -- 172.21.15.22:0/1379486782 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f05ec005450 con 0x7f05fc100df0 2023-09-20T21:46:45.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.828+0000 7f05e3fff700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05e40a1300 0x7f05e40a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:45.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.828+0000 7f05e3fff700 1 -- 172.21.15.22:0/1379486782 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f05ec0cdfe0 con 0x7f05fc100df0 2023-09-20T21:46:45.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.828+0000 7f05fa7fc700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05e40a1300 0x7f05e40a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:45.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.829+0000 7f05fa7fc700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05e40a1300 0x7f05e40a37c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009c60 tx=0x7f05f0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:45.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:45.830+0000 7f05e3fff700 1 -- 172.21.15.22:0/1379486782 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f05ec097510 con 0x7f05fc100df0 2023-09-20T21:46:45.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:45 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-20T21:46:45.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:45 smithi123 ceph-mon[134433]: osdmap e87: 12 total, 12 up, 12 in 2023-09-20T21:46:45.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:45 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1233316159' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-20T21:46:45.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:45 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-20T21:46:45.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:45 smithi123 ceph-mon[134433]: pgmap v276: 129 pgs: 24 creating+peering, 62 active+clean, 43 unknown; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2023-09-20T21:46:45.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:45 smithi123 ceph-mon[134433]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-20T21:46:45.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:45 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-20T21:46:45.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:45 smithi089 ceph-mon[134090]: osdmap e87: 12 total, 12 up, 12 in 2023-09-20T21:46:45.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:45 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1233316159' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-20T21:46:45.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:45 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-20T21:46:45.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:45 smithi089 ceph-mon[134090]: pgmap v276: 129 pgs: 24 creating+peering, 62 active+clean, 43 unknown; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2023-09-20T21:46:45.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:45 smithi089 ceph-mon[134090]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-20T21:46:45.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:45 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-09-20T21:46:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:45 smithi022 ceph-mon[97651]: osdmap e87: 12 total, 12 up, 12 in 2023-09-20T21:46:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:45 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1233316159' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-20T21:46:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:45 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-09-20T21:46:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:45 smithi022 ceph-mon[97651]: pgmap v276: 129 pgs: 24 creating+peering, 62 active+clean, 43 unknown; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2023-09-20T21:46:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:45 smithi022 ceph-mon[97651]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-09-20T21:46:46.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.089+0000 7f06013ff700 1 -- 172.21.15.22:0/1379486782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f05fc1a1490 con 0x7f05fc100df0 2023-09-20T21:46:46.360 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 21:46:45 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b[171416]: 2023-09-20T21:46:45.990+0000 7f3b247a9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-20T21:46:46.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.566+0000 7f05e3fff700 1 -- 172.21.15.22:0/1379486782 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f05ec09ab30 con 0x7f05fc100df0 2023-09-20T21:46:46.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 -- 172.21.15.22:0/1379486782 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05e40a1300 msgr2=0x7f05e40a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:46.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05e40a1300 0x7f05e40a37c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009c60 tx=0x7f05f0009260 comp rx=0 tx=0).stop 2023-09-20T21:46:46.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 -- 172.21.15.22:0/1379486782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05fc100df0 msgr2=0x7f05fc198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:46.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05fc100df0 0x7f05fc198d30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009fd0 tx=0x7f05ec00f690 comp rx=0 tx=0).stop 2023-09-20T21:46:46.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 -- 172.21.15.22:0/1379486782 shutdown_connections 2023-09-20T21:46:46.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05e40a1300 0x7f05e40a37c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:46.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05fc102980 0x7f05fc1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:46.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05fc100df0 0x7f05fc198d30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:46.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 --2- 172.21.15.22:0/1379486782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05fc101fe0 0x7f05fc199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:46.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 -- 172.21.15.22:0/1379486782 >> 172.21.15.22:0/1379486782 conn(0x7f05fc0fc360 msgr2=0x7f05fc0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:46.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 -- 172.21.15.22:0/1379486782 shutdown_connections 2023-09-20T21:46:46.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.569+0000 7f06013ff700 1 -- 172.21.15.22:0/1379486782 wait complete. 2023-09-20T21:46:46.572 INFO:teuthology.orchestra.run.smithi022.stderr:added data pool 4 to fsmap 2023-09-20T21:46:46.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:46.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.798+0000 7fcba6fff700 1 -- 172.21.15.22:0/2398296437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0101fd0 msgr2=0x7fcba0102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:46.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.798+0000 7fcba6fff700 1 --2- 172.21.15.22:0/2398296437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0101fd0 0x7fcba0102430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0098f0 tx=0x7fcb8c009c00 comp rx=0 tx=0).stop 2023-09-20T21:46:46.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.798+0000 7fcba6fff700 1 -- 172.21.15.22:0/2398296437 shutdown_connections 2023-09-20T21:46:46.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.798+0000 7fcba6fff700 1 --2- 172.21.15.22:0/2398296437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba0102970 0x7fcba0109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.798+0000 7fcba6fff700 1 --2- 172.21.15.22:0/2398296437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcba0100de0 0x7fcba0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.798+0000 7fcba6fff700 1 --2- 172.21.15.22:0/2398296437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0101fd0 0x7fcba0102430 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.798+0000 7fcba6fff700 1 -- 172.21.15.22:0/2398296437 >> 172.21.15.22:0/2398296437 conn(0x7fcba00fc310 msgr2=0x7fcba00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.798+0000 7fcba6fff700 1 -- 172.21.15.22:0/2398296437 shutdown_connections 2023-09-20T21:46:46.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.798+0000 7fcba6fff700 1 -- 172.21.15.22:0/2398296437 wait complete. 2023-09-20T21:46:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.800+0000 7fcba6fff700 1 Processor -- start 2023-09-20T21:46:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.800+0000 7fcba6fff700 1 -- start start 2023-09-20T21:46:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.800+0000 7fcba6fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba0100de0 0x7fcba0196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:46.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.800+0000 7fcba6fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcba0101fd0 0x7fcba0197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.800+0000 7fcba6fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0102970 0x7fcba019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.800+0000 7fcba6fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba019e670 con 0x7fcba0100de0 2023-09-20T21:46:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.800+0000 7fcba6fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba019e7b0 con 0x7fcba0101fd0 2023-09-20T21:46:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.800+0000 7fcba6fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba019e940 con 0x7fcba0102970 2023-09-20T21:46:46.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcba4d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba0100de0 0x7fcba0196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:46.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcba4d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba0100de0 0x7fcba0196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59880/0 (socket says 172.21.15.22:59880) 2023-09-20T21:46:46.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcb9ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcba0101fd0 0x7fcba0197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:46.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcba4d9b700 1 -- 172.21.15.22:0/3097813858 learned_addr learned my addr 172.21.15.22:0/3097813858 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:46.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcba559c700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0102970 0x7fcba019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:46.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcba4d9b700 1 -- 172.21.15.22:0/3097813858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0102970 msgr2=0x7fcba019e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:46.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcba4d9b700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0102970 0x7fcba019e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:46.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcba4d9b700 1 -- 172.21.15.22:0/3097813858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcba0101fd0 msgr2=0x7fcba0197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:46.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcba4d9b700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcba0101fd0 0x7fcba0197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:46.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcba4d9b700 1 -- 172.21.15.22:0/3097813858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb940095f0 con 0x7fcba0100de0 2023-09-20T21:46:46.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.801+0000 7fcba4d9b700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba0100de0 0x7fcba0196af0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400ebf0 tx=0x7fcb9400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.802+0000 7fcb9dffb700 1 -- 172.21.15.22:0/3097813858 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94009a10 con 0x7fcba0100de0 2023-09-20T21:46:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.802+0000 7fcba6fff700 1 -- 172.21.15.22:0/3097813858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c009520 con 0x7fcba0100de0 2023-09-20T21:46:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.802+0000 7fcb9dffb700 1 -- 172.21.15.22:0/3097813858 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb9400cc90 con 0x7fcba0100de0 2023-09-20T21:46:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.802+0000 7fcb9dffb700 1 -- 172.21.15.22:0/3097813858 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94018750 con 0x7fcba0100de0 2023-09-20T21:46:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.802+0000 7fcba6fff700 1 -- 172.21.15.22:0/3097813858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba019eff0 con 0x7fcba0100de0 2023-09-20T21:46:46.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.803+0000 7fcba6fff700 1 -- 172.21.15.22:0/3097813858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba0108370 con 0x7fcba0100de0 2023-09-20T21:46:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.805+0000 7fcb9dffb700 1 -- 172.21.15.22:0/3097813858 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fcb940188b0 con 0x7fcba0100de0 2023-09-20T21:46:46.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.807+0000 7fcb9dffb700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcb900a13d0 0x7fcb900a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.807+0000 7fcb9dffb700 1 -- 172.21.15.22:0/3097813858 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcb94014070 con 0x7fcba0100de0 2023-09-20T21:46:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.807+0000 7fcb9ffff700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcb900a13d0 0x7fcb900a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:46.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.808+0000 7fcb9ffff700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcb900a13d0 0x7fcb900a3890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005e90 tx=0x7fcb8c019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:46.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:46.810+0000 7fcb9dffb700 1 -- 172.21.15.22:0/3097813858 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcb94096f80 con 0x7fcba0100de0 2023-09-20T21:46:46.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:46 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-20T21:46:46.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:46 smithi123 ceph-mon[134433]: osdmap e88: 12 total, 12 up, 12 in 2023-09-20T21:46:46.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:46 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1379486782' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-20T21:46:46.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:46 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-20T21:46:46.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:46 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-20T21:46:46.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:46 smithi089 ceph-mon[134090]: osdmap e88: 12 total, 12 up, 12 in 2023-09-20T21:46:46.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:46 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1379486782' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-20T21:46:46.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:46 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-20T21:46:46.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:46 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-09-20T21:46:46.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:46 smithi022 ceph-mon[97651]: osdmap e88: 12 total, 12 up, 12 in 2023-09-20T21:46:46.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:46 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1379486782' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-20T21:46:46.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:46 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-09-20T21:46:47.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.069+0000 7fcba6fff700 1 -- 172.21.15.22:0/3097813858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcba010c640 con 0x7fcba0100de0 2023-09-20T21:46:47.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.070+0000 7fcb9dffb700 1 -- 172.21.15.22:0/3097813858 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11249 (secure 0 0 0) 0x7fcb9409a5a0 con 0x7fcba0100de0 2023-09-20T21:46:47.071 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:47.071 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:46:46.091716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-20T21:46:47.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.072+0000 7fcba6fff700 1 -- 172.21.15.22:0/3097813858 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcb900a13d0 msgr2=0x7fcb900a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.072+0000 7fcba6fff700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcb900a13d0 0x7fcb900a3890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005e90 tx=0x7fcb8c019040 comp rx=0 tx=0).stop 2023-09-20T21:46:47.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.072+0000 7fcba6fff700 1 -- 172.21.15.22:0/3097813858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba0100de0 msgr2=0x7fcba0196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.072+0000 7fcba6fff700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba0100de0 0x7fcba0196af0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400ebf0 tx=0x7fcb9400ef00 comp rx=0 tx=0).stop 2023-09-20T21:46:47.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.072+0000 7fcba6fff700 1 -- 172.21.15.22:0/3097813858 shutdown_connections 2023-09-20T21:46:47.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.072+0000 7fcba6fff700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0102970 0x7fcba019e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.072+0000 7fcba6fff700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcba0100de0 0x7fcba0196af0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.072+0000 7fcba6fff700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcb900a13d0 0x7fcb900a3890 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.072+0000 7fcba6fff700 1 --2- 172.21.15.22:0/3097813858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcba0101fd0 0x7fcba0197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.072+0000 7fcba6fff700 1 -- 172.21.15.22:0/3097813858 >> 172.21.15.22:0/3097813858 conn(0x7fcba00fc310 msgr2=0x7fcba01054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:47.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.073+0000 7fcba6fff700 1 -- 172.21.15.22:0/3097813858 shutdown_connections 2023-09-20T21:46:47.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.073+0000 7fcba6fff700 1 -- 172.21.15.22:0/3097813858 wait complete. 2023-09-20T21:46:47.076 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2023-09-20T21:46:47.092 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:46:47.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.295+0000 7f198fbff700 1 -- 172.21.15.22:0/1944091917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1988100df0 msgr2=0x7f1988101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.295+0000 7f198fbff700 1 --2- 172.21.15.22:0/1944091917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1988100df0 0x7f1988101210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f19840099c0 tx=0x7f1984009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.295+0000 7f198fbff700 1 -- 172.21.15.22:0/1944091917 shutdown_connections 2023-09-20T21:46:47.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.295+0000 7f198fbff700 1 --2- 172.21.15.22:0/1944091917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988102980 0x7f1988109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.295+0000 7f198fbff700 1 --2- 172.21.15.22:0/1944091917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1988100df0 0x7f1988101210 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.295+0000 7f198fbff700 1 --2- 172.21.15.22:0/1944091917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988101fe0 0x7f1988102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.295+0000 7f198fbff700 1 -- 172.21.15.22:0/1944091917 >> 172.21.15.22:0/1944091917 conn(0x7f19880fc320 msgr2=0x7f19880fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:47.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.295+0000 7f198fbff700 1 -- 172.21.15.22:0/1944091917 shutdown_connections 2023-09-20T21:46:47.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.295+0000 7f198fbff700 1 -- 172.21.15.22:0/1944091917 wait complete. 2023-09-20T21:46:47.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.296+0000 7f198fbff700 1 Processor -- start 2023-09-20T21:46:47.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.296+0000 7f198fbff700 1 -- start start 2023-09-20T21:46:47.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198fbff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988100df0 0x7f1988074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:47.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198fbff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988101fe0 0x7f19880731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:47.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198fbff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1988102980 0x7f1988073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:47.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198fbff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1988073eb0 con 0x7f1988102980 2023-09-20T21:46:47.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198fbff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1988074020 con 0x7f1988101fe0 2023-09-20T21:46:47.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198fbff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19881a0b10 con 0x7f1988100df0 2023-09-20T21:46:47.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198d99b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988100df0 0x7f1988074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:47.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198d99b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988100df0 0x7f1988074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:52158/0 (socket says 172.21.15.22:52158) 2023-09-20T21:46:47.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198d99b700 1 -- 172.21.15.22:0/3314680060 learned_addr learned my addr 172.21.15.22:0/3314680060 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:47.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198e19c700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1988102980 0x7f1988073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:47.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198d19a700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988101fe0 0x7f19880731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:47.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198d99b700 1 -- 172.21.15.22:0/3314680060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988101fe0 msgr2=0x7f19880731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198d99b700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988101fe0 0x7f19880731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198d99b700 1 -- 172.21.15.22:0/3314680060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1988102980 msgr2=0x7f1988073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198d99b700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1988102980 0x7f1988073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.297+0000 7f198d99b700 1 -- 172.21.15.22:0/3314680060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978009520 con 0x7f1988100df0 2023-09-20T21:46:47.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.298+0000 7f198d99b700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988100df0 0x7f1988074b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1984009fd0 tx=0x7f198400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:47.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.298+0000 7f197effd700 1 -- 172.21.15.22:0/3314680060 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198401d070 con 0x7f1988100df0 2023-09-20T21:46:47.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.298+0000 7f197effd700 1 -- 172.21.15.22:0/3314680060 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1984005050 con 0x7f1988100df0 2023-09-20T21:46:47.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.298+0000 7f198fbff700 1 -- 172.21.15.22:0/3314680060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19840095f0 con 0x7f1988100df0 2023-09-20T21:46:47.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.298+0000 7f197effd700 1 -- 172.21.15.22:0/3314680060 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984004a70 con 0x7f1988100df0 2023-09-20T21:46:47.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.298+0000 7f198fbff700 1 -- 172.21.15.22:0/3314680060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19881a1020 con 0x7f1988100df0 2023-09-20T21:46:47.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.299+0000 7f198fbff700 1 -- 172.21.15.22:0/3314680060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198810c650 con 0x7f1988100df0 2023-09-20T21:46:47.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.301+0000 7f197effd700 1 -- 172.21.15.22:0/3314680060 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1984005450 con 0x7f1988100df0 2023-09-20T21:46:47.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.302+0000 7f197effd700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f19740a1350 0x7f19740a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:47.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.303+0000 7f197effd700 1 -- 172.21.15.22:0/3314680060 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f19840cdff0 con 0x7f1988100df0 2023-09-20T21:46:47.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.303+0000 7f198d19a700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f19740a1350 0x7f19740a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:47.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.303+0000 7f198d19a700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f19740a1350 0x7f19740a3810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1978009df0 tx=0x7f1978009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:47.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.304+0000 7f197effd700 1 -- 172.21.15.22:0/3314680060 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1984097540 con 0x7f1988100df0 2023-09-20T21:46:47.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.518+0000 7f198fbff700 1 -- 172.21.15.22:0/3314680060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1988107ca0 con 0x7f1988100df0 2023-09-20T21:46:47.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.520+0000 7f197effd700 1 -- 172.21.15.22:0/3314680060 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23913 (secure 0 0 0) 0x7f198409ab60 con 0x7f1988100df0 2023-09-20T21:46:47.521 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:47.521 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:46:46.091544+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:46:47.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.522+0000 7f198fbff700 1 -- 172.21.15.22:0/3314680060 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f19740a1350 msgr2=0x7f19740a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.522+0000 7f198fbff700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f19740a1350 0x7f19740a3810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1978009df0 tx=0x7f1978009260 comp rx=0 tx=0).stop 2023-09-20T21:46:47.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.522+0000 7f198fbff700 1 -- 172.21.15.22:0/3314680060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988100df0 msgr2=0x7f1988074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.522+0000 7f198fbff700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988100df0 0x7f1988074b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1984009fd0 tx=0x7f198400f690 comp rx=0 tx=0).stop 2023-09-20T21:46:47.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.523+0000 7f198fbff700 1 -- 172.21.15.22:0/3314680060 shutdown_connections 2023-09-20T21:46:47.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.523+0000 7f198fbff700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1988102980 0x7f1988073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.523+0000 7f198fbff700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988100df0 0x7f1988074b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.523+0000 7f198fbff700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f19740a1350 0x7f19740a3810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.523+0000 7f198fbff700 1 --2- 172.21.15.22:0/3314680060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1988101fe0 0x7f19880731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.523+0000 7f198fbff700 1 -- 172.21.15.22:0/3314680060 >> 172.21.15.22:0/3314680060 conn(0x7f19880fc320 msgr2=0x7f19880fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:47.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.523+0000 7f198fbff700 1 -- 172.21.15.22:0/3314680060 shutdown_connections 2023-09-20T21:46:47.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.523+0000 7f198fbff700 1 -- 172.21.15.22:0/3314680060 wait complete. 2023-09-20T21:46:47.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:46:47.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.748+0000 7f84b51ff700 1 -- 172.21.15.22:0/778709740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84b0101fd0 msgr2=0x7f84b0102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.748+0000 7f84b51ff700 1 --2- 172.21.15.22:0/778709740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84b0101fd0 0x7f84b0102430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f84a40099c0 tx=0x7f84a4009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.748+0000 7f84b51ff700 1 -- 172.21.15.22:0/778709740 shutdown_connections 2023-09-20T21:46:47.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.748+0000 7f84b51ff700 1 --2- 172.21.15.22:0/778709740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b0102970 0x7f84b0109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.748+0000 7f84b51ff700 1 --2- 172.21.15.22:0/778709740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84b0100de0 0x7f84b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.748+0000 7f84b51ff700 1 --2- 172.21.15.22:0/778709740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84b0101fd0 0x7f84b0102430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.748+0000 7f84b51ff700 1 -- 172.21.15.22:0/778709740 >> 172.21.15.22:0/778709740 conn(0x7f84b00fc310 msgr2=0x7f84b00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:47.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.748+0000 7f84b51ff700 1 -- 172.21.15.22:0/778709740 shutdown_connections 2023-09-20T21:46:47.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.748+0000 7f84b51ff700 1 -- 172.21.15.22:0/778709740 wait complete. 2023-09-20T21:46:47.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.749+0000 7f84b51ff700 1 Processor -- start 2023-09-20T21:46:47.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.749+0000 7f84b51ff700 1 -- start start 2023-09-20T21:46:47.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.749+0000 7f84b51ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b0100de0 0x7f84b0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:47.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.749+0000 7f84b51ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84b0101fd0 0x7f84b01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:47.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.749+0000 7f84b51ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84b0102970 0x7f84b01a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:47.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.749+0000 7f84b51ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b01a07b0 con 0x7f84b0102970 2023-09-20T21:46:47.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.749+0000 7f84b51ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b01a0920 con 0x7f84b0101fd0 2023-09-20T21:46:47.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.749+0000 7f84b51ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b01a0ae0 con 0x7f84b0100de0 2023-09-20T21:46:47.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.749+0000 7f84af59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84b0102970 0x7f84b01a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:47.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84aed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b0100de0 0x7f84b0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:47.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84af59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84b0102970 0x7f84b01a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59928/0 (socket says 172.21.15.22:59928) 2023-09-20T21:46:47.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84af59e700 1 -- 172.21.15.22:0/501448680 learned_addr learned my addr 172.21.15.22:0/501448680 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:47.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84ae59c700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84b0101fd0 0x7f84b01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:47.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84af59e700 1 -- 172.21.15.22:0/501448680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b0100de0 msgr2=0x7f84b0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84af59e700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b0100de0 0x7f84b0198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84af59e700 1 -- 172.21.15.22:0/501448680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84b0101fd0 msgr2=0x7f84b01991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84af59e700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84b0101fd0 0x7f84b01991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84af59e700 1 -- 172.21.15.22:0/501448680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a40095f0 con 0x7f84b0102970 2023-09-20T21:46:47.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84af59e700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84b0102970 0x7f84b01a01e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f84a8007ee0 tx=0x7f84a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:47.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f8497fff700 1 -- 172.21.15.22:0/501448680 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a800dd20 con 0x7f84b0102970 2023-09-20T21:46:47.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f8497fff700 1 -- 172.21.15.22:0/501448680 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a800f040 con 0x7f84b0102970 2023-09-20T21:46:47.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f8497fff700 1 -- 172.21.15.22:0/501448680 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a80048a0 con 0x7f84b0102970 2023-09-20T21:46:47.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.750+0000 7f84b51ff700 1 -- 172.21.15.22:0/501448680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b01a0dc0 con 0x7f84b0102970 2023-09-20T21:46:47.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.751+0000 7f84b51ff700 1 -- 172.21.15.22:0/501448680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b01a11c0 con 0x7f84b0102970 2023-09-20T21:46:47.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.752+0000 7f84b51ff700 1 -- 172.21.15.22:0/501448680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b010c5f0 con 0x7f84b0102970 2023-09-20T21:46:47.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.754+0000 7f8497fff700 1 -- 172.21.15.22:0/501448680 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f84a8003a00 con 0x7f84b0102970 2023-09-20T21:46:47.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.755+0000 7f8497fff700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f84980a1350 0x7f84980a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:47.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.755+0000 7f8497fff700 1 -- 172.21.15.22:0/501448680 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f84a80ccee0 con 0x7f84b0102970 2023-09-20T21:46:47.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.755+0000 7f84aed9d700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f84980a1350 0x7f84980a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:47.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.756+0000 7f84aed9d700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f84980a1350 0x7f84980a3810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f84b019a140 tx=0x7f84a0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:47.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.757+0000 7f8497fff700 1 -- 172.21.15.22:0/501448680 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f84a8096360 con 0x7f84b0102970 2023-09-20T21:46:47.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:47 smithi123 ceph-mon[134433]: osdmap e89: 12 total, 12 up, 12 in 2023-09-20T21:46:47.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:47 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-20T21:46:47.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:47 smithi123 ceph-mon[134433]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:46:47.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:47 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3097813858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:47.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:47 smithi123 ceph-mon[134433]: pgmap v279: 129 pgs: 7 creating+peering, 105 active+clean, 17 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-09-20T21:46:47.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:47 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3314680060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:47.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:47 smithi089 ceph-mon[134090]: osdmap e89: 12 total, 12 up, 12 in 2023-09-20T21:46:47.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:47 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-20T21:46:47.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:47 smithi089 ceph-mon[134090]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:46:47.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:47 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3097813858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:47.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:47 smithi089 ceph-mon[134090]: pgmap v279: 129 pgs: 7 creating+peering, 105 active+clean, 17 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-09-20T21:46:47.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:47 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3314680060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:47.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.970+0000 7f84b51ff700 1 -- 172.21.15.22:0/501448680 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84b01a13b0 con 0x7f84b0102970 2023-09-20T21:46:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.971+0000 7f8497fff700 1 -- 172.21.15.22:0/501448680 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23913 (secure 0 0 0) 0x7f84a8023090 con 0x7f84b0102970 2023-09-20T21:46:47.972 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:47.972 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:46:46.091544+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:46:47.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 -- 172.21.15.22:0/501448680 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f84980a1350 msgr2=0x7f84980a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f84980a1350 0x7f84980a3810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f84b019a140 tx=0x7f84a0009260 comp rx=0 tx=0).stop 2023-09-20T21:46:47.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 -- 172.21.15.22:0/501448680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84b0102970 msgr2=0x7f84b01a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:47.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84b0102970 0x7f84b01a01e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f84a8007ee0 tx=0x7f84a800d3b0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 -- 172.21.15.22:0/501448680 shutdown_connections 2023-09-20T21:46:47.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84b0102970 0x7f84b01a01e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b0100de0 0x7f84b0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f84980a1350 0x7f84980a3810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 --2- 172.21.15.22:0/501448680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f84b0101fd0 0x7f84b01991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:47.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 -- 172.21.15.22:0/501448680 >> 172.21.15.22:0/501448680 conn(0x7f84b00fc310 msgr2=0x7f84b0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:47.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 -- 172.21.15.22:0/501448680 shutdown_connections 2023-09-20T21:46:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:47.974+0000 7f84b51ff700 1 -- 172.21.15.22:0/501448680 wait complete. 2023-09-20T21:46:47.990 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:47 smithi022 ceph-mon[97651]: osdmap e89: 12 total, 12 up, 12 in 2023-09-20T21:46:47.990 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:47 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-09-20T21:46:47.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:47 smithi022 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:46:47.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:47 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3097813858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:47.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:47 smithi022 ceph-mon[97651]: pgmap v279: 129 pgs: 7 creating+peering, 105 active+clean, 17 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-09-20T21:46:47.991 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:47 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3314680060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:48.006 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:46:48.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.209+0000 7f2780bff700 1 -- 172.21.15.22:0/3522035816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277c100de0 msgr2=0x7f277c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:48.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.209+0000 7f2780bff700 1 --2- 172.21.15.22:0/3522035816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277c100de0 0x7f277c101200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f276c0099c0 tx=0x7f276c009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.209+0000 7f2780bff700 1 -- 172.21.15.22:0/3522035816 shutdown_connections 2023-09-20T21:46:48.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.209+0000 7f2780bff700 1 --2- 172.21.15.22:0/3522035816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c102970 0x7f277c109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.209+0000 7f2780bff700 1 --2- 172.21.15.22:0/3522035816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277c100de0 0x7f277c101200 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.209+0000 7f2780bff700 1 --2- 172.21.15.22:0/3522035816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277c101fd0 0x7f277c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.209+0000 7f2780bff700 1 -- 172.21.15.22:0/3522035816 >> 172.21.15.22:0/3522035816 conn(0x7f277c0fc310 msgr2=0x7f277c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:48.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.209+0000 7f2780bff700 1 -- 172.21.15.22:0/3522035816 shutdown_connections 2023-09-20T21:46:48.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.209+0000 7f2780bff700 1 -- 172.21.15.22:0/3522035816 wait complete. 2023-09-20T21:46:48.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.210+0000 7f2780bff700 1 Processor -- start 2023-09-20T21:46:48.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.210+0000 7f2780bff700 1 -- start start 2023-09-20T21:46:48.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f2780bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c100de0 0x7f277c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:48.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f2780bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277c101fd0 0x7f277c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:48.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f2780bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277c102970 0x7f277c1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:48.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f2780bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1a0840 con 0x7f277c102970 2023-09-20T21:46:48.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f2780bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1a09b0 con 0x7f277c101fd0 2023-09-20T21:46:48.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f2780bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c1a0b70 con 0x7f277c100de0 2023-09-20T21:46:48.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f277ad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277c102970 0x7f277c1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:48.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f2779d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277c101fd0 0x7f277c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:48.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f277ad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277c102970 0x7f277c1a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59954/0 (socket says 172.21.15.22:59954) 2023-09-20T21:46:48.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f277ad9d700 1 -- 172.21.15.22:0/1044287343 learned_addr learned my addr 172.21.15.22:0/1044287343 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f2779d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277c101fd0 0x7f277c199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:51332/0 (socket says 172.21.15.22:51332) 2023-09-20T21:46:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f277a59c700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c100de0 0x7f277c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f277ad9d700 1 -- 172.21.15.22:0/1044287343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c100de0 msgr2=0x7f277c198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f277ad9d700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c100de0 0x7f277c198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f277ad9d700 1 -- 172.21.15.22:0/1044287343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277c101fd0 msgr2=0x7f277c199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:48.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f277ad9d700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277c101fd0 0x7f277c199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.211+0000 7f277ad9d700 1 -- 172.21.15.22:0/1044287343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c0095f0 con 0x7f277c102970 2023-09-20T21:46:48.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.212+0000 7f277ad9d700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277c102970 0x7f277c1a0270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f277400ef30 tx=0x7f277400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:48.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.212+0000 7f276b7fe700 1 -- 172.21.15.22:0/1044287343 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2774018070 con 0x7f277c102970 2023-09-20T21:46:48.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.212+0000 7f276b7fe700 1 -- 172.21.15.22:0/1044287343 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f277400f040 con 0x7f277c102970 2023-09-20T21:46:48.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.212+0000 7f2780bff700 1 -- 172.21.15.22:0/1044287343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277c1a0df0 con 0x7f277c102970 2023-09-20T21:46:48.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.212+0000 7f276b7fe700 1 -- 172.21.15.22:0/1044287343 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277400cd00 con 0x7f277c102970 2023-09-20T21:46:48.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.212+0000 7f2780bff700 1 -- 172.21.15.22:0/1044287343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c1a1210 con 0x7f277c102970 2023-09-20T21:46:48.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.213+0000 7f2780bff700 1 -- 172.21.15.22:0/1044287343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277c193430 con 0x7f277c102970 2023-09-20T21:46:48.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.215+0000 7f276b7fe700 1 -- 172.21.15.22:0/1044287343 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2774003a00 con 0x7f277c102970 2023-09-20T21:46:48.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.217+0000 7f276b7fe700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27640a13d0 0x7f27640a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:48.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.217+0000 7f276b7fe700 1 -- 172.21.15.22:0/1044287343 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f27740cd310 con 0x7f277c102970 2023-09-20T21:46:48.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.217+0000 7f277a59c700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27640a13d0 0x7f27640a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:48.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.218+0000 7f277a59c700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27640a13d0 0x7f27640a3890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f276c009990 tx=0x7f276c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:48.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.218+0000 7f276b7fe700 1 -- 172.21.15.22:0/1044287343 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2774096820 con 0x7f277c102970 2023-09-20T21:46:48.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.476+0000 7f2780bff700 1 -- 172.21.15.22:0/1044287343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f277c02d1f0 con 0x7f277c102970 2023-09-20T21:46:48.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.477+0000 7f276b7fe700 1 -- 172.21.15.22:0/1044287343 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11249 (secure 0 0 0) 0x7f2774099e40 con 0x7f277c102970 2023-09-20T21:46:48.478 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:48.478 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:46:46.091716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-20T21:46:48.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.479+0000 7f2780bff700 1 -- 172.21.15.22:0/1044287343 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27640a13d0 msgr2=0x7f27640a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:48.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.479+0000 7f2780bff700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27640a13d0 0x7f27640a3890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f276c009990 tx=0x7f276c01a040 comp rx=0 tx=0).stop 2023-09-20T21:46:48.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.479+0000 7f2780bff700 1 -- 172.21.15.22:0/1044287343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277c102970 msgr2=0x7f277c1a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:48.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.479+0000 7f2780bff700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277c102970 0x7f277c1a0270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f277400ef30 tx=0x7f277400c3a0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.479+0000 7f2780bff700 1 -- 172.21.15.22:0/1044287343 shutdown_connections 2023-09-20T21:46:48.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.479+0000 7f2780bff700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277c102970 0x7f277c1a0270 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.479+0000 7f2780bff700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f277c100de0 0x7f277c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.479+0000 7f2780bff700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27640a13d0 0x7f27640a3890 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.479+0000 7f2780bff700 1 --2- 172.21.15.22:0/1044287343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277c101fd0 0x7f277c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.479+0000 7f2780bff700 1 -- 172.21.15.22:0/1044287343 >> 172.21.15.22:0/1044287343 conn(0x7f277c0fc310 msgr2=0x7f277c1078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:48.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.480+0000 7f2780bff700 1 -- 172.21.15.22:0/1044287343 shutdown_connections 2023-09-20T21:46:48.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.480+0000 7f2780bff700 1 -- 172.21.15.22:0/1044287343 wait complete. 2023-09-20T21:46:48.482 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2023-09-20T21:46:48.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:46:48.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.705+0000 7fc76e3ff700 1 -- 172.21.15.22:0/35910779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc768102970 msgr2=0x7fc768109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:48.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.705+0000 7fc76e3ff700 1 --2- 172.21.15.22:0/35910779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc768102970 0x7fc768109220 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc76000b0d0 tx=0x7fc76000b3e0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.705+0000 7fc76e3ff700 1 -- 172.21.15.22:0/35910779 shutdown_connections 2023-09-20T21:46:48.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.705+0000 7fc76e3ff700 1 --2- 172.21.15.22:0/35910779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc768102970 0x7fc768109220 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.705+0000 7fc76e3ff700 1 --2- 172.21.15.22:0/35910779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc768100de0 0x7fc768101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.705+0000 7fc76e3ff700 1 --2- 172.21.15.22:0/35910779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc768101fd0 0x7fc768102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.705+0000 7fc76e3ff700 1 -- 172.21.15.22:0/35910779 >> 172.21.15.22:0/35910779 conn(0x7fc7680fc310 msgr2=0x7fc7680fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.705+0000 7fc76e3ff700 1 -- 172.21.15.22:0/35910779 shutdown_connections 2023-09-20T21:46:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.705+0000 7fc76e3ff700 1 -- 172.21.15.22:0/35910779 wait complete. 2023-09-20T21:46:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.706+0000 7fc76e3ff700 1 Processor -- start 2023-09-20T21:46:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.706+0000 7fc76e3ff700 1 -- start start 2023-09-20T21:46:48.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76e3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc768100de0 0x7fc768198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76e3ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc768101fd0 0x7fc7681992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76e3ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc768102970 0x7fc7681a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76e3ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681a0930 con 0x7fc768102970 2023-09-20T21:46:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76e3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681a0a70 con 0x7fc768100de0 2023-09-20T21:46:48.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76e3ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681a0c00 con 0x7fc768101fd0 2023-09-20T21:46:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc767fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc768100de0 0x7fc768198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76c99c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc768102970 0x7fc7681a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc767fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc768100de0 0x7fc768198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:51352/0 (socket says 172.21.15.22:51352) 2023-09-20T21:46:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc767fff700 1 -- 172.21.15.22:0/4157789777 learned_addr learned my addr 172.21.15.22:0/4157789777 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc7677fe700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc768101fd0 0x7fc7681992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76c99c700 1 -- 172.21.15.22:0/4157789777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc768101fd0 msgr2=0x7fc7681992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76c99c700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc768101fd0 0x7fc7681992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76c99c700 1 -- 172.21.15.22:0/4157789777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc768100de0 msgr2=0x7fc768198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76c99c700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc768100de0 0x7fc768198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76c99c700 1 -- 172.21.15.22:0/4157789777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760009d00 con 0x7fc768102970 2023-09-20T21:46:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.707+0000 7fc76c99c700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc768102970 0x7fc7681a0360 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc760007a30 tx=0x7fc760008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.708+0000 7fc7657fa700 1 -- 172.21.15.22:0/4157789777 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76000e040 con 0x7fc768102970 2023-09-20T21:46:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.708+0000 7fc7657fa700 1 -- 172.21.15.22:0/4157789777 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc76000bd30 con 0x7fc768102970 2023-09-20T21:46:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.708+0000 7fc7657fa700 1 -- 172.21.15.22:0/4157789777 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76001d470 con 0x7fc768102970 2023-09-20T21:46:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.708+0000 7fc76e3ff700 1 -- 172.21.15.22:0/4157789777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7681a0e80 con 0x7fc768102970 2023-09-20T21:46:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.708+0000 7fc76e3ff700 1 -- 172.21.15.22:0/4157789777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7681a1250 con 0x7fc768102970 2023-09-20T21:46:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.709+0000 7fc76e3ff700 1 -- 172.21.15.22:0/4157789777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76810c650 con 0x7fc768102970 2023-09-20T21:46:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.711+0000 7fc7657fa700 1 -- 172.21.15.22:0/4157789777 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fc760005020 con 0x7fc768102970 2023-09-20T21:46:48.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.712+0000 7fc7657fa700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc7500a13d0 0x7fc7500a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.712+0000 7fc7657fa700 1 -- 172.21.15.22:0/4157789777 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc7600cdfb0 con 0x7fc768102970 2023-09-20T21:46:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.712+0000 7fc767fff700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc7500a13d0 0x7fc7500a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.713+0000 7fc767fff700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc7500a13d0 0x7fc7500a3890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc758005950 tx=0x7fc758009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:48.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.716+0000 7fc7657fa700 1 -- 172.21.15.22:0/4157789777 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc7600974c0 con 0x7fc768102970 2023-09-20T21:46:48.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:48 smithi022 ceph-mon[97651]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-20T21:46:48.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:48 smithi022 ceph-mon[97651]: Cluster is now healthy 2023-09-20T21:46:48.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:48 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/501448680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:48.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:48 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1044287343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:48.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:48 smithi123 ceph-mon[134433]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-20T21:46:48.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:48 smithi123 ceph-mon[134433]: Cluster is now healthy 2023-09-20T21:46:48.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:48 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/501448680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:48.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:48 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1044287343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:48.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:48 smithi089 ceph-mon[134090]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-09-20T21:46:48.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:48 smithi089 ceph-mon[134090]: Cluster is now healthy 2023-09-20T21:46:48.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:48 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/501448680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:48.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:48 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1044287343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:46:48.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.932+0000 7fc76e3ff700 1 -- 172.21.15.22:0/4157789777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc76810c890 con 0x7fc768102970 2023-09-20T21:46:48.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.934+0000 7fc7657fa700 1 -- 172.21.15.22:0/4157789777 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23913 (secure 0 0 0) 0x7fc76009aae0 con 0x7fc768102970 2023-09-20T21:46:48.935 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:48.935 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:46:46.091544+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:46:48.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.936+0000 7fc76e3ff700 1 -- 172.21.15.22:0/4157789777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc7500a13d0 msgr2=0x7fc7500a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:48.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.936+0000 7fc76e3ff700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc7500a13d0 0x7fc7500a3890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc758005950 tx=0x7fc758009260 comp rx=0 tx=0).stop 2023-09-20T21:46:48.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.936+0000 7fc76e3ff700 1 -- 172.21.15.22:0/4157789777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc768102970 msgr2=0x7fc7681a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:48.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.936+0000 7fc76e3ff700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc768102970 0x7fc7681a0360 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc760007a30 tx=0x7fc760008eb0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.936+0000 7fc76e3ff700 1 -- 172.21.15.22:0/4157789777 shutdown_connections 2023-09-20T21:46:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.936+0000 7fc76e3ff700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc768102970 0x7fc7681a0360 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.936+0000 7fc76e3ff700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc7500a13d0 0x7fc7500a3890 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.936+0000 7fc76e3ff700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc768100de0 0x7fc768198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.936+0000 7fc76e3ff700 1 --2- 172.21.15.22:0/4157789777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc768101fd0 0x7fc7681992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.937+0000 7fc76e3ff700 1 -- 172.21.15.22:0/4157789777 >> 172.21.15.22:0/4157789777 conn(0x7fc7680fc310 msgr2=0x7fc768107830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.937+0000 7fc76e3ff700 1 -- 172.21.15.22:0/4157789777 shutdown_connections 2023-09-20T21:46:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:48.937+0000 7fc76e3ff700 1 -- 172.21.15.22:0/4157789777 wait complete. 2023-09-20T21:46:48.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-20T21:46:41.044280+0000', 'modified': '2023-09-20T21:46:46.091716+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6839/2165264535', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 2165264535}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 2165264535}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-09-20T21:46:48.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-20T21:46:48.957 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-20T21:46:49.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.160+0000 7f79519ff700 1 -- 172.21.15.22:0/2375875726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c102960 msgr2=0x7f794c109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:49.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.160+0000 7f79519ff700 1 --2- 172.21.15.22:0/2375875726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c102960 0x7f794c109210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f794400b0d0 tx=0x7f794400b3e0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.160+0000 7f79519ff700 1 -- 172.21.15.22:0/2375875726 shutdown_connections 2023-09-20T21:46:49.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.160+0000 7f79519ff700 1 --2- 172.21.15.22:0/2375875726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c102960 0x7f794c109210 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.160+0000 7f79519ff700 1 --2- 172.21.15.22:0/2375875726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c100dd0 0x7f794c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.160+0000 7f79519ff700 1 --2- 172.21.15.22:0/2375875726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c101fc0 0x7f794c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.160+0000 7f79519ff700 1 -- 172.21.15.22:0/2375875726 >> 172.21.15.22:0/2375875726 conn(0x7f794c0fc320 msgr2=0x7f794c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:49.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.161+0000 7f79519ff700 1 -- 172.21.15.22:0/2375875726 shutdown_connections 2023-09-20T21:46:49.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.161+0000 7f79519ff700 1 -- 172.21.15.22:0/2375875726 wait complete. 2023-09-20T21:46:49.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.161+0000 7f79519ff700 1 Processor -- start 2023-09-20T21:46:49.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f79519ff700 1 -- start start 2023-09-20T21:46:49.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f79519ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c100dd0 0x7f794c074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f79519ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c101fc0 0x7f794c0731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f79519ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c102960 0x7f794c073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f79519ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c073eb0 con 0x7f794c100dd0 2023-09-20T21:46:49.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f79519ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c074020 con 0x7f794c102960 2023-09-20T21:46:49.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f79519ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c1a0c40 con 0x7f794c101fc0 2023-09-20T21:46:49.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f794affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c100dd0 0x7f794c074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f794affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c100dd0 0x7f794c074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60004/0 (socket says 172.21.15.22:60004) 2023-09-20T21:46:49.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f794affd700 1 -- 172.21.15.22:0/2898895332 learned_addr learned my addr 172.21.15.22:0/2898895332 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:49.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c102960 0x7f794c073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.162+0000 7f794a7fc700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c101fc0 0x7f794c0731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f794affd700 1 -- 172.21.15.22:0/2898895332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c101fc0 msgr2=0x7f794c0731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:49.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f794affd700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c101fc0 0x7f794c0731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f794affd700 1 -- 172.21.15.22:0/2898895332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c102960 msgr2=0x7f794c073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:49.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f794affd700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c102960 0x7f794c073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f794affd700 1 -- 172.21.15.22:0/2898895332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7944009d00 con 0x7f794c100dd0 2023-09-20T21:46:49.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c102960 0x7f794c073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:46:49.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f794affd700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c100dd0 0x7f794c074b90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e970 tx=0x7f793c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:49.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f79509fd700 1 -- 172.21.15.22:0/2898895332 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c009790 con 0x7f794c100dd0 2023-09-20T21:46:49.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f79509fd700 1 -- 172.21.15.22:0/2898895332 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f793c0044e0 con 0x7f794c100dd0 2023-09-20T21:46:49.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f79519ff700 1 -- 172.21.15.22:0/2898895332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794c1a0f20 con 0x7f794c100dd0 2023-09-20T21:46:49.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f79509fd700 1 -- 172.21.15.22:0/2898895332 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c003e30 con 0x7f794c100dd0 2023-09-20T21:46:49.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.163+0000 7f79519ff700 1 -- 172.21.15.22:0/2898895332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c1a13a0 con 0x7f794c100dd0 2023-09-20T21:46:49.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.166+0000 7f79509fd700 1 -- 172.21.15.22:0/2898895332 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f793c003f90 con 0x7f794c100dd0 2023-09-20T21:46:49.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.168+0000 7f7965312b80 1 Processor -- start 2023-09-20T21:46:49.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.168+0000 7f7965312b80 1 -- start start 2023-09-20T21:46:49.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f7965312b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a357ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f79509fd700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f79340a1380 0x7f79340a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f7965312b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 0x561c7a358590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f7965312b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358b90 0x561c7a359010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f7965312b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a2cbc50 con 0x561c7a3576a0 2023-09-20T21:46:49.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f7965312b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a2afc20 con 0x561c7a358b90 2023-09-20T21:46:49.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f7965312b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a2ad540 con 0x561c7a358000 2023-09-20T21:46:49.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f79509fd700 1 -- 172.21.15.22:0/2898895332 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f793c014070 con 0x7f794c100dd0 2023-09-20T21:46:49.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f794affd700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f79340a1380 0x7f79340a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f794b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a357ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f794b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a357ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60020/0 (socket says 172.21.15.22:60020) 2023-09-20T21:46:49.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f794b7fe700 1 -- 172.21.15.22:0/2675303697 learned_addr learned my addr 172.21.15.22:0/2675303697 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:49.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.169+0000 7f794a7fc700 1 --2- 172.21.15.22:0/2675303697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 0x561c7a358590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2675303697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358b90 0x561c7a359010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f794b7fe700 1 -- 172.21.15.22:0/2675303697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 msgr2=0x561c7a358590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:46:49.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2675303697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 0x561c7a358590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f794b7fe700 1 -- 172.21.15.22:0/2675303697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358b90 msgr2=0x561c7a359010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:46:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2675303697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358b90 0x561c7a359010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f794b7fe700 1 -- 172.21.15.22:0/2675303697 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7a0d3d90 con 0x561c7a3576a0 2023-09-20T21:46:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f794affd700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f79340a1380 0x7f79340a3840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f793c09a190 tx=0x7f793c00ef70 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:49.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2675303697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a357ac0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f794400b0d0 tx=0x7f7944009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268172f15f3d3775 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:49.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f7931ffb700 1 -- 172.21.15.22:0/2675303697 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794400e070 con 0x561c7a3576a0 2023-09-20T21:46:49.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f7931ffb700 1 -- 172.21.15.22:0/2675303697 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f794400baa0 con 0x561c7a3576a0 2023-09-20T21:46:49.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.170+0000 7f7931ffb700 1 -- 172.21.15.22:0/2675303697 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794401d450 con 0x561c7a3576a0 2023-09-20T21:46:49.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.171+0000 7f7965312b80 1 -- 172.21.15.22:0/2675303697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 msgr2=0x561c7a357ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:49.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.171+0000 7f7965312b80 1 --2- 172.21.15.22:0/2675303697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a357ac0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f794400b0d0 tx=0x7f7944009b80 comp rx=0 tx=0).stop 2023-09-20T21:46:49.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.171+0000 7f7965312b80 1 -- 172.21.15.22:0/2675303697 shutdown_connections 2023-09-20T21:46:49.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.171+0000 7f7965312b80 1 --2- 172.21.15.22:0/2675303697 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358b90 0x561c7a359010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.171+0000 7f7965312b80 1 --2- 172.21.15.22:0/2675303697 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a357ac0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.171+0000 7f7965312b80 1 --2- 172.21.15.22:0/2675303697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 0x561c7a358590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.171+0000 7f7965312b80 1 -- 172.21.15.22:0/2675303697 >> 172.21.15.22:0/2675303697 conn(0x561c7a283fb0 msgr2=0x561c7a352180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:49.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.171+0000 7f7965312b80 1 -- 172.21.15.22:0/2675303697 shutdown_connections 2023-09-20T21:46:49.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.171+0000 7f7965312b80 1 -- 172.21.15.22:0/2675303697 wait complete. 2023-09-20T21:46:49.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.172+0000 7f7965312b80 1 -- start start 2023-09-20T21:46:49.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.172+0000 7f7965312b80 1 Processor -- start 2023-09-20T21:46:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.172+0000 7f7965312b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a3576a0 0x561c7a3611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.172+0000 7f7965312b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 0x561c7a3616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.172+0000 7f7965312b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a358b90 0x561c7a368730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.172+0000 7f7965312b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a2a8e30 con 0x561c7a358b90 2023-09-20T21:46:49.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.172+0000 7f7965312b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a2456f0 con 0x561c7a3576a0 2023-09-20T21:46:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.172+0000 7f7965312b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a245390 con 0x561c7a358000 2023-09-20T21:46:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f7965312b80 10 client.0 _ll_register_callbacks cb 0x561c7a18fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:46:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a358b90 0x561c7a368730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a3576a0 0x561c7a3611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 0x561c7a3616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 0x561c7a3616f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:52258/0 (socket says 172.21.15.22:52258) 2023-09-20T21:46:49.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794a7fc700 1 -- 172.21.15.22:0/3976202109 learned_addr learned my addr 172.21.15.22:0/3976202109 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:49.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a3576a0 0x561c7a3611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:51382/0 (socket says 172.21.15.22:51382) 2023-09-20T21:46:49.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794a7fc700 1 -- 172.21.15.22:0/3976202109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a3576a0 msgr2=0x561c7a3611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:46:49.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794a7fc700 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a3576a0 0x561c7a3611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794a7fc700 1 -- 172.21.15.22:0/3976202109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a358b90 msgr2=0x561c7a368730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:46:49.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794a7fc700 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a358b90 0x561c7a368730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794a7fc700 1 -- 172.21.15.22:0/3976202109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c109b60 con 0x561c7a358000 2023-09-20T21:46:49.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f794a7fc700 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 0x561c7a3616f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7940012700 tx=0x7f7940012a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d90cbcd5c25386fd server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:49.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.173+0000 7f792affd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:46:49.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.174+0000 7f792affd700 1 -- 172.21.15.22:0/3976202109 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940017070 con 0x561c7a358000 2023-09-20T21:46:49.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.174+0000 7f7965312b80 1 -- 172.21.15.22:0/3976202109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c7a219b60 con 0x561c7a358000 2023-09-20T21:46:49.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.174+0000 7f792affd700 1 -- 172.21.15.22:0/3976202109 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7940004260 con 0x561c7a358000 2023-09-20T21:46:49.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.174+0000 7f792affd700 1 -- 172.21.15.22:0/3976202109 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7940004420 con 0x561c7a358000 2023-09-20T21:46:49.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.174+0000 7f7965312b80 10 client.24661 fetch_fsmap learned FSMap version 17 2023-09-20T21:46:49.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.174+0000 7f7965312b80 1 -- 172.21.15.22:0/3976202109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x561c7a0bcc10 con 0x561c7a358000 2023-09-20T21:46:49.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.174+0000 7f792affd700 1 -- 172.21.15.22:0/3976202109 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c7a219b60 con 0x561c7a358000 2023-09-20T21:46:49.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.174+0000 7f792affd700 1 -- 172.21.15.22:0/3976202109 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f7940011820 con 0x561c7a358000 2023-09-20T21:46:49.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.175+0000 7f7965312b80 10 client.24661 fetch_fsmap finished waiting for FSMap version 17 2023-09-20T21:46:49.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.175+0000 7f7965312b80 10 client.24661 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-20T21:46:49.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.175+0000 7f7965312b80 20 client.24661 populate_metadata read hostname 'smithi022' 2023-09-20T21:46:49.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.175+0000 7f7965312b80 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561c7a36cdc0 0x561c7a36b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.175+0000 7f7965312b80 4 client.24661 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:46:49.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.175+0000 7f7965312b80 1 -- 172.21.15.22:0/3976202109 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c7a325fd0 con 0x561c7a36cdc0 2023-09-20T21:46:49.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.175+0000 7f794b7fe700 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561c7a36cdc0 0x561c7a36b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.175+0000 7f794b7fe700 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561c7a36cdc0 0x561c7a36b890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec913288aee488e server_cookie=18b4324b59d38751 in_seq=0 out_seq=0 2023-09-20T21:46:49.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.175+0000 7f792affd700 10 client.24661 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:46:49.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.176+0000 7f792affd700 1 -- 172.21.15.22:0/3976202109 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561c7a325fd0 con 0x561c7a36cdc0 2023-09-20T21:46:49.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.176+0000 7f792affd700 10 client.24661 handle_command_reply: tid=0 2023-09-20T21:46:49.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.176+0000 7f7965312b80 1 client.24661 shutdown 2023-09-20T21:46:49.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 1 -- 172.21.15.22:0/3976202109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 msgr2=0x561c7a3616f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:49.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 0x561c7a3616f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7940012700 tx=0x7f7940012a10 comp rx=0 tx=0).stop 2023-09-20T21:46:49.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 1 -- 172.21.15.22:0/3976202109 shutdown_connections 2023-09-20T21:46:49.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561c7a36cdc0 0x561c7a36b890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a358b90 0x561c7a368730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a3576a0 0x561c7a3611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 1 --2- 172.21.15.22:0/3976202109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a358000 0x561c7a3616f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 1 -- 172.21.15.22:0/3976202109 >> 172.21.15.22:0/3976202109 conn(0x561c7a283fb0 msgr2=0x561c7a351830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:49.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f792affd700 0 client.24661 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:46:49.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 1 -- 172.21.15.22:0/3976202109 shutdown_connections 2023-09-20T21:46:49.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 1 -- 172.21.15.22:0/3976202109 wait complete. 2023-09-20T21:46:49.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.177+0000 7f7965312b80 20 client.24661 trim_cache size 0 max 16384 2023-09-20T21:46:49.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.195+0000 7f7965312b80 1 Processor -- start 2023-09-20T21:46:49.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.195+0000 7f7965312b80 1 -- start start 2023-09-20T21:46:49.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f7965312b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a283fb0 0x561c7a358f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f7965312b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a3576a0 0x561c7a368650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f7965312b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a358000 0x561c7a35f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f7965312b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a1298d0 con 0x561c7a358000 2023-09-20T21:46:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f7965312b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a325fd0 con 0x561c7a283fb0 2023-09-20T21:46:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f7965312b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a135de0 con 0x561c7a3576a0 2023-09-20T21:46:49.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f794b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a283fb0 0x561c7a358f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f794b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a358000 0x561c7a35f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f794b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a283fb0 0x561c7a358f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:51392/0 (socket says 172.21.15.22:51392) 2023-09-20T21:46:49.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f794b7fe700 1 -- 172.21.15.22:0/370022134 learned_addr learned my addr 172.21.15.22:0/370022134 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:49.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.196+0000 7f794a7fc700 1 --2- 172.21.15.22:0/370022134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a3576a0 0x561c7a368650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.197+0000 7f794b7fe700 1 -- 172.21.15.22:0/370022134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a3576a0 msgr2=0x561c7a368650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:46:49.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.197+0000 7f794b7fe700 1 --2- 172.21.15.22:0/370022134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a3576a0 0x561c7a368650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.197+0000 7f794b7fe700 1 -- 172.21.15.22:0/370022134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a358000 msgr2=0x561c7a35f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:46:49.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.197+0000 7f794b7fe700 1 --2- 172.21.15.22:0/370022134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a358000 0x561c7a35f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.197+0000 7f794b7fe700 1 -- 172.21.15.22:0/370022134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c7a0fa200 con 0x561c7a283fb0 2023-09-20T21:46:49.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.197+0000 7f794b7fe700 1 --2- 172.21.15.22:0/370022134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a283fb0 0x561c7a358f70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7944009c00 tx=0x7f79440095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10a91bc6da192f44 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.197+0000 7f79327fc700 1 -- 172.21.15.22:0/370022134 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794400e040 con 0x561c7a283fb0 2023-09-20T21:46:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.197+0000 7f79327fc700 1 -- 172.21.15.22:0/370022134 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7944004b70 con 0x561c7a283fb0 2023-09-20T21:46:49.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.197+0000 7f79327fc700 1 -- 172.21.15.22:0/370022134 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794402f680 con 0x561c7a283fb0 2023-09-20T21:46:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 -- 172.21.15.22:0/370022134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a283fb0 msgr2=0x561c7a358f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 --2- 172.21.15.22:0/370022134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a283fb0 0x561c7a358f70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7944009c00 tx=0x7f79440095a0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 -- 172.21.15.22:0/370022134 shutdown_connections 2023-09-20T21:46:49.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 --2- 172.21.15.22:0/370022134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a358000 0x561c7a35f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 --2- 172.21.15.22:0/370022134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a283fb0 0x561c7a358f70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 --2- 172.21.15.22:0/370022134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a3576a0 0x561c7a368650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 -- 172.21.15.22:0/370022134 >> 172.21.15.22:0/370022134 conn(0x561c7a3524b0 msgr2=0x561c7a368f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:49.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 -- 172.21.15.22:0/370022134 shutdown_connections 2023-09-20T21:46:49.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 -- 172.21.15.22:0/370022134 wait complete. 2023-09-20T21:46:49.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 -- start start 2023-09-20T21:46:49.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.198+0000 7f7965312b80 1 Processor -- start 2023-09-20T21:46:49.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f7965312b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a283fb0 0x561c7a3678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f7965312b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a367e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f7965312b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358000 0x561c7a368380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f7965312b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a3c07b0 con 0x561c7a3576a0 2023-09-20T21:46:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f7965312b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a3c08f0 con 0x561c7a358000 2023-09-20T21:46:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f7965312b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c7a3c0a30 con 0x561c7a283fb0 2023-09-20T21:46:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f7965312b80 10 client.0 _ll_register_callbacks cb 0x561c7a18fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:46:49.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f794a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a367e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f794a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a367e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60058/0 (socket says 172.21.15.22:60058) 2023-09-20T21:46:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f794a7fc700 1 -- 172.21.15.22:0/2200652000 learned_addr learned my addr 172.21.15.22:0/2200652000 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a283fb0 0x561c7a3678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358000 0x561c7a368380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f794a7fc700 1 -- 172.21.15.22:0/2200652000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a283fb0 msgr2=0x561c7a3678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:46:49.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.199+0000 7f794a7fc700 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a283fb0 0x561c7a3678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f794a7fc700 1 -- 172.21.15.22:0/2200652000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358000 msgr2=0x561c7a368380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:46:49.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f794a7fc700 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358000 0x561c7a368380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f794a7fc700 1 -- 172.21.15.22:0/2200652000 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7944039040 con 0x561c7a3576a0 2023-09-20T21:46:49.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358000 0x561c7a368380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:46:49.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f794a7fc700 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a367e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7940011c30 tx=0x7f7940011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e538f83669a240c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:49.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f792affd700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:46:49.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f792affd700 1 -- 172.21.15.22:0/2200652000 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940017070 con 0x561c7a3576a0 2023-09-20T21:46:49.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f7965312b80 1 -- 172.21.15.22:0/2200652000 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f794400baa0 con 0x561c7a3576a0 2023-09-20T21:46:49.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f792affd700 1 -- 172.21.15.22:0/2200652000 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7940011500 con 0x561c7a3576a0 2023-09-20T21:46:49.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f792affd700 1 -- 172.21.15.22:0/2200652000 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79400116e0 con 0x561c7a3576a0 2023-09-20T21:46:49.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f7965312b80 10 client.14637 fetch_fsmap learned FSMap version 17 2023-09-20T21:46:49.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.200+0000 7f7965312b80 1 -- 172.21.15.22:0/2200652000 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x561c7a0d2260 con 0x561c7a3576a0 2023-09-20T21:46:49.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.201+0000 7f792affd700 1 -- 172.21.15.22:0/2200652000 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f794400baa0 con 0x561c7a3576a0 2023-09-20T21:46:49.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.201+0000 7f792affd700 1 -- 172.21.15.22:0/2200652000 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f794000f4c0 con 0x561c7a3576a0 2023-09-20T21:46:49.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.201+0000 7f7965312b80 10 client.14637 fetch_fsmap finished waiting for FSMap version 17 2023-09-20T21:46:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.201+0000 7f7965312b80 10 client.14637 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-20T21:46:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.201+0000 7f7965312b80 20 client.14637 populate_metadata read hostname 'smithi022' 2023-09-20T21:46:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.201+0000 7f7965312b80 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561c7a358b90 0x561c7a3c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.201+0000 7f7965312b80 4 client.14637 mds_command: new command op to 24578 tid=0[{"prefix": "status"}] 2023-09-20T21:46:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.201+0000 7f7965312b80 1 -- 172.21.15.22:0/2200652000 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561c7a2a1e10 con 0x561c7a358b90 2023-09-20T21:46:49.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.201+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561c7a358b90 0x561c7a3c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f792affd700 10 client.14637 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:46:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f794b7fe700 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561c7a358b90 0x561c7a3c2120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2187950a12a9a5a server_cookie=89884b1c7fd7a45b in_seq=0 out_seq=0 2023-09-20T21:46:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f792affd700 1 -- 172.21.15.22:0/2200652000 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x561c7a2a1e10 con 0x561c7a358b90 2023-09-20T21:46:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f792affd700 10 client.14637 handle_command_reply: tid=0 2023-09-20T21:46:49.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 client.14637 shutdown 2023-09-20T21:46:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 -- 172.21.15.22:0/2200652000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 msgr2=0x561c7a367e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a367e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7940011c30 tx=0x7f7940011fd0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 -- 172.21.15.22:0/2200652000 shutdown_connections 2023-09-20T21:46:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561c7a358b90 0x561c7a3c2120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561c7a358000 0x561c7a368380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c7a283fb0 0x561c7a3678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 --2- 172.21.15.22:0/2200652000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561c7a3576a0 0x561c7a367e20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f792affd700 0 client.14637 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:46:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 -- 172.21.15.22:0/2200652000 >> 172.21.15.22:0/2200652000 conn(0x561c7a3524b0 msgr2=0x561c7a368f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 -- 172.21.15.22:0/2200652000 shutdown_connections 2023-09-20T21:46:49.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.202+0000 7f7965312b80 1 -- 172.21.15.22:0/2200652000 wait complete. 2023-09-20T21:46:49.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f7965312b80 20 client.14637 trim_cache size 0 max 16384 2023-09-20T21:46:49.217 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:46:49.217 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:46:49.218 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2023-09-20T21:46:49.218 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24578, 2023-09-20T21:46:49.218 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:46:49.218 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:46:49.218 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:46:49.219 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 7.6888329019999997, 2023-09-20T21:46:49.219 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 17, 2023-09-20T21:46:49.219 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 89, 2023-09-20T21:46:49.219 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 88, 2023-09-20T21:46:49.219 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 13.953680185 2023-09-20T21:46:49.219 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:46:49.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 -- 172.21.15.22:0/2898895332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f79340a1380 msgr2=0x7f79340a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:49.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f79340a1380 0x7f79340a3840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f793c09a190 tx=0x7f793c00ef70 comp rx=0 tx=0).stop 2023-09-20T21:46:49.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 -- 172.21.15.22:0/2898895332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c100dd0 msgr2=0x7f794c074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:49.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c100dd0 0x7f794c074b90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e970 tx=0x7f793c00ed30 comp rx=0 tx=0).stop 2023-09-20T21:46:49.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 -- 172.21.15.22:0/2898895332 shutdown_connections 2023-09-20T21:46:49.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f79340a1380 0x7f79340a3840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f794c102960 0x7f794c073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c100dd0 0x7f794c074b90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 --2- 172.21.15.22:0/2898895332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c101fc0 0x7f794c0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:49.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 -- 172.21.15.22:0/2898895332 >> 172.21.15.22:0/2898895332 conn(0x7f794c0fc320 msgr2=0x7f794c105460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:49.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 -- 172.21.15.22:0/2898895332 shutdown_connections 2023-09-20T21:46:49.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:49.203+0000 7f79519ff700 1 -- 172.21.15.22:0/2898895332 wait complete. 2023-09-20T21:46:49.223 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-20T21:46:49.224 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-20T21:46:49.225 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph fs ls 2023-09-20T21:46:49.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:49 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4157789777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:49.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:49 smithi123 ceph-mon[134433]: pgmap v280: 129 pgs: 7 creating+peering, 117 active+clean, 5 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 4 op/s 2023-09-20T21:46:49.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:49 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4157789777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:49.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:49 smithi089 ceph-mon[134090]: pgmap v280: 129 pgs: 7 creating+peering, 117 active+clean, 5 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 4 op/s 2023-09-20T21:46:49.950 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:46:49.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:49 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4157789777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:49.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:49 smithi022 ceph-mon[97651]: pgmap v280: 129 pgs: 7 creating+peering, 117 active+clean, 5 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 4 op/s 2023-09-20T21:46:51.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.455+0000 7f260718b700 1 -- 172.21.15.22:0/3453571579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2600101060 msgr2=0x7f2600101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.455+0000 7f260718b700 1 --2- 172.21.15.22:0/3453571579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2600101060 0x7f2600101440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009990 tx=0x7f25f4009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.456+0000 7f260718b700 1 -- 172.21.15.22:0/3453571579 shutdown_connections 2023-09-20T21:46:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.456+0000 7f260718b700 1 --2- 172.21.15.22:0/3453571579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2600106140 0x7f260010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:51.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.456+0000 7f260718b700 1 --2- 172.21.15.22:0/3453571579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2600101060 0x7f2600101440 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.456+0000 7f260718b700 1 --2- 172.21.15.22:0/3453571579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2600101a10 0x7f2600105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.456+0000 7f260718b700 1 -- 172.21.15.22:0/3453571579 >> 172.21.15.22:0/3453571579 conn(0x7f26000fc8d0 msgr2=0x7f26000fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.456+0000 7f260718b700 1 -- 172.21.15.22:0/3453571579 shutdown_connections 2023-09-20T21:46:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.456+0000 7f260718b700 1 -- 172.21.15.22:0/3453571579 wait complete. 2023-09-20T21:46:51.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.457+0000 7f260718b700 1 Processor -- start 2023-09-20T21:46:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.457+0000 7f260718b700 1 -- start start 2023-09-20T21:46:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.457+0000 7f260718b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2600101060 0x7f26000730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.457+0000 7f260718b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2600101a10 0x7f2600073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.457+0000 7f260718b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2600106140 0x7f2600073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:51.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.457+0000 7f260718b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26001a5370 con 0x7f2600106140 2023-09-20T21:46:51.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.457+0000 7f260718b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26001a54e0 con 0x7f2600101a10 2023-09-20T21:46:51.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.457+0000 7f260718b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26001a56a0 con 0x7f2600101060 2023-09-20T21:46:51.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f2605728700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2600106140 0x7f2600073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:51.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f2604f27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2600101060 0x7f26000730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:51.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f25fffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2600101a10 0x7f2600073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:51.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f25fffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2600101a10 0x7f2600073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:51416/0 (socket says 172.21.15.22:51416) 2023-09-20T21:46:51.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f2604f27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2600101060 0x7f26000730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:52290/0 (socket says 172.21.15.22:52290) 2023-09-20T21:46:51.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f25fffff700 1 -- 172.21.15.22:0/1612140505 learned_addr learned my addr 172.21.15.22:0/1612140505 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:51.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f25fffff700 1 -- 172.21.15.22:0/1612140505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2600101060 msgr2=0x7f26000730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:51.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f25fffff700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2600101060 0x7f26000730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:51.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f25fffff700 1 -- 172.21.15.22:0/1612140505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2600106140 msgr2=0x7f2600073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:51.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f25fffff700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2600106140 0x7f2600073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:51.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f25fffff700 1 -- 172.21.15.22:0/1612140505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f40095f0 con 0x7f2600101a10 2023-09-20T21:46:51.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.458+0000 7f25fffff700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2600101a10 0x7f2600073620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00d980 tx=0x7f25ec00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:51.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.459+0000 7f25fdffb700 1 -- 172.21.15.22:0/1612140505 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec00f890 con 0x7f2600101a10 2023-09-20T21:46:51.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.459+0000 7f260718b700 1 -- 172.21.15.22:0/1612140505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26001a5980 con 0x7f2600101a10 2023-09-20T21:46:51.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.459+0000 7f25fdffb700 1 -- 172.21.15.22:0/1612140505 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25ec010460 con 0x7f2600101a10 2023-09-20T21:46:51.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.459+0000 7f25fdffb700 1 -- 172.21.15.22:0/1612140505 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec00e610 con 0x7f2600101a10 2023-09-20T21:46:51.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.459+0000 7f260718b700 1 -- 172.21.15.22:0/1612140505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26001a5ed0 con 0x7f2600101a10 2023-09-20T21:46:51.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.460+0000 7f260718b700 1 -- 172.21.15.22:0/1612140505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260004eae0 con 0x7f2600101a10 2023-09-20T21:46:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.462+0000 7f25fdffb700 1 -- 172.21.15.22:0/1612140505 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f25ec0105d0 con 0x7f2600101a10 2023-09-20T21:46:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.463+0000 7f25fdffb700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25f00a1300 0x7f25f00a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.463+0000 7f25fdffb700 1 -- 172.21.15.22:0/1612140505 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f25ec0cc8b0 con 0x7f2600101a10 2023-09-20T21:46:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.463+0000 7f2604f27700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25f00a1300 0x7f25f00a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.464+0000 7f2604f27700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25f00a1300 0x7f25f00a37c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b2e0 tx=0x7f25f4005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.465+0000 7f25fdffb700 1 -- 172.21.15.22:0/1612140505 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f25ec095d30 con 0x7f2600101a10 2023-09-20T21:46:51.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.723+0000 7f260718b700 1 -- 172.21.15.22:0/1612140505 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2600110fb0 con 0x7f2600101a10 2023-09-20T21:46:51.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.723+0000 7f25fdffb700 1 -- 172.21.15.22:0/1612140505 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f25ec020070 con 0x7f2600101a10 2023-09-20T21:46:51.725 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-20T21:46:51.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 -- 172.21.15.22:0/1612140505 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25f00a1300 msgr2=0x7f25f00a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:51.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25f00a1300 0x7f25f00a37c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f25f400b2e0 tx=0x7f25f4005960 comp rx=0 tx=0).stop 2023-09-20T21:46:51.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 -- 172.21.15.22:0/1612140505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2600101a10 msgr2=0x7f2600073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:51.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2600101a10 0x7f2600073620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00d980 tx=0x7f25ec00dc90 comp rx=0 tx=0).stop 2023-09-20T21:46:51.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 -- 172.21.15.22:0/1612140505 shutdown_connections 2023-09-20T21:46:51.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2600106140 0x7f2600073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:51.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2600101060 0x7f26000730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:51.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25f00a1300 0x7f25f00a37c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 --2- 172.21.15.22:0/1612140505 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2600101a10 0x7f2600073620 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 -- 172.21.15.22:0/1612140505 >> 172.21.15.22:0/1612140505 conn(0x7f26000fc8d0 msgr2=0x7f2600104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 -- 172.21.15.22:0/1612140505 shutdown_connections 2023-09-20T21:46:51.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:51.726+0000 7f260718b700 1 -- 172.21.15.22:0/1612140505 wait complete. 2023-09-20T21:46:52.229 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-09-20T21:46:52.230 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-20T21:46:52.230 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2023-09-20T21:46:52.230 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-09-20T21:46:52.231 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-20T21:46:52.231 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-09-20T21:46:52.231 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-20T21:46:52.232 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-20T21:46:52.232 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-09-20T21:46:52.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:52.233 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2023-09-20T21:46:52.279 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-20T21:46:52.280 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-20T21:46:52.280 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-09-20T21:46:52.280 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-09-20T21:46:52.280 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-09-20T21:46:52.281 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-09-20T21:46:52.281 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-20T21:46:52.281 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2023-09-20T21:46:52.282 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2023-09-20T21:46:52.282 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-20T21:46:52.283 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2023-09-20T21:46:52.283 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2023-09-20T21:46:52.283 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-20T21:46:52.284 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2023-09-20T21:46:52.284 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2023-09-20T21:46:52.284 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-20T21:46:52.285 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 41730sec preferred_lft 41730sec 2023-09-20T21:46:52.285 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-09-20T21:46:52.285 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-09-20T21:46:52.286 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-20T21:46:52.286 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2023-09-20T21:46:52.286 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2023-09-20T21:46:52.287 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-20T21:46:52.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:52.287 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-09-20T21:46:52.288 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-09-20T21:46:52.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2023-09-20T21:46:52.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2023-09-20T21:46:52.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2023-09-20T21:46:52.290 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-20T21:46:52.290 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-09-20T21:46:52.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:52 smithi022 ceph-mon[97651]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2023-09-20T21:46:52.425 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:52 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1612140505' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-20T21:46:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:52 smithi123 ceph-mon[134433]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2023-09-20T21:46:52.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:52 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1612140505' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-20T21:46:52.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:52 smithi089 ceph-mon[134090]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2023-09-20T21:46:52.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:52 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1612140505' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-20T21:46:52.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:52.760 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-20T21:46:52.835 INFO:teuthology.orchestra.run.smithi022.stdout:1 2023-09-20T21:46:52.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:52.838 DEBUG:teuthology.orchestra.run.smithi022:> route 2023-09-20T21:46:52.904 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2023-09-20T21:46:52.904 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-20T21:46:52.904 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-20T21:46:52.904 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-20T21:46:52.905 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-20T21:46:52.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:52.906 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-09-20T21:46:52.907 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-09-20T21:46:52.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-20T21:46:52.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-20T21:46:52.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-20T21:46:52.908 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-09-20T21:46:53.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:53.617 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-09-20T21:46:53.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:53.664 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2023-09-20T21:46:53.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:53.747 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-09-20T21:46:53.748 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-09-20T21:46:53.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-20T21:46:53.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-09-20T21:46:53.749 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-09-20T21:46:53.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:53 smithi123 ceph-mon[134433]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2023-09-20T21:46:53.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:53 smithi089 ceph-mon[134090]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2023-09-20T21:46:53.890 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:53 smithi022 ceph-mon[97651]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2023-09-20T21:46:53.936 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-09-20T21:46:53.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:53.937 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-09-20T21:46:53.937 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-09-20T21:46:53.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-09-20T21:46:53.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-20T21:46:53.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-09-20T21:46:53.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-09-20T21:46:53.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-09-20T21:46:53.939 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-09-20T21:46:54.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:54.215 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-09-20T21:46:54.216 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-09-20T21:46:54.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2023-09-20T21:46:54.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2023-09-20T21:46:54.217 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-09-20T21:46:54.392 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-09-20T21:46:54.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-20T21:46:54.393 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-09-20T21:46:54.437 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-09-20T21:46:54.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-20T21:46:54.438 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-09-20T21:46:54.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-09-20T21:46:54.631 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-20T21:46:54.632 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-20T21:46:54.702 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-20T21:46:54.705 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-20T21:46:54.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:54.709 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-09-20T21:46:54.752 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-20T21:46:54.753 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-09-20T21:46:54.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-09-20T21:46:54.870 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-20T21:46:54.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-20T21:46:54.942 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-20T21:46:54.944 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-20T21:46:54.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:54.945 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-09-20T21:46:55.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:2023-09-20T21:46:55.018+0000 7fba39c7e5c0 -1 init, newargv = 0x55ad6ae8ac10 newargc=15 2023-09-20T21:46:55.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[151459]: starting ceph client 2023-09-20T21:46:55.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[151459]: starting fuse 2023-09-20T21:46:56.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo modprobe fuse 2023-09-20T21:46:56.170 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-09-20T21:46:56.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-09-20T21:46:56.242 INFO:teuthology.orchestra.run.smithi022.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-09-20T21:46:56.245 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-09-20T21:46:56.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:56.246 DEBUG:teuthology.orchestra.run.smithi022:> ls /sys/fs/fuse/connections 2023-09-20T21:46:56.290 INFO:teuthology.orchestra.run.smithi022.stdout:121 2023-09-20T21:46:56.291 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-09-20T21:46:56.291 DEBUG:teuthology.orchestra.run.smithi022:> sudo stdin-killer -- python3 -c ' 2023-09-20T21:46:56.292 DEBUG:teuthology.orchestra.run.smithi022:> import glob 2023-09-20T21:46:56.292 DEBUG:teuthology.orchestra.run.smithi022:> import re 2023-09-20T21:46:56.292 DEBUG:teuthology.orchestra.run.smithi022:> import os 2023-09-20T21:46:56.293 DEBUG:teuthology.orchestra.run.smithi022:> import subprocess 2023-09-20T21:46:56.293 DEBUG:teuthology.orchestra.run.smithi022:> 2023-09-20T21:46:56.293 DEBUG:teuthology.orchestra.run.smithi022:> def _find_admin_socket(client_name): 2023-09-20T21:46:56.294 DEBUG:teuthology.orchestra.run.smithi022:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-09-20T21:46:56.294 DEBUG:teuthology.orchestra.run.smithi022:> files = glob.glob(asok_path) 2023-09-20T21:46:56.294 DEBUG:teuthology.orchestra.run.smithi022:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-09-20T21:46:56.295 DEBUG:teuthology.orchestra.run.smithi022:> 2023-09-20T21:46:56.295 DEBUG:teuthology.orchestra.run.smithi022:> # Given a non-glob path, it better be there 2023-09-20T21:46:56.295 DEBUG:teuthology.orchestra.run.smithi022:> if "*" not in asok_path: 2023-09-20T21:46:56.296 DEBUG:teuthology.orchestra.run.smithi022:> assert(len(files) == 1) 2023-09-20T21:46:56.296 DEBUG:teuthology.orchestra.run.smithi022:> return files[0] 2023-09-20T21:46:56.296 DEBUG:teuthology.orchestra.run.smithi022:> 2023-09-20T21:46:56.297 DEBUG:teuthology.orchestra.run.smithi022:> for f in files: 2023-09-20T21:46:56.297 DEBUG:teuthology.orchestra.run.smithi022:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-20T21:46:56.297 DEBUG:teuthology.orchestra.run.smithi022:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-20T21:46:56.298 DEBUG:teuthology.orchestra.run.smithi022:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-20T21:46:56.298 DEBUG:teuthology.orchestra.run.smithi022:> contents = proc_f.read() 2023-09-20T21:46:56.299 DEBUG:teuthology.orchestra.run.smithi022:> if mountpoint in contents: 2023-09-20T21:46:56.299 DEBUG:teuthology.orchestra.run.smithi022:> return f 2023-09-20T21:46:56.299 DEBUG:teuthology.orchestra.run.smithi022:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-20T21:46:56.299 DEBUG:teuthology.orchestra.run.smithi022:> 2023-09-20T21:46:56.300 DEBUG:teuthology.orchestra.run.smithi022:> print(_find_admin_socket("client.admin")) 2023-09-20T21:46:56.300 DEBUG:teuthology.orchestra.run.smithi022:> ' 2023-09-20T21:46:56.475 INFO:teuthology.orchestra.run.smithi022.stdout:/var/run/ceph/ceph-client.admin.151459.asok 2023-09-20T21:46:56.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-20T21:46:56.486 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:56 smithi022 ceph-mon[97651]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 1 op/s 2023-09-20T21:46:56.488 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.151459.asok 2023-09-20T21:46:56.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:56.489 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.151459.asok status 2023-09-20T21:46:56.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:56 smithi123 ceph-mon[134433]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 1 op/s 2023-09-20T21:46:56.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:56 smithi089 ceph-mon[134090]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 1 op/s 2023-09-20T21:46:56.720 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:46:56.721 INFO:teuthology.orchestra.run.smithi022.stdout: "metadata": { 2023-09-20T21:46:56.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_sha1": "bf8a061ff16ac1d393cf108f740d92dfa49f2e54", 2023-09-20T21:46:56.721 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 18.0.0-6213-gbf8a061f (bf8a061ff16ac1d393cf108f740d92dfa49f2e54) reef (dev)", 2023-09-20T21:46:56.722 INFO:teuthology.orchestra.run.smithi022.stdout: "entity_id": "admin", 2023-09-20T21:46:56.722 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": "smithi022", 2023-09-20T21:46:56.722 INFO:teuthology.orchestra.run.smithi022.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-09-20T21:46:56.722 INFO:teuthology.orchestra.run.smithi022.stdout: "pid": "151459", 2023-09-20T21:46:56.723 INFO:teuthology.orchestra.run.smithi022.stdout: "root": "/" 2023-09-20T21:46:56.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T21:46:56.723 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_count": 0, 2023-09-20T21:46:56.724 INFO:teuthology.orchestra.run.smithi022.stdout: "dentry_pinned_count": 0, 2023-09-20T21:46:56.724 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24704, 2023-09-20T21:46:56.725 INFO:teuthology.orchestra.run.smithi022.stdout: "inst": { 2023-09-20T21:46:56.725 INFO:teuthology.orchestra.run.smithi022.stdout: "name": { 2023-09-20T21:46:56.725 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "client", 2023-09-20T21:46:56.726 INFO:teuthology.orchestra.run.smithi022.stdout: "num": 24704 2023-09-20T21:46:56.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T21:46:56.726 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-09-20T21:46:56.727 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-09-20T21:46:56.727 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2023-09-20T21:46:56.728 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4242110207 2023-09-20T21:46:56.728 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:46:56.728 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T21:46:56.729 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": { 2023-09-20T21:46:56.729 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-09-20T21:46:56.729 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2023-09-20T21:46:56.729 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 4242110207 2023-09-20T21:46:56.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T21:46:56.730 INFO:teuthology.orchestra.run.smithi022.stdout: "inst_str": "client.24704 172.21.15.22:0/4242110207", 2023-09-20T21:46:56.730 INFO:teuthology.orchestra.run.smithi022.stdout: "addr_str": "172.21.15.22:0/4242110207", 2023-09-20T21:46:56.730 INFO:teuthology.orchestra.run.smithi022.stdout: "inode_count": 1, 2023-09-20T21:46:56.730 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_epoch": 17, 2023-09-20T21:46:56.730 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch": 89, 2023-09-20T21:46:56.731 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_epoch_barrier": 0, 2023-09-20T21:46:56.731 INFO:teuthology.orchestra.run.smithi022.stdout: "blocklisted": false, 2023-09-20T21:46:56.731 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs" 2023-09-20T21:46:56.731 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:46:56.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:56.735 DEBUG:teuthology.orchestra.run.smithi022:> stat --file-system '--printf=%T 2023-09-20T21:46:56.736 DEBUG:teuthology.orchestra.run.smithi022:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-09-20T21:46:56.781 INFO:teuthology.orchestra.run.smithi022.stdout:fuseblk 2023-09-20T21:46:56.781 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-09-20T21:46:56.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:56.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-09-20T21:46:56.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:46:56.893 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-09-20T21:46:57.015 INFO:teuthology.orchestra.run.smithi022.stdout:# file: . 2023-09-20T21:46:57.016 INFO:teuthology.orchestra.run.smithi022.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-09-20T21:46:57.016 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:57.017 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:46:57.017 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-20T21:46:57.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:46:57.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.305+0000 7f0f507ff700 1 -- 172.21.15.22:0/2627449574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f48102980 msgr2=0x7f0f48109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:57.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.305+0000 7f0f507ff700 1 --2- 172.21.15.22:0/2627449574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f48102980 0x7f0f48109230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000b0a0 tx=0x7f0f4000b3b0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.305+0000 7f0f507ff700 1 -- 172.21.15.22:0/2627449574 shutdown_connections 2023-09-20T21:46:57.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.305+0000 7f0f507ff700 1 --2- 172.21.15.22:0/2627449574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f48102980 0x7f0f48109230 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.305+0000 7f0f507ff700 1 --2- 172.21.15.22:0/2627449574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f48100df0 0x7f0f48101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.305+0000 7f0f507ff700 1 --2- 172.21.15.22:0/2627449574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f48101fe0 0x7f0f48102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.305+0000 7f0f507ff700 1 -- 172.21.15.22:0/2627449574 >> 172.21.15.22:0/2627449574 conn(0x7f0f480fc320 msgr2=0x7f0f480fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:57.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.305+0000 7f0f507ff700 1 -- 172.21.15.22:0/2627449574 shutdown_connections 2023-09-20T21:46:57.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.305+0000 7f0f507ff700 1 -- 172.21.15.22:0/2627449574 wait complete. 2023-09-20T21:46:57.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.306+0000 7f0f507ff700 1 Processor -- start 2023-09-20T21:46:57.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.306+0000 7f0f507ff700 1 -- start start 2023-09-20T21:46:57.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.306+0000 7f0f507ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f48100df0 0x7f0f48196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:57.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.306+0000 7f0f507ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f48101fe0 0x7f0f481970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:57.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.306+0000 7f0f507ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f48102980 0x7f0f4819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:57.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f507ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4819e720 con 0x7f0f48101fe0 2023-09-20T21:46:57.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f507ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4819e860 con 0x7f0f48102980 2023-09-20T21:46:57.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f507ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4819e9f0 con 0x7f0f48100df0 2023-09-20T21:46:57.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4dd9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f48101fe0 0x7f0f481970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:57.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4e59b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f48100df0 0x7f0f48196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:57.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4dd9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f48101fe0 0x7f0f481970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41214/0 (socket says 172.21.15.22:41214) 2023-09-20T21:46:57.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4dd9a700 1 -- 172.21.15.22:0/767552700 learned_addr learned my addr 172.21.15.22:0/767552700 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:57.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4ed9c700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f48102980 0x7f0f4819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:57.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4e59b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f48100df0 0x7f0f48196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:55136/0 (socket says 172.21.15.22:55136) 2023-09-20T21:46:57.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4dd9a700 1 -- 172.21.15.22:0/767552700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f48100df0 msgr2=0x7f0f48196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:57.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4dd9a700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f48100df0 0x7f0f48196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4dd9a700 1 -- 172.21.15.22:0/767552700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f48102980 msgr2=0x7f0f4819e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:57.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4dd9a700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f48102980 0x7f0f4819e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4dd9a700 1 -- 172.21.15.22:0/767552700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f40009d00 con 0x7f0f48101fe0 2023-09-20T21:46:57.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.307+0000 7f0f4e59b700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f48100df0 0x7f0f48196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:57.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.308+0000 7f0f4dd9a700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f48101fe0 0x7f0f481970e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800d950 tx=0x7f0f3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.308+0000 7f0f3f7fe700 1 -- 172.21.15.22:0/767552700 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800f890 con 0x7f0f48101fe0 2023-09-20T21:46:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.308+0000 7f0f507ff700 1 -- 172.21.15.22:0/767552700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4819ecd0 con 0x7f0f48101fe0 2023-09-20T21:46:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.308+0000 7f0f3f7fe700 1 -- 172.21.15.22:0/767552700 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f38010460 con 0x7f0f48101fe0 2023-09-20T21:46:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.308+0000 7f0f3f7fe700 1 -- 172.21.15.22:0/767552700 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800e610 con 0x7f0f48101fe0 2023-09-20T21:46:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.308+0000 7f0f507ff700 1 -- 172.21.15.22:0/767552700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4819f1d0 con 0x7f0f48101fe0 2023-09-20T21:46:57.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.309+0000 7f0f507ff700 1 -- 172.21.15.22:0/767552700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f481082f0 con 0x7f0f48101fe0 2023-09-20T21:46:57.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.311+0000 7f0f3f7fe700 1 -- 172.21.15.22:0/767552700 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f0f380108d0 con 0x7f0f48101fe0 2023-09-20T21:46:57.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.312+0000 7f0f3f7fe700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f340a13d0 0x7f0f340a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:57.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.312+0000 7f0f3f7fe700 1 -- 172.21.15.22:0/767552700 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0f380ccb90 con 0x7f0f48101fe0 2023-09-20T21:46:57.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.312+0000 7f0f4e59b700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f340a13d0 0x7f0f340a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:57.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.313+0000 7f0f4e59b700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f340a13d0 0x7f0f340a3890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f48101e40 tx=0x7f0f44009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.317+0000 7f0f3f7fe700 1 -- 172.21.15.22:0/767552700 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0f38096010 con 0x7f0f48101fe0 2023-09-20T21:46:57.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.532+0000 7f0f507ff700 1 -- 172.21.15.22:0/767552700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f4810c660 con 0x7f0f48101fe0 2023-09-20T21:46:57.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.533+0000 7f0f3f7fe700 1 -- 172.21.15.22:0/767552700 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23913 (secure 0 0 0) 0x7f0f38099630 con 0x7f0f48101fe0 2023-09-20T21:46:57.534 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:46:57.534 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:46:46.091544+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:46:57.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 -- 172.21.15.22:0/767552700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f340a13d0 msgr2=0x7f0f340a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:57.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f340a13d0 0x7f0f340a3890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f48101e40 tx=0x7f0f44009260 comp rx=0 tx=0).stop 2023-09-20T21:46:57.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 -- 172.21.15.22:0/767552700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f48101fe0 msgr2=0x7f0f481970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:57.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f48101fe0 0x7f0f481970e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800d950 tx=0x7f0f3800dc60 comp rx=0 tx=0).stop 2023-09-20T21:46:57.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 -- 172.21.15.22:0/767552700 shutdown_connections 2023-09-20T21:46:57.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f340a13d0 0x7f0f340a3890 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f48102980 0x7f0f4819e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f48100df0 0x7f0f48196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 --2- 172.21.15.22:0/767552700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f48101fe0 0x7f0f481970e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 -- 172.21.15.22:0/767552700 >> 172.21.15.22:0/767552700 conn(0x7f0f480fc320 msgr2=0x7f0f48105430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:57.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 -- 172.21.15.22:0/767552700 shutdown_connections 2023-09-20T21:46:57.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.536+0000 7f0f507ff700 1 -- 172.21.15.22:0/767552700 wait complete. 2023-09-20T21:46:57.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T21:46:57.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:46:57.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.764+0000 7f4c9edff700 1 -- 172.21.15.22:0/1172122902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c98101fc0 msgr2=0x7f4c98102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.764+0000 7f4c9edff700 1 --2- 172.21.15.22:0/1172122902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c98101fc0 0x7f4c98102420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c840099c0 tx=0x7f4c84009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.764+0000 7f4c9edff700 1 -- 172.21.15.22:0/1172122902 shutdown_connections 2023-09-20T21:46:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.764+0000 7f4c9edff700 1 --2- 172.21.15.22:0/1172122902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c98102960 0x7f4c98109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.764+0000 7f4c9edff700 1 --2- 172.21.15.22:0/1172122902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c98100dd0 0x7f4c981011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.764+0000 7f4c9edff700 1 --2- 172.21.15.22:0/1172122902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c98101fc0 0x7f4c98102420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.764+0000 7f4c9edff700 1 -- 172.21.15.22:0/1172122902 >> 172.21.15.22:0/1172122902 conn(0x7f4c980fc320 msgr2=0x7f4c980fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.764+0000 7f4c9edff700 1 -- 172.21.15.22:0/1172122902 shutdown_connections 2023-09-20T21:46:57.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.764+0000 7f4c9edff700 1 -- 172.21.15.22:0/1172122902 wait complete. 2023-09-20T21:46:57.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.765+0000 7f4c9edff700 1 Processor -- start 2023-09-20T21:46:57.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.765+0000 7f4c9edff700 1 -- start start 2023-09-20T21:46:57.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.765+0000 7f4c9edff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c98100dd0 0x7f4c98198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:57.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.765+0000 7f4c9edff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c98101fc0 0x7f4c98199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:57.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.765+0000 7f4c9edff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c98102960 0x7f4c981a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:57.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.765+0000 7f4c9edff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c981a08d0 con 0x7f4c98100dd0 2023-09-20T21:46:57.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.765+0000 7f4c9edff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c981a0a10 con 0x7f4c98102960 2023-09-20T21:46:57.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.765+0000 7f4c9edff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c981a0ba0 con 0x7f4c98101fc0 2023-09-20T21:46:57.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9cb9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c98100dd0 0x7f4c98198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:57.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9cb9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c98100dd0 0x7f4c98198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41238/0 (socket says 172.21.15.22:41238) 2023-09-20T21:46:57.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9cb9b700 1 -- 172.21.15.22:0/2034383066 learned_addr learned my addr 172.21.15.22:0/2034383066 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:57.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9d39c700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c98102960 0x7f4c981a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:57.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c97fff700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c98101fc0 0x7f4c98199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:57.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9d39c700 1 -- 172.21.15.22:0/2034383066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c98101fc0 msgr2=0x7f4c98199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:57.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9d39c700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c98101fc0 0x7f4c98199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9d39c700 1 -- 172.21.15.22:0/2034383066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c98100dd0 msgr2=0x7f4c98198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:57.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9d39c700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c98100dd0 0x7f4c98198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9d39c700 1 -- 172.21.15.22:0/2034383066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c840095f0 con 0x7f4c98102960 2023-09-20T21:46:57.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9cb9b700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c98100dd0 0x7f4c98198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:57.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.766+0000 7f4c9d39c700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c98102960 0x7f4c981a0300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000efc0 tx=0x7f4c9000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:57.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.767+0000 7f4c95ffb700 1 -- 172.21.15.22:0/2034383066 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90018070 con 0x7f4c98102960 2023-09-20T21:46:57.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.767+0000 7f4c95ffb700 1 -- 172.21.15.22:0/2034383066 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c9000f040 con 0x7f4c98102960 2023-09-20T21:46:57.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.767+0000 7f4c95ffb700 1 -- 172.21.15.22:0/2034383066 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9000cd40 con 0x7f4c98102960 2023-09-20T21:46:57.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.767+0000 7f4c9edff700 1 -- 172.21.15.22:0/2034383066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c981a0e20 con 0x7f4c98102960 2023-09-20T21:46:57.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.767+0000 7f4c9edff700 1 -- 172.21.15.22:0/2034383066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c981a12a0 con 0x7f4c98102960 2023-09-20T21:46:57.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.768+0000 7f4c9edff700 1 -- 172.21.15.22:0/2034383066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9810c630 con 0x7f4c98102960 2023-09-20T21:46:57.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.770+0000 7f4c95ffb700 1 -- 172.21.15.22:0/2034383066 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f4c90003a00 con 0x7f4c98102960 2023-09-20T21:46:57.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.771+0000 7f4c95ffb700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4c880a1380 0x7f4c880a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:57.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.771+0000 7f4c95ffb700 1 -- 172.21.15.22:0/2034383066 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4c900ccae0 con 0x7f4c98102960 2023-09-20T21:46:57.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.771+0000 7f4c9cb9b700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4c880a1380 0x7f4c880a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:57.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.772+0000 7f4c9cb9b700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4c880a1380 0x7f4c880a3840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c005950 tx=0x7f4c8c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:57.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.774+0000 7f4c95ffb700 1 -- 172.21.15.22:0/2034383066 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4c90095ff0 con 0x7f4c98102960 2023-09-20T21:46:57.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.983+0000 7f4c9edff700 1 -- 172.21.15.22:0/2034383066 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c981a1490 con 0x7f4c98102960 2023-09-20T21:46:57.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.984+0000 7f4c95ffb700 1 -- 172.21.15.22:0/2034383066 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c90099610 con 0x7f4c98102960 2023-09-20T21:46:57.986 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:46:57.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 -- 172.21.15.22:0/2034383066 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4c880a1380 msgr2=0x7f4c880a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4c880a1380 0x7f4c880a3840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c005950 tx=0x7f4c8c009260 comp rx=0 tx=0).stop 2023-09-20T21:46:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 -- 172.21.15.22:0/2034383066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c98102960 msgr2=0x7f4c981a0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c98102960 0x7f4c981a0300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000efc0 tx=0x7f4c9000c3a0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 -- 172.21.15.22:0/2034383066 shutdown_connections 2023-09-20T21:46:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4c880a1380 0x7f4c880a3840 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c98102960 0x7f4c981a0300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c98100dd0 0x7f4c98198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 --2- 172.21.15.22:0/2034383066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c98101fc0 0x7f4c98199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 -- 172.21.15.22:0/2034383066 >> 172.21.15.22:0/2034383066 conn(0x7f4c980fc320 msgr2=0x7f4c980fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 -- 172.21.15.22:0/2034383066 shutdown_connections 2023-09-20T21:46:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:57.986+0000 7f4c9edff700 1 -- 172.21.15.22:0/2034383066 wait complete. 2023-09-20T21:46:58.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T21:46:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.212+0000 7fe3609ff700 1 -- 172.21.15.22:0/814114622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c100dd0 msgr2=0x7fe35c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.212+0000 7fe3609ff700 1 --2- 172.21.15.22:0/814114622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c100dd0 0x7fe35c1011f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0099c0 tx=0x7fe34c009cd0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.212+0000 7fe3609ff700 1 -- 172.21.15.22:0/814114622 shutdown_connections 2023-09-20T21:46:58.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.212+0000 7fe3609ff700 1 --2- 172.21.15.22:0/814114622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe35c102960 0x7fe35c109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.212+0000 7fe3609ff700 1 --2- 172.21.15.22:0/814114622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c100dd0 0x7fe35c1011f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.212+0000 7fe3609ff700 1 --2- 172.21.15.22:0/814114622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c101fc0 0x7fe35c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.212+0000 7fe3609ff700 1 -- 172.21.15.22:0/814114622 >> 172.21.15.22:0/814114622 conn(0x7fe35c0fc320 msgr2=0x7fe35c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:58.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.213+0000 7fe3609ff700 1 -- 172.21.15.22:0/814114622 shutdown_connections 2023-09-20T21:46:58.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.213+0000 7fe3609ff700 1 -- 172.21.15.22:0/814114622 wait complete. 2023-09-20T21:46:58.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.213+0000 7fe3609ff700 1 Processor -- start 2023-09-20T21:46:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe3609ff700 1 -- start start 2023-09-20T21:46:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe3609ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c100dd0 0x7fe35c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe3609ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c101fc0 0x7fe35c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe3609ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe35c102960 0x7fe35c1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:58.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe3609ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c1a0880 con 0x7fe35c100dd0 2023-09-20T21:46:58.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe3609ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c1a09f0 con 0x7fe35c101fc0 2023-09-20T21:46:58.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe3609ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c1a0bb0 con 0x7fe35c102960 2023-09-20T21:46:58.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe35a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c100dd0 0x7fe35c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:58.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe35a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c100dd0 0x7fe35c198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41252/0 (socket says 172.21.15.22:41252) 2023-09-20T21:46:58.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe35a59c700 1 -- 172.21.15.22:0/1559858582 learned_addr learned my addr 172.21.15.22:0/1559858582 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:58.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe359d9b700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c101fc0 0x7fe35c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:58.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.214+0000 7fe35ad9d700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe35c102960 0x7fe35c1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:58.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe35a59c700 1 -- 172.21.15.22:0/1559858582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe35c102960 msgr2=0x7fe35c1a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:58.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe35a59c700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe35c102960 0x7fe35c1a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe35a59c700 1 -- 172.21.15.22:0/1559858582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c101fc0 msgr2=0x7fe35c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:58.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe35a59c700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c101fc0 0x7fe35c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe35a59c700 1 -- 172.21.15.22:0/1559858582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c0095f0 con 0x7fe35c100dd0 2023-09-20T21:46:58.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe359d9b700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c101fc0 0x7fe35c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:58.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe35a59c700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c100dd0 0x7fe35c198d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009fd0 tx=0x7fe34c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:58.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe34b7fe700 1 -- 172.21.15.22:0/1559858582 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c01d070 con 0x7fe35c100dd0 2023-09-20T21:46:58.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe3609ff700 1 -- 172.21.15.22:0/1559858582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c1a0e30 con 0x7fe35c100dd0 2023-09-20T21:46:58.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe34b7fe700 1 -- 172.21.15.22:0/1559858582 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe34c005050 con 0x7fe35c100dd0 2023-09-20T21:46:58.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.215+0000 7fe34b7fe700 1 -- 172.21.15.22:0/1559858582 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c004a70 con 0x7fe35c100dd0 2023-09-20T21:46:58.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.216+0000 7fe3609ff700 1 -- 172.21.15.22:0/1559858582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c1a11a0 con 0x7fe35c100dd0 2023-09-20T21:46:58.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.217+0000 7fe3609ff700 1 -- 172.21.15.22:0/1559858582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c10c630 con 0x7fe35c100dd0 2023-09-20T21:46:58.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.219+0000 7fe34b7fe700 1 -- 172.21.15.22:0/1559858582 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe34c005450 con 0x7fe35c100dd0 2023-09-20T21:46:58.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.220+0000 7fe34b7fe700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe3440a1300 0x7fe3440a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:58.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.220+0000 7fe34b7fe700 1 -- 172.21.15.22:0/1559858582 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe34c0cdfe0 con 0x7fe35c100dd0 2023-09-20T21:46:58.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.220+0000 7fe359d9b700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe3440a1300 0x7fe3440a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:58.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.221+0000 7fe359d9b700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe3440a1300 0x7fe3440a37c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe350011490 tx=0x7fe350009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:58.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.222+0000 7fe34b7fe700 1 -- 172.21.15.22:0/1559858582 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe34c097510 con 0x7fe35c100dd0 2023-09-20T21:46:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.433+0000 7fe3609ff700 1 -- 172.21.15.22:0/1559858582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe35c1a1390 con 0x7fe35c100dd0 2023-09-20T21:46:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.433+0000 7fe34b7fe700 1 -- 172.21.15.22:0/1559858582 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe34c09ab30 con 0x7fe35c100dd0 2023-09-20T21:46:58.434 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:46:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 -- 172.21.15.22:0/1559858582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe3440a1300 msgr2=0x7fe3440a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe3440a1300 0x7fe3440a37c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe350011490 tx=0x7fe350009260 comp rx=0 tx=0).stop 2023-09-20T21:46:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 -- 172.21.15.22:0/1559858582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c100dd0 msgr2=0x7fe35c198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c100dd0 0x7fe35c198d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009fd0 tx=0x7fe34c00f690 comp rx=0 tx=0).stop 2023-09-20T21:46:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 -- 172.21.15.22:0/1559858582 shutdown_connections 2023-09-20T21:46:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe35c102960 0x7fe35c1a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe35c100dd0 0x7fe35c198d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe3440a1300 0x7fe3440a37c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 --2- 172.21.15.22:0/1559858582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe35c101fc0 0x7fe35c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 -- 172.21.15.22:0/1559858582 >> 172.21.15.22:0/1559858582 conn(0x7fe35c0fc320 msgr2=0x7fe35c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:58.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 -- 172.21.15.22:0/1559858582 shutdown_connections 2023-09-20T21:46:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.436+0000 7fe3609ff700 1 -- 172.21.15.22:0/1559858582 wait complete. 2023-09-20T21:46:58.452 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:58 smithi022 ceph-mon[97651]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 97 B/s wr, 0 op/s 2023-09-20T21:46:58.452 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:58 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/767552700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:58.453 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:58 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2034383066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:58.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T21:46:58.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:58 smithi123 ceph-mon[134433]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 97 B/s wr, 0 op/s 2023-09-20T21:46:58.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:58 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/767552700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:58.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:58 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2034383066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:58 smithi089 ceph-mon[134090]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 97 B/s wr, 0 op/s 2023-09-20T21:46:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:58 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/767552700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:46:58.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:58 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2034383066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:46:58.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.661+0000 7f1ab99ff700 1 -- 172.21.15.22:0/246548857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4102960 msgr2=0x7f1ab4109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:58.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.661+0000 7f1ab99ff700 1 --2- 172.21.15.22:0/246548857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4102960 0x7f1ab4109210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009fc0 tx=0x7f1aac00b330 comp rx=0 tx=0).stop 2023-09-20T21:46:58.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.661+0000 7f1ab99ff700 1 -- 172.21.15.22:0/246548857 shutdown_connections 2023-09-20T21:46:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.661+0000 7f1ab99ff700 1 --2- 172.21.15.22:0/246548857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4102960 0x7f1ab4109210 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.661+0000 7f1ab99ff700 1 --2- 172.21.15.22:0/246548857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab4100dd0 0x7f1ab41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.661+0000 7f1ab99ff700 1 --2- 172.21.15.22:0/246548857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4101fc0 0x7f1ab4102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.661+0000 7f1ab99ff700 1 -- 172.21.15.22:0/246548857 >> 172.21.15.22:0/246548857 conn(0x7f1ab40fc320 msgr2=0x7f1ab40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.661+0000 7f1ab99ff700 1 -- 172.21.15.22:0/246548857 shutdown_connections 2023-09-20T21:46:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.661+0000 7f1ab99ff700 1 -- 172.21.15.22:0/246548857 wait complete. 2023-09-20T21:46:58.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.662+0000 7f1ab99ff700 1 Processor -- start 2023-09-20T21:46:58.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.662+0000 7f1ab99ff700 1 -- start start 2023-09-20T21:46:58.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab99ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4100dd0 0x7f1ab4196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:58.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab99ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab4101fc0 0x7f1ab41970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:58.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab99ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4102960 0x7f1ab419e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:58.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab99ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419e710 con 0x7f1ab4102960 2023-09-20T21:46:58.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab99ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419e850 con 0x7f1ab4101fc0 2023-09-20T21:46:58.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab99ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419e9e0 con 0x7f1ab4100dd0 2023-09-20T21:46:58.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4102960 0x7f1ab419e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:58.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab27fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab4101fc0 0x7f1ab41970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:58.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4102960 0x7f1ab419e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41256/0 (socket says 172.21.15.22:41256) 2023-09-20T21:46:58.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4100dd0 0x7f1ab4196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:58.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab37fe700 1 -- 172.21.15.22:0/3666684425 learned_addr learned my addr 172.21.15.22:0/3666684425 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:58.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab27fc700 1 -- 172.21.15.22:0/3666684425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4100dd0 msgr2=0x7f1ab4196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:58.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab27fc700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4100dd0 0x7f1ab4196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab27fc700 1 -- 172.21.15.22:0/3666684425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4102960 msgr2=0x7f1ab419e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab27fc700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4102960 0x7f1ab419e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab27fc700 1 -- 172.21.15.22:0/3666684425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac009c40 con 0x7f1ab4101fc0 2023-09-20T21:46:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab37fe700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4102960 0x7f1ab419e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.663+0000 7f1ab2ffd700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4100dd0 0x7f1ab4196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:58.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.664+0000 7f1ab27fc700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab4101fc0 0x7f1ab41970d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800e9c0 tx=0x7f1aa800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:58.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.664+0000 7f1ab89fd700 1 -- 172.21.15.22:0/3666684425 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa800c9c0 con 0x7f1ab4101fc0 2023-09-20T21:46:58.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.664+0000 7f1ab89fd700 1 -- 172.21.15.22:0/3666684425 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aa8010430 con 0x7f1ab4101fc0 2023-09-20T21:46:58.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.664+0000 7f1ab89fd700 1 -- 172.21.15.22:0/3666684425 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8018680 con 0x7f1ab4101fc0 2023-09-20T21:46:58.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.664+0000 7f1ab99ff700 1 -- 172.21.15.22:0/3666684425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab419ecc0 con 0x7f1ab4101fc0 2023-09-20T21:46:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.664+0000 7f1ab99ff700 1 -- 172.21.15.22:0/3666684425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab419f180 con 0x7f1ab4101fc0 2023-09-20T21:46:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.665+0000 7f1ab99ff700 1 -- 172.21.15.22:0/3666684425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab4108320 con 0x7f1ab4101fc0 2023-09-20T21:46:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.667+0000 7f1ab89fd700 1 -- 172.21.15.22:0/3666684425 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1aa8010860 con 0x7f1ab4101fc0 2023-09-20T21:46:58.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.669+0000 7f1ab89fd700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1a9c0a1300 0x7f1a9c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:58.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.669+0000 7f1ab89fd700 1 -- 172.21.15.22:0/3666684425 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1aa8014070 con 0x7f1ab4101fc0 2023-09-20T21:46:58.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.669+0000 7f1ab2ffd700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1a9c0a1300 0x7f1a9c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:58.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.669+0000 7f1ab2ffd700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1a9c0a1300 0x7f1a9c0a37c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4101e20 tx=0x7f1aa4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:58.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.671+0000 7f1ab89fd700 1 -- 172.21.15.22:0/3666684425 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1aa8096500 con 0x7f1ab4101fc0 2023-09-20T21:46:58.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.882+0000 7f1ab99ff700 1 -- 172.21.15.22:0/3666684425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ab402cfa0 con 0x7f1ab4101fc0 2023-09-20T21:46:58.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.882+0000 7f1ab89fd700 1 -- 172.21.15.22:0/3666684425 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1aa8026020 con 0x7f1ab4101fc0 2023-09-20T21:46:58.883 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T21:46:58.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 -- 172.21.15.22:0/3666684425 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1a9c0a1300 msgr2=0x7f1a9c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:58.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1a9c0a1300 0x7f1a9c0a37c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4101e20 tx=0x7f1aa4009260 comp rx=0 tx=0).stop 2023-09-20T21:46:58.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 -- 172.21.15.22:0/3666684425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab4101fc0 msgr2=0x7f1ab41970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:58.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab4101fc0 0x7f1ab41970d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800e9c0 tx=0x7f1aa800ecd0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 -- 172.21.15.22:0/3666684425 shutdown_connections 2023-09-20T21:46:58.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ab4102960 0x7f1ab419e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4100dd0 0x7f1ab4196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1a9c0a1300 0x7f1a9c0a37c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 --2- 172.21.15.22:0/3666684425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ab4101fc0 0x7f1ab41970d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:58.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 -- 172.21.15.22:0/3666684425 >> 172.21.15.22:0/3666684425 conn(0x7f1ab40fc320 msgr2=0x7f1ab4105460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:58.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 -- 172.21.15.22:0/3666684425 shutdown_connections 2023-09-20T21:46:58.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:58.885+0000 7f1ab99ff700 1 -- 172.21.15.22:0/3666684425 wait complete. 2023-09-20T21:46:58.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T21:46:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.118+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/946049628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb04102960 msgr2=0x7ffb04109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:59.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.118+0000 7ffb0a7ff700 1 --2- 172.21.15.22:0/946049628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb04102960 0x7ffb04109210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00b0d0 tx=0x7ffafc00b3e0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.118+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/946049628 shutdown_connections 2023-09-20T21:46:59.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.118+0000 7ffb0a7ff700 1 --2- 172.21.15.22:0/946049628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb04102960 0x7ffb04109210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.118+0000 7ffb0a7ff700 1 --2- 172.21.15.22:0/946049628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04100dd0 0x7ffb041011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.118+0000 7ffb0a7ff700 1 --2- 172.21.15.22:0/946049628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04101fc0 0x7ffb04102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.118+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/946049628 >> 172.21.15.22:0/946049628 conn(0x7ffb040fc320 msgr2=0x7ffb040fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.118+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/946049628 shutdown_connections 2023-09-20T21:46:59.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.118+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/946049628 wait complete. 2023-09-20T21:46:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.120+0000 7ffb0a7ff700 1 Processor -- start 2023-09-20T21:46:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.120+0000 7ffb0a7ff700 1 -- start start 2023-09-20T21:46:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb0a7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04100dd0 0x7ffb04074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:59.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb0a7ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb04101fc0 0x7ffb040731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb0a7ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04102960 0x7ffb04073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb0a7ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04073eb0 con 0x7ffb04101fc0 2023-09-20T21:46:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb0a7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04074020 con 0x7ffb04100dd0 2023-09-20T21:46:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb0a7ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb041a0c50 con 0x7ffb04102960 2023-09-20T21:46:59.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb03fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04100dd0 0x7ffb04074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb03fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04100dd0 0x7ffb04074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:33160/0 (socket says 172.21.15.22:33160) 2023-09-20T21:46:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb03fff700 1 -- 172.21.15.22:0/3807191763 learned_addr learned my addr 172.21.15.22:0/3807191763 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb037fe700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb04101fc0 0x7ffb040731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:59.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb08d9c700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04102960 0x7ffb04073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb03fff700 1 -- 172.21.15.22:0/3807191763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04102960 msgr2=0x7ffb04073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb03fff700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04102960 0x7ffb04073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb03fff700 1 -- 172.21.15.22:0/3807191763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb04101fc0 msgr2=0x7ffb040731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb03fff700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb04101fc0 0x7ffb040731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.121+0000 7ffb03fff700 1 -- 172.21.15.22:0/3807191763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc009d00 con 0x7ffb04100dd0 2023-09-20T21:46:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.122+0000 7ffb03fff700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04100dd0 0x7ffb04074b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400d980 tx=0x7ffaf400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.122+0000 7ffb017fa700 1 -- 172.21.15.22:0/3807191763 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4004020 con 0x7ffb04100dd0 2023-09-20T21:46:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.122+0000 7ffb017fa700 1 -- 172.21.15.22:0/3807191763 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaf4009730 con 0x7ffb04100dd0 2023-09-20T21:46:59.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.122+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/3807191763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb041a0f30 con 0x7ffb04100dd0 2023-09-20T21:46:59.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.122+0000 7ffb017fa700 1 -- 172.21.15.22:0/3807191763 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4003a30 con 0x7ffb04100dd0 2023-09-20T21:46:59.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.122+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/3807191763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb041a1350 con 0x7ffb04100dd0 2023-09-20T21:46:59.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.123+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/3807191763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0410c6b0 con 0x7ffb04100dd0 2023-09-20T21:46:59.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.125+0000 7ffb017fa700 1 -- 172.21.15.22:0/3807191763 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ffaf40098a0 con 0x7ffb04100dd0 2023-09-20T21:46:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.126+0000 7ffb017fa700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffaec0a1300 0x7ffaec0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.127+0000 7ffb017fa700 1 -- 172.21.15.22:0/3807191763 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffaf40ccee0 con 0x7ffb04100dd0 2023-09-20T21:46:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.127+0000 7ffb037fe700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffaec0a1300 0x7ffaec0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.127+0000 7ffb037fe700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffaec0a1300 0x7ffaec0a37c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009c60 tx=0x7ffaf8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:59.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.128+0000 7ffb017fa700 1 -- 172.21.15.22:0/3807191763 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffaf4096360 con 0x7ffb04100dd0 2023-09-20T21:46:59.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.341+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/3807191763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ffb04107cb0 con 0x7ffb04100dd0 2023-09-20T21:46:59.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.342+0000 7ffb017fa700 1 -- 172.21.15.22:0/3807191763 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7ffaf4099980 con 0x7ffb04100dd0 2023-09-20T21:46:59.342 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:46:59.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/3807191763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffaec0a1300 msgr2=0x7ffaec0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:59.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffaec0a1300 0x7ffaec0a37c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009c60 tx=0x7ffaf8009260 comp rx=0 tx=0).stop 2023-09-20T21:46:59.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/3807191763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04100dd0 msgr2=0x7ffb04074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:59.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04100dd0 0x7ffb04074b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400d980 tx=0x7ffaf400dc90 comp rx=0 tx=0).stop 2023-09-20T21:46:59.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/3807191763 shutdown_connections 2023-09-20T21:46:59.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb04102960 0x7ffb04073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffaec0a1300 0x7ffaec0a37c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb04100dd0 0x7ffb04074b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 --2- 172.21.15.22:0/3807191763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb04101fc0 0x7ffb040731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/3807191763 >> 172.21.15.22:0/3807191763 conn(0x7ffb040fc320 msgr2=0x7ffb040fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:59.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/3807191763 shutdown_connections 2023-09-20T21:46:59.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.344+0000 7ffb0a7ff700 1 -- 172.21.15.22:0/3807191763 wait complete. 2023-09-20T21:46:59.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-20T21:46:59.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:59 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1559858582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:46:59.424 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:46:59 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3666684425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:46:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.568+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1222209235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf0073840 msgr2=0x7fcbf0073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.568+0000 7fcbf59ff700 1 --2- 172.21.15.22:0/1222209235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf0073840 0x7fcbf0073cc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009990 tx=0x7fcbe4009ca0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.568+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1222209235 shutdown_connections 2023-09-20T21:46:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.568+0000 7fcbf59ff700 1 --2- 172.21.15.22:0/1222209235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf0108c90 0x7fcbf010b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.568+0000 7fcbf59ff700 1 --2- 172.21.15.22:0/1222209235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbf0074e20 0x7fcbf0073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.568+0000 7fcbf59ff700 1 --2- 172.21.15.22:0/1222209235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf0073840 0x7fcbf0073cc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.568+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1222209235 >> 172.21.15.22:0/1222209235 conn(0x7fcbf00fc320 msgr2=0x7fcbf00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:59.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.568+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1222209235 shutdown_connections 2023-09-20T21:46:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.569+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1222209235 wait complete. 2023-09-20T21:46:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.570+0000 7fcbf59ff700 1 Processor -- start 2023-09-20T21:46:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.570+0000 7fcbf59ff700 1 -- start start 2023-09-20T21:46:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbf59ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf0073840 0x7fcbf0198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbf59ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf0074e20 0x7fcbf0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbf59ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbf0108c90 0x7fcbf01a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbf59ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01a0840 con 0x7fcbf0074e20 2023-09-20T21:46:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbf59ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01a09b0 con 0x7fcbf0073840 2023-09-20T21:46:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbf59ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01a0b70 con 0x7fcbf0108c90 2023-09-20T21:46:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbee7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf0074e20 0x7fcbf0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbeeffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf0073840 0x7fcbf0198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbee7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf0074e20 0x7fcbf0199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41282/0 (socket says 172.21.15.22:41282) 2023-09-20T21:46:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbee7fc700 1 -- 172.21.15.22:0/1100977990 learned_addr learned my addr 172.21.15.22:0/1100977990 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:46:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbef7fe700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbf0108c90 0x7fcbf01a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbee7fc700 1 -- 172.21.15.22:0/1100977990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbf0108c90 msgr2=0x7fcbf01a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbee7fc700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbf0108c90 0x7fcbf01a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbee7fc700 1 -- 172.21.15.22:0/1100977990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf0073840 msgr2=0x7fcbf0198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbee7fc700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf0073840 0x7fcbf0198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbee7fc700 1 -- 172.21.15.22:0/1100977990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe40095f0 con 0x7fcbf0074e20 2023-09-20T21:46:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.571+0000 7fcbeeffd700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf0073840 0x7fcbf0198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:46:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.572+0000 7fcbee7fc700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf0074e20 0x7fcbf0199240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4004710 tx=0x7fcbe4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.572+0000 7fcbf49fd700 1 -- 172.21.15.22:0/1100977990 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe401d070 con 0x7fcbf0074e20 2023-09-20T21:46:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.572+0000 7fcbf49fd700 1 -- 172.21.15.22:0/1100977990 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbe4021410 con 0x7fcbf0074e20 2023-09-20T21:46:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.572+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1100977990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf01a0df0 con 0x7fcbf0074e20 2023-09-20T21:46:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.572+0000 7fcbf49fd700 1 -- 172.21.15.22:0/1100977990 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4017650 con 0x7fcbf0074e20 2023-09-20T21:46:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.572+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1100977990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf01a11c0 con 0x7fcbf0074e20 2023-09-20T21:46:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.573+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1100977990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf0107340 con 0x7fcbf0074e20 2023-09-20T21:46:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.575+0000 7fcbf49fd700 1 -- 172.21.15.22:0/1100977990 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fcbe4021580 con 0x7fcbf0074e20 2023-09-20T21:46:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.576+0000 7fcbf49fd700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcbd80a1380 0x7fcbd80a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:46:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.576+0000 7fcbf49fd700 1 -- 172.21.15.22:0/1100977990 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcbe40ce1d0 con 0x7fcbf0074e20 2023-09-20T21:46:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.577+0000 7fcbeeffd700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcbd80a1380 0x7fcbd80a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:46:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.577+0000 7fcbeeffd700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcbd80a1380 0x7fcbd80a3840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcbf019a1d0 tx=0x7fcbe0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:46:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.581+0000 7fcbf49fd700 1 -- 172.21.15.22:0/1100977990 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcbe40976e0 con 0x7fcbf0074e20 2023-09-20T21:46:59.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:59 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1559858582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:46:59.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:46:59 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3666684425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:46:59.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:59 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1559858582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:46:59.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:46:59 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3666684425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:46:59.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.810+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1100977990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcbf010e7c0 con 0x7fcbf0074e20 2023-09-20T21:46:59.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.810+0000 7fcbf49fd700 1 -- 172.21.15.22:0/1100977990 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+591 (secure 0 0 0) 0x7fcbe409ad00 con 0x7fcbf0074e20 2023-09-20T21:46:59.811 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3260510802 2023-09-21T21:40:07.017182+0000 2023-09-20T21:46:59.811 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3260510802 2023-09-21T21:40:07.017182+0000 2023-09-20T21:46:59.811 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/1128310681 2023-09-21T21:39:39.363251+0000 2023-09-20T21:46:59.812 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3419565055 2023-09-21T21:40:07.017182+0000 2023-09-20T21:46:59.812 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/1128310681 2023-09-21T21:39:39.363251+0000 2023-09-20T21:46:59.812 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/4209005662 2023-09-21T21:39:39.363251+0000 2023-09-20T21:46:59.812 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1007451083 2023-09-21T21:39:39.363251+0000 2023-09-20T21:46:59.812 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1994934339 2023-09-21T21:40:07.017182+0000 2023-09-20T21:46:59.813 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/875874054 2023-09-21T21:40:07.017182+0000 2023-09-20T21:46:59.813 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2863894527 2023-09-21T21:39:39.363251+0000 2023-09-20T21:46:59.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.812+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1100977990 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcbd80a1380 msgr2=0x7fcbd80a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:59.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.812+0000 7fcbf59ff700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcbd80a1380 0x7fcbd80a3840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcbf019a1d0 tx=0x7fcbe0009260 comp rx=0 tx=0).stop 2023-09-20T21:46:59.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.812+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1100977990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf0074e20 msgr2=0x7fcbf0199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:46:59.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.812+0000 7fcbf59ff700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf0074e20 0x7fcbf0199240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4004710 tx=0x7fcbe4009580 comp rx=0 tx=0).stop 2023-09-20T21:46:59.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.813+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1100977990 shutdown_connections 2023-09-20T21:46:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.813+0000 7fcbf59ff700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbf0108c90 0x7fcbf01a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.813+0000 7fcbf59ff700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fcbd80a1380 0x7fcbd80a3840 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.813+0000 7fcbf59ff700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbf0073840 0x7fcbf0198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.813+0000 7fcbf59ff700 1 --2- 172.21.15.22:0/1100977990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbf0074e20 0x7fcbf0199240 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:46:59.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.813+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1100977990 >> 172.21.15.22:0/1100977990 conn(0x7fcbf00fc320 msgr2=0x7fcbf00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:46:59.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.813+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1100977990 shutdown_connections 2023-09-20T21:46:59.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:46:59.813+0000 7fcbf59ff700 1 -- 172.21.15.22:0/1100977990 wait complete. 2023-09-20T21:46:59.817 INFO:teuthology.orchestra.run.smithi022.stderr:listed 10 entries 2023-09-20T21:46:59.831 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:46:59.832 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-20T21:46:59.876 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:47:00.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.120+0000 7f3b417ff700 1 -- 172.21.15.22:0/2242205735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b3c100df0 msgr2=0x7f3b3c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:00.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.120+0000 7f3b417ff700 1 --2- 172.21.15.22:0/2242205735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b3c100df0 0x7f3b3c101210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0099c0 tx=0x7f3b2c009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.120+0000 7f3b417ff700 1 -- 172.21.15.22:0/2242205735 shutdown_connections 2023-09-20T21:47:00.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.120+0000 7f3b417ff700 1 --2- 172.21.15.22:0/2242205735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b3c102980 0x7f3b3c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.120+0000 7f3b417ff700 1 --2- 172.21.15.22:0/2242205735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b3c100df0 0x7f3b3c101210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.120+0000 7f3b417ff700 1 --2- 172.21.15.22:0/2242205735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b3c101fe0 0x7f3b3c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.120+0000 7f3b417ff700 1 -- 172.21.15.22:0/2242205735 >> 172.21.15.22:0/2242205735 conn(0x7f3b3c0fc320 msgr2=0x7f3b3c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:00.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.120+0000 7f3b417ff700 1 -- 172.21.15.22:0/2242205735 shutdown_connections 2023-09-20T21:47:00.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.120+0000 7f3b417ff700 1 -- 172.21.15.22:0/2242205735 wait complete. 2023-09-20T21:47:00.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.121+0000 7f3b417ff700 1 Processor -- start 2023-09-20T21:47:00.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.121+0000 7f3b417ff700 1 -- start start 2023-09-20T21:47:00.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.121+0000 7f3b417ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b3c100df0 0x7f3b3c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:00.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.121+0000 7f3b417ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b3c101fe0 0x7f3b3c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:00.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.121+0000 7f3b417ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b3c102980 0x7f3b3c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:00.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.121+0000 7f3b417ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c1a0830 con 0x7f3b3c102980 2023-09-20T21:47:00.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.121+0000 7f3b417ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c1a09a0 con 0x7f3b3c100df0 2023-09-20T21:47:00.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.121+0000 7f3b417ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c1a0b60 con 0x7f3b3c101fe0 2023-09-20T21:47:00.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.121+0000 7f3b3b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b3c102980 0x7f3b3c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:00.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3affd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b3c100df0 0x7f3b3c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:00.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b3c101fe0 0x7f3b3c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:00.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b3c102980 0x7f3b3c1a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41300/0 (socket says 172.21.15.22:41300) 2023-09-20T21:47:00.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3b7fe700 1 -- 172.21.15.22:0/3340218249 learned_addr learned my addr 172.21.15.22:0/3340218249 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:00.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3a7fc700 1 -- 172.21.15.22:0/3340218249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b3c100df0 msgr2=0x7f3b3c198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:00.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3a7fc700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b3c100df0 0x7f3b3c198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3a7fc700 1 -- 172.21.15.22:0/3340218249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b3c102980 msgr2=0x7f3b3c1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3a7fc700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b3c102980 0x7f3b3c1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3a7fc700 1 -- 172.21.15.22:0/3340218249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c0095f0 con 0x7f3b3c101fe0 2023-09-20T21:47:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3b7fe700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b3c102980 0x7f3b3c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b3a7fc700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b3c101fe0 0x7f3b3c199230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000e9f0 tx=0x7f3b3000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:00.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.122+0000 7f3b23fff700 1 -- 172.21.15.22:0/3340218249 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3000c9c0 con 0x7f3b3c101fe0 2023-09-20T21:47:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.123+0000 7f3b23fff700 1 -- 172.21.15.22:0/3340218249 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b30010430 con 0x7f3b3c101fe0 2023-09-20T21:47:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.123+0000 7f3b417ff700 1 -- 172.21.15.22:0/3340218249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c1a0e40 con 0x7f3b3c101fe0 2023-09-20T21:47:00.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.123+0000 7f3b23fff700 1 -- 172.21.15.22:0/3340218249 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30018680 con 0x7f3b3c101fe0 2023-09-20T21:47:00.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.123+0000 7f3b417ff700 1 -- 172.21.15.22:0/3340218249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c1a1260 con 0x7f3b3c101fe0 2023-09-20T21:47:00.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.124+0000 7f3b417ff700 1 -- 172.21.15.22:0/3340218249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c10c650 con 0x7f3b3c101fe0 2023-09-20T21:47:00.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.126+0000 7f3b23fff700 1 -- 172.21.15.22:0/3340218249 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f3b30010860 con 0x7f3b3c101fe0 2023-09-20T21:47:00.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.127+0000 7f3b23fff700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3b240a1350 0x7f3b240a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:00.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.127+0000 7f3b23fff700 1 -- 172.21.15.22:0/3340218249 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3b30014070 con 0x7f3b3c101fe0 2023-09-20T21:47:00.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.127+0000 7f3b3affd700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3b240a1350 0x7f3b240a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:00.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.128+0000 7f3b3affd700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3b240a1350 0x7f3b240a3810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c19a1c0 tx=0x7f3b2c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:00.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.129+0000 7f3b23fff700 1 -- 172.21.15.22:0/3340218249 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3b300965f0 con 0x7f3b3c101fe0 2023-09-20T21:47:00.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.345+0000 7f3b417ff700 1 -- 172.21.15.22:0/3340218249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b3c1a1480 con 0x7f3b3c101fe0 2023-09-20T21:47:00.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.347+0000 7f3b23fff700 1 -- 172.21.15.22:0/3340218249 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23913 (secure 0 0 0) 0x7f3b30099c10 con 0x7f3b3c101fe0 2023-09-20T21:47:00.347 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:00.348 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:46:46.091544+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:47:00.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 -- 172.21.15.22:0/3340218249 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3b240a1350 msgr2=0x7f3b240a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:00.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3b240a1350 0x7f3b240a3810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c19a1c0 tx=0x7f3b2c005fb0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 -- 172.21.15.22:0/3340218249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b3c101fe0 msgr2=0x7f3b3c199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b3c101fe0 0x7f3b3c199230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000e9f0 tx=0x7f3b3000ed00 comp rx=0 tx=0).stop 2023-09-20T21:47:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 -- 172.21.15.22:0/3340218249 shutdown_connections 2023-09-20T21:47:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b3c102980 0x7f3b3c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3b240a1350 0x7f3b240a3810 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3b3c100df0 0x7f3b3c198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 --2- 172.21.15.22:0/3340218249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b3c101fe0 0x7f3b3c199230 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 -- 172.21.15.22:0/3340218249 >> 172.21.15.22:0/3340218249 conn(0x7f3b3c0fc320 msgr2=0x7f3b3c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:00.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 -- 172.21.15.22:0/3340218249 shutdown_connections 2023-09-20T21:47:00.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.349+0000 7f3b417ff700 1 -- 172.21.15.22:0/3340218249 wait complete. 2023-09-20T21:47:00.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T21:47:00.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:47:00.430 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:00 smithi022 ceph-mon[97651]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-20T21:47:00.430 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:00 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3807191763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:47:00.430 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:00 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1100977990' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-20T21:47:00.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.575+0000 7f5ab7dff700 1 -- 172.21.15.22:0/3548977634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ab0101fc0 msgr2=0x7f5ab0102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:00.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.575+0000 7f5ab7dff700 1 --2- 172.21.15.22:0/3548977634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ab0101fc0 0x7f5ab0102420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00099c0 tx=0x7f5aa0009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.575+0000 7f5ab7dff700 1 -- 172.21.15.22:0/3548977634 shutdown_connections 2023-09-20T21:47:00.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.575+0000 7f5ab7dff700 1 --2- 172.21.15.22:0/3548977634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0102960 0x7f5ab0109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.575+0000 7f5ab7dff700 1 --2- 172.21.15.22:0/3548977634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab0100dd0 0x7f5ab01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.575+0000 7f5ab7dff700 1 --2- 172.21.15.22:0/3548977634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ab0101fc0 0x7f5ab0102420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.575+0000 7f5ab7dff700 1 -- 172.21.15.22:0/3548977634 >> 172.21.15.22:0/3548977634 conn(0x7f5ab00fc320 msgr2=0x7f5ab00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:00.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.575+0000 7f5ab7dff700 1 -- 172.21.15.22:0/3548977634 shutdown_connections 2023-09-20T21:47:00.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.575+0000 7f5ab7dff700 1 -- 172.21.15.22:0/3548977634 wait complete. 2023-09-20T21:47:00.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.576+0000 7f5ab7dff700 1 Processor -- start 2023-09-20T21:47:00.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab7dff700 1 -- start start 2023-09-20T21:47:00.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab7dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab0100dd0 0x7f5ab0074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:00.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab7dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0101fc0 0x7f5ab00731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:00.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab7dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ab0102960 0x7f5ab0073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:00.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab7dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab0073eb0 con 0x7f5ab0102960 2023-09-20T21:47:00.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab7dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab0074020 con 0x7f5ab0100dd0 2023-09-20T21:47:00.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab7dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab01a0b80 con 0x7f5ab0101fc0 2023-09-20T21:47:00.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab639c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ab0102960 0x7f5ab0073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:00.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab5b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab0100dd0 0x7f5ab0074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:00.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab639c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ab0102960 0x7f5ab0073710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41318/0 (socket says 172.21.15.22:41318) 2023-09-20T21:47:00.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab639c700 1 -- 172.21.15.22:0/718807326 learned_addr learned my addr 172.21.15.22:0/718807326 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:00.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.577+0000 7f5ab539a700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0101fc0 0x7f5ab00731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:00.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5ab5b9b700 1 -- 172.21.15.22:0/718807326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0101fc0 msgr2=0x7f5ab00731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:00.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5ab5b9b700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0101fc0 0x7f5ab00731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5ab5b9b700 1 -- 172.21.15.22:0/718807326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ab0102960 msgr2=0x7f5ab0073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:00.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5ab5b9b700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ab0102960 0x7f5ab0073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5ab5b9b700 1 -- 172.21.15.22:0/718807326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00095f0 con 0x7f5ab0100dd0 2023-09-20T21:47:00.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5ab639c700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ab0102960 0x7f5ab0073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:00.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5ab5b9b700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab0100dd0 0x7f5ab0074b90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00e9c0 tx=0x7f5aac00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:00.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5aa6ffd700 1 -- 172.21.15.22:0/718807326 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac004220 con 0x7f5ab0100dd0 2023-09-20T21:47:00.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5ab7dff700 1 -- 172.21.15.22:0/718807326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab01a0e60 con 0x7f5ab0100dd0 2023-09-20T21:47:00.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5aa6ffd700 1 -- 172.21.15.22:0/718807326 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5aac004380 con 0x7f5ab0100dd0 2023-09-20T21:47:00.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5aa6ffd700 1 -- 172.21.15.22:0/718807326 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac003be0 con 0x7f5ab0100dd0 2023-09-20T21:47:00.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.578+0000 7f5ab7dff700 1 -- 172.21.15.22:0/718807326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab01a1280 con 0x7f5ab0100dd0 2023-09-20T21:47:00.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.579+0000 7f5ab7dff700 1 -- 172.21.15.22:0/718807326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab0107d50 con 0x7f5ab0100dd0 2023-09-20T21:47:00.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.581+0000 7f5aa6ffd700 1 -- 172.21.15.22:0/718807326 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5aac00c7c0 con 0x7f5ab0100dd0 2023-09-20T21:47:00.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.583+0000 7f5aa6ffd700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5a9c0a1300 0x7f5a9c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:00.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.583+0000 7f5aa6ffd700 1 -- 172.21.15.22:0/718807326 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5aac014070 con 0x7f5ab0100dd0 2023-09-20T21:47:00.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.583+0000 7f5ab539a700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5a9c0a1300 0x7f5a9c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:00.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.583+0000 7f5ab539a700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5a9c0a1300 0x7f5a9c0a37c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b580 tx=0x7f5aa0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:00.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.585+0000 7f5aa6ffd700 1 -- 172.21.15.22:0/718807326 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5aac0965a0 con 0x7f5ab0100dd0 2023-09-20T21:47:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:00 smithi123 ceph-mon[134433]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-20T21:47:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:00 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3807191763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:47:00.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:00 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1100977990' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-20T21:47:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:00 smithi089 ceph-mon[134090]: pgmap v285: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-20T21:47:00.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:00 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3807191763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:47:00.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:00 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1100977990' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-20T21:47:00.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.794+0000 7f5ab7dff700 1 -- 172.21.15.22:0/718807326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ab002d010 con 0x7f5ab0100dd0 2023-09-20T21:47:00.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.795+0000 7f5aa6ffd700 1 -- 172.21.15.22:0/718807326 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5aac017e90 con 0x7f5ab0100dd0 2023-09-20T21:47:00.795 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:47:00.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.797+0000 7f5ab7dff700 1 -- 172.21.15.22:0/718807326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5a9c0a1300 msgr2=0x7f5a9c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:00.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.797+0000 7f5ab7dff700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5a9c0a1300 0x7f5a9c0a37c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b580 tx=0x7f5aa0005fb0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.797+0000 7f5ab7dff700 1 -- 172.21.15.22:0/718807326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab0100dd0 msgr2=0x7f5ab0074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:00.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.797+0000 7f5ab7dff700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab0100dd0 0x7f5ab0074b90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00e9c0 tx=0x7f5aac00ecd0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.798+0000 7f5ab7dff700 1 -- 172.21.15.22:0/718807326 shutdown_connections 2023-09-20T21:47:00.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.798+0000 7f5ab7dff700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ab0102960 0x7f5ab0073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.798+0000 7f5ab7dff700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5a9c0a1300 0x7f5a9c0a37c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.798+0000 7f5ab7dff700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ab0100dd0 0x7f5ab0074b90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.798+0000 7f5ab7dff700 1 --2- 172.21.15.22:0/718807326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0101fc0 0x7f5ab00731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:00.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.798+0000 7f5ab7dff700 1 -- 172.21.15.22:0/718807326 >> 172.21.15.22:0/718807326 conn(0x7f5ab00fc320 msgr2=0x7f5ab00fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:00.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.798+0000 7f5ab7dff700 1 -- 172.21.15.22:0/718807326 shutdown_connections 2023-09-20T21:47:00.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:00.798+0000 7f5ab7dff700 1 -- 172.21.15.22:0/718807326 wait complete. 2023-09-20T21:47:00.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T21:47:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.023+0000 7f12e71ff700 1 -- 172.21.15.22:0/787945247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12e0074e20 msgr2=0x7f12e0073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.023+0000 7f12e71ff700 1 --2- 172.21.15.22:0/787945247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12e0074e20 0x7f12e0073270 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009990 tx=0x7f12d4009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.023+0000 7f12e71ff700 1 -- 172.21.15.22:0/787945247 shutdown_connections 2023-09-20T21:47:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.023+0000 7f12e71ff700 1 --2- 172.21.15.22:0/787945247 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12e0108c90 0x7f12e010b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.023+0000 7f12e71ff700 1 --2- 172.21.15.22:0/787945247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12e0074e20 0x7f12e0073270 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.023+0000 7f12e71ff700 1 --2- 172.21.15.22:0/787945247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12e0073840 0x7f12e0073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.023+0000 7f12e71ff700 1 -- 172.21.15.22:0/787945247 >> 172.21.15.22:0/787945247 conn(0x7f12e00fc320 msgr2=0x7f12e00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.023+0000 7f12e71ff700 1 -- 172.21.15.22:0/787945247 shutdown_connections 2023-09-20T21:47:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.023+0000 7f12e71ff700 1 -- 172.21.15.22:0/787945247 wait complete. 2023-09-20T21:47:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.024+0000 7f12e71ff700 1 Processor -- start 2023-09-20T21:47:01.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.024+0000 7f12e71ff700 1 -- start start 2023-09-20T21:47:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.024+0000 7f12e71ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12e0073840 0x7f12e0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e71ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12e0074e20 0x7f12e0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e71ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12e0108c90 0x7f12e01a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e71ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e01a0890 con 0x7f12e0108c90 2023-09-20T21:47:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e71ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e01a0a00 con 0x7f12e0073840 2023-09-20T21:47:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e71ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e01a0bc0 con 0x7f12e0074e20 2023-09-20T21:47:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e579c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12e0108c90 0x7f12e01a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e579c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12e0108c90 0x7f12e01a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41334/0 (socket says 172.21.15.22:41334) 2023-09-20T21:47:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e579c700 1 -- 172.21.15.22:0/877485132 learned_addr learned my addr 172.21.15.22:0/877485132 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e4f9b700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12e0073840 0x7f12e0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12dffff700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12e0074e20 0x7f12e0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e579c700 1 -- 172.21.15.22:0/877485132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12e0074e20 msgr2=0x7f12e0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e579c700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12e0074e20 0x7f12e0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e579c700 1 -- 172.21.15.22:0/877485132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12e0073840 msgr2=0x7f12e0198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e579c700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12e0073840 0x7f12e0198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e579c700 1 -- 172.21.15.22:0/877485132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d40095f0 con 0x7f12e0108c90 2023-09-20T21:47:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.025+0000 7f12e4f9b700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12e0073840 0x7f12e0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.026+0000 7f12e579c700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12e0108c90 0x7f12e01a02c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f12d800ef30 tx=0x7f12d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.026+0000 7f12ddffb700 1 -- 172.21.15.22:0/877485132 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8018070 con 0x7f12e0108c90 2023-09-20T21:47:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.026+0000 7f12ddffb700 1 -- 172.21.15.22:0/877485132 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12d800f040 con 0x7f12e0108c90 2023-09-20T21:47:01.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.026+0000 7f12e71ff700 1 -- 172.21.15.22:0/877485132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e01a0e40 con 0x7f12e0108c90 2023-09-20T21:47:01.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.026+0000 7f12ddffb700 1 -- 172.21.15.22:0/877485132 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d800cd00 con 0x7f12e0108c90 2023-09-20T21:47:01.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.026+0000 7f12e71ff700 1 -- 172.21.15.22:0/877485132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e01a12c0 con 0x7f12e0108c90 2023-09-20T21:47:01.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.027+0000 7f12e71ff700 1 -- 172.21.15.22:0/877485132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e0107340 con 0x7f12e0108c90 2023-09-20T21:47:01.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.029+0000 7f12ddffb700 1 -- 172.21.15.22:0/877485132 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f12d8003a00 con 0x7f12e0108c90 2023-09-20T21:47:01.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.031+0000 7f12ddffb700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12d00a1380 0x7f12d00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.031+0000 7f12ddffb700 1 -- 172.21.15.22:0/877485132 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f12d80cc910 con 0x7f12e0108c90 2023-09-20T21:47:01.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.031+0000 7f12e4f9b700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12d00a1380 0x7f12d00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.032+0000 7f12e4f9b700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12d00a1380 0x7f12d00a3840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12d400b2e0 tx=0x7f12d4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:01.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.035+0000 7f12ddffb700 1 -- 172.21.15.22:0/877485132 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f12d8095e20 con 0x7f12e0108c90 2023-09-20T21:47:01.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.249+0000 7f12e71ff700 1 -- 172.21.15.22:0/877485132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12e002d170 con 0x7f12e0108c90 2023-09-20T21:47:01.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.249+0000 7f12ddffb700 1 -- 172.21.15.22:0/877485132 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12d8099440 con 0x7f12e0108c90 2023-09-20T21:47:01.250 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:47:01.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 -- 172.21.15.22:0/877485132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12d00a1380 msgr2=0x7f12d00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12d00a1380 0x7f12d00a3840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12d400b2e0 tx=0x7f12d4005fb0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 -- 172.21.15.22:0/877485132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12e0108c90 msgr2=0x7f12e01a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12e0108c90 0x7f12e01a02c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f12d800ef30 tx=0x7f12d800c3a0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 -- 172.21.15.22:0/877485132 shutdown_connections 2023-09-20T21:47:01.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12e0108c90 0x7f12e01a02c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f12d00a1380 0x7f12d00a3840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f12e0073840 0x7f12e0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 --2- 172.21.15.22:0/877485132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12e0074e20 0x7f12e0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 -- 172.21.15.22:0/877485132 >> 172.21.15.22:0/877485132 conn(0x7f12e00fc320 msgr2=0x7f12e00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:01.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 -- 172.21.15.22:0/877485132 shutdown_connections 2023-09-20T21:47:01.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.252+0000 7f12e71ff700 1 -- 172.21.15.22:0/877485132 wait complete. 2023-09-20T21:47:01.271 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T21:47:01.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.478+0000 7f4e229ff700 1 -- 172.21.15.22:0/2411524597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c102960 msgr2=0x7f4e1c109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.478+0000 7f4e229ff700 1 --2- 172.21.15.22:0/2411524597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c102960 0x7f4e1c109210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400b0d0 tx=0x7f4e1400b3e0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.478+0000 7f4e229ff700 1 -- 172.21.15.22:0/2411524597 shutdown_connections 2023-09-20T21:47:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.478+0000 7f4e229ff700 1 --2- 172.21.15.22:0/2411524597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c102960 0x7f4e1c109210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.478+0000 7f4e229ff700 1 --2- 172.21.15.22:0/2411524597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e1c100dd0 0x7f4e1c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.478+0000 7f4e229ff700 1 --2- 172.21.15.22:0/2411524597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e1c101fc0 0x7f4e1c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.478+0000 7f4e229ff700 1 -- 172.21.15.22:0/2411524597 >> 172.21.15.22:0/2411524597 conn(0x7f4e1c0fc320 msgr2=0x7f4e1c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:01.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.478+0000 7f4e229ff700 1 -- 172.21.15.22:0/2411524597 shutdown_connections 2023-09-20T21:47:01.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.478+0000 7f4e229ff700 1 -- 172.21.15.22:0/2411524597 wait complete. 2023-09-20T21:47:01.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.479+0000 7f4e229ff700 1 Processor -- start 2023-09-20T21:47:01.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.479+0000 7f4e229ff700 1 -- start start 2023-09-20T21:47:01.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.479+0000 7f4e229ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e1c100dd0 0x7f4e1c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.479+0000 7f4e229ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e1c101fc0 0x7f4e1c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.479+0000 7f4e229ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c102960 0x7f4e1c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.479+0000 7f4e229ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c1a0a10 con 0x7f4e1c102960 2023-09-20T21:47:01.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.479+0000 7f4e229ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c1a0b50 con 0x7f4e1c101fc0 2023-09-20T21:47:01.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.479+0000 7f4e229ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c1a0ce0 con 0x7f4e1c100dd0 2023-09-20T21:47:01.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e20f9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c102960 0x7f4e1c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e1b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e1c101fc0 0x7f4e1c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e1b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e1c101fc0 0x7f4e1c1993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:33228/0 (socket says 172.21.15.22:33228) 2023-09-20T21:47:01.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e20f9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c102960 0x7f4e1c1a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41346/0 (socket says 172.21.15.22:41346) 2023-09-20T21:47:01.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e1b7fe700 1 -- 172.21.15.22:0/1783735777 learned_addr learned my addr 172.21.15.22:0/1783735777 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:01.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e1bfff700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e1c100dd0 0x7f4e1c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e1b7fe700 1 -- 172.21.15.22:0/1783735777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e1c100dd0 msgr2=0x7f4e1c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e1b7fe700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e1c100dd0 0x7f4e1c198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e1b7fe700 1 -- 172.21.15.22:0/1783735777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c102960 msgr2=0x7f4e1c1a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e1b7fe700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c102960 0x7f4e1c1a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e1b7fe700 1 -- 172.21.15.22:0/1783735777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c009520 con 0x7f4e1c101fc0 2023-09-20T21:47:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e20f9c700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c102960 0x7f4e1c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.480+0000 7f4e1b7fe700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e1c101fc0 0x7f4e1c1993d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000e9a0 tx=0x7f4e1000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:01.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.481+0000 7f4e197fa700 1 -- 172.21.15.22:0/1783735777 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10009790 con 0x7f4e1c101fc0 2023-09-20T21:47:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.481+0000 7f4e229ff700 1 -- 172.21.15.22:0/1783735777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e14009d00 con 0x7f4e1c101fc0 2023-09-20T21:47:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.481+0000 7f4e197fa700 1 -- 172.21.15.22:0/1783735777 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e10010430 con 0x7f4e1c101fc0 2023-09-20T21:47:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.481+0000 7f4e197fa700 1 -- 172.21.15.22:0/1783735777 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10018680 con 0x7f4e1c101fc0 2023-09-20T21:47:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.481+0000 7f4e229ff700 1 -- 172.21.15.22:0/1783735777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1c1a1300 con 0x7f4e1c101fc0 2023-09-20T21:47:01.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.482+0000 7f4e229ff700 1 -- 172.21.15.22:0/1783735777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1c193430 con 0x7f4e1c101fc0 2023-09-20T21:47:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.484+0000 7f4e197fa700 1 -- 172.21.15.22:0/1783735777 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f4e100105a0 con 0x7f4e1c101fc0 2023-09-20T21:47:01.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.485+0000 7f4e197fa700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4e040a1300 0x7f4e040a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.485+0000 7f4e197fa700 1 -- 172.21.15.22:0/1783735777 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4e10014070 con 0x7f4e1c101fc0 2023-09-20T21:47:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.485+0000 7f4e1bfff700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4e040a1300 0x7f4e040a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.486+0000 7f4e1bfff700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4e040a1300 0x7f4e040a37c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c101e20 tx=0x7f4e0c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:01.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.487+0000 7f4e197fa700 1 -- 172.21.15.22:0/1783735777 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4e10096390 con 0x7f4e1c101fc0 2023-09-20T21:47:01.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:01 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3340218249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:01.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:01 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/718807326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:47:01.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:01 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3340218249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:01.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:01 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/718807326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:47:01.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:01 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3340218249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:01.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:01 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/718807326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:47:01.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.702+0000 7f4e229ff700 1 -- 172.21.15.22:0/1783735777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e1c10c640 con 0x7f4e1c101fc0 2023-09-20T21:47:01.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.702+0000 7f4e197fa700 1 -- 172.21.15.22:0/1783735777 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e100999b0 con 0x7f4e1c101fc0 2023-09-20T21:47:01.703 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T21:47:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 -- 172.21.15.22:0/1783735777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4e040a1300 msgr2=0x7f4e040a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4e040a1300 0x7f4e040a37c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c101e20 tx=0x7f4e0c009260 comp rx=0 tx=0).stop 2023-09-20T21:47:01.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 -- 172.21.15.22:0/1783735777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e1c101fc0 msgr2=0x7f4e1c1993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e1c101fc0 0x7f4e1c1993d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000e9a0 tx=0x7f4e1000ecb0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 -- 172.21.15.22:0/1783735777 shutdown_connections 2023-09-20T21:47:01.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e1c102960 0x7f4e1c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e1c100dd0 0x7f4e1c198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4e040a1300 0x7f4e040a37c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 --2- 172.21.15.22:0/1783735777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e1c101fc0 0x7f4e1c1993d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 -- 172.21.15.22:0/1783735777 >> 172.21.15.22:0/1783735777 conn(0x7f4e1c0fc320 msgr2=0x7f4e1c1078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:01.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 -- 172.21.15.22:0/1783735777 shutdown_connections 2023-09-20T21:47:01.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.705+0000 7f4e229ff700 1 -- 172.21.15.22:0/1783735777 wait complete. 2023-09-20T21:47:01.723 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T21:47:01.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.929+0000 7f78b3fff700 1 -- 172.21.15.22:0/2162769965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78ac100dd0 msgr2=0x7f78ac1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.929+0000 7f78b3fff700 1 --2- 172.21.15.22:0/2162769965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78ac100dd0 0x7f78ac1011f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009990 tx=0x7f78a8009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.930+0000 7f78b3fff700 1 -- 172.21.15.22:0/2162769965 shutdown_connections 2023-09-20T21:47:01.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.930+0000 7f78b3fff700 1 --2- 172.21.15.22:0/2162769965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78ac102960 0x7f78ac109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.930+0000 7f78b3fff700 1 --2- 172.21.15.22:0/2162769965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78ac100dd0 0x7f78ac1011f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.930+0000 7f78b3fff700 1 --2- 172.21.15.22:0/2162769965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78ac101fc0 0x7f78ac102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.930+0000 7f78b3fff700 1 -- 172.21.15.22:0/2162769965 >> 172.21.15.22:0/2162769965 conn(0x7f78ac0fc320 msgr2=0x7f78ac0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:01.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.930+0000 7f78b3fff700 1 -- 172.21.15.22:0/2162769965 shutdown_connections 2023-09-20T21:47:01.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.930+0000 7f78b3fff700 1 -- 172.21.15.22:0/2162769965 wait complete. 2023-09-20T21:47:01.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b3fff700 1 Processor -- start 2023-09-20T21:47:01.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b3fff700 1 -- start start 2023-09-20T21:47:01.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78ac100dd0 0x7f78ac198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78ac101fc0 0x7f78ac199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78ac102960 0x7f78ac1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b3fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac1a0880 con 0x7f78ac102960 2023-09-20T21:47:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b3fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac1a09f0 con 0x7f78ac101fc0 2023-09-20T21:47:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b3fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac1a0bb0 con 0x7f78ac100dd0 2023-09-20T21:47:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78ac102960 0x7f78ac1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78ac102960 0x7f78ac1a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41366/0 (socket says 172.21.15.22:41366) 2023-09-20T21:47:01.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b259c700 1 -- 172.21.15.22:0/3562871286 learned_addr learned my addr 172.21.15.22:0/3562871286 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b159a700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78ac101fc0 0x7f78ac199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.931+0000 7f78b1d9b700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78ac100dd0 0x7f78ac198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78b259c700 1 -- 172.21.15.22:0/3562871286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78ac100dd0 msgr2=0x7f78ac198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78b259c700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78ac100dd0 0x7f78ac198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78b259c700 1 -- 172.21.15.22:0/3562871286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78ac101fc0 msgr2=0x7f78ac199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78b259c700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78ac101fc0 0x7f78ac199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78b259c700 1 -- 172.21.15.22:0/3562871286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80095f0 con 0x7f78ac102960 2023-09-20T21:47:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78b159a700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78ac101fc0 0x7f78ac199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:01.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78b259c700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78ac102960 0x7f78ac1a02b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f78a400ef30 tx=0x7f78a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:01.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78a2ffd700 1 -- 172.21.15.22:0/3562871286 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4018070 con 0x7f78ac102960 2023-09-20T21:47:01.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78b3fff700 1 -- 172.21.15.22:0/3562871286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac1a0e30 con 0x7f78ac102960 2023-09-20T21:47:01.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78a2ffd700 1 -- 172.21.15.22:0/3562871286 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a400f040 con 0x7f78ac102960 2023-09-20T21:47:01.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78a2ffd700 1 -- 172.21.15.22:0/3562871286 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a400cd00 con 0x7f78ac102960 2023-09-20T21:47:01.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.932+0000 7f78b3fff700 1 -- 172.21.15.22:0/3562871286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac1a1250 con 0x7f78ac102960 2023-09-20T21:47:01.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.934+0000 7f78b3fff700 1 -- 172.21.15.22:0/3562871286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ac193420 con 0x7f78ac102960 2023-09-20T21:47:01.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.936+0000 7f78a2ffd700 1 -- 172.21.15.22:0/3562871286 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f78a4003a00 con 0x7f78ac102960 2023-09-20T21:47:01.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.937+0000 7f78a2ffd700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f78980a1380 0x7f78980a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:01.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.937+0000 7f78a2ffd700 1 -- 172.21.15.22:0/3562871286 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f78a40cc960 con 0x7f78ac102960 2023-09-20T21:47:01.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.937+0000 7f78b1d9b700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f78980a1380 0x7f78980a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:01.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.938+0000 7f78b1d9b700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f78980a1380 0x7f78980a3840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f78a8000c00 tx=0x7f78a8005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:01.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:01.941+0000 7f78a2ffd700 1 -- 172.21.15.22:0/3562871286 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f78a4095e70 con 0x7f78ac102960 2023-09-20T21:47:02.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.153+0000 7f78b3fff700 1 -- 172.21.15.22:0/3562871286 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f78ac02d140 con 0x7f78ac102960 2023-09-20T21:47:02.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.154+0000 7f78a2ffd700 1 -- 172.21.15.22:0/3562871286 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f78a4099490 con 0x7f78ac102960 2023-09-20T21:47:02.155 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:47:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 -- 172.21.15.22:0/3562871286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f78980a1380 msgr2=0x7f78980a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f78980a1380 0x7f78980a3840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f78a8000c00 tx=0x7f78a8005fb0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 -- 172.21.15.22:0/3562871286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78ac102960 msgr2=0x7f78ac1a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:02.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78ac102960 0x7f78ac1a02b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f78a400ef30 tx=0x7f78a400c3a0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 -- 172.21.15.22:0/3562871286 shutdown_connections 2023-09-20T21:47:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78ac102960 0x7f78ac1a02b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78ac100dd0 0x7f78ac198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f78980a1380 0x7f78980a3840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 --2- 172.21.15.22:0/3562871286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78ac101fc0 0x7f78ac199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 -- 172.21.15.22:0/3562871286 >> 172.21.15.22:0/3562871286 conn(0x7f78ac0fc320 msgr2=0x7f78ac1078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:02.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 -- 172.21.15.22:0/3562871286 shutdown_connections 2023-09-20T21:47:02.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.156+0000 7f78b3fff700 1 -- 172.21.15.22:0/3562871286 wait complete. 2023-09-20T21:47:02.176 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-20T21:47:02.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.380+0000 7f00653ff700 1 -- 172.21.15.22:0/2194662283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060100dd0 msgr2=0x7f00601011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.380+0000 7f00653ff700 1 --2- 172.21.15.22:0/2194662283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060100dd0 0x7f00601011f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f00500099c0 tx=0x7f0050009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.380+0000 7f00653ff700 1 -- 172.21.15.22:0/2194662283 shutdown_connections 2023-09-20T21:47:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.380+0000 7f00653ff700 1 --2- 172.21.15.22:0/2194662283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0060102960 0x7f0060109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.380+0000 7f00653ff700 1 --2- 172.21.15.22:0/2194662283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060100dd0 0x7f00601011f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.380+0000 7f00653ff700 1 --2- 172.21.15.22:0/2194662283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0060101fc0 0x7f0060102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.381+0000 7f00653ff700 1 -- 172.21.15.22:0/2194662283 >> 172.21.15.22:0/2194662283 conn(0x7f00600fc320 msgr2=0x7f00600fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:02.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.381+0000 7f00653ff700 1 -- 172.21.15.22:0/2194662283 shutdown_connections 2023-09-20T21:47:02.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.381+0000 7f00653ff700 1 -- 172.21.15.22:0/2194662283 wait complete. 2023-09-20T21:47:02.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f00653ff700 1 Processor -- start 2023-09-20T21:47:02.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f00653ff700 1 -- start start 2023-09-20T21:47:02.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f00653ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060100dd0 0x7f0060196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:02.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f00653ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0060101fc0 0x7f0060197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:02.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f00653ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0060102960 0x7f006019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:02.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f00653ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006019e6a0 con 0x7f0060102960 2023-09-20T21:47:02.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f00653ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006019e7e0 con 0x7f0060100dd0 2023-09-20T21:47:02.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f00653ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006019e970 con 0x7f0060101fc0 2023-09-20T21:47:02.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f005effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060100dd0 0x7f0060196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:02.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f005effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060100dd0 0x7f0060196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:33258/0 (socket says 172.21.15.22:33258) 2023-09-20T21:47:02.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f005effd700 1 -- 172.21.15.22:0/1883402610 learned_addr learned my addr 172.21.15.22:0/1883402610 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:02.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.382+0000 7f005f7fe700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0060102960 0x7f006019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:02.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f005e7fc700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0060101fc0 0x7f0060197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:02.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f005effd700 1 -- 172.21.15.22:0/1883402610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0060101fc0 msgr2=0x7f0060197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:02.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f005effd700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0060101fc0 0x7f0060197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f005effd700 1 -- 172.21.15.22:0/1883402610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0060102960 msgr2=0x7f006019e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:02.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f005effd700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0060102960 0x7f006019e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f005effd700 1 -- 172.21.15.22:0/1883402610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00500095f0 con 0x7f0060100dd0 2023-09-20T21:47:02.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f005effd700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060100dd0 0x7f0060196b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0050009fd0 tx=0x7f005000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:02.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f0047fff700 1 -- 172.21.15.22:0/1883402610 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005001d070 con 0x7f0060100dd0 2023-09-20T21:47:02.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f00653ff700 1 -- 172.21.15.22:0/1883402610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006019ebf0 con 0x7f0060100dd0 2023-09-20T21:47:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f0047fff700 1 -- 172.21.15.22:0/1883402610 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0050005050 con 0x7f0060100dd0 2023-09-20T21:47:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.383+0000 7f0047fff700 1 -- 172.21.15.22:0/1883402610 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050004a70 con 0x7f0060100dd0 2023-09-20T21:47:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.384+0000 7f00653ff700 1 -- 172.21.15.22:0/1883402610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006019f020 con 0x7f0060100dd0 2023-09-20T21:47:02.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.384+0000 7f00653ff700 1 -- 172.21.15.22:0/1883402610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0060107d80 con 0x7f0060100dd0 2023-09-20T21:47:02.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.386+0000 7f0047fff700 1 -- 172.21.15.22:0/1883402610 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f0050005450 con 0x7f0060100dd0 2023-09-20T21:47:02.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.388+0000 7f0047fff700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f00480a1300 0x7f00480a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:02.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.388+0000 7f0047fff700 1 -- 172.21.15.22:0/1883402610 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f00500cdbf0 con 0x7f0060100dd0 2023-09-20T21:47:02.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.388+0000 7f005e7fc700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f00480a1300 0x7f00480a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:02.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.388+0000 7f005e7fc700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f00480a1300 0x7f00480a37c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0054009c60 tx=0x7f0054009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:02.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.390+0000 7f0047fff700 1 -- 172.21.15.22:0/1883402610 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0050097100 con 0x7f0060100dd0 2023-09-20T21:47:02.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:02 smithi022 ceph-mon[97651]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-20T21:47:02.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:02 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/877485132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:47:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:02 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1783735777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:47:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:02 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3562871286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:47:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:02 smithi123 ceph-mon[134433]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-20T21:47:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:02 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/877485132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:47:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:02 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1783735777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:47:02.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:02 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3562871286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:47:02.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:02 smithi089 ceph-mon[134090]: pgmap v286: 129 pgs: 129 active+clean; 579 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-09-20T21:47:02.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:02 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/877485132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:47:02.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:02 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1783735777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:47:02.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:02 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3562871286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:47:02.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.617+0000 7f00653ff700 1 -- 172.21.15.22:0/1883402610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f006002d120 con 0x7f0060100dd0 2023-09-20T21:47:02.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.618+0000 7f0047fff700 1 -- 172.21.15.22:0/1883402610 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+591 (secure 0 0 0) 0x7f005009a720 con 0x7f0060100dd0 2023-09-20T21:47:02.618 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3260510802 2023-09-21T21:40:07.017182+0000 2023-09-20T21:47:02.619 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3260510802 2023-09-21T21:40:07.017182+0000 2023-09-20T21:47:02.619 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/1128310681 2023-09-21T21:39:39.363251+0000 2023-09-20T21:47:02.619 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3419565055 2023-09-21T21:40:07.017182+0000 2023-09-20T21:47:02.619 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/1128310681 2023-09-21T21:39:39.363251+0000 2023-09-20T21:47:02.619 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/4209005662 2023-09-21T21:39:39.363251+0000 2023-09-20T21:47:02.620 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1007451083 2023-09-21T21:39:39.363251+0000 2023-09-20T21:47:02.620 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1994934339 2023-09-21T21:40:07.017182+0000 2023-09-20T21:47:02.620 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/875874054 2023-09-21T21:40:07.017182+0000 2023-09-20T21:47:02.620 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2863894527 2023-09-21T21:39:39.363251+0000 2023-09-20T21:47:02.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.620+0000 7f00653ff700 1 -- 172.21.15.22:0/1883402610 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f00480a1300 msgr2=0x7f00480a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:02.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.620+0000 7f00653ff700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f00480a1300 0x7f00480a37c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0054009c60 tx=0x7f0054009260 comp rx=0 tx=0).stop 2023-09-20T21:47:02.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.620+0000 7f00653ff700 1 -- 172.21.15.22:0/1883402610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060100dd0 msgr2=0x7f0060196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:02.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.621+0000 7f00653ff700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060100dd0 0x7f0060196b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0050009fd0 tx=0x7f005000f690 comp rx=0 tx=0).stop 2023-09-20T21:47:02.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.621+0000 7f00653ff700 1 -- 172.21.15.22:0/1883402610 shutdown_connections 2023-09-20T21:47:02.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.621+0000 7f00653ff700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0060102960 0x7f006019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.621+0000 7f00653ff700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f00480a1300 0x7f00480a37c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.621+0000 7f00653ff700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0060100dd0 0x7f0060196b20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.621+0000 7f00653ff700 1 --2- 172.21.15.22:0/1883402610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0060101fc0 0x7f0060197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:02.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.621+0000 7f00653ff700 1 -- 172.21.15.22:0/1883402610 >> 172.21.15.22:0/1883402610 conn(0x7f00600fc320 msgr2=0x7f00600fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:02.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.621+0000 7f00653ff700 1 -- 172.21.15.22:0/1883402610 shutdown_connections 2023-09-20T21:47:02.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:02.621+0000 7f00653ff700 1 -- 172.21.15.22:0/1883402610 wait complete. 2023-09-20T21:47:02.624 INFO:teuthology.orchestra.run.smithi022.stderr:listed 10 entries 2023-09-20T21:47:02.639 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi022.front.sepia.ceph.com... 2023-09-20T21:47:02.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:47:02.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-09-20T21:47:02.706 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi022.stderr:ceph-fuse[151459]: fuse finished with error 0 and tester_r 0 2023-09-20T21:47:02.710 INFO:teuthology.orchestra.run:waiting for 300 2023-09-20T21:47:02.768 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-09-20T21:47:02.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:47:02.769 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-09-20T21:47:02.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:47:03.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.026+0000 7f93941ff700 1 -- 172.21.15.22:0/1926064460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f938c101fd0 msgr2=0x7f938c102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.026+0000 7f93941ff700 1 --2- 172.21.15.22:0/1926064460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f938c101fd0 0x7f938c102430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f937c0099c0 tx=0x7f937c009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.026+0000 7f93941ff700 1 -- 172.21.15.22:0/1926064460 shutdown_connections 2023-09-20T21:47:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.026+0000 7f93941ff700 1 --2- 172.21.15.22:0/1926064460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f938c102970 0x7f938c109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.026+0000 7f93941ff700 1 --2- 172.21.15.22:0/1926064460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c100de0 0x7f938c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.026+0000 7f93941ff700 1 --2- 172.21.15.22:0/1926064460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f938c101fd0 0x7f938c102430 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.026+0000 7f93941ff700 1 -- 172.21.15.22:0/1926064460 >> 172.21.15.22:0/1926064460 conn(0x7f938c0fc310 msgr2=0x7f938c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:03.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.026+0000 7f93941ff700 1 -- 172.21.15.22:0/1926064460 shutdown_connections 2023-09-20T21:47:03.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.026+0000 7f93941ff700 1 -- 172.21.15.22:0/1926064460 wait complete. 2023-09-20T21:47:03.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.027+0000 7f93941ff700 1 Processor -- start 2023-09-20T21:47:03.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.027+0000 7f93941ff700 1 -- start start 2023-09-20T21:47:03.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.027+0000 7f93941ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f938c100de0 0x7f938c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.027+0000 7f93941ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f938c101fd0 0x7f938c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.027+0000 7f93941ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c102970 0x7f938c1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f93941ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1a0800 con 0x7f938c100de0 2023-09-20T21:47:03.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f93941ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1a0970 con 0x7f938c101fd0 2023-09-20T21:47:03.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f93941ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1a0b30 con 0x7f938c102970 2023-09-20T21:47:03.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f9391f9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f938c100de0 0x7f938c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f9391f9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f938c100de0 0x7f938c198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41396/0 (socket says 172.21.15.22:41396) 2023-09-20T21:47:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f939179a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f938c101fd0 0x7f938c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f9391f9b700 1 -- 172.21.15.22:0/3937754430 learned_addr learned my addr 172.21.15.22:0/3937754430 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f939279c700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c102970 0x7f938c1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f9391f9b700 1 -- 172.21.15.22:0/3937754430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c102970 msgr2=0x7f938c1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f9391f9b700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c102970 0x7f938c1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f9391f9b700 1 -- 172.21.15.22:0/3937754430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f938c101fd0 msgr2=0x7f938c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f9391f9b700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f938c101fd0 0x7f938c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f9391f9b700 1 -- 172.21.15.22:0/3937754430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c0095f0 con 0x7f938c100de0 2023-09-20T21:47:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f939179a700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f938c101fd0 0x7f938c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:03.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.028+0000 7f9391f9b700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f938c100de0 0x7f938c198c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f938800e970 tx=0x7f938800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.029+0000 7f9382ffd700 1 -- 172.21.15.22:0/3937754430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388009790 con 0x7f938c100de0 2023-09-20T21:47:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.029+0000 7f9382ffd700 1 -- 172.21.15.22:0/3937754430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f938800c7e0 con 0x7f938c100de0 2023-09-20T21:47:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.029+0000 7f93941ff700 1 -- 172.21.15.22:0/3937754430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c1a0e10 con 0x7f938c100de0 2023-09-20T21:47:03.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.029+0000 7f9382ffd700 1 -- 172.21.15.22:0/3937754430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388003c70 con 0x7f938c100de0 2023-09-20T21:47:03.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.029+0000 7f93941ff700 1 -- 172.21.15.22:0/3937754430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c1a1240 con 0x7f938c100de0 2023-09-20T21:47:03.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.030+0000 7f93941ff700 1 -- 172.21.15.22:0/3937754430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938c10c640 con 0x7f938c100de0 2023-09-20T21:47:03.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.032+0000 7f9382ffd700 1 -- 172.21.15.22:0/3937754430 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f938800cc50 con 0x7f938c100de0 2023-09-20T21:47:03.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.033+0000 7f9382ffd700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f93780a13d0 0x7f93780a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.033+0000 7f9382ffd700 1 -- 172.21.15.22:0/3937754430 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9388014070 con 0x7f938c100de0 2023-09-20T21:47:03.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.034+0000 7f939179a700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f93780a13d0 0x7f93780a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.034+0000 7f939179a700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f93780a13d0 0x7f93780a3890 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f937c00b580 tx=0x7f937c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:03.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.037+0000 7f9382ffd700 1 -- 172.21.15.22:0/3937754430 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f93880d0030 con 0x7f938c100de0 2023-09-20T21:47:03.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.251+0000 7f93941ff700 1 -- 172.21.15.22:0/3937754430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f938c10c850 con 0x7f938c100de0 2023-09-20T21:47:03.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.252+0000 7f9382ffd700 1 -- 172.21.15.22:0/3937754430 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23913 (secure 0 0 0) 0x7f9388096450 con 0x7f938c100de0 2023-09-20T21:47:03.253 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:03.254 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:46:46.091544+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:47:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 -- 172.21.15.22:0/3937754430 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f93780a13d0 msgr2=0x7f93780a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f93780a13d0 0x7f93780a3890 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f937c00b580 tx=0x7f937c005fb0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 -- 172.21.15.22:0/3937754430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f938c100de0 msgr2=0x7f938c198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f938c100de0 0x7f938c198c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f938800e970 tx=0x7f938800ec80 comp rx=0 tx=0).stop 2023-09-20T21:47:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 -- 172.21.15.22:0/3937754430 shutdown_connections 2023-09-20T21:47:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f938c102970 0x7f938c1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f938c100de0 0x7f938c198c90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f93780a13d0 0x7f93780a3890 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 --2- 172.21.15.22:0/3937754430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f938c101fd0 0x7f938c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 -- 172.21.15.22:0/3937754430 >> 172.21.15.22:0/3937754430 conn(0x7f938c0fc310 msgr2=0x7f938c107850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 -- 172.21.15.22:0/3937754430 shutdown_connections 2023-09-20T21:47:03.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.255+0000 7f93941ff700 1 -- 172.21.15.22:0/3937754430 wait complete. 2023-09-20T21:47:03.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:47:03.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.478+0000 7faaf2bff700 1 -- 172.21.15.22:0/460848819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaec102970 msgr2=0x7faaec109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.478+0000 7faaf2bff700 1 --2- 172.21.15.22:0/460848819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaec102970 0x7faaec109220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faae400b0d0 tx=0x7faae400b3e0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.478+0000 7faaf2bff700 1 -- 172.21.15.22:0/460848819 shutdown_connections 2023-09-20T21:47:03.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.478+0000 7faaf2bff700 1 --2- 172.21.15.22:0/460848819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaec102970 0x7faaec109220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.478+0000 7faaf2bff700 1 --2- 172.21.15.22:0/460848819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec100de0 0x7faaec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.478+0000 7faaf2bff700 1 --2- 172.21.15.22:0/460848819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaec101fd0 0x7faaec102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.478+0000 7faaf2bff700 1 -- 172.21.15.22:0/460848819 >> 172.21.15.22:0/460848819 conn(0x7faaec0fc310 msgr2=0x7faaec0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:03.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.478+0000 7faaf2bff700 1 -- 172.21.15.22:0/460848819 shutdown_connections 2023-09-20T21:47:03.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.478+0000 7faaf2bff700 1 -- 172.21.15.22:0/460848819 wait complete. 2023-09-20T21:47:03.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.479+0000 7faaf2bff700 1 Processor -- start 2023-09-20T21:47:03.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaf2bff700 1 -- start start 2023-09-20T21:47:03.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaf2bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec100de0 0x7faaec198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaf2bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaec101fd0 0x7faaec199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaf2bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaec102970 0x7faaec1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaf2bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec1a0850 con 0x7faaec101fd0 2023-09-20T21:47:03.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaf2bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec1a09c0 con 0x7faaec102970 2023-09-20T21:47:03.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaf2bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec1a0b80 con 0x7faaec100de0 2023-09-20T21:47:03.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaebfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaec101fd0 0x7faaec199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaf099b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec100de0 0x7faaec198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaebfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaec101fd0 0x7faaec199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41418/0 (socket says 172.21.15.22:41418) 2023-09-20T21:47:03.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaebfff700 1 -- 172.21.15.22:0/1209565130 learned_addr learned my addr 172.21.15.22:0/1209565130 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:03.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.480+0000 7faaf119c700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaec102970 0x7faaec1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.481+0000 7faaf099b700 1 -- 172.21.15.22:0/1209565130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaec102970 msgr2=0x7faaec1a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.481+0000 7faaf099b700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaec102970 0x7faaec1a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.481+0000 7faaf099b700 1 -- 172.21.15.22:0/1209565130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaec101fd0 msgr2=0x7faaec199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.481+0000 7faaf099b700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaec101fd0 0x7faaec199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.481+0000 7faaf099b700 1 -- 172.21.15.22:0/1209565130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0009520 con 0x7faaec100de0 2023-09-20T21:47:03.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.481+0000 7faaebfff700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaec101fd0 0x7faaec199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:03.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.481+0000 7faaf099b700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec100de0 0x7faaec198d10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faae000e9c0 tx=0x7faae000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:03.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.481+0000 7faae9ffb700 1 -- 172.21.15.22:0/1209565130 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae000cb70 con 0x7faaec100de0 2023-09-20T21:47:03.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.481+0000 7faaf2bff700 1 -- 172.21.15.22:0/1209565130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae4009d00 con 0x7faaec100de0 2023-09-20T21:47:03.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.482+0000 7faae9ffb700 1 -- 172.21.15.22:0/1209565130 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faae000ee60 con 0x7faaec100de0 2023-09-20T21:47:03.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.482+0000 7faae9ffb700 1 -- 172.21.15.22:0/1209565130 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0018680 con 0x7faaec100de0 2023-09-20T21:47:03.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.482+0000 7faaf2bff700 1 -- 172.21.15.22:0/1209565130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaec1a1180 con 0x7faaec100de0 2023-09-20T21:47:03.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.483+0000 7faaf2bff700 1 -- 172.21.15.22:0/1209565130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaec10c650 con 0x7faaec100de0 2023-09-20T21:47:03.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.485+0000 7faae9ffb700 1 -- 172.21.15.22:0/1209565130 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7faae0010430 con 0x7faaec100de0 2023-09-20T21:47:03.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.486+0000 7faae9ffb700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faadc0a1350 0x7faadc0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.486+0000 7faae9ffb700 1 -- 172.21.15.22:0/1209565130 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faae0014070 con 0x7faaec100de0 2023-09-20T21:47:03.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.487+0000 7faaebfff700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faadc0a1350 0x7faadc0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.487+0000 7faaebfff700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faadc0a1350 0x7faadc0a3810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faad800f4d0 tx=0x7faad8005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:03.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.488+0000 7faae9ffb700 1 -- 172.21.15.22:0/1209565130 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7faae0096820 con 0x7faaec100de0 2023-09-20T21:47:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.699+0000 7faaf2bff700 1 -- 172.21.15.22:0/1209565130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faaec1a1370 con 0x7faaec100de0 2023-09-20T21:47:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.702+0000 7faae9ffb700 1 -- 172.21.15.22:0/1209565130 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23913 (secure 0 0 0) 0x7faae0099e40 con 0x7faaec100de0 2023-09-20T21:47:03.702 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:03.702 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:46:46.091544+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:47:03.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 -- 172.21.15.22:0/1209565130 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faadc0a1350 msgr2=0x7faadc0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faadc0a1350 0x7faadc0a3810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faad800f4d0 tx=0x7faad8005f90 comp rx=0 tx=0).stop 2023-09-20T21:47:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 -- 172.21.15.22:0/1209565130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec100de0 msgr2=0x7faaec198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec100de0 0x7faaec198d10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faae000e9c0 tx=0x7faae000ecd0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 -- 172.21.15.22:0/1209565130 shutdown_connections 2023-09-20T21:47:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7faadc0a1350 0x7faadc0a3810 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaec102970 0x7faaec1a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec100de0 0x7faaec198d10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 --2- 172.21.15.22:0/1209565130 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaec101fd0 0x7faaec199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 -- 172.21.15.22:0/1209565130 >> 172.21.15.22:0/1209565130 conn(0x7faaec0fc310 msgr2=0x7faaec107780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 -- 172.21.15.22:0/1209565130 shutdown_connections 2023-09-20T21:47:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.704+0000 7faaf2bff700 1 -- 172.21.15.22:0/1209565130 wait complete. 2023-09-20T21:47:03.723 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-20T21:47:03.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:03 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1883402610' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-20T21:47:03.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:03 smithi123 ceph-mon[134433]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-20T21:47:03.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:03 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3937754430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:03.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:03 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1883402610' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-20T21:47:03.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:03 smithi089 ceph-mon[134090]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-20T21:47:03.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:03 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3937754430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.932+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/2657823987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f44101fc0 msgr2=0x7f6f44102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.932+0000 7f6f4b1ff700 1 --2- 172.21.15.22:0/2657823987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f44101fc0 0x7f6f44102420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009920 tx=0x7f6f30009c30 comp rx=0 tx=0).stop 2023-09-20T21:47:03.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.932+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/2657823987 shutdown_connections 2023-09-20T21:47:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.932+0000 7f6f4b1ff700 1 --2- 172.21.15.22:0/2657823987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f44102960 0x7f6f44109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.932+0000 7f6f4b1ff700 1 --2- 172.21.15.22:0/2657823987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f44100dd0 0x7f6f441011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.932+0000 7f6f4b1ff700 1 --2- 172.21.15.22:0/2657823987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f44101fc0 0x7f6f44102420 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.932+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/2657823987 >> 172.21.15.22:0/2657823987 conn(0x7f6f440fc320 msgr2=0x7f6f440fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:03.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.932+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/2657823987 shutdown_connections 2023-09-20T21:47:03.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.932+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/2657823987 wait complete. 2023-09-20T21:47:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.933+0000 7f6f4b1ff700 1 Processor -- start 2023-09-20T21:47:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.933+0000 7f6f4b1ff700 1 -- start start 2023-09-20T21:47:03.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f4b1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f44100dd0 0x7f6f44198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f4b1ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f44101fc0 0x7f6f44199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f4b1ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f44102960 0x7f6f441a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f4b1ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f441a09b0 con 0x7f6f44101fc0 2023-09-20T21:47:03.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f4b1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f441a0af0 con 0x7f6f44100dd0 2023-09-20T21:47:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f4b1ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f441a0c80 con 0x7f6f44102960 2023-09-20T21:47:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f43fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f44101fc0 0x7f6f44199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f43fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f44101fc0 0x7f6f44199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41442/0 (socket says 172.21.15.22:41442) 2023-09-20T21:47:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f43fff700 1 -- 172.21.15.22:0/3652317728 learned_addr learned my addr 172.21.15.22:0/3652317728 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:03.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f48f9b700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f44100dd0 0x7f6f44198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f4979c700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f44102960 0x7f6f441a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f43fff700 1 -- 172.21.15.22:0/3652317728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f44102960 msgr2=0x7f6f441a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f43fff700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f44102960 0x7f6f441a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f43fff700 1 -- 172.21.15.22:0/3652317728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f44100dd0 msgr2=0x7f6f44198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f43fff700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f44100dd0 0x7f6f44198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.934+0000 7f6f43fff700 1 -- 172.21.15.22:0/3652317728 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f30009520 con 0x7f6f44101fc0 2023-09-20T21:47:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.935+0000 7f6f43fff700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f44101fc0 0x7f6f44199370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009230 tx=0x7f6f3000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.935+0000 7f6f41ffb700 1 -- 172.21.15.22:0/3652317728 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3001c070 con 0x7f6f44101fc0 2023-09-20T21:47:03.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.935+0000 7f6f41ffb700 1 -- 172.21.15.22:0/3652317728 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f30005040 con 0x7f6f44101fc0 2023-09-20T21:47:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.935+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/3652317728 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f441a0f00 con 0x7f6f44101fc0 2023-09-20T21:47:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.935+0000 7f6f41ffb700 1 -- 172.21.15.22:0/3652317728 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30003a20 con 0x7f6f44101fc0 2023-09-20T21:47:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.935+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/3652317728 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f441a1270 con 0x7f6f44101fc0 2023-09-20T21:47:03.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.936+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/3652317728 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f44193420 con 0x7f6f44101fc0 2023-09-20T21:47:03.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.938+0000 7f6f41ffb700 1 -- 172.21.15.22:0/3652317728 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f6f30020030 con 0x7f6f44101fc0 2023-09-20T21:47:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.939+0000 7f6f41ffb700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f340a1300 0x7f6f340a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.939+0000 7f6f41ffb700 1 -- 172.21.15.22:0/3652317728 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6f300cde40 con 0x7f6f44101fc0 2023-09-20T21:47:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.940+0000 7f6f48f9b700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f340a1300 0x7f6f340a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.940+0000 7f6f48f9b700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f340a1300 0x7f6f340a37c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f44101e20 tx=0x7f6f38009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:03.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:03.943+0000 7f6f41ffb700 1 -- 172.21.15.22:0/3652317728 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6f30097350 con 0x7f6f44101fc0 2023-09-20T21:47:03.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:03 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1883402610' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-20T21:47:03.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:03 smithi022 ceph-mon[97651]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-20T21:47:03.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:03 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3937754430' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:04.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.201+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/3652317728 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6f4410c630 con 0x7f6f44101fc0 2023-09-20T21:47:04.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.561+0000 7f6f41ffb700 1 -- 172.21.15.22:0/3652317728 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6f3009a970 con 0x7f6f44101fc0 2023-09-20T21:47:04.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/3652317728 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f340a1300 msgr2=0x7f6f340a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:04.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f340a1300 0x7f6f340a37c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f44101e20 tx=0x7f6f38009310 comp rx=0 tx=0).stop 2023-09-20T21:47:04.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/3652317728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f44101fc0 msgr2=0x7f6f44199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f44101fc0 0x7f6f44199370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009230 tx=0x7f6f3000b560 comp rx=0 tx=0).stop 2023-09-20T21:47:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/3652317728 shutdown_connections 2023-09-20T21:47:04.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f44102960 0x7f6f441a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f340a1300 0x7f6f340a37c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f44100dd0 0x7f6f44198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 --2- 172.21.15.22:0/3652317728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f44101fc0 0x7f6f44199370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/3652317728 >> 172.21.15.22:0/3652317728 conn(0x7f6f440fc320 msgr2=0x7f6f44107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:04.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/3652317728 shutdown_connections 2023-09-20T21:47:04.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.564+0000 7f6f4b1ff700 1 -- 172.21.15.22:0/3652317728 wait complete. 2023-09-20T21:47:04.584 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True} 2023-09-20T21:47:04.584 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-09-20T21:47:04.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.791+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/106393640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb28101fe0 msgr2=0x7fbb28102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:04.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.791+0000 7fbb2f7ff700 1 --2- 172.21.15.22:0/106393640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb28101fe0 0x7fbb28102440 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbb180099c0 tx=0x7fbb18009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:04.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.791+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/106393640 shutdown_connections 2023-09-20T21:47:04.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.791+0000 7fbb2f7ff700 1 --2- 172.21.15.22:0/106393640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb28102980 0x7fbb28109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:04.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.791+0000 7fbb2f7ff700 1 --2- 172.21.15.22:0/106393640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28100df0 0x7fbb28101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:04.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.791+0000 7fbb2f7ff700 1 --2- 172.21.15.22:0/106393640 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb28101fe0 0x7fbb28102440 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:04.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.791+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/106393640 >> 172.21.15.22:0/106393640 conn(0x7fbb280fc360 msgr2=0x7fbb280fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:04.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.791+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/106393640 shutdown_connections 2023-09-20T21:47:04.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.791+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/106393640 wait complete. 2023-09-20T21:47:04.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.792+0000 7fbb2f7ff700 1 Processor -- start 2023-09-20T21:47:04.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.792+0000 7fbb2f7ff700 1 -- start start 2023-09-20T21:47:04.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.792+0000 7fbb2f7ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb28100df0 0x7fbb28198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:04.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.792+0000 7fbb2f7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28101fe0 0x7fbb28199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:04.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2f7ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb28102980 0x7fbb281a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:04.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2f7ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb281a0850 con 0x7fbb28100df0 2023-09-20T21:47:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2f7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb281a09c0 con 0x7fbb28101fe0 2023-09-20T21:47:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2f7ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb281a0b80 con 0x7fbb28102980 2023-09-20T21:47:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2d59b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb28100df0 0x7fbb28198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2cd9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28101fe0 0x7fbb28199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:04.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2cd9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28101fe0 0x7fbb28199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:33310/0 (socket says 172.21.15.22:33310) 2023-09-20T21:47:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2cd9a700 1 -- 172.21.15.22:0/1881404105 learned_addr learned my addr 172.21.15.22:0/1881404105 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2dd9c700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb28102980 0x7fbb281a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2cd9a700 1 -- 172.21.15.22:0/1881404105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb28102980 msgr2=0x7fbb281a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2cd9a700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb28102980 0x7fbb281a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:04.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2cd9a700 1 -- 172.21.15.22:0/1881404105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb28100df0 msgr2=0x7fbb28198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:04.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2cd9a700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb28100df0 0x7fbb28198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:04.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.793+0000 7fbb2cd9a700 1 -- 172.21.15.22:0/1881404105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180095f0 con 0x7fbb28101fe0 2023-09-20T21:47:04.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.794+0000 7fbb2cd9a700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28101fe0 0x7fbb28199250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009990 tx=0x7fbb1800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:04.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.794+0000 7fbb1e7fc700 1 -- 172.21.15.22:0/1881404105 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1801d070 con 0x7fbb28101fe0 2023-09-20T21:47:04.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.794+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/1881404105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb281a0e00 con 0x7fbb28101fe0 2023-09-20T21:47:04.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.794+0000 7fbb1e7fc700 1 -- 172.21.15.22:0/1881404105 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb180054f0 con 0x7fbb28101fe0 2023-09-20T21:47:04.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.794+0000 7fbb1e7fc700 1 -- 172.21.15.22:0/1881404105 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18004b00 con 0x7fbb28101fe0 2023-09-20T21:47:04.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.794+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/1881404105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb281a11d0 con 0x7fbb28101fe0 2023-09-20T21:47:04.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.795+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/1881404105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2810c650 con 0x7fbb28101fe0 2023-09-20T21:47:04.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.797+0000 7fbb1e7fc700 1 -- 172.21.15.22:0/1881404105 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fbb1800fb80 con 0x7fbb28101fe0 2023-09-20T21:47:04.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.798+0000 7fbb1e7fc700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb140a1380 0x7fbb140a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:04.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.798+0000 7fbb1e7fc700 1 -- 172.21.15.22:0/1881404105 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbb180cdbd0 con 0x7fbb28101fe0 2023-09-20T21:47:04.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.799+0000 7fbb2d59b700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb140a1380 0x7fbb140a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:04.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.799+0000 7fbb2d59b700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb140a1380 0x7fbb140a3840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbb2819a1e0 tx=0x7fbb24009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:04.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:04.800+0000 7fbb1e7fc700 1 -- 172.21.15.22:0/1881404105 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbb180970e0 con 0x7fbb28101fe0 2023-09-20T21:47:04.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:04 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1209565130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:04.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:04 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3652317728' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-20T21:47:04.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:04 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1209565130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:04.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:04 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3652317728' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-20T21:47:04.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:04 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1209565130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:04.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:04 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3652317728' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-09-20T21:47:05.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.058+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/1881404105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fbb281a1480 con 0x7fbb28101fe0 2023-09-20T21:47:05.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.574+0000 7fbb1e7fc700 1 -- 172.21.15.22:0/1881404105 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fbb180260b0 con 0x7fbb28101fe0 2023-09-20T21:47:05.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.577+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/1881404105 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb140a1380 msgr2=0x7fbb140a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:05.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.577+0000 7fbb2f7ff700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb140a1380 0x7fbb140a3840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbb2819a1e0 tx=0x7fbb24009260 comp rx=0 tx=0).stop 2023-09-20T21:47:05.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.577+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/1881404105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28101fe0 msgr2=0x7fbb28199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:05.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.577+0000 7fbb2f7ff700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28101fe0 0x7fbb28199250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009990 tx=0x7fbb1800f690 comp rx=0 tx=0).stop 2023-09-20T21:47:05.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.577+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/1881404105 shutdown_connections 2023-09-20T21:47:05.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.577+0000 7fbb2f7ff700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb28102980 0x7fbb281a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:05.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.577+0000 7fbb2f7ff700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb28100df0 0x7fbb28198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:05.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.577+0000 7fbb2f7ff700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb140a1380 0x7fbb140a3840 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:05.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.578+0000 7fbb2f7ff700 1 --2- 172.21.15.22:0/1881404105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb28101fe0 0x7fbb28199250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:05.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.578+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/1881404105 >> 172.21.15.22:0/1881404105 conn(0x7fbb280fc360 msgr2=0x7fbb280fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:05.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.578+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/1881404105 shutdown_connections 2023-09-20T21:47:05.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.578+0000 7fbb2f7ff700 1 -- 172.21.15.22:0/1881404105 wait complete. 2023-09-20T21:47:05.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-09-20T21:47:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.811+0000 7ffae03ff700 1 -- 172.21.15.22:0/11001487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad8100dd0 msgr2=0x7ffad81011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.811+0000 7ffae03ff700 1 --2- 172.21.15.22:0/11001487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad8100dd0 0x7ffad81011f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffad40099c0 tx=0x7ffad4009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.811+0000 7ffae03ff700 1 -- 172.21.15.22:0/11001487 shutdown_connections 2023-09-20T21:47:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.811+0000 7ffae03ff700 1 --2- 172.21.15.22:0/11001487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffad8102960 0x7ffad8109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.811+0000 7ffae03ff700 1 --2- 172.21.15.22:0/11001487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad8100dd0 0x7ffad81011f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.811+0000 7ffae03ff700 1 --2- 172.21.15.22:0/11001487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad8101fc0 0x7ffad8102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:05.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.811+0000 7ffae03ff700 1 -- 172.21.15.22:0/11001487 >> 172.21.15.22:0/11001487 conn(0x7ffad80fc320 msgr2=0x7ffad80fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:05.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.811+0000 7ffae03ff700 1 -- 172.21.15.22:0/11001487 shutdown_connections 2023-09-20T21:47:05.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.811+0000 7ffae03ff700 1 -- 172.21.15.22:0/11001487 wait complete. 2023-09-20T21:47:05.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.812+0000 7ffae03ff700 1 Processor -- start 2023-09-20T21:47:05.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.812+0000 7ffae03ff700 1 -- start start 2023-09-20T21:47:05.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffae03ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad8100dd0 0x7ffad8074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:05.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffae03ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad8101fc0 0x7ffad80731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:05.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffae03ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffad8102960 0x7ffad8073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:05.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffae03ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad8073eb0 con 0x7ffad8100dd0 2023-09-20T21:47:05.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffae03ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad8074020 con 0x7ffad8101fc0 2023-09-20T21:47:05.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffae03ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81a0b60 con 0x7ffad8102960 2023-09-20T21:47:05.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffade19b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad8100dd0 0x7ffad8074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:05.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffadd99a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad8101fc0 0x7ffad80731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:05.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffadd99a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad8101fc0 0x7ffad80731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:33330/0 (socket says 172.21.15.22:33330) 2023-09-20T21:47:05.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffadd99a700 1 -- 172.21.15.22:0/1279100401 learned_addr learned my addr 172.21.15.22:0/1279100401 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:05.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffade99c700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffad8102960 0x7ffad8073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:05.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffadd99a700 1 -- 172.21.15.22:0/1279100401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffad8102960 msgr2=0x7ffad8073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:05.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffadd99a700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffad8102960 0x7ffad8073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:05.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffadd99a700 1 -- 172.21.15.22:0/1279100401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad8100dd0 msgr2=0x7ffad8074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:05.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffadd99a700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad8100dd0 0x7ffad8074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:05.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffadd99a700 1 -- 172.21.15.22:0/1279100401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad40095f0 con 0x7ffad8101fc0 2023-09-20T21:47:05.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.813+0000 7ffade19b700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad8100dd0 0x7ffad8074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:05.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.814+0000 7ffadd99a700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad8101fc0 0x7ffad80731d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9a0 tx=0x7ffac800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.814+0000 7ffacf7fe700 1 -- 172.21.15.22:0/1279100401 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8009790 con 0x7ffad8101fc0 2023-09-20T21:47:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.814+0000 7ffae03ff700 1 -- 172.21.15.22:0/1279100401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad81a0e40 con 0x7ffad8101fc0 2023-09-20T21:47:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.814+0000 7ffacf7fe700 1 -- 172.21.15.22:0/1279100401 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac8010430 con 0x7ffad8101fc0 2023-09-20T21:47:05.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.814+0000 7ffacf7fe700 1 -- 172.21.15.22:0/1279100401 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8018680 con 0x7ffad8101fc0 2023-09-20T21:47:05.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.814+0000 7ffae03ff700 1 -- 172.21.15.22:0/1279100401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad81a12c0 con 0x7ffad8101fc0 2023-09-20T21:47:05.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.815+0000 7ffae03ff700 1 -- 172.21.15.22:0/1279100401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad810c5e0 con 0x7ffad8101fc0 2023-09-20T21:47:05.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.817+0000 7ffacf7fe700 1 -- 172.21.15.22:0/1279100401 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ffac80105a0 con 0x7ffad8101fc0 2023-09-20T21:47:05.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.818+0000 7ffacf7fe700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffac40a1300 0x7ffac40a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:05.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.819+0000 7ffacf7fe700 1 -- 172.21.15.22:0/1279100401 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffac8014070 con 0x7ffad8101fc0 2023-09-20T21:47:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.819+0000 7ffade19b700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffac40a1300 0x7ffac40a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.819+0000 7ffade19b700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffac40a1300 0x7ffac40a37c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009fd0 tx=0x7ffad4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:05.820+0000 7ffacf7fe700 1 -- 172.21.15.22:0/1279100401 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffac80966a0 con 0x7ffad8101fc0 2023-09-20T21:47:05.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:05 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3652317728' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-20T21:47:05.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:05 smithi123 ceph-mon[134433]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:47:05.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:05 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1881404105' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-20T21:47:05.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:05 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-20T21:47:05.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:05 smithi123 ceph-mon[134433]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-20T21:47:05.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:05 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3652317728' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-20T21:47:05.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:05 smithi089 ceph-mon[134090]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:47:05.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:05 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1881404105' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-20T21:47:05.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:05 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-20T21:47:05.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:05 smithi089 ceph-mon[134090]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-20T21:47:05.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:05 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3652317728' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-09-20T21:47:05.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:05 smithi022 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:47:05.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:05 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1881404105' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-20T21:47:05.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:05 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-09-20T21:47:05.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:05 smithi022 ceph-mon[97651]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-09-20T21:47:06.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.079+0000 7ffae03ff700 1 -- 172.21.15.22:0/1279100401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7ffad8107d80 con 0x7ffad8101fc0 2023-09-20T21:47:06.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.582+0000 7ffacf7fe700 1 -- 172.21.15.22:0/1279100401 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7ffac8099cc0 con 0x7ffad8101fc0 2023-09-20T21:47:06.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 -- 172.21.15.22:0/1279100401 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffac40a1300 msgr2=0x7ffac40a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:06.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffac40a1300 0x7ffac40a37c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009fd0 tx=0x7ffad4005fb0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 -- 172.21.15.22:0/1279100401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad8101fc0 msgr2=0x7ffad80731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad8101fc0 0x7ffad80731d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9a0 tx=0x7ffac800ecb0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 -- 172.21.15.22:0/1279100401 shutdown_connections 2023-09-20T21:47:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffad8102960 0x7ffad8073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffad8100dd0 0x7ffad8074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffac40a1300 0x7ffac40a37c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 --2- 172.21.15.22:0/1279100401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffad8101fc0 0x7ffad80731d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 -- 172.21.15.22:0/1279100401 >> 172.21.15.22:0/1279100401 conn(0x7ffad80fc320 msgr2=0x7ffad80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:06.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 -- 172.21.15.22:0/1279100401 shutdown_connections 2023-09-20T21:47:06.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.586+0000 7ffae03ff700 1 -- 172.21.15.22:0/1279100401 wait complete. 2023-09-20T21:47:06.613 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-09-20T21:47:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.824+0000 7fc101dff700 1 -- 172.21.15.22:0/143107323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0fc1018b0 msgr2=0x7fc0fc103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.824+0000 7fc101dff700 1 --2- 172.21.15.22:0/143107323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0fc1018b0 0x7fc0fc103ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009990 tx=0x7fc0ec009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.824+0000 7fc101dff700 1 -- 172.21.15.22:0/143107323 shutdown_connections 2023-09-20T21:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.824+0000 7fc101dff700 1 --2- 172.21.15.22:0/143107323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0fc106b10 0x7fc0fc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.824+0000 7fc101dff700 1 --2- 172.21.15.22:0/143107323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0fc1018b0 0x7fc0fc103ca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.824+0000 7fc101dff700 1 --2- 172.21.15.22:0/143107323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0fc1041e0 0x7fc0fc1065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.824+0000 7fc101dff700 1 -- 172.21.15.22:0/143107323 >> 172.21.15.22:0/143107323 conn(0x7fc0fc0fb240 msgr2=0x7fc0fc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.824+0000 7fc101dff700 1 -- 172.21.15.22:0/143107323 shutdown_connections 2023-09-20T21:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.824+0000 7fc101dff700 1 -- 172.21.15.22:0/143107323 wait complete. 2023-09-20T21:47:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.825+0000 7fc101dff700 1 Processor -- start 2023-09-20T21:47:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.825+0000 7fc101dff700 1 -- start start 2023-09-20T21:47:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc101dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0fc1018b0 0x7fc0fc198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc101dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0fc1041e0 0x7fc0fc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc101dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0fc106b10 0x7fc0fc1a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc101dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc1a0890 con 0x7fc0fc1018b0 2023-09-20T21:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc101dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc1a0a00 con 0x7fc0fc1041e0 2023-09-20T21:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc101dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc1a0bc0 con 0x7fc0fc106b10 2023-09-20T21:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc0fb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0fc1018b0 0x7fc0fc198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc0fb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0fc1018b0 0x7fc0fc198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50722/0 (socket says 172.21.15.22:50722) 2023-09-20T21:47:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc0fb7fe700 1 -- 172.21.15.22:0/270276030 learned_addr learned my addr 172.21.15.22:0/270276030 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc0faffd700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0fc1041e0 0x7fc0fc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.826+0000 7fc0fbfff700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0fc106b10 0x7fc0fc1a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc0fb7fe700 1 -- 172.21.15.22:0/270276030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0fc106b10 msgr2=0x7fc0fc1a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc0fb7fe700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0fc106b10 0x7fc0fc1a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc0fb7fe700 1 -- 172.21.15.22:0/270276030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0fc1041e0 msgr2=0x7fc0fc199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc0fb7fe700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0fc1041e0 0x7fc0fc199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc0fb7fe700 1 -- 172.21.15.22:0/270276030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec0095f0 con 0x7fc0fc1018b0 2023-09-20T21:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc0fb7fe700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0fc1018b0 0x7fc0fc198d50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b680 tx=0x7fc0ec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc0f8ff9700 1 -- 172.21.15.22:0/270276030 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec01d070 con 0x7fc0fc1018b0 2023-09-20T21:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc0f8ff9700 1 -- 172.21.15.22:0/270276030 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0ec022470 con 0x7fc0fc1018b0 2023-09-20T21:47:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc101dff700 1 -- 172.21.15.22:0/270276030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0fc1a0e40 con 0x7fc0fc1018b0 2023-09-20T21:47:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc0f8ff9700 1 -- 172.21.15.22:0/270276030 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec017650 con 0x7fc0fc1018b0 2023-09-20T21:47:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.827+0000 7fc101dff700 1 -- 172.21.15.22:0/270276030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0fc1a1200 con 0x7fc0fc1018b0 2023-09-20T21:47:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.828+0000 7fc101dff700 1 -- 172.21.15.22:0/270276030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0fc108690 con 0x7fc0fc1018b0 2023-09-20T21:47:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.830+0000 7fc0f8ff9700 1 -- 172.21.15.22:0/270276030 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fc0ec0225e0 con 0x7fc0fc1018b0 2023-09-20T21:47:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.832+0000 7fc0f8ff9700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0e40a1380 0x7fc0e40a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.832+0000 7fc0f8ff9700 1 -- 172.21.15.22:0/270276030 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc0ec0ce9e0 con 0x7fc0fc1018b0 2023-09-20T21:47:06.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.832+0000 7fc0faffd700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0e40a1380 0x7fc0e40a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:06.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.832+0000 7fc0faffd700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0e40a1380 0x7fc0e40a3840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009c60 tx=0x7fc0f0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:06.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:06.834+0000 7fc0f8ff9700 1 -- 172.21.15.22:0/270276030 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc0ec097ef0 con 0x7fc0fc1018b0 2023-09-20T21:47:06.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:06 smithi123 ceph-mon[134433]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-20T21:47:06.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:06 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-20T21:47:06.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:06 smithi123 ceph-mon[134433]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-20T21:47:06.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:06 smithi123 ceph-mon[134433]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:47:06.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:06 smithi123 ceph-mon[134433]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-09-20T21:47:06.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:06 smithi123 ceph-mon[134433]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-09-20T21:47:06.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:06 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1279100401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-20T21:47:06.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:06 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-20T21:47:06.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:06 smithi089 ceph-mon[134090]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-20T21:47:06.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:06 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-20T21:47:06.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:06 smithi089 ceph-mon[134090]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-20T21:47:06.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:06 smithi089 ceph-mon[134090]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:47:06.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:06 smithi089 ceph-mon[134090]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-09-20T21:47:06.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:06 smithi089 ceph-mon[134090]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-09-20T21:47:06.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:06 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1279100401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-20T21:47:06.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:06 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-20T21:47:06.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:06 smithi022 ceph-mon[97651]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-09-20T21:47:06.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:06 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-09-20T21:47:06.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:06 smithi022 ceph-mon[97651]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-09-20T21:47:06.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:06 smithi022 ceph-mon[97651]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-09-20T21:47:06.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:06 smithi022 ceph-mon[97651]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-09-20T21:47:06.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:06 smithi022 ceph-mon[97651]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-09-20T21:47:06.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:06 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1279100401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-20T21:47:06.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:06 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-09-20T21:47:07.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.096+0000 7fc101dff700 1 -- 172.21.15.22:0/270276030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fc0fc10c850 con 0x7fc0fc1018b0 2023-09-20T21:47:07.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.556+0000 7fc0f8ff9700 1 -- 172.21.15.22:0/270276030 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc0ec09b510 con 0x7fc0fc1018b0 2023-09-20T21:47:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.559+0000 7fc101dff700 1 -- 172.21.15.22:0/270276030 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0e40a1380 msgr2=0x7fc0e40a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.559+0000 7fc101dff700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0e40a1380 0x7fc0e40a3840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009c60 tx=0x7fc0f0009260 comp rx=0 tx=0).stop 2023-09-20T21:47:07.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.559+0000 7fc101dff700 1 -- 172.21.15.22:0/270276030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0fc1018b0 msgr2=0x7fc0fc198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.559+0000 7fc101dff700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0fc1018b0 0x7fc0fc198d50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00b680 tx=0x7fc0ec005740 comp rx=0 tx=0).stop 2023-09-20T21:47:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.559+0000 7fc101dff700 1 -- 172.21.15.22:0/270276030 shutdown_connections 2023-09-20T21:47:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.559+0000 7fc101dff700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0fc106b10 0x7fc0fc1a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:07.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.559+0000 7fc101dff700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0fc1018b0 0x7fc0fc198d50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:07.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.559+0000 7fc101dff700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0e40a1380 0x7fc0e40a3840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:07.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.559+0000 7fc101dff700 1 --2- 172.21.15.22:0/270276030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0fc1041e0 0x7fc0fc199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:07.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.559+0000 7fc101dff700 1 -- 172.21.15.22:0/270276030 >> 172.21.15.22:0/270276030 conn(0x7fc0fc0fb240 msgr2=0x7fc0fc102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:07.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.560+0000 7fc101dff700 1 -- 172.21.15.22:0/270276030 shutdown_connections 2023-09-20T21:47:07.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.560+0000 7fc101dff700 1 -- 172.21.15.22:0/270276030 wait complete. 2023-09-20T21:47:07.586 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-20T21:47:07.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:47:07.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.796+0000 7f05627ff700 1 -- 172.21.15.22:0/2429592203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f055c102970 msgr2=0x7f055c109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:07.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.796+0000 7f05627ff700 1 --2- 172.21.15.22:0/2429592203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f055c102970 0x7f055c109220 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f055400b0d0 tx=0x7f055400b3e0 comp rx=0 tx=0).stop 2023-09-20T21:47:07.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.797+0000 7f05627ff700 1 -- 172.21.15.22:0/2429592203 shutdown_connections 2023-09-20T21:47:07.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.797+0000 7f05627ff700 1 --2- 172.21.15.22:0/2429592203 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f055c102970 0x7f055c109220 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:07.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.797+0000 7f05627ff700 1 --2- 172.21.15.22:0/2429592203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c100de0 0x7f055c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:07.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.797+0000 7f05627ff700 1 --2- 172.21.15.22:0/2429592203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c101fd0 0x7f055c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:07.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.797+0000 7f05627ff700 1 -- 172.21.15.22:0/2429592203 >> 172.21.15.22:0/2429592203 conn(0x7f055c0fc310 msgr2=0x7f055c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:07.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.797+0000 7f05627ff700 1 -- 172.21.15.22:0/2429592203 shutdown_connections 2023-09-20T21:47:07.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.797+0000 7f05627ff700 1 -- 172.21.15.22:0/2429592203 wait complete. 2023-09-20T21:47:07.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.798+0000 7f05627ff700 1 Processor -- start 2023-09-20T21:47:07.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.798+0000 7f05627ff700 1 -- start start 2023-09-20T21:47:07.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.798+0000 7f05627ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c100de0 0x7f055c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:07.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.798+0000 7f05627ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c101fd0 0x7f055c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:07.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.798+0000 7f05627ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f055c102970 0x7f055c1a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:07.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.798+0000 7f05627ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1a0890 con 0x7f055c102970 2023-09-20T21:47:07.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.798+0000 7f05627ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1a0a00 con 0x7f055c101fd0 2023-09-20T21:47:07.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.798+0000 7f05627ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1a0bc0 con 0x7f055c100de0 2023-09-20T21:47:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.798+0000 7f055bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c100de0 0x7f055c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.798+0000 7f055bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c100de0 0x7f055c198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:56532/0 (socket says 172.21.15.22:56532) 2023-09-20T21:47:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f055bfff700 1 -- 172.21.15.22:0/3901384317 learned_addr learned my addr 172.21.15.22:0/3901384317 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f0560d9c700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f055c102970 0x7f055c1a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:07.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f055b7fe700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c101fd0 0x7f055c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f055bfff700 1 -- 172.21.15.22:0/3901384317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c101fd0 msgr2=0x7f055c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f055bfff700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c101fd0 0x7f055c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f055bfff700 1 -- 172.21.15.22:0/3901384317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f055c102970 msgr2=0x7f055c1a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f055bfff700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f055c102970 0x7f055c1a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f055bfff700 1 -- 172.21.15.22:0/3901384317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0554009d00 con 0x7f055c100de0 2023-09-20T21:47:07.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f055bfff700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c100de0 0x7f055c198d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e970 tx=0x7f054c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f05597fa700 1 -- 172.21.15.22:0/3901384317 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c009790 con 0x7f055c100de0 2023-09-20T21:47:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f05597fa700 1 -- 172.21.15.22:0/3901384317 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f054c00c7e0 con 0x7f055c100de0 2023-09-20T21:47:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.799+0000 7f05627ff700 1 -- 172.21.15.22:0/3901384317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c1a0ea0 con 0x7f055c100de0 2023-09-20T21:47:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.800+0000 7f05597fa700 1 -- 172.21.15.22:0/3901384317 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c003c70 con 0x7f055c100de0 2023-09-20T21:47:07.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.800+0000 7f05627ff700 1 -- 172.21.15.22:0/3901384317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c1a12d0 con 0x7f055c100de0 2023-09-20T21:47:07.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.800+0000 7f05627ff700 1 -- 172.21.15.22:0/3901384317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c10c600 con 0x7f055c100de0 2023-09-20T21:47:07.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.803+0000 7f05597fa700 1 -- 172.21.15.22:0/3901384317 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f054c00cc50 con 0x7f055c100de0 2023-09-20T21:47:07.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.804+0000 7f05597fa700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05440a13d0 0x7f05440a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:07.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.804+0000 7f05597fa700 1 -- 172.21.15.22:0/3901384317 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f054c014070 con 0x7f055c100de0 2023-09-20T21:47:07.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.804+0000 7f055b7fe700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05440a13d0 0x7f05440a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:07.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.805+0000 7f055b7fe700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05440a13d0 0x7f05440a3890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0550009c60 tx=0x7f0550009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:07.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:07.807+0000 7f05597fa700 1 -- 172.21.15.22:0/3901384317 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f054c0965a0 con 0x7f055c100de0 2023-09-20T21:47:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] up:active 2023-09-20T21:47:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-20T21:47:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-20T21:47:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-09-20T21:47:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2023-09-20T21:47:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-20T21:47:07.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]: dispatch 2023-09-20T21:47:07.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-09-20T21:47:07.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-09-20T21:47:07.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-09-20T21:47:07.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2023-09-20T21:47:07.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-09-20T21:47:07.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2023-09-20T21:47:07.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-09-20T21:47:07.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2023-09-20T21:47:07.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/270276030' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-20T21:47:07.847 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2023-09-20T21:47:07.847 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] up:active 2023-09-20T21:47:07.847 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/270276030' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-20T21:47:07.847 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-20T21:47:07.847 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-20T21:47:07.847 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:creating} 3 up:standby-replay 5 up:standby 2023-09-20T21:47:07.848 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:07 smithi123 ceph-mon[134433]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-09-20T21:47:07.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] up:active 2023-09-20T21:47:07.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-20T21:47:07.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-20T21:47:07.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-09-20T21:47:07.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2023-09-20T21:47:07.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-20T21:47:07.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]: dispatch 2023-09-20T21:47:07.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-09-20T21:47:07.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-09-20T21:47:07.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-09-20T21:47:07.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2023-09-20T21:47:07.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-09-20T21:47:07.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2023-09-20T21:47:07.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-09-20T21:47:07.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2023-09-20T21:47:07.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/270276030' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-20T21:47:07.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2023-09-20T21:47:07.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] up:active 2023-09-20T21:47:07.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/270276030' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-20T21:47:07.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-20T21:47:07.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-20T21:47:07.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:creating} 3 up:standby-replay 5 up:standby 2023-09-20T21:47:07.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:07 smithi089 ceph-mon[134090]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-09-20T21:47:07.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] up:active 2023-09-20T21:47:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-09-20T21:47:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-09-20T21:47:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-09-20T21:47:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2023-09-20T21:47:07.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-09-20T21:47:07.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]: dispatch 2023-09-20T21:47:07.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2023-09-20T21:47:07.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-09-20T21:47:07.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-09-20T21:47:07.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2023-09-20T21:47:08.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-09-20T21:47:08.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2023-09-20T21:47:08.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2023-09-20T21:47:08.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2023-09-20T21:47:08.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/270276030' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-09-20T21:47:08.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: pgmap v289: 129 pgs: 129 active+clean; 581 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2023-09-20T21:47:08.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] up:active 2023-09-20T21:47:08.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/270276030' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-09-20T21:47:08.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-09-20T21:47:08.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-09-20T21:47:08.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:creating} 3 up:standby-replay 5 up:standby 2023-09-20T21:47:08.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:07 smithi022 ceph-mon[97651]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-09-20T21:47:08.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.062+0000 7f05627ff700 1 -- 172.21.15.22:0/3901384317 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f055c1a1780 con 0x7f055c100de0 2023-09-20T21:47:08.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.063+0000 7f05597fa700 1 -- 172.21.15.22:0/3901384317 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11328 (secure 0 0 0) 0x7f054c003dd0 con 0x7f055c100de0 2023-09-20T21:47:08.064 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:08.064 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24473,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:47:07.557004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:creating","state_seq":1,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T21:47:08.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 -- 172.21.15.22:0/3901384317 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05440a13d0 msgr2=0x7f05440a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:08.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05440a13d0 0x7f05440a3890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0550009c60 tx=0x7f0550009260 comp rx=0 tx=0).stop 2023-09-20T21:47:08.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 -- 172.21.15.22:0/3901384317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c100de0 msgr2=0x7f055c198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:08.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c100de0 0x7f055c198d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e970 tx=0x7f054c00ec80 comp rx=0 tx=0).stop 2023-09-20T21:47:08.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 -- 172.21.15.22:0/3901384317 shutdown_connections 2023-09-20T21:47:08.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f055c102970 0x7f055c1a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f055c100de0 0x7f055c198d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05440a13d0 0x7f05440a3890 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 --2- 172.21.15.22:0/3901384317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f055c101fd0 0x7f055c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 -- 172.21.15.22:0/3901384317 >> 172.21.15.22:0/3901384317 conn(0x7f055c0fc310 msgr2=0x7f055c1077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:08.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 -- 172.21.15.22:0/3901384317 shutdown_connections 2023-09-20T21:47:08.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.066+0000 7f05627ff700 1 -- 172.21.15.22:0/3901384317 wait complete. 2023-09-20T21:47:08.069 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2023-09-20T21:47:08.085 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:47:08.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.288+0000 7f1bf5fff700 1 -- 172.21.15.22:0/426805318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bf0100de0 msgr2=0x7f1bf0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:08.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.288+0000 7f1bf5fff700 1 --2- 172.21.15.22:0/426805318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bf0100de0 0x7f1bf0101200 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1be00099c0 tx=0x7f1be0009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.288+0000 7f1bf5fff700 1 -- 172.21.15.22:0/426805318 shutdown_connections 2023-09-20T21:47:08.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.288+0000 7f1bf5fff700 1 --2- 172.21.15.22:0/426805318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf0102970 0x7f1bf0109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.288+0000 7f1bf5fff700 1 --2- 172.21.15.22:0/426805318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bf0100de0 0x7f1bf0101200 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.288+0000 7f1bf5fff700 1 --2- 172.21.15.22:0/426805318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bf0101fd0 0x7f1bf0102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.288+0000 7f1bf5fff700 1 -- 172.21.15.22:0/426805318 >> 172.21.15.22:0/426805318 conn(0x7f1bf00fc310 msgr2=0x7f1bf00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:08.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.288+0000 7f1bf5fff700 1 -- 172.21.15.22:0/426805318 shutdown_connections 2023-09-20T21:47:08.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.288+0000 7f1bf5fff700 1 -- 172.21.15.22:0/426805318 wait complete. 2023-09-20T21:47:08.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.289+0000 7f1bf5fff700 1 Processor -- start 2023-09-20T21:47:08.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.289+0000 7f1bf5fff700 1 -- start start 2023-09-20T21:47:08.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bf5fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bf0100de0 0x7f1bf0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:08.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bf5fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bf0101fd0 0x7f1bf0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:08.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bf5fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf0102970 0x7f1bf01a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:08.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bf5fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf01a0830 con 0x7f1bf0100de0 2023-09-20T21:47:08.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bf5fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf01a09a0 con 0x7f1bf0101fd0 2023-09-20T21:47:08.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bf5fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf01a0b60 con 0x7f1bf0102970 2023-09-20T21:47:08.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bef7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bf0100de0 0x7f1bf0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:08.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bef7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bf0100de0 0x7f1bf0198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50760/0 (socket says 172.21.15.22:50760) 2023-09-20T21:47:08.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bef7fe700 1 -- 172.21.15.22:0/142460400 learned_addr learned my addr 172.21.15.22:0/142460400 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:08.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1beffff700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf0102970 0x7f1bf01a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:08.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1beeffd700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bf0101fd0 0x7f1bf0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:08.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bef7fe700 1 -- 172.21.15.22:0/142460400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf0102970 msgr2=0x7f1bf01a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:08.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bef7fe700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf0102970 0x7f1bf01a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bef7fe700 1 -- 172.21.15.22:0/142460400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bf0101fd0 msgr2=0x7f1bf0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:08.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bef7fe700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bf0101fd0 0x7f1bf0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1bef7fe700 1 -- 172.21.15.22:0/142460400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00095f0 con 0x7f1bf0100de0 2023-09-20T21:47:08.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.290+0000 7f1beeffd700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bf0101fd0 0x7f1bf0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:47:08.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.291+0000 7f1bef7fe700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bf0100de0 0x7f1bf0198cf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1be00098e0 tx=0x7f1be000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:08.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.291+0000 7f1becff9700 1 -- 172.21.15.22:0/142460400 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be001d070 con 0x7f1bf0100de0 2023-09-20T21:47:08.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.291+0000 7f1becff9700 1 -- 172.21.15.22:0/142460400 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be0005510 con 0x7f1bf0100de0 2023-09-20T21:47:08.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.291+0000 7f1bf5fff700 1 -- 172.21.15.22:0/142460400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf01a0de0 con 0x7f1bf0100de0 2023-09-20T21:47:08.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.291+0000 7f1becff9700 1 -- 172.21.15.22:0/142460400 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0004c00 con 0x7f1bf0100de0 2023-09-20T21:47:08.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.291+0000 7f1bf5fff700 1 -- 172.21.15.22:0/142460400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf01a11b0 con 0x7f1bf0100de0 2023-09-20T21:47:08.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.292+0000 7f1bf5fff700 1 -- 172.21.15.22:0/142460400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf0193430 con 0x7f1bf0100de0 2023-09-20T21:47:08.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.295+0000 7f1becff9700 1 -- 172.21.15.22:0/142460400 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1be000fb80 con 0x7f1bf0100de0 2023-09-20T21:47:08.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.295+0000 7f1becff9700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bd80a1350 0x7f1bd80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:08.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.296+0000 7f1becff9700 1 -- 172.21.15.22:0/142460400 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1be00cde40 con 0x7f1bf0100de0 2023-09-20T21:47:08.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.296+0000 7f1beeffd700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bd80a1350 0x7f1bd80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:08.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.296+0000 7f1beeffd700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bd80a1350 0x7f1bd80a3810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009cb0 tx=0x7f1be4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:08.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.300+0000 7f1becff9700 1 -- 172.21.15.22:0/142460400 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1be0097240 con 0x7f1bf0100de0 2023-09-20T21:47:08.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.515+0000 7f1bf5fff700 1 -- 172.21.15.22:0/142460400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bf010c640 con 0x7f1bf0100de0 2023-09-20T21:47:08.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.517+0000 7f1becff9700 1 -- 172.21.15.22:0/142460400 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24411 (secure 0 0 0) 0x7f1be009a860 con 0x7f1bf0100de0 2023-09-20T21:47:08.517 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:08.518 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":90,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:47:07.585720+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:47:08.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 -- 172.21.15.22:0/142460400 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bd80a1350 msgr2=0x7f1bd80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:08.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bd80a1350 0x7f1bd80a3810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009cb0 tx=0x7f1be4009260 comp rx=0 tx=0).stop 2023-09-20T21:47:08.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 -- 172.21.15.22:0/142460400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bf0100de0 msgr2=0x7f1bf0198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:08.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bf0100de0 0x7f1bf0198cf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1be00098e0 tx=0x7f1be000f690 comp rx=0 tx=0).stop 2023-09-20T21:47:08.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 -- 172.21.15.22:0/142460400 shutdown_connections 2023-09-20T21:47:08.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bf0102970 0x7f1bf01a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1bf0100de0 0x7f1bf0198cf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1bd80a1350 0x7f1bd80a3810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 --2- 172.21.15.22:0/142460400 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1bf0101fd0 0x7f1bf0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 -- 172.21.15.22:0/142460400 >> 172.21.15.22:0/142460400 conn(0x7f1bf00fc310 msgr2=0x7f1bf01078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:08.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 -- 172.21.15.22:0/142460400 shutdown_connections 2023-09-20T21:47:08.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.519+0000 7f1bf5fff700 1 -- 172.21.15.22:0/142460400 wait complete. 2023-09-20T21:47:08.538 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-20T21:47:08.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-20T21:47:08.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.756+0000 7fed08bff700 1 -- 172.21.15.22:0/3349340680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed04102970 msgr2=0x7fed04109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.756+0000 7fed08bff700 1 --2- 172.21.15.22:0/3349340680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed04102970 0x7fed04109220 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00b0d0 tx=0x7fecfc00b3e0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.756+0000 7fed08bff700 1 -- 172.21.15.22:0/3349340680 shutdown_connections 2023-09-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.756+0000 7fed08bff700 1 --2- 172.21.15.22:0/3349340680 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed04102970 0x7fed04109220 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.756+0000 7fed08bff700 1 --2- 172.21.15.22:0/3349340680 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed04100de0 0x7fed04101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.756+0000 7fed08bff700 1 --2- 172.21.15.22:0/3349340680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed04101fd0 0x7fed04102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.756+0000 7fed08bff700 1 -- 172.21.15.22:0/3349340680 >> 172.21.15.22:0/3349340680 conn(0x7fed040fc310 msgr2=0x7fed040fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.756+0000 7fed08bff700 1 -- 172.21.15.22:0/3349340680 shutdown_connections 2023-09-20T21:47:08.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.756+0000 7fed08bff700 1 -- 172.21.15.22:0/3349340680 wait complete. 2023-09-20T21:47:08.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.757+0000 7fed08bff700 1 Processor -- start 2023-09-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.757+0000 7fed08bff700 1 -- start start 2023-09-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.757+0000 7fed08bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed04100de0 0x7fed04196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed08bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed04101fd0 0x7fed041970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed08bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed04102970 0x7fed0419e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:08.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed08bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0419e710 con 0x7fed04102970 2023-09-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed08bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0419e850 con 0x7fed04101fd0 2023-09-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed08bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0419e9e0 con 0x7fed04100de0 2023-09-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed0259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed04100de0 0x7fed04196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed0259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed04100de0 0x7fed04196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:56574/0 (socket says 172.21.15.22:56574) 2023-09-20T21:47:08.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed0259c700 1 -- 172.21.15.22:0/779739354 learned_addr learned my addr 172.21.15.22:0/779739354 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed01d9b700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed04101fd0 0x7fed041970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed02d9d700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed04102970 0x7fed0419e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed0259c700 1 -- 172.21.15.22:0/779739354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed04101fd0 msgr2=0x7fed041970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed0259c700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed04101fd0 0x7fed041970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed0259c700 1 -- 172.21.15.22:0/779739354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed04102970 msgr2=0x7fed0419e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed0259c700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed04102970 0x7fed0419e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.758+0000 7fed0259c700 1 -- 172.21.15.22:0/779739354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf4009520 con 0x7fed04100de0 2023-09-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.759+0000 7fed0259c700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed04100de0 0x7fed04196b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fecf400ea50 tx=0x7fecf400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.759+0000 7fecf37fe700 1 -- 172.21.15.22:0/779739354 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf400cbd0 con 0x7fed04100de0 2023-09-20T21:47:08.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.759+0000 7fecf37fe700 1 -- 172.21.15.22:0/779739354 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecf400cd30 con 0x7fed04100de0 2023-09-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.759+0000 7fed08bff700 1 -- 172.21.15.22:0/779739354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecfc009d00 con 0x7fed04100de0 2023-09-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.759+0000 7fecf37fe700 1 -- 172.21.15.22:0/779739354 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4018680 con 0x7fed04100de0 2023-09-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.759+0000 7fed08bff700 1 -- 172.21.15.22:0/779739354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0419f090 con 0x7fed04100de0 2023-09-20T21:47:08.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.760+0000 7fed08bff700 1 -- 172.21.15.22:0/779739354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed041082e0 con 0x7fed04100de0 2023-09-20T21:47:08.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.762+0000 7fecf37fe700 1 -- 172.21.15.22:0/779739354 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fecf4010430 con 0x7fed04100de0 2023-09-20T21:47:08.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.764+0000 7fecf37fe700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fecec0a13d0 0x7fecec0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.764+0000 7fecf37fe700 1 -- 172.21.15.22:0/779739354 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fecf4014070 con 0x7fed04100de0 2023-09-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.764+0000 7fed01d9b700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fecec0a13d0 0x7fecec0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.764+0000 7fed01d9b700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fecec0a13d0 0x7fecec0a3890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009c60 tx=0x7fecf8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:08.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.766+0000 7fecf37fe700 1 -- 172.21.15.22:0/779739354 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fecf4096d30 con 0x7fed04100de0 2023-09-20T21:47:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]': finished 2023-09-20T21:47:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-09-20T21:47:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-09-20T21:47:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-09-20T21:47:08.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2023-09-20T21:47:08.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-09-20T21:47:08.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2023-09-20T21:47:08.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-09-20T21:47:08.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2023-09-20T21:47:08.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: osdmap e90: 12 total, 12 up, 12 in 2023-09-20T21:47:08.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3901384317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:47:08.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/142460400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:08.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] up:active 2023-09-20T21:47:08.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: daemon mds.j assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-20T21:47:08.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-20T21:47:08.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: Cluster is now healthy 2023-09-20T21:47:08.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:active} 3 up:standby-replay 5 up:standby 2023-09-20T21:47:08.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:08 smithi089 ceph-mon[134090]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-09-20T21:47:08.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.976+0000 7fed08bff700 1 -- 172.21.15.22:0/779739354 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed0402d1f0 con 0x7fecec0a13d0 2023-09-20T21:47:08.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.992+0000 7fecf37fe700 1 -- 172.21.15.22:0/779739354 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+421915 (secure 0 0 0) 0x7fed0402d1f0 con 0x7fecec0a13d0 2023-09-20T21:47:08.997 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 -- 172.21.15.22:0/779739354 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fecec0a13d0 msgr2=0x7fecec0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fecec0a13d0 0x7fecec0a3890 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009c60 tx=0x7fecf8009260 comp rx=0 tx=0).stop 2023-09-20T21:47:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 -- 172.21.15.22:0/779739354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed04100de0 msgr2=0x7fed04196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed04100de0 0x7fed04196b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fecf400ea50 tx=0x7fecf400ed60 comp rx=0 tx=0).stop 2023-09-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 -- 172.21.15.22:0/779739354 shutdown_connections 2023-09-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed04102970 0x7fed0419e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed04100de0 0x7fed04196b90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fecec0a13d0 0x7fecec0a3890 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 --2- 172.21.15.22:0/779739354 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed04101fd0 0x7fed041970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 -- 172.21.15.22:0/779739354 >> 172.21.15.22:0/779739354 conn(0x7fed040fc310 msgr2=0x7fed04105420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 -- 172.21.15.22:0/779739354 shutdown_connections 2023-09-20T21:47:09.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:08.997+0000 7fed08bff700 1 -- 172.21.15.22:0/779739354 wait complete. 2023-09-20T21:47:09.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]': finished 2023-09-20T21:47:09.003 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-09-20T21:47:09.003 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-09-20T21:47:09.003 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-09-20T21:47:09.003 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2023-09-20T21:47:09.003 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-09-20T21:47:09.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2023-09-20T21:47:09.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-09-20T21:47:09.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2023-09-20T21:47:09.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: osdmap e90: 12 total, 12 up, 12 in 2023-09-20T21:47:09.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3901384317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:47:09.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/142460400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:09.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] up:active 2023-09-20T21:47:09.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: daemon mds.j assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-20T21:47:09.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-20T21:47:09.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: Cluster is now healthy 2023-09-20T21:47:09.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:active} 3 up:standby-replay 5 up:standby 2023-09-20T21:47:09.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:08 smithi022 ceph-mon[97651]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-09-20T21:47:09.006 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-09-20T21:47:09.015 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-09-20T21:47:08.592344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595322,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":129,"num_read_kb":112,"num_write":268,"num_write_kb":2968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337608,"kb_used_data":13388,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604216,"statfs":{"total":1151940427776,"available":1151594717184,"internally_reserved":0,"allocated":13709312,"data_stored":7517083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2095,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":41,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.492374"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564150+0000","last_change":"2023-09-20T21:46:41.506092+0000","last_active":"2023-09-20T21:46:45.564150+0000","last_peered":"2023-09-20T21:46:45.564150+0000","last_clean":"2023-09-20T21:46:45.564150+0000","last_became_active":"2023-09-20T21:46:41.505779+0000","last_became_peered":"2023-09-20T21:46:41.505779+0000","last_unstale":"2023-09-20T21:46:45.564150+0000","last_undegraded":"2023-09-20T21:46:45.564150+0000","last_fullsized":"2023-09-20T21:46:45.564150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:25.284068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571607+0000","last_change":"2023-09-20T21:46:41.505000+0000","last_active":"2023-09-20T21:46:45.571607+0000","last_peered":"2023-09-20T21:46:45.571607+0000","last_clean":"2023-09-20T21:46:45.571607+0000","last_became_active":"2023-09-20T21:46:41.504464+0000","last_became_peered":"2023-09-20T21:46:41.504464+0000","last_unstale":"2023-09-20T21:46:45.571607+0000","last_undegraded":"2023-09-20T21:46:45.571607+0000","last_fullsized":"2023-09-20T21:46:45.571607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:42:44.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569994+0000","last_change":"2023-09-20T21:46:41.513134+0000","last_active":"2023-09-20T21:46:45.569994+0000","last_peered":"2023-09-20T21:46:45.569994+0000","last_clean":"2023-09-20T21:46:45.569994+0000","last_became_active":"2023-09-20T21:46:41.512793+0000","last_became_peered":"2023-09-20T21:46:41.512793+0000","last_unstale":"2023-09-20T21:46:45.569994+0000","last_undegraded":"2023-09-20T21:46:45.569994+0000","last_fullsized":"2023-09-20T21:46:45.569994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:59.978630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.572104+0000","last_change":"2023-09-20T21:46:41.500200+0000","last_active":"2023-09-20T21:46:45.572104+0000","last_peered":"2023-09-20T21:46:45.572104+0000","last_clean":"2023-09-20T21:46:45.572104+0000","last_became_active":"2023-09-20T21:46:41.499866+0000","last_became_peered":"2023-09-20T21:46:41.499866+0000","last_unstale":"2023-09-20T21:46:45.572104+0000","last_undegraded":"2023-09-20T21:46:45.572104+0000","last_fullsized":"2023-09-20T21:46:45.572104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:20:52.989008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501872+0000","last_change":"2023-09-20T21:46:41.507584+0000","last_active":"2023-09-20T21:46:47.501872+0000","last_peered":"2023-09-20T21:46:47.501872+0000","last_clean":"2023-09-20T21:46:47.501872+0000","last_became_active":"2023-09-20T21:46:41.507298+0000","last_became_peered":"2023-09-20T21:46:41.507298+0000","last_unstale":"2023-09-20T21:46:47.501872+0000","last_undegraded":"2023-09-20T21:46:47.501872+0000","last_fullsized":"2023-09-20T21:46:47.501872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:32:14.042414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564527+0000","last_change":"2023-09-20T21:46:41.497002+0000","last_active":"2023-09-20T21:46:45.564527+0000","last_peered":"2023-09-20T21:46:45.564527+0000","last_clean":"2023-09-20T21:46:45.564527+0000","last_became_active":"2023-09-20T21:46:41.496485+0000","last_became_peered":"2023-09-20T21:46:41.496485+0000","last_unstale":"2023-09-20T21:46:45.564527+0000","last_undegraded":"2023-09-20T21:46:45.564527+0000","last_fullsized":"2023-09-20T21:46:45.564527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:48:24.481575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.567850+0000","last_change":"2023-09-20T21:46:41.503303+0000","last_active":"2023-09-20T21:46:45.567850+0000","last_peered":"2023-09-20T21:46:45.567850+0000","last_clean":"2023-09-20T21:46:45.567850+0000","last_became_active":"2023-09-20T21:46:41.502982+0000","last_became_peered":"2023-09-20T21:46:41.502982+0000","last_unstale":"2023-09-20T21:46:45.567850+0000","last_undegraded":"2023-09-20T21:46:45.567850+0000","last_fullsized":"2023-09-20T21:46:45.567850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:23:27.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569351+0000","last_change":"2023-09-20T21:46:41.508243+0000","last_active":"2023-09-20T21:46:45.569351+0000","last_peered":"2023-09-20T21:46:45.569351+0000","last_clean":"2023-09-20T21:46:45.569351+0000","last_became_active":"2023-09-20T21:46:41.506407+0000","last_became_peered":"2023-09-20T21:46:41.506407+0000","last_unstale":"2023-09-20T21:46:45.569351+0000","last_undegraded":"2023-09-20T21:46:45.569351+0000","last_fullsized":"2023-09-20T21:46:45.569351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:39.388716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568936+0000","last_change":"2023-09-20T21:46:41.501513+0000","last_active":"2023-09-20T21:46:45.568936+0000","last_peered":"2023-09-20T21:46:45.568936+0000","last_clean":"2023-09-20T21:46:45.568936+0000","last_became_active":"2023-09-20T21:46:41.500051+0000","last_became_peered":"2023-09-20T21:46:41.500051+0000","last_unstale":"2023-09-20T21:46:45.568936+0000","last_undegraded":"2023-09-20T21:46:45.568936+0000","last_fullsized":"2023-09-20T21:46:45.568936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:40.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564777+0000","last_change":"2023-09-20T21:46:41.501045+0000","last_active":"2023-09-20T21:46:45.564777+0000","last_peered":"2023-09-20T21:46:45.564777+0000","last_clean":"2023-09-20T21:46:45.564777+0000","last_became_active":"2023-09-20T21:46:41.500744+0000","last_became_peered":"2023-09-20T21:46:41.500744+0000","last_unstale":"2023-09-20T21:46:45.564777+0000","last_undegraded":"2023-09-20T21:46:45.564777+0000","last_fullsized":"2023-09-20T21:46:45.564777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:57:29.120472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570586+0000","last_change":"2023-09-20T21:46:41.510235+0000","last_active":"2023-09-20T21:46:45.570586+0000","last_peered":"2023-09-20T21:46:45.570586+0000","last_clean":"2023-09-20T21:46:45.570586+0000","last_became_active":"2023-09-20T21:46:41.509943+0000","last_became_peered":"2023-09-20T21:46:41.509943+0000","last_unstale":"2023-09-20T21:46:45.570586+0000","last_undegraded":"2023-09-20T21:46:45.570586+0000","last_fullsized":"2023-09-20T21:46:45.570586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:09:19.383505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.572012+0000","last_change":"2023-09-20T21:46:41.501194+0000","last_active":"2023-09-20T21:46:45.572012+0000","last_peered":"2023-09-20T21:46:45.572012+0000","last_clean":"2023-09-20T21:46:45.572012+0000","last_became_active":"2023-09-20T21:46:41.500095+0000","last_became_peered":"2023-09-20T21:46:41.500095+0000","last_unstale":"2023-09-20T21:46:45.572012+0000","last_undegraded":"2023-09-20T21:46:45.572012+0000","last_fullsized":"2023-09-20T21:46:45.572012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:46:51.343332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575166+0000","last_change":"2023-09-20T21:46:41.507562+0000","last_active":"2023-09-20T21:46:45.575166+0000","last_peered":"2023-09-20T21:46:45.575166+0000","last_clean":"2023-09-20T21:46:45.575166+0000","last_became_active":"2023-09-20T21:46:41.507094+0000","last_became_peered":"2023-09-20T21:46:41.507094+0000","last_unstale":"2023-09-20T21:46:45.575166+0000","last_undegraded":"2023-09-20T21:46:45.575166+0000","last_fullsized":"2023-09-20T21:46:45.575166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:34:32.401415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569178+0000","last_change":"2023-09-20T21:46:41.502864+0000","last_active":"2023-09-20T21:46:45.569178+0000","last_peered":"2023-09-20T21:46:45.569178+0000","last_clean":"2023-09-20T21:46:45.569178+0000","last_became_active":"2023-09-20T21:46:41.502543+0000","last_became_peered":"2023-09-20T21:46:41.502543+0000","last_unstale":"2023-09-20T21:46:45.569178+0000","last_undegraded":"2023-09-20T21:46:45.569178+0000","last_fullsized":"2023-09-20T21:46:45.569178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:02:04.631874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.572035+0000","last_change":"2023-09-20T21:46:41.512189+0000","last_active":"2023-09-20T21:46:45.572035+0000","last_peered":"2023-09-20T21:46:45.572035+0000","last_clean":"2023-09-20T21:46:45.572035+0000","last_became_active":"2023-09-20T21:46:41.511888+0000","last_became_peered":"2023-09-20T21:46:41.511888+0000","last_unstale":"2023-09-20T21:46:45.572035+0000","last_undegraded":"2023-09-20T21:46:45.572035+0000","last_fullsized":"2023-09-20T21:46:45.572035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:17:20.417468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571768+0000","last_change":"2023-09-20T21:46:41.502036+0000","last_active":"2023-09-20T21:46:45.571768+0000","last_peered":"2023-09-20T21:46:45.571768+0000","last_clean":"2023-09-20T21:46:45.571768+0000","last_became_active":"2023-09-20T21:46:41.501745+0000","last_became_peered":"2023-09-20T21:46:41.501745+0000","last_unstale":"2023-09-20T21:46:45.571768+0000","last_undegraded":"2023-09-20T21:46:45.571768+0000","last_fullsized":"2023-09-20T21:46:45.571768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:08:06.699543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563697+0000","last_change":"2023-09-20T21:46:41.506666+0000","last_active":"2023-09-20T21:46:45.563697+0000","last_peered":"2023-09-20T21:46:45.563697+0000","last_clean":"2023-09-20T21:46:45.563697+0000","last_became_active":"2023-09-20T21:46:41.506266+0000","last_became_peered":"2023-09-20T21:46:41.506266+0000","last_unstale":"2023-09-20T21:46:45.563697+0000","last_undegraded":"2023-09-20T21:46:45.563697+0000","last_fullsized":"2023-09-20T21:46:45.563697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:06:16.809609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563418+0000","last_change":"2023-09-20T21:46:41.497391+0000","last_active":"2023-09-20T21:46:45.563418+0000","last_peered":"2023-09-20T21:46:45.563418+0000","last_clean":"2023-09-20T21:46:45.563418+0000","last_became_active":"2023-09-20T21:46:41.497064+0000","last_became_peered":"2023-09-20T21:46:41.497064+0000","last_unstale":"2023-09-20T21:46:45.563418+0000","last_undegraded":"2023-09-20T21:46:45.563418+0000","last_fullsized":"2023-09-20T21:46:45.563418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:56:11.489585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569164+0000","last_change":"2023-09-20T21:46:41.510602+0000","last_active":"2023-09-20T21:46:45.569164+0000","last_peered":"2023-09-20T21:46:45.569164+0000","last_clean":"2023-09-20T21:46:45.569164+0000","last_became_active":"2023-09-20T21:46:41.510308+0000","last_became_peered":"2023-09-20T21:46:41.510308+0000","last_unstale":"2023-09-20T21:46:45.569164+0000","last_undegraded":"2023-09-20T21:46:45.569164+0000","last_fullsized":"2023-09-20T21:46:45.569164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:43:41.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565451+0000","last_change":"2023-09-20T21:46:45.581583+0000","last_active":"2023-09-20T21:46:46.565451+0000","last_peered":"2023-09-20T21:46:46.565451+0000","last_clean":"2023-09-20T21:46:46.565451+0000","last_became_active":"2023-09-20T21:46:45.581165+0000","last_became_peered":"2023-09-20T21:46:45.581165+0000","last_unstale":"2023-09-20T21:46:46.565451+0000","last_undegraded":"2023-09-20T21:46:46.565451+0000","last_fullsized":"2023-09-20T21:46:46.565451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:39:36.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564314+0000","last_change":"2023-09-20T21:46:40.500010+0000","last_active":"2023-09-20T21:46:45.564314+0000","last_peered":"2023-09-20T21:46:45.564314+0000","last_clean":"2023-09-20T21:46:45.564314+0000","last_became_active":"2023-09-20T21:46:40.499632+0000","last_became_peered":"2023-09-20T21:46:40.499632+0000","last_unstale":"2023-09-20T21:46:45.564314+0000","last_undegraded":"2023-09-20T21:46:45.564314+0000","last_fullsized":"2023-09-20T21:46:45.564314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:41:35.590982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569101+0000","last_change":"2023-09-20T21:46:41.516465+0000","last_active":"2023-09-20T21:46:45.569101+0000","last_peered":"2023-09-20T21:46:45.569101+0000","last_clean":"2023-09-20T21:46:45.569101+0000","last_became_active":"2023-09-20T21:46:41.516150+0000","last_became_peered":"2023-09-20T21:46:41.516150+0000","last_unstale":"2023-09-20T21:46:45.569101+0000","last_undegraded":"2023-09-20T21:46:45.569101+0000","last_fullsized":"2023-09-20T21:46:45.569101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:33:37.507049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.503015+0000","last_change":"2023-09-20T21:46:45.582916+0000","last_active":"2023-09-20T21:46:47.503015+0000","last_peered":"2023-09-20T21:46:47.503015+0000","last_clean":"2023-09-20T21:46:47.503015+0000","last_became_active":"2023-09-20T21:46:45.582511+0000","last_became_peered":"2023-09-20T21:46:45.582511+0000","last_unstale":"2023-09-20T21:46:47.503015+0000","last_undegraded":"2023-09-20T21:46:47.503015+0000","last_fullsized":"2023-09-20T21:46:47.503015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:43:47.006768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569207+0000","last_change":"2023-09-20T21:46:40.516867+0000","last_active":"2023-09-20T21:46:45.569207+0000","last_peered":"2023-09-20T21:46:45.569207+0000","last_clean":"2023-09-20T21:46:45.569207+0000","last_became_active":"2023-09-20T21:46:40.516343+0000","last_became_peered":"2023-09-20T21:46:40.516343+0000","last_unstale":"2023-09-20T21:46:45.569207+0000","last_undegraded":"2023-09-20T21:46:45.569207+0000","last_fullsized":"2023-09-20T21:46:45.569207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:17:18.348921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573554+0000","last_change":"2023-09-20T21:46:41.507093+0000","last_active":"2023-09-20T21:46:45.573554+0000","last_peered":"2023-09-20T21:46:45.573554+0000","last_clean":"2023-09-20T21:46:45.573554+0000","last_became_active":"2023-09-20T21:46:41.506587+0000","last_became_peered":"2023-09-20T21:46:41.506587+0000","last_unstale":"2023-09-20T21:46:45.573554+0000","last_undegraded":"2023-09-20T21:46:45.573554+0000","last_fullsized":"2023-09-20T21:46:45.573554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:26:44.825499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502877+0000","last_change":"2023-09-20T21:46:45.583942+0000","last_active":"2023-09-20T21:46:47.502877+0000","last_peered":"2023-09-20T21:46:47.502877+0000","last_clean":"2023-09-20T21:46:47.502877+0000","last_became_active":"2023-09-20T21:46:45.583484+0000","last_became_peered":"2023-09-20T21:46:45.583484+0000","last_unstale":"2023-09-20T21:46:47.502877+0000","last_undegraded":"2023-09-20T21:46:47.502877+0000","last_fullsized":"2023-09-20T21:46:47.502877+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:48.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"peering","last_fresh":"2023-09-20T21:47:07.573536+0000","last_change":"2023-09-20T21:47:07.596775+0000","last_active":"2023-09-20T21:47:07.596775+0000","last_peered":"2023-09-20T21:47:07.573536+0000","last_clean":"2023-09-20T21:47:07.573536+0000","last_became_active":"2023-09-20T21:46:40.501028+0000","last_became_peered":"2023-09-20T21:46:40.501028+0000","last_unstale":"2023-09-20T21:47:07.573536+0000","last_undegraded":"2023-09-20T21:47:07.573536+0000","last_fullsized":"2023-09-20T21:47:07.573536+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,7],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568146+0000","last_change":"2023-09-20T21:46:41.504449+0000","last_active":"2023-09-20T21:46:45.568146+0000","last_peered":"2023-09-20T21:46:45.568146+0000","last_clean":"2023-09-20T21:46:45.568146+0000","last_became_active":"2023-09-20T21:46:41.504169+0000","last_became_peered":"2023-09-20T21:46:41.504169+0000","last_unstale":"2023-09-20T21:46:45.568146+0000","last_undegraded":"2023-09-20T21:46:45.568146+0000","last_fullsized":"2023-09-20T21:46:45.568146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:36:43.980902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502954+0000","last_change":"2023-09-20T21:46:45.582269+0000","last_active":"2023-09-20T21:46:47.502954+0000","last_peered":"2023-09-20T21:46:47.502954+0000","last_clean":"2023-09-20T21:46:47.502954+0000","last_became_active":"2023-09-20T21:46:45.581782+0000","last_became_peered":"2023-09-20T21:46:45.581782+0000","last_unstale":"2023-09-20T21:46:47.502954+0000","last_undegraded":"2023-09-20T21:46:47.502954+0000","last_fullsized":"2023-09-20T21:46:47.502954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:32:09.518715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502079+0000","last_change":"2023-09-20T21:46:40.504099+0000","last_active":"2023-09-20T21:46:47.502079+0000","last_peered":"2023-09-20T21:46:47.502079+0000","last_clean":"2023-09-20T21:46:47.502079+0000","last_became_active":"2023-09-20T21:46:40.503774+0000","last_became_peered":"2023-09-20T21:46:40.503774+0000","last_unstale":"2023-09-20T21:46:47.502079+0000","last_undegraded":"2023-09-20T21:46:47.502079+0000","last_fullsized":"2023-09-20T21:46:47.502079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:17.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568776+0000","last_change":"2023-09-20T21:46:41.499802+0000","last_active":"2023-09-20T21:46:45.568776+0000","last_peered":"2023-09-20T21:46:45.568776+0000","last_clean":"2023-09-20T21:46:45.568776+0000","last_became_active":"2023-09-20T21:46:41.499491+0000","last_became_peered":"2023-09-20T21:46:41.499491+0000","last_unstale":"2023-09-20T21:46:45.568776+0000","last_undegraded":"2023-09-20T21:46:45.568776+0000","last_fullsized":"2023-09-20T21:46:45.568776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:44.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565986+0000","last_change":"2023-09-20T21:46:45.578753+0000","last_active":"2023-09-20T21:46:46.565986+0000","last_peered":"2023-09-20T21:46:46.565986+0000","last_clean":"2023-09-20T21:46:46.565986+0000","last_became_active":"2023-09-20T21:46:45.578297+0000","last_became_peered":"2023-09-20T21:46:45.578297+0000","last_unstale":"2023-09-20T21:46:46.565986+0000","last_undegraded":"2023-09-20T21:46:46.565986+0000","last_fullsized":"2023-09-20T21:46:46.565986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:18:22.596550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571690+0000","last_change":"2023-09-20T21:46:40.505675+0000","last_active":"2023-09-20T21:46:45.571690+0000","last_peered":"2023-09-20T21:46:45.571690+0000","last_clean":"2023-09-20T21:46:45.571690+0000","last_became_active":"2023-09-20T21:46:40.505432+0000","last_became_peered":"2023-09-20T21:46:40.505432+0000","last_unstale":"2023-09-20T21:46:45.571690+0000","last_undegraded":"2023-09-20T21:46:45.571690+0000","last_fullsized":"2023-09-20T21:46:45.571690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:14:53.652904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501949+0000","last_change":"2023-09-20T21:46:41.505924+0000","last_active":"2023-09-20T21:46:47.501949+0000","last_peered":"2023-09-20T21:46:47.501949+0000","last_clean":"2023-09-20T21:46:47.501949+0000","last_became_active":"2023-09-20T21:46:41.505609+0000","last_became_peered":"2023-09-20T21:46:41.505609+0000","last_unstale":"2023-09-20T21:46:47.501949+0000","last_undegraded":"2023-09-20T21:46:47.501949+0000","last_fullsized":"2023-09-20T21:46:47.501949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:13:46.506786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565886+0000","last_change":"2023-09-20T21:46:45.587818+0000","last_active":"2023-09-20T21:46:46.565886+0000","last_peered":"2023-09-20T21:46:46.565886+0000","last_clean":"2023-09-20T21:46:46.565886+0000","last_became_active":"2023-09-20T21:46:45.587376+0000","last_became_peered":"2023-09-20T21:46:45.587376+0000","last_unstale":"2023-09-20T21:46:46.565886+0000","last_undegraded":"2023-09-20T21:46:46.565886+0000","last_fullsized":"2023-09-20T21:46:46.565886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:28:53.605158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564415+0000","last_change":"2023-09-20T21:46:40.505610+0000","last_active":"2023-09-20T21:46:45.564415+0000","last_peered":"2023-09-20T21:46:45.564415+0000","last_clean":"2023-09-20T21:46:45.564415+0000","last_became_active":"2023-09-20T21:46:40.505279+0000","last_became_peered":"2023-09-20T21:46:40.505279+0000","last_unstale":"2023-09-20T21:46:45.564415+0000","last_undegraded":"2023-09-20T21:46:45.564415+0000","last_fullsized":"2023-09-20T21:46:45.564415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:41:19.575154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571794+0000","last_change":"2023-09-20T21:46:41.505337+0000","last_active":"2023-09-20T21:46:45.571794+0000","last_peered":"2023-09-20T21:46:45.571794+0000","last_clean":"2023-09-20T21:46:45.571794+0000","last_became_active":"2023-09-20T21:46:41.504959+0000","last_became_peered":"2023-09-20T21:46:41.504959+0000","last_unstale":"2023-09-20T21:46:45.571794+0000","last_undegraded":"2023-09-20T21:46:45.571794+0000","last_fullsized":"2023-09-20T21:46:45.571794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:53:37.797319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.297727+0000","last_change":"2023-09-20T21:46:45.581738+0000","last_active":"2023-09-20T21:46:47.297727+0000","last_peered":"2023-09-20T21:46:47.297727+0000","last_clean":"2023-09-20T21:46:47.297727+0000","last_became_active":"2023-09-20T21:46:45.581279+0000","last_became_peered":"2023-09-20T21:46:45.581279+0000","last_unstale":"2023-09-20T21:46:47.297727+0000","last_undegraded":"2023-09-20T21:46:47.297727+0000","last_fullsized":"2023-09-20T21:46:47.297727+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:41:41.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570018+0000","last_change":"2023-09-20T21:46:40.510539+0000","last_active":"2023-09-20T21:46:45.570018+0000","last_peered":"2023-09-20T21:46:45.570018+0000","last_clean":"2023-09-20T21:46:45.570018+0000","last_became_active":"2023-09-20T21:46:40.509880+0000","last_became_peered":"2023-09-20T21:46:40.509880+0000","last_unstale":"2023-09-20T21:46:45.570018+0000","last_undegraded":"2023-09-20T21:46:45.570018+0000","last_fullsized":"2023-09-20T21:46:45.570018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:52.877786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569259+0000","last_change":"2023-09-20T21:46:41.505105+0000","last_active":"2023-09-20T21:46:45.569259+0000","last_peered":"2023-09-20T21:46:45.569259+0000","last_clean":"2023-09-20T21:46:45.569259+0000","last_became_active":"2023-09-20T21:46:41.504785+0000","last_became_peered":"2023-09-20T21:46:41.504785+0000","last_unstale":"2023-09-20T21:46:45.569259+0000","last_undegraded":"2023-09-20T21:46:45.569259+0000","last_fullsized":"2023-09-20T21:46:45.569259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:07:57.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502213+0000","last_change":"2023-09-20T21:46:45.583160+0000","last_active":"2023-09-20T21:46:47.502213+0000","last_peered":"2023-09-20T21:46:47.502213+0000","last_clean":"2023-09-20T21:46:47.502213+0000","last_became_active":"2023-09-20T21:46:45.582709+0000","last_became_peered":"2023-09-20T21:46:45.582709+0000","last_unstale":"2023-09-20T21:46:47.502213+0000","last_undegraded":"2023-09-20T21:46:47.502213+0000","last_fullsized":"2023-09-20T21:46:47.502213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:24:57.563682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568131+0000","last_change":"2023-09-20T21:46:40.502591+0000","last_active":"2023-09-20T21:46:45.568131+0000","last_peered":"2023-09-20T21:46:45.568131+0000","last_clean":"2023-09-20T21:46:45.568131+0000","last_became_active":"2023-09-20T21:46:40.502288+0000","last_became_peered":"2023-09-20T21:46:40.502288+0000","last_unstale":"2023-09-20T21:46:45.568131+0000","last_undegraded":"2023-09-20T21:46:45.568131+0000","last_fullsized":"2023-09-20T21:46:45.568131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:09:37.983866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563192+0000","last_change":"2023-09-20T21:46:41.511367+0000","last_active":"2023-09-20T21:46:45.563192+0000","last_peered":"2023-09-20T21:46:45.563192+0000","last_clean":"2023-09-20T21:46:45.563192+0000","last_became_active":"2023-09-20T21:46:41.511005+0000","last_became_peered":"2023-09-20T21:46:41.511005+0000","last_unstale":"2023-09-20T21:46:45.563192+0000","last_undegraded":"2023-09-20T21:46:45.563192+0000","last_fullsized":"2023-09-20T21:46:45.563192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:37:59.684808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.862394+0000","last_change":"2023-09-20T21:46:45.580917+0000","last_active":"2023-09-20T21:46:46.862394+0000","last_peered":"2023-09-20T21:46:46.862394+0000","last_clean":"2023-09-20T21:46:46.862394+0000","last_became_active":"2023-09-20T21:46:45.580505+0000","last_became_peered":"2023-09-20T21:46:45.580505+0000","last_unstale":"2023-09-20T21:46:46.862394+0000","last_undegraded":"2023-09-20T21:46:46.862394+0000","last_fullsized":"2023-09-20T21:46:46.862394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:52:19.733919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.591401+0000","last_change":"2023-09-20T21:46:40.507040+0000","last_active":"2023-09-20T21:47:05.591401+0000","last_peered":"2023-09-20T21:47:05.591401+0000","last_clean":"2023-09-20T21:47:05.591401+0000","last_became_active":"2023-09-20T21:46:40.506712+0000","last_became_peered":"2023-09-20T21:46:40.506712+0000","last_unstale":"2023-09-20T21:47:05.591401+0000","last_undegraded":"2023-09-20T21:47:05.591401+0000","last_fullsized":"2023-09-20T21:47:05.591401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:02:56.040911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568722+0000","last_change":"2023-09-20T21:46:41.504794+0000","last_active":"2023-09-20T21:46:45.568722+0000","last_peered":"2023-09-20T21:46:45.568722+0000","last_clean":"2023-09-20T21:46:45.568722+0000","last_became_active":"2023-09-20T21:46:41.504372+0000","last_became_peered":"2023-09-20T21:46:41.504372+0000","last_unstale":"2023-09-20T21:46:45.568722+0000","last_undegraded":"2023-09-20T21:46:45.568722+0000","last_fullsized":"2023-09-20T21:46:45.568722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:42:20.527306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.564942+0000","last_change":"2023-09-20T21:46:45.586364+0000","last_active":"2023-09-20T21:46:46.564942+0000","last_peered":"2023-09-20T21:46:46.564942+0000","last_clean":"2023-09-20T21:46:46.564942+0000","last_became_active":"2023-09-20T21:46:45.584620+0000","last_became_peered":"2023-09-20T21:46:45.584620+0000","last_unstale":"2023-09-20T21:46:46.564942+0000","last_undegraded":"2023-09-20T21:46:46.564942+0000","last_fullsized":"2023-09-20T21:46:46.564942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:05:20.158118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.588953+0000","last_change":"2023-09-20T21:46:40.500797+0000","last_active":"2023-09-20T21:47:05.588953+0000","last_peered":"2023-09-20T21:47:05.588953+0000","last_clean":"2023-09-20T21:47:05.588953+0000","last_became_active":"2023-09-20T21:46:40.500337+0000","last_became_peered":"2023-09-20T21:46:40.500337+0000","last_unstale":"2023-09-20T21:47:05.588953+0000","last_undegraded":"2023-09-20T21:47:05.588953+0000","last_fullsized":"2023-09-20T21:47:05.588953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:19.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573645+0000","last_change":"2023-09-20T21:46:41.507805+0000","last_active":"2023-09-20T21:46:45.573645+0000","last_peered":"2023-09-20T21:46:45.573645+0000","last_clean":"2023-09-20T21:46:45.573645+0000","last_became_active":"2023-09-20T21:46:41.507441+0000","last_became_peered":"2023-09-20T21:46:41.507441+0000","last_unstale":"2023-09-20T21:46:45.573645+0000","last_undegraded":"2023-09-20T21:46:45.573645+0000","last_fullsized":"2023-09-20T21:46:45.573645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:51:33.240534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.503294+0000","last_change":"2023-09-20T21:46:45.583081+0000","last_active":"2023-09-20T21:46:47.503294+0000","last_peered":"2023-09-20T21:46:47.503294+0000","last_clean":"2023-09-20T21:46:47.503294+0000","last_became_active":"2023-09-20T21:46:45.580496+0000","last_became_peered":"2023-09-20T21:46:45.580496+0000","last_unstale":"2023-09-20T21:46:47.503294+0000","last_undegraded":"2023-09-20T21:46:47.503294+0000","last_fullsized":"2023-09-20T21:46:47.503294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:42.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568676+0000","last_change":"2023-09-20T21:46:40.512739+0000","last_active":"2023-09-20T21:46:45.568676+0000","last_peered":"2023-09-20T21:46:45.568676+0000","last_clean":"2023-09-20T21:46:45.568676+0000","last_became_active":"2023-09-20T21:46:40.512485+0000","last_became_peered":"2023-09-20T21:46:40.512485+0000","last_unstale":"2023-09-20T21:46:45.568676+0000","last_undegraded":"2023-09-20T21:46:45.568676+0000","last_fullsized":"2023-09-20T21:46:45.568676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:02:49.392560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569091+0000","last_change":"2023-09-20T21:46:41.503093+0000","last_active":"2023-09-20T21:46:45.569091+0000","last_peered":"2023-09-20T21:46:45.569091+0000","last_clean":"2023-09-20T21:46:45.569091+0000","last_became_active":"2023-09-20T21:46:41.502806+0000","last_became_peered":"2023-09-20T21:46:41.502806+0000","last_unstale":"2023-09-20T21:46:45.569091+0000","last_undegraded":"2023-09-20T21:46:45.569091+0000","last_fullsized":"2023-09-20T21:46:45.569091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:53:21.143258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565158+0000","last_change":"2023-09-20T21:46:45.579478+0000","last_active":"2023-09-20T21:46:46.565158+0000","last_peered":"2023-09-20T21:46:46.565158+0000","last_clean":"2023-09-20T21:46:46.565158+0000","last_became_active":"2023-09-20T21:46:45.579055+0000","last_became_peered":"2023-09-20T21:46:45.579055+0000","last_unstale":"2023-09-20T21:46:46.565158+0000","last_undegraded":"2023-09-20T21:46:46.565158+0000","last_fullsized":"2023-09-20T21:46:46.565158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:17:32.013280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.590075+0000","last_change":"2023-09-20T21:46:40.507321+0000","last_active":"2023-09-20T21:47:05.590075+0000","last_peered":"2023-09-20T21:47:05.590075+0000","last_clean":"2023-09-20T21:47:05.590075+0000","last_became_active":"2023-09-20T21:46:40.506983+0000","last_became_peered":"2023-09-20T21:46:40.506983+0000","last_unstale":"2023-09-20T21:47:05.590075+0000","last_undegraded":"2023-09-20T21:47:05.590075+0000","last_fullsized":"2023-09-20T21:47:05.590075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:46:40.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502157+0000","last_change":"2023-09-20T21:46:41.509646+0000","last_active":"2023-09-20T21:46:47.502157+0000","last_peered":"2023-09-20T21:46:47.502157+0000","last_clean":"2023-09-20T21:46:47.502157+0000","last_became_active":"2023-09-20T21:46:41.509183+0000","last_became_peered":"2023-09-20T21:46:41.509183+0000","last_unstale":"2023-09-20T21:46:47.502157+0000","last_undegraded":"2023-09-20T21:46:47.502157+0000","last_fullsized":"2023-09-20T21:46:47.502157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:17:21.084830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794668+0000","last_change":"2023-09-20T21:46:45.583709+0000","last_active":"2023-09-20T21:46:46.794668+0000","last_peered":"2023-09-20T21:46:46.794668+0000","last_clean":"2023-09-20T21:46:46.794668+0000","last_became_active":"2023-09-20T21:46:45.583302+0000","last_became_peered":"2023-09-20T21:46:45.583302+0000","last_unstale":"2023-09-20T21:46:46.794668+0000","last_undegraded":"2023-09-20T21:46:46.794668+0000","last_fullsized":"2023-09-20T21:46:46.794668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:31:35.472750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.590965+0000","last_change":"2023-09-20T21:46:40.506411+0000","last_active":"2023-09-20T21:47:05.590965+0000","last_peered":"2023-09-20T21:47:05.590965+0000","last_clean":"2023-09-20T21:47:05.590965+0000","last_became_active":"2023-09-20T21:46:40.506116+0000","last_became_peered":"2023-09-20T21:46:40.506116+0000","last_unstale":"2023-09-20T21:47:05.590965+0000","last_undegraded":"2023-09-20T21:47:05.590965+0000","last_fullsized":"2023-09-20T21:47:05.590965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:23:19.175007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575844+0000","last_change":"2023-09-20T21:46:41.513793+0000","last_active":"2023-09-20T21:46:45.575844+0000","last_peered":"2023-09-20T21:46:45.575844+0000","last_clean":"2023-09-20T21:46:45.575844+0000","last_became_active":"2023-09-20T21:46:41.513434+0000","last_became_peered":"2023-09-20T21:46:41.513434+0000","last_unstale":"2023-09-20T21:46:45.575844+0000","last_undegraded":"2023-09-20T21:46:45.575844+0000","last_fullsized":"2023-09-20T21:46:45.575844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:00:57.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.297451+0000","last_change":"2023-09-20T21:46:45.578188+0000","last_active":"2023-09-20T21:46:47.297451+0000","last_peered":"2023-09-20T21:46:47.297451+0000","last_clean":"2023-09-20T21:46:47.297451+0000","last_became_active":"2023-09-20T21:46:45.577548+0000","last_became_peered":"2023-09-20T21:46:45.577548+0000","last_unstale":"2023-09-20T21:46:47.297451+0000","last_undegraded":"2023-09-20T21:46:47.297451+0000","last_fullsized":"2023-09-20T21:46:47.297451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:54:58.894269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573689+0000","last_change":"2023-09-20T21:46:40.505465+0000","last_active":"2023-09-20T21:46:45.573689+0000","last_peered":"2023-09-20T21:46:45.573689+0000","last_clean":"2023-09-20T21:46:45.573689+0000","last_became_active":"2023-09-20T21:46:40.505174+0000","last_became_peered":"2023-09-20T21:46:40.505174+0000","last_unstale":"2023-09-20T21:46:45.573689+0000","last_undegraded":"2023-09-20T21:46:45.573689+0000","last_fullsized":"2023-09-20T21:46:45.573689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:50:53.191963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568918+0000","last_change":"2023-09-20T21:46:41.507370+0000","last_active":"2023-09-20T21:46:45.568918+0000","last_peered":"2023-09-20T21:46:45.568918+0000","last_clean":"2023-09-20T21:46:45.568918+0000","last_became_active":"2023-09-20T21:46:41.507035+0000","last_became_peered":"2023-09-20T21:46:41.507035+0000","last_unstale":"2023-09-20T21:46:45.568918+0000","last_undegraded":"2023-09-20T21:46:45.568918+0000","last_fullsized":"2023-09-20T21:46:45.568918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:50.298899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565547+0000","last_change":"2023-09-20T21:46:45.584458+0000","last_active":"2023-09-20T21:46:46.565547+0000","last_peered":"2023-09-20T21:46:46.565547+0000","last_clean":"2023-09-20T21:46:46.565547+0000","last_became_active":"2023-09-20T21:46:45.584033+0000","last_became_peered":"2023-09-20T21:46:45.584033+0000","last_unstale":"2023-09-20T21:46:46.565547+0000","last_undegraded":"2023-09-20T21:46:46.565547+0000","last_fullsized":"2023-09-20T21:46:46.565547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:48:44.653880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570682+0000","last_change":"2023-09-20T21:46:40.501329+0000","last_active":"2023-09-20T21:46:45.570682+0000","last_peered":"2023-09-20T21:46:45.570682+0000","last_clean":"2023-09-20T21:46:45.570682+0000","last_became_active":"2023-09-20T21:46:40.500999+0000","last_became_peered":"2023-09-20T21:46:40.500999+0000","last_unstale":"2023-09-20T21:46:45.570682+0000","last_undegraded":"2023-09-20T21:46:45.570682+0000","last_fullsized":"2023-09-20T21:46:45.570682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:38.028398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568141+0000","last_change":"2023-09-20T21:46:41.504583+0000","last_active":"2023-09-20T21:46:45.568141+0000","last_peered":"2023-09-20T21:46:45.568141+0000","last_clean":"2023-09-20T21:46:45.568141+0000","last_became_active":"2023-09-20T21:46:41.504273+0000","last_became_peered":"2023-09-20T21:46:41.504273+0000","last_unstale":"2023-09-20T21:46:45.568141+0000","last_undegraded":"2023-09-20T21:46:45.568141+0000","last_fullsized":"2023-09-20T21:46:45.568141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:37.100382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502476+0000","last_change":"2023-09-20T21:46:45.587116+0000","last_active":"2023-09-20T21:46:47.502476+0000","last_peered":"2023-09-20T21:46:47.502476+0000","last_clean":"2023-09-20T21:46:47.502476+0000","last_became_active":"2023-09-20T21:46:45.586629+0000","last_became_peered":"2023-09-20T21:46:45.586629+0000","last_unstale":"2023-09-20T21:46:47.502476+0000","last_undegraded":"2023-09-20T21:46:47.502476+0000","last_fullsized":"2023-09-20T21:46:47.502476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:10:33.024235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'16","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.572218+0000","last_change":"2023-09-20T21:46:40.499985+0000","last_active":"2023-09-20T21:47:07.572218+0000","last_peered":"2023-09-20T21:47:07.572218+0000","last_clean":"2023-09-20T21:47:07.572218+0000","last_became_active":"2023-09-20T21:46:40.499646+0000","last_became_peered":"2023-09-20T21:46:40.499646+0000","last_unstale":"2023-09-20T21:47:07.572218+0000","last_undegraded":"2023-09-20T21:47:07.572218+0000","last_fullsized":"2023-09-20T21:47:07.572218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:52:02.758556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568813+0000","last_change":"2023-09-20T21:46:41.501241+0000","last_active":"2023-09-20T21:46:45.568813+0000","last_peered":"2023-09-20T21:46:45.568813+0000","last_clean":"2023-09-20T21:46:45.568813+0000","last_became_active":"2023-09-20T21:46:41.500902+0000","last_became_peered":"2023-09-20T21:46:41.500902+0000","last_unstale":"2023-09-20T21:46:45.568813+0000","last_undegraded":"2023-09-20T21:46:45.568813+0000","last_fullsized":"2023-09-20T21:46:45.568813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:21:55.210065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794066+0000","last_change":"2023-09-20T21:46:45.583405+0000","last_active":"2023-09-20T21:46:46.794066+0000","last_peered":"2023-09-20T21:46:46.794066+0000","last_clean":"2023-09-20T21:46:46.794066+0000","last_became_active":"2023-09-20T21:46:45.582945+0000","last_became_peered":"2023-09-20T21:46:45.582945+0000","last_unstale":"2023-09-20T21:46:46.794066+0000","last_undegraded":"2023-09-20T21:46:46.794066+0000","last_fullsized":"2023-09-20T21:46:46.794066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:27:14.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.561084+0000","last_change":"2023-09-20T21:46:40.507744+0000","last_active":"2023-09-20T21:47:07.561084+0000","last_peered":"2023-09-20T21:47:07.561084+0000","last_clean":"2023-09-20T21:47:07.561084+0000","last_became_active":"2023-09-20T21:46:40.507454+0000","last_became_peered":"2023-09-20T21:46:40.507454+0000","last_unstale":"2023-09-20T21:47:07.561084+0000","last_undegraded":"2023-09-20T21:47:07.561084+0000","last_fullsized":"2023-09-20T21:47:07.561084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:00:24.492789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563417+0000","last_change":"2023-09-20T21:46:41.504263+0000","last_active":"2023-09-20T21:46:45.563417+0000","last_peered":"2023-09-20T21:46:45.563417+0000","last_clean":"2023-09-20T21:46:45.563417+0000","last_became_active":"2023-09-20T21:46:41.503961+0000","last_became_peered":"2023-09-20T21:46:41.503961+0000","last_unstale":"2023-09-20T21:46:45.563417+0000","last_undegraded":"2023-09-20T21:46:45.563417+0000","last_fullsized":"2023-09-20T21:46:45.563417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:31:58.035517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.944063+0000","last_change":"2023-09-20T21:46:45.581343+0000","last_active":"2023-09-20T21:46:47.944063+0000","last_peered":"2023-09-20T21:46:47.944063+0000","last_clean":"2023-09-20T21:46:47.944063+0000","last_became_active":"2023-09-20T21:46:45.580830+0000","last_became_peered":"2023-09-20T21:46:45.580830+0000","last_unstale":"2023-09-20T21:46:47.944063+0000","last_undegraded":"2023-09-20T21:46:47.944063+0000","last_fullsized":"2023-09-20T21:46:47.944063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:20:49.865825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563359+0000","last_change":"2023-09-20T21:46:40.501850+0000","last_active":"2023-09-20T21:46:45.563359+0000","last_peered":"2023-09-20T21:46:45.563359+0000","last_clean":"2023-09-20T21:46:45.563359+0000","last_became_active":"2023-09-20T21:46:40.501500+0000","last_became_peered":"2023-09-20T21:46:40.501500+0000","last_unstale":"2023-09-20T21:46:45.563359+0000","last_undegraded":"2023-09-20T21:46:45.563359+0000","last_fullsized":"2023-09-20T21:46:45.563359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:14:15.907335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564629+0000","last_change":"2023-09-20T21:46:41.507432+0000","last_active":"2023-09-20T21:46:45.564629+0000","last_peered":"2023-09-20T21:46:45.564629+0000","last_clean":"2023-09-20T21:46:45.564629+0000","last_became_active":"2023-09-20T21:46:41.507119+0000","last_became_peered":"2023-09-20T21:46:41.507119+0000","last_unstale":"2023-09-20T21:46:45.564629+0000","last_undegraded":"2023-09-20T21:46:45.564629+0000","last_fullsized":"2023-09-20T21:46:45.564629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:14:10.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502815+0000","last_change":"2023-09-20T21:46:45.588455+0000","last_active":"2023-09-20T21:46:47.502815+0000","last_peered":"2023-09-20T21:46:47.502815+0000","last_clean":"2023-09-20T21:46:47.502815+0000","last_became_active":"2023-09-20T21:46:45.587980+0000","last_became_peered":"2023-09-20T21:46:45.587980+0000","last_unstale":"2023-09-20T21:46:47.502815+0000","last_undegraded":"2023-09-20T21:46:47.502815+0000","last_fullsized":"2023-09-20T21:46:47.502815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:07:35.697330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.568603+0000","last_change":"2023-09-20T21:46:40.506260+0000","last_active":"2023-09-20T21:46:46.568603+0000","last_peered":"2023-09-20T21:46:46.568603+0000","last_clean":"2023-09-20T21:46:46.568603+0000","last_became_active":"2023-09-20T21:46:40.505999+0000","last_became_peered":"2023-09-20T21:46:40.505999+0000","last_unstale":"2023-09-20T21:46:46.568603+0000","last_undegraded":"2023-09-20T21:46:46.568603+0000","last_fullsized":"2023-09-20T21:46:46.568603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:39:01.748038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564591+0000","last_change":"2023-09-20T21:46:41.496822+0000","last_active":"2023-09-20T21:46:45.564591+0000","last_peered":"2023-09-20T21:46:45.564591+0000","last_clean":"2023-09-20T21:46:45.564591+0000","last_became_active":"2023-09-20T21:46:41.496517+0000","last_became_peered":"2023-09-20T21:46:41.496517+0000","last_unstale":"2023-09-20T21:46:45.564591+0000","last_undegraded":"2023-09-20T21:46:45.564591+0000","last_fullsized":"2023-09-20T21:46:45.564591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:12.592040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502793+0000","last_change":"2023-09-20T21:46:45.587410+0000","last_active":"2023-09-20T21:46:47.502793+0000","last_peered":"2023-09-20T21:46:47.502793+0000","last_clean":"2023-09-20T21:46:47.502793+0000","last_became_active":"2023-09-20T21:46:45.586944+0000","last_became_peered":"2023-09-20T21:46:45.586944+0000","last_unstale":"2023-09-20T21:46:47.502793+0000","last_undegraded":"2023-09-20T21:46:47.502793+0000","last_fullsized":"2023-09-20T21:46:47.502793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:57.053765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568334+0000","last_change":"2023-09-20T21:46:41.511526+0000","last_active":"2023-09-20T21:46:45.568334+0000","last_peered":"2023-09-20T21:46:45.568334+0000","last_clean":"2023-09-20T21:46:45.568334+0000","last_became_active":"2023-09-20T21:46:41.511213+0000","last_became_peered":"2023-09-20T21:46:41.511213+0000","last_unstale":"2023-09-20T21:46:45.568334+0000","last_undegraded":"2023-09-20T21:46:45.568334+0000","last_fullsized":"2023-09-20T21:46:45.568334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:29:04.199190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569931+0000","last_change":"2023-09-20T21:46:40.514916+0000","last_active":"2023-09-20T21:46:45.569931+0000","last_peered":"2023-09-20T21:46:45.569931+0000","last_clean":"2023-09-20T21:46:45.569931+0000","last_became_active":"2023-09-20T21:46:40.514553+0000","last_became_peered":"2023-09-20T21:46:40.514553+0000","last_unstale":"2023-09-20T21:46:45.569931+0000","last_undegraded":"2023-09-20T21:46:45.569931+0000","last_fullsized":"2023-09-20T21:46:45.569931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:12:32.481852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.568779+0000","last_change":"2023-09-20T21:46:45.587489+0000","last_active":"2023-09-20T21:46:46.568779+0000","last_peered":"2023-09-20T21:46:46.568779+0000","last_clean":"2023-09-20T21:46:46.568779+0000","last_became_active":"2023-09-20T21:46:45.587022+0000","last_became_peered":"2023-09-20T21:46:45.587022+0000","last_unstale":"2023-09-20T21:46:46.568779+0000","last_undegraded":"2023-09-20T21:46:46.568779+0000","last_fullsized":"2023-09-20T21:46:46.568779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:09:05.870763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568768+0000","last_change":"2023-09-20T21:46:41.508886+0000","last_active":"2023-09-20T21:46:45.568768+0000","last_peered":"2023-09-20T21:46:45.568768+0000","last_clean":"2023-09-20T21:46:45.568768+0000","last_became_active":"2023-09-20T21:46:41.508470+0000","last_became_peered":"2023-09-20T21:46:41.508470+0000","last_unstale":"2023-09-20T21:46:45.568768+0000","last_undegraded":"2023-09-20T21:46:45.568768+0000","last_fullsized":"2023-09-20T21:46:45.568768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:45:04.888300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569676+0000","last_change":"2023-09-20T21:46:40.503350+0000","last_active":"2023-09-20T21:46:45.569676+0000","last_peered":"2023-09-20T21:46:45.569676+0000","last_clean":"2023-09-20T21:46:45.569676+0000","last_became_active":"2023-09-20T21:46:40.503045+0000","last_became_peered":"2023-09-20T21:46:40.503045+0000","last_unstale":"2023-09-20T21:46:45.569676+0000","last_undegraded":"2023-09-20T21:46:45.569676+0000","last_fullsized":"2023-09-20T21:46:45.569676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:31:09.004342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.567932+0000","last_change":"2023-09-20T21:45:35.180242+0000","last_active":"2023-09-20T21:46:45.567932+0000","last_peered":"2023-09-20T21:46:45.567932+0000","last_clean":"2023-09-20T21:46:45.567932+0000","last_became_active":"2023-09-20T21:45:35.179705+0000","last_became_peered":"2023-09-20T21:45:35.179705+0000","last_unstale":"2023-09-20T21:46:45.567932+0000","last_undegraded":"2023-09-20T21:46:45.567932+0000","last_fullsized":"2023-09-20T21:46:45.567932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_clean_scrub_stamp":"2023-09-20T21:42:25.826683+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:02:03.448188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794591+0000","last_change":"2023-09-20T21:46:45.581396+0000","last_active":"2023-09-20T21:46:46.794591+0000","last_peered":"2023-09-20T21:46:46.794591+0000","last_clean":"2023-09-20T21:46:46.794591+0000","last_became_active":"2023-09-20T21:46:45.581018+0000","last_became_peered":"2023-09-20T21:46:45.581018+0000","last_unstale":"2023-09-20T21:46:46.794591+0000","last_undegraded":"2023-09-20T21:46:46.794591+0000","last_fullsized":"2023-09-20T21:46:46.794591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:16:35.934460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569837+0000","last_change":"2023-09-20T21:46:41.514392+0000","last_active":"2023-09-20T21:46:45.569837+0000","last_peered":"2023-09-20T21:46:45.569837+0000","last_clean":"2023-09-20T21:46:45.569837+0000","last_became_active":"2023-09-20T21:46:41.514090+0000","last_became_peered":"2023-09-20T21:46:41.514090+0000","last_unstale":"2023-09-20T21:46:45.569837+0000","last_undegraded":"2023-09-20T21:46:45.569837+0000","last_fullsized":"2023-09-20T21:46:45.569837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:05:47.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:06.596564+0000","last_change":"2023-09-20T21:46:40.507346+0000","last_active":"2023-09-20T21:47:06.596564+0000","last_peered":"2023-09-20T21:47:06.596564+0000","last_clean":"2023-09-20T21:47:06.596564+0000","last_became_active":"2023-09-20T21:46:40.506969+0000","last_became_peered":"2023-09-20T21:46:40.506969+0000","last_unstale":"2023-09-20T21:47:06.596564+0000","last_undegraded":"2023-09-20T21:47:06.596564+0000","last_fullsized":"2023-09-20T21:47:06.596564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:01:20.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.297500+0000","last_change":"2023-09-20T21:46:45.578033+0000","last_active":"2023-09-20T21:46:47.297500+0000","last_peered":"2023-09-20T21:46:47.297500+0000","last_clean":"2023-09-20T21:46:47.297500+0000","last_became_active":"2023-09-20T21:46:45.577540+0000","last_became_peered":"2023-09-20T21:46:45.577540+0000","last_unstale":"2023-09-20T21:46:47.297500+0000","last_undegraded":"2023-09-20T21:46:47.297500+0000","last_fullsized":"2023-09-20T21:46:47.297500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:24:31.020660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.573672+0000","last_change":"2023-09-20T21:46:40.510634+0000","last_active":"2023-09-20T21:47:07.573672+0000","last_peered":"2023-09-20T21:47:07.573672+0000","last_clean":"2023-09-20T21:47:07.573672+0000","last_became_active":"2023-09-20T21:46:40.510390+0000","last_became_peered":"2023-09-20T21:46:40.510390+0000","last_unstale":"2023-09-20T21:47:07.573672+0000","last_undegraded":"2023-09-20T21:47:07.573672+0000","last_fullsized":"2023-09-20T21:47:07.573672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:44:49.272987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571610+0000","last_change":"2023-09-20T21:46:41.502981+0000","last_active":"2023-09-20T21:46:45.571610+0000","last_peered":"2023-09-20T21:46:45.571610+0000","last_clean":"2023-09-20T21:46:45.571610+0000","last_became_active":"2023-09-20T21:46:41.502594+0000","last_became_peered":"2023-09-20T21:46:41.502594+0000","last_unstale":"2023-09-20T21:46:45.571610+0000","last_undegraded":"2023-09-20T21:46:45.571610+0000","last_fullsized":"2023-09-20T21:46:45.571610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:41:15.261478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.862185+0000","last_change":"2023-09-20T21:46:45.582903+0000","last_active":"2023-09-20T21:46:46.862185+0000","last_peered":"2023-09-20T21:46:46.862185+0000","last_clean":"2023-09-20T21:46:46.862185+0000","last_became_active":"2023-09-20T21:46:45.582433+0000","last_became_peered":"2023-09-20T21:46:45.582433+0000","last_unstale":"2023-09-20T21:46:46.862185+0000","last_undegraded":"2023-09-20T21:46:46.862185+0000","last_fullsized":"2023-09-20T21:46:46.862185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:38:22.174903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563444+0000","last_change":"2023-09-20T21:46:40.507671+0000","last_active":"2023-09-20T21:46:45.563444+0000","last_peered":"2023-09-20T21:46:45.563444+0000","last_clean":"2023-09-20T21:46:45.563444+0000","last_became_active":"2023-09-20T21:46:40.507288+0000","last_became_peered":"2023-09-20T21:46:40.507288+0000","last_unstale":"2023-09-20T21:46:45.563444+0000","last_undegraded":"2023-09-20T21:46:45.563444+0000","last_fullsized":"2023-09-20T21:46:45.563444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:27:02.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570001+0000","last_change":"2023-09-20T21:46:41.507654+0000","last_active":"2023-09-20T21:46:45.570001+0000","last_peered":"2023-09-20T21:46:45.570001+0000","last_clean":"2023-09-20T21:46:45.570001+0000","last_became_active":"2023-09-20T21:46:41.507362+0000","last_became_peered":"2023-09-20T21:46:41.507362+0000","last_unstale":"2023-09-20T21:46:45.570001+0000","last_undegraded":"2023-09-20T21:46:45.570001+0000","last_fullsized":"2023-09-20T21:46:45.570001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:32:08.973379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565429+0000","last_change":"2023-09-20T21:46:45.578437+0000","last_active":"2023-09-20T21:46:46.565429+0000","last_peered":"2023-09-20T21:46:46.565429+0000","last_clean":"2023-09-20T21:46:46.565429+0000","last_became_active":"2023-09-20T21:46:45.578010+0000","last_became_peered":"2023-09-20T21:46:45.578010+0000","last_unstale":"2023-09-20T21:46:46.565429+0000","last_undegraded":"2023-09-20T21:46:46.565429+0000","last_fullsized":"2023-09-20T21:46:46.565429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:59:05.494861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573073+0000","last_change":"2023-09-20T21:46:40.508937+0000","last_active":"2023-09-20T21:46:45.573073+0000","last_peered":"2023-09-20T21:46:45.573073+0000","last_clean":"2023-09-20T21:46:45.573073+0000","last_became_active":"2023-09-20T21:46:40.508639+0000","last_became_peered":"2023-09-20T21:46:40.508639+0000","last_unstale":"2023-09-20T21:46:45.573073+0000","last_undegraded":"2023-09-20T21:46:45.573073+0000","last_fullsized":"2023-09-20T21:46:45.573073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:50:48.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571508+0000","last_change":"2023-09-20T21:46:41.510702+0000","last_active":"2023-09-20T21:46:45.571508+0000","last_peered":"2023-09-20T21:46:45.571508+0000","last_clean":"2023-09-20T21:46:45.571508+0000","last_became_active":"2023-09-20T21:46:41.510357+0000","last_became_peered":"2023-09-20T21:46:41.510357+0000","last_unstale":"2023-09-20T21:46:45.571508+0000","last_undegraded":"2023-09-20T21:46:45.571508+0000","last_fullsized":"2023-09-20T21:46:45.571508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:35:44.455981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.503739+0000","last_change":"2023-09-20T21:46:45.581110+0000","last_active":"2023-09-20T21:46:47.503739+0000","last_peered":"2023-09-20T21:46:47.503739+0000","last_clean":"2023-09-20T21:46:47.503739+0000","last_became_active":"2023-09-20T21:46:45.580498+0000","last_became_peered":"2023-09-20T21:46:45.580498+0000","last_unstale":"2023-09-20T21:46:47.503739+0000","last_undegraded":"2023-09-20T21:46:47.503739+0000","last_fullsized":"2023-09-20T21:46:47.503739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:19:22.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571650+0000","last_change":"2023-09-20T21:46:40.497017+0000","last_active":"2023-09-20T21:46:45.571650+0000","last_peered":"2023-09-20T21:46:45.571650+0000","last_clean":"2023-09-20T21:46:45.571650+0000","last_became_active":"2023-09-20T21:46:40.496722+0000","last_became_peered":"2023-09-20T21:46:40.496722+0000","last_unstale":"2023-09-20T21:46:45.571650+0000","last_undegraded":"2023-09-20T21:46:45.571650+0000","last_fullsized":"2023-09-20T21:46:45.571650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:12:07.017318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501752+0000","last_change":"2023-09-20T21:46:41.508793+0000","last_active":"2023-09-20T21:46:47.501752+0000","last_peered":"2023-09-20T21:46:47.501752+0000","last_clean":"2023-09-20T21:46:47.501752+0000","last_became_active":"2023-09-20T21:46:41.508453+0000","last_became_peered":"2023-09-20T21:46:41.508453+0000","last_unstale":"2023-09-20T21:46:47.501752+0000","last_undegraded":"2023-09-20T21:46:47.501752+0000","last_fullsized":"2023-09-20T21:46:47.501752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:35:12.711570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565653+0000","last_change":"2023-09-20T21:46:45.583519+0000","last_active":"2023-09-20T21:46:46.565653+0000","last_peered":"2023-09-20T21:46:46.565653+0000","last_clean":"2023-09-20T21:46:46.565653+0000","last_became_active":"2023-09-20T21:46:45.582933+0000","last_became_peered":"2023-09-20T21:46:45.582933+0000","last_unstale":"2023-09-20T21:46:46.565653+0000","last_undegraded":"2023-09-20T21:46:46.565653+0000","last_fullsized":"2023-09-20T21:46:46.565653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:13:20.237865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.591317+0000","last_change":"2023-09-20T21:46:40.507384+0000","last_active":"2023-09-20T21:47:05.591317+0000","last_peered":"2023-09-20T21:47:05.591317+0000","last_clean":"2023-09-20T21:47:05.591317+0000","last_became_active":"2023-09-20T21:46:40.507021+0000","last_became_peered":"2023-09-20T21:46:40.507021+0000","last_unstale":"2023-09-20T21:47:05.591317+0000","last_undegraded":"2023-09-20T21:47:05.591317+0000","last_fullsized":"2023-09-20T21:47:05.591317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:48:04.360664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563191+0000","last_change":"2023-09-20T21:46:41.511537+0000","last_active":"2023-09-20T21:46:45.563191+0000","last_peered":"2023-09-20T21:46:45.563191+0000","last_clean":"2023-09-20T21:46:45.563191+0000","last_became_active":"2023-09-20T21:46:41.511238+0000","last_became_peered":"2023-09-20T21:46:41.511238+0000","last_unstale":"2023-09-20T21:46:45.563191+0000","last_undegraded":"2023-09-20T21:46:45.563191+0000","last_fullsized":"2023-09-20T21:46:45.563191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:01:41.492824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.503278+0000","last_change":"2023-09-20T21:46:45.585750+0000","last_active":"2023-09-20T21:46:47.503278+0000","last_peered":"2023-09-20T21:46:47.503278+0000","last_clean":"2023-09-20T21:46:47.503278+0000","last_became_active":"2023-09-20T21:46:45.585213+0000","last_became_peered":"2023-09-20T21:46:45.585213+0000","last_unstale":"2023-09-20T21:46:47.503278+0000","last_undegraded":"2023-09-20T21:46:47.503278+0000","last_fullsized":"2023-09-20T21:46:47.503278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:31:15.650766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568158+0000","last_change":"2023-09-20T21:46:40.506394+0000","last_active":"2023-09-20T21:46:45.568158+0000","last_peered":"2023-09-20T21:46:45.568158+0000","last_clean":"2023-09-20T21:46:45.568158+0000","last_became_active":"2023-09-20T21:46:40.506085+0000","last_became_peered":"2023-09-20T21:46:40.506085+0000","last_unstale":"2023-09-20T21:46:45.568158+0000","last_undegraded":"2023-09-20T21:46:45.568158+0000","last_fullsized":"2023-09-20T21:46:45.568158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:02.738600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570438+0000","last_change":"2023-09-20T21:46:41.509491+0000","last_active":"2023-09-20T21:46:45.570438+0000","last_peered":"2023-09-20T21:46:45.570438+0000","last_clean":"2023-09-20T21:46:45.570438+0000","last_became_active":"2023-09-20T21:46:41.509039+0000","last_became_peered":"2023-09-20T21:46:41.509039+0000","last_unstale":"2023-09-20T21:46:45.570438+0000","last_undegraded":"2023-09-20T21:46:45.570438+0000","last_fullsized":"2023-09-20T21:46:45.570438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:57:39.003154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.503259+0000","last_change":"2023-09-20T21:46:45.581651+0000","last_active":"2023-09-20T21:46:47.503259+0000","last_peered":"2023-09-20T21:46:47.503259+0000","last_clean":"2023-09-20T21:46:47.503259+0000","last_became_active":"2023-09-20T21:46:45.581224+0000","last_became_peered":"2023-09-20T21:46:45.581224+0000","last_unstale":"2023-09-20T21:46:47.503259+0000","last_undegraded":"2023-09-20T21:46:47.503259+0000","last_fullsized":"2023-09-20T21:46:47.503259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:32:17.778301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569410+0000","last_change":"2023-09-20T21:46:40.501133+0000","last_active":"2023-09-20T21:46:45.569410+0000","last_peered":"2023-09-20T21:46:45.569410+0000","last_clean":"2023-09-20T21:46:45.569410+0000","last_became_active":"2023-09-20T21:46:40.500831+0000","last_became_peered":"2023-09-20T21:46:40.500831+0000","last_unstale":"2023-09-20T21:46:45.569410+0000","last_undegraded":"2023-09-20T21:46:45.569410+0000","last_fullsized":"2023-09-20T21:46:45.569410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:20.498532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568903+0000","last_change":"2023-09-20T21:46:41.507218+0000","last_active":"2023-09-20T21:46:45.568903+0000","last_peered":"2023-09-20T21:46:45.568903+0000","last_clean":"2023-09-20T21:46:45.568903+0000","last_became_active":"2023-09-20T21:46:41.506889+0000","last_became_peered":"2023-09-20T21:46:41.506889+0000","last_unstale":"2023-09-20T21:46:45.568903+0000","last_undegraded":"2023-09-20T21:46:45.568903+0000","last_fullsized":"2023-09-20T21:46:45.568903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:08:40.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.503610+0000","last_change":"2023-09-20T21:46:45.579269+0000","last_active":"2023-09-20T21:46:47.503610+0000","last_peered":"2023-09-20T21:46:47.503610+0000","last_clean":"2023-09-20T21:46:47.503610+0000","last_became_active":"2023-09-20T21:46:45.578541+0000","last_became_peered":"2023-09-20T21:46:45.578541+0000","last_unstale":"2023-09-20T21:46:47.503610+0000","last_undegraded":"2023-09-20T21:46:47.503610+0000","last_fullsized":"2023-09-20T21:46:47.503610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:07:50.241947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569525+0000","last_change":"2023-09-20T21:46:40.508774+0000","last_active":"2023-09-20T21:46:45.569525+0000","last_peered":"2023-09-20T21:46:45.569525+0000","last_clean":"2023-09-20T21:46:45.569525+0000","last_became_active":"2023-09-20T21:46:40.508480+0000","last_became_peered":"2023-09-20T21:46:40.508480+0000","last_unstale":"2023-09-20T21:46:45.569525+0000","last_undegraded":"2023-09-20T21:46:45.569525+0000","last_fullsized":"2023-09-20T21:46:45.569525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:22.643724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570698+0000","last_change":"2023-09-20T21:46:41.503763+0000","last_active":"2023-09-20T21:46:45.570698+0000","last_peered":"2023-09-20T21:46:45.570698+0000","last_clean":"2023-09-20T21:46:45.570698+0000","last_became_active":"2023-09-20T21:46:41.503360+0000","last_became_peered":"2023-09-20T21:46:41.503360+0000","last_unstale":"2023-09-20T21:46:45.570698+0000","last_undegraded":"2023-09-20T21:46:45.570698+0000","last_fullsized":"2023-09-20T21:46:45.570698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:03:00.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502032+0000","last_change":"2023-09-20T21:46:45.586702+0000","last_active":"2023-09-20T21:46:47.502032+0000","last_peered":"2023-09-20T21:46:47.502032+0000","last_clean":"2023-09-20T21:46:47.502032+0000","last_became_active":"2023-09-20T21:46:45.586177+0000","last_became_peered":"2023-09-20T21:46:45.586177+0000","last_unstale":"2023-09-20T21:46:47.502032+0000","last_undegraded":"2023-09-20T21:46:47.502032+0000","last_fullsized":"2023-09-20T21:46:47.502032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:19:03.883440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568892+0000","last_change":"2023-09-20T21:46:40.507270+0000","last_active":"2023-09-20T21:46:45.568892+0000","last_peered":"2023-09-20T21:46:45.568892+0000","last_clean":"2023-09-20T21:46:45.568892+0000","last_became_active":"2023-09-20T21:46:40.506933+0000","last_became_peered":"2023-09-20T21:46:40.506933+0000","last_unstale":"2023-09-20T21:46:45.568892+0000","last_undegraded":"2023-09-20T21:46:45.568892+0000","last_fullsized":"2023-09-20T21:46:45.568892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:59.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569676+0000","last_change":"2023-09-20T21:46:41.507868+0000","last_active":"2023-09-20T21:46:45.569676+0000","last_peered":"2023-09-20T21:46:45.569676+0000","last_clean":"2023-09-20T21:46:45.569676+0000","last_became_active":"2023-09-20T21:46:41.507568+0000","last_became_peered":"2023-09-20T21:46:41.507568+0000","last_unstale":"2023-09-20T21:46:45.569676+0000","last_undegraded":"2023-09-20T21:46:45.569676+0000","last_fullsized":"2023-09-20T21:46:45.569676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:44.615620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.793795+0000","last_change":"2023-09-20T21:46:45.578726+0000","last_active":"2023-09-20T21:46:46.793795+0000","last_peered":"2023-09-20T21:46:46.793795+0000","last_clean":"2023-09-20T21:46:46.793795+0000","last_became_active":"2023-09-20T21:46:45.578326+0000","last_became_peered":"2023-09-20T21:46:45.578326+0000","last_unstale":"2023-09-20T21:46:46.793795+0000","last_undegraded":"2023-09-20T21:46:46.793795+0000","last_fullsized":"2023-09-20T21:46:46.793795+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:21:34.564840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570141+0000","last_change":"2023-09-20T21:46:40.509024+0000","last_active":"2023-09-20T21:46:45.570141+0000","last_peered":"2023-09-20T21:46:45.570141+0000","last_clean":"2023-09-20T21:46:45.570141+0000","last_became_active":"2023-09-20T21:46:40.508659+0000","last_became_peered":"2023-09-20T21:46:40.508659+0000","last_unstale":"2023-09-20T21:46:45.570141+0000","last_undegraded":"2023-09-20T21:46:45.570141+0000","last_fullsized":"2023-09-20T21:46:45.570141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:12:08.302068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569100+0000","last_change":"2023-09-20T21:46:41.510307+0000","last_active":"2023-09-20T21:46:45.569100+0000","last_peered":"2023-09-20T21:46:45.569100+0000","last_clean":"2023-09-20T21:46:45.569100+0000","last_became_active":"2023-09-20T21:46:41.510014+0000","last_became_peered":"2023-09-20T21:46:41.510014+0000","last_unstale":"2023-09-20T21:46:45.569100+0000","last_undegraded":"2023-09-20T21:46:45.569100+0000","last_fullsized":"2023-09-20T21:46:45.569100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:05:55.062515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563226+0000","last_change":"2023-09-20T21:46:41.511525+0000","last_active":"2023-09-20T21:46:45.563226+0000","last_peered":"2023-09-20T21:46:45.563226+0000","last_clean":"2023-09-20T21:46:45.563226+0000","last_became_active":"2023-09-20T21:46:41.511203+0000","last_became_peered":"2023-09-20T21:46:41.511203+0000","last_unstale":"2023-09-20T21:46:45.563226+0000","last_undegraded":"2023-09-20T21:46:45.563226+0000","last_fullsized":"2023-09-20T21:46:45.563226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:42:15.757422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570189+0000","last_change":"2023-09-20T21:46:41.502936+0000","last_active":"2023-09-20T21:46:45.570189+0000","last_peered":"2023-09-20T21:46:45.570189+0000","last_clean":"2023-09-20T21:46:45.570189+0000","last_became_active":"2023-09-20T21:46:41.502617+0000","last_became_peered":"2023-09-20T21:46:41.502617+0000","last_unstale":"2023-09-20T21:46:45.570189+0000","last_undegraded":"2023-09-20T21:46:45.570189+0000","last_fullsized":"2023-09-20T21:46:45.570189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:45:21.197652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568057+0000","last_change":"2023-09-20T21:46:41.510766+0000","last_active":"2023-09-20T21:46:45.568057+0000","last_peered":"2023-09-20T21:46:45.568057+0000","last_clean":"2023-09-20T21:46:45.568057+0000","last_became_active":"2023-09-20T21:46:41.510433+0000","last_became_peered":"2023-09-20T21:46:41.510433+0000","last_unstale":"2023-09-20T21:46:45.568057+0000","last_undegraded":"2023-09-20T21:46:45.568057+0000","last_fullsized":"2023-09-20T21:46:45.568057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:56:12.085926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.572421+0000","last_change":"2023-09-20T21:46:41.508846+0000","last_active":"2023-09-20T21:46:45.572421+0000","last_peered":"2023-09-20T21:46:45.572421+0000","last_clean":"2023-09-20T21:46:45.572421+0000","last_became_active":"2023-09-20T21:46:41.508518+0000","last_became_peered":"2023-09-20T21:46:41.508518+0000","last_unstale":"2023-09-20T21:46:45.572421+0000","last_undegraded":"2023-09-20T21:46:45.572421+0000","last_fullsized":"2023-09-20T21:46:45.572421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:01:58.195327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571696+0000","last_change":"2023-09-20T21:46:41.505929+0000","last_active":"2023-09-20T21:46:45.571696+0000","last_peered":"2023-09-20T21:46:45.571696+0000","last_clean":"2023-09-20T21:46:45.571696+0000","last_became_active":"2023-09-20T21:46:41.505542+0000","last_became_peered":"2023-09-20T21:46:41.505542+0000","last_unstale":"2023-09-20T21:46:45.571696+0000","last_undegraded":"2023-09-20T21:46:45.571696+0000","last_fullsized":"2023-09-20T21:46:45.571696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:24:47.520344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563371+0000","last_change":"2023-09-20T21:46:41.507603+0000","last_active":"2023-09-20T21:46:45.563371+0000","last_peered":"2023-09-20T21:46:45.563371+0000","last_clean":"2023-09-20T21:46:45.563371+0000","last_became_active":"2023-09-20T21:46:41.507340+0000","last_became_peered":"2023-09-20T21:46:41.507340+0000","last_unstale":"2023-09-20T21:46:45.563371+0000","last_undegraded":"2023-09-20T21:46:45.563371+0000","last_fullsized":"2023-09-20T21:46:45.563371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:50:19.756491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568500+0000","last_change":"2023-09-20T21:46:41.509625+0000","last_active":"2023-09-20T21:46:45.568500+0000","last_peered":"2023-09-20T21:46:45.568500+0000","last_clean":"2023-09-20T21:46:45.568500+0000","last_became_active":"2023-09-20T21:46:41.509301+0000","last_became_peered":"2023-09-20T21:46:41.509301+0000","last_unstale":"2023-09-20T21:46:45.568500+0000","last_undegraded":"2023-09-20T21:46:45.568500+0000","last_fullsized":"2023-09-20T21:46:45.568500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:15:03.108060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570117+0000","last_change":"2023-09-20T21:46:41.516781+0000","last_active":"2023-09-20T21:46:45.570117+0000","last_peered":"2023-09-20T21:46:45.570117+0000","last_clean":"2023-09-20T21:46:45.570117+0000","last_became_active":"2023-09-20T21:46:41.516496+0000","last_became_peered":"2023-09-20T21:46:41.516496+0000","last_unstale":"2023-09-20T21:46:45.570117+0000","last_undegraded":"2023-09-20T21:46:45.570117+0000","last_fullsized":"2023-09-20T21:46:45.570117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:17:26.492851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568221+0000","last_change":"2023-09-20T21:46:41.499927+0000","last_active":"2023-09-20T21:46:45.568221+0000","last_peered":"2023-09-20T21:46:45.568221+0000","last_clean":"2023-09-20T21:46:45.568221+0000","last_became_active":"2023-09-20T21:46:41.499605+0000","last_became_peered":"2023-09-20T21:46:41.499605+0000","last_unstale":"2023-09-20T21:46:45.568221+0000","last_undegraded":"2023-09-20T21:46:45.568221+0000","last_fullsized":"2023-09-20T21:46:45.568221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:18:16.306349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570531+0000","last_change":"2023-09-20T21:46:41.515842+0000","last_active":"2023-09-20T21:46:45.570531+0000","last_peered":"2023-09-20T21:46:45.570531+0000","last_clean":"2023-09-20T21:46:45.570531+0000","last_became_active":"2023-09-20T21:46:41.515600+0000","last_became_peered":"2023-09-20T21:46:41.515600+0000","last_unstale":"2023-09-20T21:46:45.570531+0000","last_undegraded":"2023-09-20T21:46:45.570531+0000","last_fullsized":"2023-09-20T21:46:45.570531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:50:58.088103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569630+0000","last_change":"2023-09-20T21:46:41.510695+0000","last_active":"2023-09-20T21:46:45.569630+0000","last_peered":"2023-09-20T21:46:45.569630+0000","last_clean":"2023-09-20T21:46:45.569630+0000","last_became_active":"2023-09-20T21:46:41.510345+0000","last_became_peered":"2023-09-20T21:46:41.510345+0000","last_unstale":"2023-09-20T21:46:45.569630+0000","last_undegraded":"2023-09-20T21:46:45.569630+0000","last_fullsized":"2023-09-20T21:46:45.569630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:38:24.201382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575033+0000","last_change":"2023-09-20T21:46:41.507809+0000","last_active":"2023-09-20T21:46:45.575033+0000","last_peered":"2023-09-20T21:46:45.575033+0000","last_clean":"2023-09-20T21:46:45.575033+0000","last_became_active":"2023-09-20T21:46:41.507513+0000","last_became_peered":"2023-09-20T21:46:41.507513+0000","last_unstale":"2023-09-20T21:46:45.575033+0000","last_undegraded":"2023-09-20T21:46:45.575033+0000","last_fullsized":"2023-09-20T21:46:45.575033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:30:43.132728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563662+0000","last_change":"2023-09-20T21:46:41.503590+0000","last_active":"2023-09-20T21:46:45.563662+0000","last_peered":"2023-09-20T21:46:45.563662+0000","last_clean":"2023-09-20T21:46:45.563662+0000","last_became_active":"2023-09-20T21:46:41.503295+0000","last_became_peered":"2023-09-20T21:46:41.503295+0000","last_unstale":"2023-09-20T21:46:45.563662+0000","last_undegraded":"2023-09-20T21:46:45.563662+0000","last_fullsized":"2023-09-20T21:46:45.563662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:28:33.088873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4954,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":87,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":18126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.65200000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47599999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.496}]},{"osd":6,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.58199999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.56999999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.75900000000000001}]},{"osd":10,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.69099999999999995}]}]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1071729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46100000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54700000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.59599999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.63200000000000001}]},{"osd":11,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71499999999999997}]}]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":477145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.68500000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.622},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.75}]},{"osd":4,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.79700000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.82899999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:46:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.7719999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.872}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":486769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77200000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.34100000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.371}]},{"osd":2,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65500000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64700000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":1.121}]},{"osd":3,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.64300000000000002,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.92600000000000005}]},{"osd":4,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79500000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.878,"15min":0.878},"last":0.84199999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.313,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78600000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.94599999999999995,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.71999999999999997}]},{"osd":7,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.876,"5min":1.018,"15min":1.018},"last":0.57499999999999996}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.96599999999999997}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.627},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1068723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41799999999999998},"min":{"1min":0.27500000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.438,"15min":0.433},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.623},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70799999999999996}]},{"osd":6,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.747},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.64000000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65300000000000002,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.53000000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.78400000000000003}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":1.018,"15min":1.018},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64500000000000002,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":1.2,"15min":1.2},"last":0.88700000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.57199999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.466,"15min":1.466},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":1.012,"15min":1.012},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.175,"5min":1.179,"15min":1.179},"last":0.79900000000000004}]},{"osd":4,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0640000000000001,"5min":1.093,"15min":1.093},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78700000000000003,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.022,"5min":1.407,"15min":1.407},"last":0.93400000000000005}]},{"osd":5,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0549999999999999,"5min":1.266,"15min":1.266},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.046,"5min":1.073,"15min":1.073},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.84799999999999998,"15min":0.83699999999999997},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.85899999999999999,"15min":0.84599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.95099999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:46:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.84499999999999997,"15min":0.85899999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0149999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.751},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.85599999999999998,"15min":0.86399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1240000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0509999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.085},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.91400000000000003}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":480405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.39400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86499999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.52000000000000002,"15min":0.5},"min":{"1min":0.40000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77700000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.41899999999999998,"15min":0.39300000000000002},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.83699999999999997}]},{"osd":6,"last update":"Wed Sep 20 21:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.55900000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.92200000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.372,"15min":0.372},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.81699999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.89200000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.85599999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":1},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":478041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.503,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92300000000000004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.44900000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66600000000000004,"15min":0.64200000000000002},"min":{"1min":0.52000000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.62},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.67400000000000004,"15min":0.627},"min":{"1min":0.64600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.752}]},{"osd":5,"last update":"Wed Sep 20 21:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86899999999999999,"5min":1.149,"15min":1.149},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.67200000000000004},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.94199999999999995,"5min":1.194,"15min":1.194},"last":0.92000000000000004}]},{"osd":6,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.69599999999999995,"15min":0.67000000000000004},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.026},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82399999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.012,"5min":1.214,"15min":1.214},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.90500000000000003,"15min":0.91600000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0940000000000001,"5min":1.232,"15min":1.232},"last":1.073}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.091},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.90000000000000002}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.111},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.96099999999999997}]},{"osd":11,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.129}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78900000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.39400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.59299999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85299999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.44,"15min":0.39900000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.55500000000000005,"15min":0.49399999999999999},"min":{"1min":0.38300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":7.1500000000000004,"5min":7.1500000000000004,"15min":7.1500000000000004},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.56999999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.65400000000000003,"15min":0.64100000000000001},"min":{"1min":0.53000000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73699999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.63}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.88}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.753},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":476667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.432,"15min":0.39000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81699999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":1,"15min":1},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.84899999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63300000000000001,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65300000000000002,"15min":0.626},"min":{"1min":0.53500000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68799999999999994,"15min":0.64500000000000002},"min":{"1min":0.60399999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed Sep 20 21:46:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":1.024,"15min":1.024},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75900000000000001,"15min":0.747},"min":{"1min":0.61099999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98399999999999999,"5min":1.167,"15min":1.167},"last":0.69599999999999995}]},{"osd":6,"last update":"Wed Sep 20 21:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.52500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.075,"5min":1.147,"15min":1.147},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.627}]},{"osd":7,"last update":"Wed Sep 20 21:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.55400000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.177,"5min":1.248,"15min":1.248},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0549999999999999,"5min":1.228,"15min":1.228},"last":0.874}]},{"osd":8,"last update":"Wed Sep 20 21:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.74299999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.85099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920816,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":476585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57699999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.495},{"interface":"front","average":{"1min":0.438,"5min":0.38,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.46600000000000003}]},{"osd":2,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.40899999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.46400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68700000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.48499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.51900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58599999999999997,"15min":0.56799999999999995},"min":{"1min":0.503,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed Sep 20 21:46:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.58399999999999996,"15min":0.56799999999999995},"min":{"1min":0.55000000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.65800000000000003,"15min":0.64300000000000002},"min":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.86499999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.82799999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":1,"15min":1},"last":0.746},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88200000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81100000000000005}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.79600000000000004}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.91900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1581056,"data_stored":1066832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85499999999999998,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.753,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.42699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81299999999999994,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.36699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.434},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40400000000000003,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.59899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79100000000000004,"5min":1.615,"15min":1.615},"last":0.496},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.008,"15min":1.008},"last":0.77300000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65900000000000003,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85999999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.75800000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.749,"5min":0.879,"15min":0.879},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59599999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.78900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-20T21:47:09.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-20T21:47:09.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]': finished 2023-09-20T21:47:09.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2023-09-20T21:47:09.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-09-20T21:47:09.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-09-20T21:47:09.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2023-09-20T21:47:09.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-09-20T21:47:09.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2023-09-20T21:47:09.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2023-09-20T21:47:09.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2023-09-20T21:47:09.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: osdmap e90: 12 total, 12 up, 12 in 2023-09-20T21:47:09.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3901384317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:47:09.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/142460400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:09.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] up:active 2023-09-20T21:47:09.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: daemon mds.j assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-09-20T21:47:09.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-09-20T21:47:09.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: Cluster is now healthy 2023-09-20T21:47:09.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:active} 3 up:standby-replay 5 up:standby 2023-09-20T21:47:09.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:08 smithi123 ceph-mon[134433]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-09-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.261+0000 7fe5e65ff700 1 -- 172.21.15.22:0/4220551487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e0100df0 msgr2=0x7fe5e0101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.261+0000 7fe5e65ff700 1 --2- 172.21.15.22:0/4220551487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e0100df0 0x7fe5e0101210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00099c0 tx=0x7fe5d0009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.261+0000 7fe5e65ff700 1 -- 172.21.15.22:0/4220551487 shutdown_connections 2023-09-20T21:47:09.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.261+0000 7fe5e65ff700 1 --2- 172.21.15.22:0/4220551487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0102980 0x7fe5e0109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.261+0000 7fe5e65ff700 1 --2- 172.21.15.22:0/4220551487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e0100df0 0x7fe5e0101210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.261+0000 7fe5e65ff700 1 --2- 172.21.15.22:0/4220551487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e0101fe0 0x7fe5e0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.261+0000 7fe5e65ff700 1 -- 172.21.15.22:0/4220551487 >> 172.21.15.22:0/4220551487 conn(0x7fe5e00fc320 msgr2=0x7fe5e00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:09.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.261+0000 7fe5e65ff700 1 -- 172.21.15.22:0/4220551487 shutdown_connections 2023-09-20T21:47:09.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.261+0000 7fe5e65ff700 1 -- 172.21.15.22:0/4220551487 wait complete. 2023-09-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.262+0000 7fe5e65ff700 1 Processor -- start 2023-09-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.262+0000 7fe5e65ff700 1 -- start start 2023-09-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e65ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e0100df0 0x7fe5e0196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e65ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101fe0 0x7fe5e0196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:09.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e65ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e0102980 0x7fe5e019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e65ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e019e630 con 0x7fe5e0102980 2023-09-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e65ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e019e770 con 0x7fe5e0100df0 2023-09-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e65ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e019e900 con 0x7fe5e0101fe0 2023-09-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e4b9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e0102980 0x7fe5e019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:09.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e4b9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e0102980 0x7fe5e019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50794/0 (socket says 172.21.15.22:50794) 2023-09-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5dffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e0100df0 0x7fe5e0196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e4b9c700 1 -- 172.21.15.22:0/3914164627 learned_addr learned my addr 172.21.15.22:0/3914164627 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5df7fe700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101fe0 0x7fe5e0196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e4b9c700 1 -- 172.21.15.22:0/3914164627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101fe0 msgr2=0x7fe5e0196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:09.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.263+0000 7fe5e4b9c700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101fe0 0x7fe5e0196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.264+0000 7fe5e4b9c700 1 -- 172.21.15.22:0/3914164627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e0100df0 msgr2=0x7fe5e0196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.264+0000 7fe5e4b9c700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e0100df0 0x7fe5e0196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.264+0000 7fe5e4b9c700 1 -- 172.21.15.22:0/3914164627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d00095f0 con 0x7fe5e0102980 2023-09-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.264+0000 7fe5e4b9c700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e0102980 0x7fe5e019e060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800ef30 tx=0x7fe5d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:09.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.264+0000 7fe5dd7fa700 1 -- 172.21.15.22:0/3914164627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8018070 con 0x7fe5e0102980 2023-09-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.264+0000 7fe5dd7fa700 1 -- 172.21.15.22:0/3914164627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5d800f040 con 0x7fe5e0102980 2023-09-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.264+0000 7fe5e65ff700 1 -- 172.21.15.22:0/3914164627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e019eb80 con 0x7fe5e0102980 2023-09-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.264+0000 7fe5dd7fa700 1 -- 172.21.15.22:0/3914164627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800cd00 con 0x7fe5e0102980 2023-09-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.264+0000 7fe5e65ff700 1 -- 172.21.15.22:0/3914164627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e019f0e0 con 0x7fe5e0102980 2023-09-20T21:47:09.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.265+0000 7fe5e65ff700 1 -- 172.21.15.22:0/3914164627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e0107d20 con 0x7fe5e0102980 2023-09-20T21:47:09.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.267+0000 7fe5dd7fa700 1 -- 172.21.15.22:0/3914164627 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe5d8003a00 con 0x7fe5e0102980 2023-09-20T21:47:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.269+0000 7fe5dd7fa700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5c80a13d0 0x7fe5c80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.269+0000 7fe5dd7fa700 1 -- 172.21.15.22:0/3914164627 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe5d80cc9f0 con 0x7fe5e0102980 2023-09-20T21:47:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.269+0000 7fe5dffff700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5c80a13d0 0x7fe5c80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:09.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.269+0000 7fe5dffff700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5c80a13d0 0x7fe5c80a3890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009990 tx=0x7fe5d0019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:09.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.272+0000 7fe5dd7fa700 1 -- 172.21.15.22:0/3914164627 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe5d8095df0 con 0x7fe5e0102980 2023-09-20T21:47:09.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.483+0000 7fe5e65ff700 1 -- 172.21.15.22:0/3914164627 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5e002d230 con 0x7fe5c80a13d0 2023-09-20T21:47:09.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.496+0000 7fe5dd7fa700 1 -- 172.21.15.22:0/3914164627 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+421925 (secure 0 0 0) 0x7fe5e002d230 con 0x7fe5c80a13d0 2023-09-20T21:47:09.500 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 -- 172.21.15.22:0/3914164627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5c80a13d0 msgr2=0x7fe5c80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5c80a13d0 0x7fe5c80a3890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009990 tx=0x7fe5d0019040 comp rx=0 tx=0).stop 2023-09-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 -- 172.21.15.22:0/3914164627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e0102980 msgr2=0x7fe5e019e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e0102980 0x7fe5e019e060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800ef30 tx=0x7fe5d800c3a0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 -- 172.21.15.22:0/3914164627 shutdown_connections 2023-09-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e0102980 0x7fe5e019e060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5c80a13d0 0x7fe5c80a3890 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e0100df0 0x7fe5e0196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 --2- 172.21.15.22:0/3914164627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e0101fe0 0x7fe5e0196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 -- 172.21.15.22:0/3914164627 >> 172.21.15.22:0/3914164627 conn(0x7fe5e00fc320 msgr2=0x7fe5e00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 -- 172.21.15.22:0/3914164627 shutdown_connections 2023-09-20T21:47:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.501+0000 7fe5e65ff700 1 -- 172.21.15.22:0/3914164627 wait complete. 2023-09-20T21:47:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-09-20T21:47:09.519 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-09-20T21:47:09.102716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596000,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":137,"num_read_kb":118,"num_write":291,"num_write_kb":2978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337664,"kb_used_data":13444,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604160,"statfs":{"total":1151940427776,"available":1151594659840,"internally_reserved":0,"allocated":13766656,"data_stored":7537855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2773,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":64,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002240"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598937+0000","last_change":"2023-09-20T21:46:41.506092+0000","last_active":"2023-09-20T21:47:07.598937+0000","last_peered":"2023-09-20T21:47:07.598937+0000","last_clean":"2023-09-20T21:47:07.598937+0000","last_became_active":"2023-09-20T21:46:41.505779+0000","last_became_peered":"2023-09-20T21:46:41.505779+0000","last_unstale":"2023-09-20T21:47:07.598937+0000","last_undegraded":"2023-09-20T21:47:07.598937+0000","last_fullsized":"2023-09-20T21:47:07.598937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:25.284068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571607+0000","last_change":"2023-09-20T21:46:41.505000+0000","last_active":"2023-09-20T21:46:45.571607+0000","last_peered":"2023-09-20T21:46:45.571607+0000","last_clean":"2023-09-20T21:46:45.571607+0000","last_became_active":"2023-09-20T21:46:41.504464+0000","last_became_peered":"2023-09-20T21:46:41.504464+0000","last_unstale":"2023-09-20T21:46:45.571607+0000","last_undegraded":"2023-09-20T21:46:45.571607+0000","last_fullsized":"2023-09-20T21:46:45.571607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:42:44.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569994+0000","last_change":"2023-09-20T21:46:41.513134+0000","last_active":"2023-09-20T21:46:45.569994+0000","last_peered":"2023-09-20T21:46:45.569994+0000","last_clean":"2023-09-20T21:46:45.569994+0000","last_became_active":"2023-09-20T21:46:41.512793+0000","last_became_peered":"2023-09-20T21:46:41.512793+0000","last_unstale":"2023-09-20T21:46:45.569994+0000","last_undegraded":"2023-09-20T21:46:45.569994+0000","last_fullsized":"2023-09-20T21:46:45.569994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:59.978630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597341+0000","last_change":"2023-09-20T21:46:41.500200+0000","last_active":"2023-09-20T21:47:07.597341+0000","last_peered":"2023-09-20T21:47:07.597341+0000","last_clean":"2023-09-20T21:47:07.597341+0000","last_became_active":"2023-09-20T21:46:41.499866+0000","last_became_peered":"2023-09-20T21:46:41.499866+0000","last_unstale":"2023-09-20T21:47:07.597341+0000","last_undegraded":"2023-09-20T21:47:07.597341+0000","last_fullsized":"2023-09-20T21:47:07.597341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:20:52.989008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501872+0000","last_change":"2023-09-20T21:46:41.507584+0000","last_active":"2023-09-20T21:46:47.501872+0000","last_peered":"2023-09-20T21:46:47.501872+0000","last_clean":"2023-09-20T21:46:47.501872+0000","last_became_active":"2023-09-20T21:46:41.507298+0000","last_became_peered":"2023-09-20T21:46:41.507298+0000","last_unstale":"2023-09-20T21:46:47.501872+0000","last_undegraded":"2023-09-20T21:46:47.501872+0000","last_fullsized":"2023-09-20T21:46:47.501872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:32:14.042414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564527+0000","last_change":"2023-09-20T21:46:41.497002+0000","last_active":"2023-09-20T21:46:45.564527+0000","last_peered":"2023-09-20T21:46:45.564527+0000","last_clean":"2023-09-20T21:46:45.564527+0000","last_became_active":"2023-09-20T21:46:41.496485+0000","last_became_peered":"2023-09-20T21:46:41.496485+0000","last_unstale":"2023-09-20T21:46:45.564527+0000","last_undegraded":"2023-09-20T21:46:45.564527+0000","last_fullsized":"2023-09-20T21:46:45.564527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:48:24.481575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.567850+0000","last_change":"2023-09-20T21:46:41.503303+0000","last_active":"2023-09-20T21:46:45.567850+0000","last_peered":"2023-09-20T21:46:45.567850+0000","last_clean":"2023-09-20T21:46:45.567850+0000","last_became_active":"2023-09-20T21:46:41.502982+0000","last_became_peered":"2023-09-20T21:46:41.502982+0000","last_unstale":"2023-09-20T21:46:45.567850+0000","last_undegraded":"2023-09-20T21:46:45.567850+0000","last_fullsized":"2023-09-20T21:46:45.567850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:23:27.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569351+0000","last_change":"2023-09-20T21:46:41.508243+0000","last_active":"2023-09-20T21:46:45.569351+0000","last_peered":"2023-09-20T21:46:45.569351+0000","last_clean":"2023-09-20T21:46:45.569351+0000","last_became_active":"2023-09-20T21:46:41.506407+0000","last_became_peered":"2023-09-20T21:46:41.506407+0000","last_unstale":"2023-09-20T21:46:45.569351+0000","last_undegraded":"2023-09-20T21:46:45.569351+0000","last_fullsized":"2023-09-20T21:46:45.569351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:39.388716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568936+0000","last_change":"2023-09-20T21:46:41.501513+0000","last_active":"2023-09-20T21:46:45.568936+0000","last_peered":"2023-09-20T21:46:45.568936+0000","last_clean":"2023-09-20T21:46:45.568936+0000","last_became_active":"2023-09-20T21:46:41.500051+0000","last_became_peered":"2023-09-20T21:46:41.500051+0000","last_unstale":"2023-09-20T21:46:45.568936+0000","last_undegraded":"2023-09-20T21:46:45.568936+0000","last_fullsized":"2023-09-20T21:46:45.568936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:40.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564777+0000","last_change":"2023-09-20T21:46:41.501045+0000","last_active":"2023-09-20T21:46:45.564777+0000","last_peered":"2023-09-20T21:46:45.564777+0000","last_clean":"2023-09-20T21:46:45.564777+0000","last_became_active":"2023-09-20T21:46:41.500744+0000","last_became_peered":"2023-09-20T21:46:41.500744+0000","last_unstale":"2023-09-20T21:46:45.564777+0000","last_undegraded":"2023-09-20T21:46:45.564777+0000","last_fullsized":"2023-09-20T21:46:45.564777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:57:29.120472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570586+0000","last_change":"2023-09-20T21:46:41.510235+0000","last_active":"2023-09-20T21:46:45.570586+0000","last_peered":"2023-09-20T21:46:45.570586+0000","last_clean":"2023-09-20T21:46:45.570586+0000","last_became_active":"2023-09-20T21:46:41.509943+0000","last_became_peered":"2023-09-20T21:46:41.509943+0000","last_unstale":"2023-09-20T21:46:45.570586+0000","last_undegraded":"2023-09-20T21:46:45.570586+0000","last_fullsized":"2023-09-20T21:46:45.570586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:09:19.383505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597247+0000","last_change":"2023-09-20T21:46:41.501194+0000","last_active":"2023-09-20T21:47:07.597247+0000","last_peered":"2023-09-20T21:47:07.597247+0000","last_clean":"2023-09-20T21:47:07.597247+0000","last_became_active":"2023-09-20T21:46:41.500095+0000","last_became_peered":"2023-09-20T21:46:41.500095+0000","last_unstale":"2023-09-20T21:47:07.597247+0000","last_undegraded":"2023-09-20T21:47:07.597247+0000","last_fullsized":"2023-09-20T21:47:07.597247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:46:51.343332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575166+0000","last_change":"2023-09-20T21:46:41.507562+0000","last_active":"2023-09-20T21:46:45.575166+0000","last_peered":"2023-09-20T21:46:45.575166+0000","last_clean":"2023-09-20T21:46:45.575166+0000","last_became_active":"2023-09-20T21:46:41.507094+0000","last_became_peered":"2023-09-20T21:46:41.507094+0000","last_unstale":"2023-09-20T21:46:45.575166+0000","last_undegraded":"2023-09-20T21:46:45.575166+0000","last_fullsized":"2023-09-20T21:46:45.575166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:34:32.401415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569178+0000","last_change":"2023-09-20T21:46:41.502864+0000","last_active":"2023-09-20T21:46:45.569178+0000","last_peered":"2023-09-20T21:46:45.569178+0000","last_clean":"2023-09-20T21:46:45.569178+0000","last_became_active":"2023-09-20T21:46:41.502543+0000","last_became_peered":"2023-09-20T21:46:41.502543+0000","last_unstale":"2023-09-20T21:46:45.569178+0000","last_undegraded":"2023-09-20T21:46:45.569178+0000","last_fullsized":"2023-09-20T21:46:45.569178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:02:04.631874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.572035+0000","last_change":"2023-09-20T21:46:41.512189+0000","last_active":"2023-09-20T21:46:45.572035+0000","last_peered":"2023-09-20T21:46:45.572035+0000","last_clean":"2023-09-20T21:46:45.572035+0000","last_became_active":"2023-09-20T21:46:41.511888+0000","last_became_peered":"2023-09-20T21:46:41.511888+0000","last_unstale":"2023-09-20T21:46:45.572035+0000","last_undegraded":"2023-09-20T21:46:45.572035+0000","last_fullsized":"2023-09-20T21:46:45.572035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:17:20.417468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571768+0000","last_change":"2023-09-20T21:46:41.502036+0000","last_active":"2023-09-20T21:46:45.571768+0000","last_peered":"2023-09-20T21:46:45.571768+0000","last_clean":"2023-09-20T21:46:45.571768+0000","last_became_active":"2023-09-20T21:46:41.501745+0000","last_became_peered":"2023-09-20T21:46:41.501745+0000","last_unstale":"2023-09-20T21:46:45.571768+0000","last_undegraded":"2023-09-20T21:46:45.571768+0000","last_fullsized":"2023-09-20T21:46:45.571768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:08:06.699543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598739+0000","last_change":"2023-09-20T21:46:41.506666+0000","last_active":"2023-09-20T21:47:07.598739+0000","last_peered":"2023-09-20T21:47:07.598739+0000","last_clean":"2023-09-20T21:47:07.598739+0000","last_became_active":"2023-09-20T21:46:41.506266+0000","last_became_peered":"2023-09-20T21:46:41.506266+0000","last_unstale":"2023-09-20T21:47:07.598739+0000","last_undegraded":"2023-09-20T21:47:07.598739+0000","last_fullsized":"2023-09-20T21:47:07.598739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:06:16.809609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563418+0000","last_change":"2023-09-20T21:46:41.497391+0000","last_active":"2023-09-20T21:46:45.563418+0000","last_peered":"2023-09-20T21:46:45.563418+0000","last_clean":"2023-09-20T21:46:45.563418+0000","last_became_active":"2023-09-20T21:46:41.497064+0000","last_became_peered":"2023-09-20T21:46:41.497064+0000","last_unstale":"2023-09-20T21:46:45.563418+0000","last_undegraded":"2023-09-20T21:46:45.563418+0000","last_fullsized":"2023-09-20T21:46:45.563418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:56:11.489585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569164+0000","last_change":"2023-09-20T21:46:41.510602+0000","last_active":"2023-09-20T21:46:45.569164+0000","last_peered":"2023-09-20T21:46:45.569164+0000","last_clean":"2023-09-20T21:46:45.569164+0000","last_became_active":"2023-09-20T21:46:41.510308+0000","last_became_peered":"2023-09-20T21:46:41.510308+0000","last_unstale":"2023-09-20T21:46:45.569164+0000","last_undegraded":"2023-09-20T21:46:45.569164+0000","last_fullsized":"2023-09-20T21:46:45.569164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:43:41.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565451+0000","last_change":"2023-09-20T21:46:45.581583+0000","last_active":"2023-09-20T21:46:46.565451+0000","last_peered":"2023-09-20T21:46:46.565451+0000","last_clean":"2023-09-20T21:46:46.565451+0000","last_became_active":"2023-09-20T21:46:45.581165+0000","last_became_peered":"2023-09-20T21:46:45.581165+0000","last_unstale":"2023-09-20T21:46:46.565451+0000","last_undegraded":"2023-09-20T21:46:46.565451+0000","last_fullsized":"2023-09-20T21:46:46.565451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:39:36.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564314+0000","last_change":"2023-09-20T21:46:40.500010+0000","last_active":"2023-09-20T21:46:45.564314+0000","last_peered":"2023-09-20T21:46:45.564314+0000","last_clean":"2023-09-20T21:46:45.564314+0000","last_became_active":"2023-09-20T21:46:40.499632+0000","last_became_peered":"2023-09-20T21:46:40.499632+0000","last_unstale":"2023-09-20T21:46:45.564314+0000","last_undegraded":"2023-09-20T21:46:45.564314+0000","last_fullsized":"2023-09-20T21:46:45.564314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:41:35.590982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569101+0000","last_change":"2023-09-20T21:46:41.516465+0000","last_active":"2023-09-20T21:46:45.569101+0000","last_peered":"2023-09-20T21:46:45.569101+0000","last_clean":"2023-09-20T21:46:45.569101+0000","last_became_active":"2023-09-20T21:46:41.516150+0000","last_became_peered":"2023-09-20T21:46:41.516150+0000","last_unstale":"2023-09-20T21:46:45.569101+0000","last_undegraded":"2023-09-20T21:46:45.569101+0000","last_fullsized":"2023-09-20T21:46:45.569101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:33:37.507049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596541+0000","last_change":"2023-09-20T21:46:45.582916+0000","last_active":"2023-09-20T21:47:07.596541+0000","last_peered":"2023-09-20T21:47:07.596541+0000","last_clean":"2023-09-20T21:47:07.596541+0000","last_became_active":"2023-09-20T21:46:45.582511+0000","last_became_peered":"2023-09-20T21:46:45.582511+0000","last_unstale":"2023-09-20T21:47:07.596541+0000","last_undegraded":"2023-09-20T21:47:07.596541+0000","last_fullsized":"2023-09-20T21:47:07.596541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:43:47.006768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569207+0000","last_change":"2023-09-20T21:46:40.516867+0000","last_active":"2023-09-20T21:46:45.569207+0000","last_peered":"2023-09-20T21:46:45.569207+0000","last_clean":"2023-09-20T21:46:45.569207+0000","last_became_active":"2023-09-20T21:46:40.516343+0000","last_became_peered":"2023-09-20T21:46:40.516343+0000","last_unstale":"2023-09-20T21:46:45.569207+0000","last_undegraded":"2023-09-20T21:46:45.569207+0000","last_fullsized":"2023-09-20T21:46:45.569207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:17:18.348921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573554+0000","last_change":"2023-09-20T21:46:41.507093+0000","last_active":"2023-09-20T21:46:45.573554+0000","last_peered":"2023-09-20T21:46:45.573554+0000","last_clean":"2023-09-20T21:46:45.573554+0000","last_became_active":"2023-09-20T21:46:41.506587+0000","last_became_peered":"2023-09-20T21:46:41.506587+0000","last_unstale":"2023-09-20T21:46:45.573554+0000","last_undegraded":"2023-09-20T21:46:45.573554+0000","last_fullsized":"2023-09-20T21:46:45.573554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:26:44.825499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597188+0000","last_change":"2023-09-20T21:46:45.583942+0000","last_active":"2023-09-20T21:47:07.597188+0000","last_peered":"2023-09-20T21:47:07.597188+0000","last_clean":"2023-09-20T21:47:07.597188+0000","last_became_active":"2023-09-20T21:46:45.583484+0000","last_became_peered":"2023-09-20T21:46:45.583484+0000","last_unstale":"2023-09-20T21:47:07.597188+0000","last_undegraded":"2023-09-20T21:47:07.597188+0000","last_fullsized":"2023-09-20T21:47:07.597188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:48.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"peering","last_fresh":"2023-09-20T21:47:07.573536+0000","last_change":"2023-09-20T21:47:07.596775+0000","last_active":"2023-09-20T21:47:07.596775+0000","last_peered":"2023-09-20T21:47:07.573536+0000","last_clean":"2023-09-20T21:47:07.573536+0000","last_became_active":"2023-09-20T21:46:40.501028+0000","last_became_peered":"2023-09-20T21:46:40.501028+0000","last_unstale":"2023-09-20T21:47:07.573536+0000","last_undegraded":"2023-09-20T21:47:07.573536+0000","last_fullsized":"2023-09-20T21:47:07.573536+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,7],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568146+0000","last_change":"2023-09-20T21:46:41.504449+0000","last_active":"2023-09-20T21:46:45.568146+0000","last_peered":"2023-09-20T21:46:45.568146+0000","last_clean":"2023-09-20T21:46:45.568146+0000","last_became_active":"2023-09-20T21:46:41.504169+0000","last_became_peered":"2023-09-20T21:46:41.504169+0000","last_unstale":"2023-09-20T21:46:45.568146+0000","last_undegraded":"2023-09-20T21:46:45.568146+0000","last_fullsized":"2023-09-20T21:46:45.568146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:36:43.980902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596618+0000","last_change":"2023-09-20T21:46:45.582269+0000","last_active":"2023-09-20T21:47:07.596618+0000","last_peered":"2023-09-20T21:47:07.596618+0000","last_clean":"2023-09-20T21:47:07.596618+0000","last_became_active":"2023-09-20T21:46:45.581782+0000","last_became_peered":"2023-09-20T21:46:45.581782+0000","last_unstale":"2023-09-20T21:47:07.596618+0000","last_undegraded":"2023-09-20T21:47:07.596618+0000","last_fullsized":"2023-09-20T21:47:07.596618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:32:09.518715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502079+0000","last_change":"2023-09-20T21:46:40.504099+0000","last_active":"2023-09-20T21:46:47.502079+0000","last_peered":"2023-09-20T21:46:47.502079+0000","last_clean":"2023-09-20T21:46:47.502079+0000","last_became_active":"2023-09-20T21:46:40.503774+0000","last_became_peered":"2023-09-20T21:46:40.503774+0000","last_unstale":"2023-09-20T21:46:47.502079+0000","last_undegraded":"2023-09-20T21:46:47.502079+0000","last_fullsized":"2023-09-20T21:46:47.502079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:17.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568776+0000","last_change":"2023-09-20T21:46:41.499802+0000","last_active":"2023-09-20T21:46:45.568776+0000","last_peered":"2023-09-20T21:46:45.568776+0000","last_clean":"2023-09-20T21:46:45.568776+0000","last_became_active":"2023-09-20T21:46:41.499491+0000","last_became_peered":"2023-09-20T21:46:41.499491+0000","last_unstale":"2023-09-20T21:46:45.568776+0000","last_undegraded":"2023-09-20T21:46:45.568776+0000","last_fullsized":"2023-09-20T21:46:45.568776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:44.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565986+0000","last_change":"2023-09-20T21:46:45.578753+0000","last_active":"2023-09-20T21:46:46.565986+0000","last_peered":"2023-09-20T21:46:46.565986+0000","last_clean":"2023-09-20T21:46:46.565986+0000","last_became_active":"2023-09-20T21:46:45.578297+0000","last_became_peered":"2023-09-20T21:46:45.578297+0000","last_unstale":"2023-09-20T21:46:46.565986+0000","last_undegraded":"2023-09-20T21:46:46.565986+0000","last_fullsized":"2023-09-20T21:46:46.565986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:18:22.596550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571690+0000","last_change":"2023-09-20T21:46:40.505675+0000","last_active":"2023-09-20T21:46:45.571690+0000","last_peered":"2023-09-20T21:46:45.571690+0000","last_clean":"2023-09-20T21:46:45.571690+0000","last_became_active":"2023-09-20T21:46:40.505432+0000","last_became_peered":"2023-09-20T21:46:40.505432+0000","last_unstale":"2023-09-20T21:46:45.571690+0000","last_undegraded":"2023-09-20T21:46:45.571690+0000","last_fullsized":"2023-09-20T21:46:45.571690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:14:53.652904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501949+0000","last_change":"2023-09-20T21:46:41.505924+0000","last_active":"2023-09-20T21:46:47.501949+0000","last_peered":"2023-09-20T21:46:47.501949+0000","last_clean":"2023-09-20T21:46:47.501949+0000","last_became_active":"2023-09-20T21:46:41.505609+0000","last_became_peered":"2023-09-20T21:46:41.505609+0000","last_unstale":"2023-09-20T21:46:47.501949+0000","last_undegraded":"2023-09-20T21:46:47.501949+0000","last_fullsized":"2023-09-20T21:46:47.501949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:13:46.506786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565886+0000","last_change":"2023-09-20T21:46:45.587818+0000","last_active":"2023-09-20T21:46:46.565886+0000","last_peered":"2023-09-20T21:46:46.565886+0000","last_clean":"2023-09-20T21:46:46.565886+0000","last_became_active":"2023-09-20T21:46:45.587376+0000","last_became_peered":"2023-09-20T21:46:45.587376+0000","last_unstale":"2023-09-20T21:46:46.565886+0000","last_undegraded":"2023-09-20T21:46:46.565886+0000","last_fullsized":"2023-09-20T21:46:46.565886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:28:53.605158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564415+0000","last_change":"2023-09-20T21:46:40.505610+0000","last_active":"2023-09-20T21:46:45.564415+0000","last_peered":"2023-09-20T21:46:45.564415+0000","last_clean":"2023-09-20T21:46:45.564415+0000","last_became_active":"2023-09-20T21:46:40.505279+0000","last_became_peered":"2023-09-20T21:46:40.505279+0000","last_unstale":"2023-09-20T21:46:45.564415+0000","last_undegraded":"2023-09-20T21:46:45.564415+0000","last_fullsized":"2023-09-20T21:46:45.564415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:41:19.575154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571794+0000","last_change":"2023-09-20T21:46:41.505337+0000","last_active":"2023-09-20T21:46:45.571794+0000","last_peered":"2023-09-20T21:46:45.571794+0000","last_clean":"2023-09-20T21:46:45.571794+0000","last_became_active":"2023-09-20T21:46:41.504959+0000","last_became_peered":"2023-09-20T21:46:41.504959+0000","last_unstale":"2023-09-20T21:46:45.571794+0000","last_undegraded":"2023-09-20T21:46:45.571794+0000","last_fullsized":"2023-09-20T21:46:45.571794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:53:37.797319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598682+0000","last_change":"2023-09-20T21:46:45.581738+0000","last_active":"2023-09-20T21:47:07.598682+0000","last_peered":"2023-09-20T21:47:07.598682+0000","last_clean":"2023-09-20T21:47:07.598682+0000","last_became_active":"2023-09-20T21:46:45.581279+0000","last_became_peered":"2023-09-20T21:46:45.581279+0000","last_unstale":"2023-09-20T21:47:07.598682+0000","last_undegraded":"2023-09-20T21:47:07.598682+0000","last_fullsized":"2023-09-20T21:47:07.598682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:41:41.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570018+0000","last_change":"2023-09-20T21:46:40.510539+0000","last_active":"2023-09-20T21:46:45.570018+0000","last_peered":"2023-09-20T21:46:45.570018+0000","last_clean":"2023-09-20T21:46:45.570018+0000","last_became_active":"2023-09-20T21:46:40.509880+0000","last_became_peered":"2023-09-20T21:46:40.509880+0000","last_unstale":"2023-09-20T21:46:45.570018+0000","last_undegraded":"2023-09-20T21:46:45.570018+0000","last_fullsized":"2023-09-20T21:46:45.570018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:52.877786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569259+0000","last_change":"2023-09-20T21:46:41.505105+0000","last_active":"2023-09-20T21:46:45.569259+0000","last_peered":"2023-09-20T21:46:45.569259+0000","last_clean":"2023-09-20T21:46:45.569259+0000","last_became_active":"2023-09-20T21:46:41.504785+0000","last_became_peered":"2023-09-20T21:46:41.504785+0000","last_unstale":"2023-09-20T21:46:45.569259+0000","last_undegraded":"2023-09-20T21:46:45.569259+0000","last_fullsized":"2023-09-20T21:46:45.569259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:07:57.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502213+0000","last_change":"2023-09-20T21:46:45.583160+0000","last_active":"2023-09-20T21:46:47.502213+0000","last_peered":"2023-09-20T21:46:47.502213+0000","last_clean":"2023-09-20T21:46:47.502213+0000","last_became_active":"2023-09-20T21:46:45.582709+0000","last_became_peered":"2023-09-20T21:46:45.582709+0000","last_unstale":"2023-09-20T21:46:47.502213+0000","last_undegraded":"2023-09-20T21:46:47.502213+0000","last_fullsized":"2023-09-20T21:46:47.502213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:24:57.563682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568131+0000","last_change":"2023-09-20T21:46:40.502591+0000","last_active":"2023-09-20T21:46:45.568131+0000","last_peered":"2023-09-20T21:46:45.568131+0000","last_clean":"2023-09-20T21:46:45.568131+0000","last_became_active":"2023-09-20T21:46:40.502288+0000","last_became_peered":"2023-09-20T21:46:40.502288+0000","last_unstale":"2023-09-20T21:46:45.568131+0000","last_undegraded":"2023-09-20T21:46:45.568131+0000","last_fullsized":"2023-09-20T21:46:45.568131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:09:37.983866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563192+0000","last_change":"2023-09-20T21:46:41.511367+0000","last_active":"2023-09-20T21:46:45.563192+0000","last_peered":"2023-09-20T21:46:45.563192+0000","last_clean":"2023-09-20T21:46:45.563192+0000","last_became_active":"2023-09-20T21:46:41.511005+0000","last_became_peered":"2023-09-20T21:46:41.511005+0000","last_unstale":"2023-09-20T21:46:45.563192+0000","last_undegraded":"2023-09-20T21:46:45.563192+0000","last_fullsized":"2023-09-20T21:46:45.563192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:37:59.684808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.862394+0000","last_change":"2023-09-20T21:46:45.580917+0000","last_active":"2023-09-20T21:46:46.862394+0000","last_peered":"2023-09-20T21:46:46.862394+0000","last_clean":"2023-09-20T21:46:46.862394+0000","last_became_active":"2023-09-20T21:46:45.580505+0000","last_became_peered":"2023-09-20T21:46:45.580505+0000","last_unstale":"2023-09-20T21:46:46.862394+0000","last_undegraded":"2023-09-20T21:46:46.862394+0000","last_fullsized":"2023-09-20T21:46:46.862394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:52:19.733919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.591401+0000","last_change":"2023-09-20T21:46:40.507040+0000","last_active":"2023-09-20T21:47:05.591401+0000","last_peered":"2023-09-20T21:47:05.591401+0000","last_clean":"2023-09-20T21:47:05.591401+0000","last_became_active":"2023-09-20T21:46:40.506712+0000","last_became_peered":"2023-09-20T21:46:40.506712+0000","last_unstale":"2023-09-20T21:47:05.591401+0000","last_undegraded":"2023-09-20T21:47:05.591401+0000","last_fullsized":"2023-09-20T21:47:05.591401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:02:56.040911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568722+0000","last_change":"2023-09-20T21:46:41.504794+0000","last_active":"2023-09-20T21:46:45.568722+0000","last_peered":"2023-09-20T21:46:45.568722+0000","last_clean":"2023-09-20T21:46:45.568722+0000","last_became_active":"2023-09-20T21:46:41.504372+0000","last_became_peered":"2023-09-20T21:46:41.504372+0000","last_unstale":"2023-09-20T21:46:45.568722+0000","last_undegraded":"2023-09-20T21:46:45.568722+0000","last_fullsized":"2023-09-20T21:46:45.568722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:42:20.527306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.564942+0000","last_change":"2023-09-20T21:46:45.586364+0000","last_active":"2023-09-20T21:46:46.564942+0000","last_peered":"2023-09-20T21:46:46.564942+0000","last_clean":"2023-09-20T21:46:46.564942+0000","last_became_active":"2023-09-20T21:46:45.584620+0000","last_became_peered":"2023-09-20T21:46:45.584620+0000","last_unstale":"2023-09-20T21:46:46.564942+0000","last_undegraded":"2023-09-20T21:46:46.564942+0000","last_fullsized":"2023-09-20T21:46:46.564942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:05:20.158118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.588953+0000","last_change":"2023-09-20T21:46:40.500797+0000","last_active":"2023-09-20T21:47:05.588953+0000","last_peered":"2023-09-20T21:47:05.588953+0000","last_clean":"2023-09-20T21:47:05.588953+0000","last_became_active":"2023-09-20T21:46:40.500337+0000","last_became_peered":"2023-09-20T21:46:40.500337+0000","last_unstale":"2023-09-20T21:47:05.588953+0000","last_undegraded":"2023-09-20T21:47:05.588953+0000","last_fullsized":"2023-09-20T21:47:05.588953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:19.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573645+0000","last_change":"2023-09-20T21:46:41.507805+0000","last_active":"2023-09-20T21:46:45.573645+0000","last_peered":"2023-09-20T21:46:45.573645+0000","last_clean":"2023-09-20T21:46:45.573645+0000","last_became_active":"2023-09-20T21:46:41.507441+0000","last_became_peered":"2023-09-20T21:46:41.507441+0000","last_unstale":"2023-09-20T21:46:45.573645+0000","last_undegraded":"2023-09-20T21:46:45.573645+0000","last_fullsized":"2023-09-20T21:46:45.573645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:51:33.240534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597442+0000","last_change":"2023-09-20T21:46:45.583081+0000","last_active":"2023-09-20T21:47:07.597442+0000","last_peered":"2023-09-20T21:47:07.597442+0000","last_clean":"2023-09-20T21:47:07.597442+0000","last_became_active":"2023-09-20T21:46:45.580496+0000","last_became_peered":"2023-09-20T21:46:45.580496+0000","last_unstale":"2023-09-20T21:47:07.597442+0000","last_undegraded":"2023-09-20T21:47:07.597442+0000","last_fullsized":"2023-09-20T21:47:07.597442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:42.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568676+0000","last_change":"2023-09-20T21:46:40.512739+0000","last_active":"2023-09-20T21:46:45.568676+0000","last_peered":"2023-09-20T21:46:45.568676+0000","last_clean":"2023-09-20T21:46:45.568676+0000","last_became_active":"2023-09-20T21:46:40.512485+0000","last_became_peered":"2023-09-20T21:46:40.512485+0000","last_unstale":"2023-09-20T21:46:45.568676+0000","last_undegraded":"2023-09-20T21:46:45.568676+0000","last_fullsized":"2023-09-20T21:46:45.568676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:02:49.392560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569091+0000","last_change":"2023-09-20T21:46:41.503093+0000","last_active":"2023-09-20T21:46:45.569091+0000","last_peered":"2023-09-20T21:46:45.569091+0000","last_clean":"2023-09-20T21:46:45.569091+0000","last_became_active":"2023-09-20T21:46:41.502806+0000","last_became_peered":"2023-09-20T21:46:41.502806+0000","last_unstale":"2023-09-20T21:46:45.569091+0000","last_undegraded":"2023-09-20T21:46:45.569091+0000","last_fullsized":"2023-09-20T21:46:45.569091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:53:21.143258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565158+0000","last_change":"2023-09-20T21:46:45.579478+0000","last_active":"2023-09-20T21:46:46.565158+0000","last_peered":"2023-09-20T21:46:46.565158+0000","last_clean":"2023-09-20T21:46:46.565158+0000","last_became_active":"2023-09-20T21:46:45.579055+0000","last_became_peered":"2023-09-20T21:46:45.579055+0000","last_unstale":"2023-09-20T21:46:46.565158+0000","last_undegraded":"2023-09-20T21:46:46.565158+0000","last_fullsized":"2023-09-20T21:46:46.565158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:17:32.013280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.590075+0000","last_change":"2023-09-20T21:46:40.507321+0000","last_active":"2023-09-20T21:47:05.590075+0000","last_peered":"2023-09-20T21:47:05.590075+0000","last_clean":"2023-09-20T21:47:05.590075+0000","last_became_active":"2023-09-20T21:46:40.506983+0000","last_became_peered":"2023-09-20T21:46:40.506983+0000","last_unstale":"2023-09-20T21:47:05.590075+0000","last_undegraded":"2023-09-20T21:47:05.590075+0000","last_fullsized":"2023-09-20T21:47:05.590075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:46:40.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502157+0000","last_change":"2023-09-20T21:46:41.509646+0000","last_active":"2023-09-20T21:46:47.502157+0000","last_peered":"2023-09-20T21:46:47.502157+0000","last_clean":"2023-09-20T21:46:47.502157+0000","last_became_active":"2023-09-20T21:46:41.509183+0000","last_became_peered":"2023-09-20T21:46:41.509183+0000","last_unstale":"2023-09-20T21:46:47.502157+0000","last_undegraded":"2023-09-20T21:46:47.502157+0000","last_fullsized":"2023-09-20T21:46:47.502157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:17:21.084830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794668+0000","last_change":"2023-09-20T21:46:45.583709+0000","last_active":"2023-09-20T21:46:46.794668+0000","last_peered":"2023-09-20T21:46:46.794668+0000","last_clean":"2023-09-20T21:46:46.794668+0000","last_became_active":"2023-09-20T21:46:45.583302+0000","last_became_peered":"2023-09-20T21:46:45.583302+0000","last_unstale":"2023-09-20T21:46:46.794668+0000","last_undegraded":"2023-09-20T21:46:46.794668+0000","last_fullsized":"2023-09-20T21:46:46.794668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:31:35.472750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.590965+0000","last_change":"2023-09-20T21:46:40.506411+0000","last_active":"2023-09-20T21:47:05.590965+0000","last_peered":"2023-09-20T21:47:05.590965+0000","last_clean":"2023-09-20T21:47:05.590965+0000","last_became_active":"2023-09-20T21:46:40.506116+0000","last_became_peered":"2023-09-20T21:46:40.506116+0000","last_unstale":"2023-09-20T21:47:05.590965+0000","last_undegraded":"2023-09-20T21:47:05.590965+0000","last_fullsized":"2023-09-20T21:47:05.590965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:23:19.175007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575844+0000","last_change":"2023-09-20T21:46:41.513793+0000","last_active":"2023-09-20T21:46:45.575844+0000","last_peered":"2023-09-20T21:46:45.575844+0000","last_clean":"2023-09-20T21:46:45.575844+0000","last_became_active":"2023-09-20T21:46:41.513434+0000","last_became_peered":"2023-09-20T21:46:41.513434+0000","last_unstale":"2023-09-20T21:46:45.575844+0000","last_undegraded":"2023-09-20T21:46:45.575844+0000","last_fullsized":"2023-09-20T21:46:45.575844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:00:57.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598091+0000","last_change":"2023-09-20T21:46:45.578188+0000","last_active":"2023-09-20T21:47:07.598091+0000","last_peered":"2023-09-20T21:47:07.598091+0000","last_clean":"2023-09-20T21:47:07.598091+0000","last_became_active":"2023-09-20T21:46:45.577548+0000","last_became_peered":"2023-09-20T21:46:45.577548+0000","last_unstale":"2023-09-20T21:47:07.598091+0000","last_undegraded":"2023-09-20T21:47:07.598091+0000","last_fullsized":"2023-09-20T21:47:07.598091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:54:58.894269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573689+0000","last_change":"2023-09-20T21:46:40.505465+0000","last_active":"2023-09-20T21:46:45.573689+0000","last_peered":"2023-09-20T21:46:45.573689+0000","last_clean":"2023-09-20T21:46:45.573689+0000","last_became_active":"2023-09-20T21:46:40.505174+0000","last_became_peered":"2023-09-20T21:46:40.505174+0000","last_unstale":"2023-09-20T21:46:45.573689+0000","last_undegraded":"2023-09-20T21:46:45.573689+0000","last_fullsized":"2023-09-20T21:46:45.573689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:50:53.191963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568918+0000","last_change":"2023-09-20T21:46:41.507370+0000","last_active":"2023-09-20T21:46:45.568918+0000","last_peered":"2023-09-20T21:46:45.568918+0000","last_clean":"2023-09-20T21:46:45.568918+0000","last_became_active":"2023-09-20T21:46:41.507035+0000","last_became_peered":"2023-09-20T21:46:41.507035+0000","last_unstale":"2023-09-20T21:46:45.568918+0000","last_undegraded":"2023-09-20T21:46:45.568918+0000","last_fullsized":"2023-09-20T21:46:45.568918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:50.298899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565547+0000","last_change":"2023-09-20T21:46:45.584458+0000","last_active":"2023-09-20T21:46:46.565547+0000","last_peered":"2023-09-20T21:46:46.565547+0000","last_clean":"2023-09-20T21:46:46.565547+0000","last_became_active":"2023-09-20T21:46:45.584033+0000","last_became_peered":"2023-09-20T21:46:45.584033+0000","last_unstale":"2023-09-20T21:46:46.565547+0000","last_undegraded":"2023-09-20T21:46:46.565547+0000","last_fullsized":"2023-09-20T21:46:46.565547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:48:44.653880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.567221+0000","last_change":"2023-09-20T21:46:40.501329+0000","last_active":"2023-09-20T21:47:08.567221+0000","last_peered":"2023-09-20T21:47:08.567221+0000","last_clean":"2023-09-20T21:47:08.567221+0000","last_became_active":"2023-09-20T21:46:40.500999+0000","last_became_peered":"2023-09-20T21:46:40.500999+0000","last_unstale":"2023-09-20T21:47:08.567221+0000","last_undegraded":"2023-09-20T21:47:08.567221+0000","last_fullsized":"2023-09-20T21:47:08.567221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:38.028398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568141+0000","last_change":"2023-09-20T21:46:41.504583+0000","last_active":"2023-09-20T21:46:45.568141+0000","last_peered":"2023-09-20T21:46:45.568141+0000","last_clean":"2023-09-20T21:46:45.568141+0000","last_became_active":"2023-09-20T21:46:41.504273+0000","last_became_peered":"2023-09-20T21:46:41.504273+0000","last_unstale":"2023-09-20T21:46:45.568141+0000","last_undegraded":"2023-09-20T21:46:45.568141+0000","last_fullsized":"2023-09-20T21:46:45.568141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:37.100382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502476+0000","last_change":"2023-09-20T21:46:45.587116+0000","last_active":"2023-09-20T21:46:47.502476+0000","last_peered":"2023-09-20T21:46:47.502476+0000","last_clean":"2023-09-20T21:46:47.502476+0000","last_became_active":"2023-09-20T21:46:45.586629+0000","last_became_peered":"2023-09-20T21:46:45.586629+0000","last_unstale":"2023-09-20T21:46:47.502476+0000","last_undegraded":"2023-09-20T21:46:47.502476+0000","last_fullsized":"2023-09-20T21:46:47.502476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:10:33.024235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'16","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.572218+0000","last_change":"2023-09-20T21:46:40.499985+0000","last_active":"2023-09-20T21:47:07.572218+0000","last_peered":"2023-09-20T21:47:07.572218+0000","last_clean":"2023-09-20T21:47:07.572218+0000","last_became_active":"2023-09-20T21:46:40.499646+0000","last_became_peered":"2023-09-20T21:46:40.499646+0000","last_unstale":"2023-09-20T21:47:07.572218+0000","last_undegraded":"2023-09-20T21:47:07.572218+0000","last_fullsized":"2023-09-20T21:47:07.572218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:52:02.758556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568813+0000","last_change":"2023-09-20T21:46:41.501241+0000","last_active":"2023-09-20T21:46:45.568813+0000","last_peered":"2023-09-20T21:46:45.568813+0000","last_clean":"2023-09-20T21:46:45.568813+0000","last_became_active":"2023-09-20T21:46:41.500902+0000","last_became_peered":"2023-09-20T21:46:41.500902+0000","last_unstale":"2023-09-20T21:46:45.568813+0000","last_undegraded":"2023-09-20T21:46:45.568813+0000","last_fullsized":"2023-09-20T21:46:45.568813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:21:55.210065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794066+0000","last_change":"2023-09-20T21:46:45.583405+0000","last_active":"2023-09-20T21:46:46.794066+0000","last_peered":"2023-09-20T21:46:46.794066+0000","last_clean":"2023-09-20T21:46:46.794066+0000","last_became_active":"2023-09-20T21:46:45.582945+0000","last_became_peered":"2023-09-20T21:46:45.582945+0000","last_unstale":"2023-09-20T21:46:46.794066+0000","last_undegraded":"2023-09-20T21:46:46.794066+0000","last_fullsized":"2023-09-20T21:46:46.794066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:27:14.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.561084+0000","last_change":"2023-09-20T21:46:40.507744+0000","last_active":"2023-09-20T21:47:07.561084+0000","last_peered":"2023-09-20T21:47:07.561084+0000","last_clean":"2023-09-20T21:47:07.561084+0000","last_became_active":"2023-09-20T21:46:40.507454+0000","last_became_peered":"2023-09-20T21:46:40.507454+0000","last_unstale":"2023-09-20T21:47:07.561084+0000","last_undegraded":"2023-09-20T21:47:07.561084+0000","last_fullsized":"2023-09-20T21:47:07.561084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:00:24.492789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598829+0000","last_change":"2023-09-20T21:46:41.504263+0000","last_active":"2023-09-20T21:47:07.598829+0000","last_peered":"2023-09-20T21:47:07.598829+0000","last_clean":"2023-09-20T21:47:07.598829+0000","last_became_active":"2023-09-20T21:46:41.503961+0000","last_became_peered":"2023-09-20T21:46:41.503961+0000","last_unstale":"2023-09-20T21:47:07.598829+0000","last_undegraded":"2023-09-20T21:47:07.598829+0000","last_fullsized":"2023-09-20T21:47:07.598829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:31:58.035517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.944063+0000","last_change":"2023-09-20T21:46:45.581343+0000","last_active":"2023-09-20T21:46:47.944063+0000","last_peered":"2023-09-20T21:46:47.944063+0000","last_clean":"2023-09-20T21:46:47.944063+0000","last_became_active":"2023-09-20T21:46:45.580830+0000","last_became_peered":"2023-09-20T21:46:45.580830+0000","last_unstale":"2023-09-20T21:46:47.944063+0000","last_undegraded":"2023-09-20T21:46:47.944063+0000","last_fullsized":"2023-09-20T21:46:47.944063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:20:49.865825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563359+0000","last_change":"2023-09-20T21:46:40.501850+0000","last_active":"2023-09-20T21:46:45.563359+0000","last_peered":"2023-09-20T21:46:45.563359+0000","last_clean":"2023-09-20T21:46:45.563359+0000","last_became_active":"2023-09-20T21:46:40.501500+0000","last_became_peered":"2023-09-20T21:46:40.501500+0000","last_unstale":"2023-09-20T21:46:45.563359+0000","last_undegraded":"2023-09-20T21:46:45.563359+0000","last_fullsized":"2023-09-20T21:46:45.563359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:14:15.907335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564629+0000","last_change":"2023-09-20T21:46:41.507432+0000","last_active":"2023-09-20T21:46:45.564629+0000","last_peered":"2023-09-20T21:46:45.564629+0000","last_clean":"2023-09-20T21:46:45.564629+0000","last_became_active":"2023-09-20T21:46:41.507119+0000","last_became_peered":"2023-09-20T21:46:41.507119+0000","last_unstale":"2023-09-20T21:46:45.564629+0000","last_undegraded":"2023-09-20T21:46:45.564629+0000","last_fullsized":"2023-09-20T21:46:45.564629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:14:10.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502815+0000","last_change":"2023-09-20T21:46:45.588455+0000","last_active":"2023-09-20T21:46:47.502815+0000","last_peered":"2023-09-20T21:46:47.502815+0000","last_clean":"2023-09-20T21:46:47.502815+0000","last_became_active":"2023-09-20T21:46:45.587980+0000","last_became_peered":"2023-09-20T21:46:45.587980+0000","last_unstale":"2023-09-20T21:46:47.502815+0000","last_undegraded":"2023-09-20T21:46:47.502815+0000","last_fullsized":"2023-09-20T21:46:47.502815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:07:35.697330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.568603+0000","last_change":"2023-09-20T21:46:40.506260+0000","last_active":"2023-09-20T21:46:46.568603+0000","last_peered":"2023-09-20T21:46:46.568603+0000","last_clean":"2023-09-20T21:46:46.568603+0000","last_became_active":"2023-09-20T21:46:40.505999+0000","last_became_peered":"2023-09-20T21:46:40.505999+0000","last_unstale":"2023-09-20T21:46:46.568603+0000","last_undegraded":"2023-09-20T21:46:46.568603+0000","last_fullsized":"2023-09-20T21:46:46.568603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:39:01.748038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564591+0000","last_change":"2023-09-20T21:46:41.496822+0000","last_active":"2023-09-20T21:46:45.564591+0000","last_peered":"2023-09-20T21:46:45.564591+0000","last_clean":"2023-09-20T21:46:45.564591+0000","last_became_active":"2023-09-20T21:46:41.496517+0000","last_became_peered":"2023-09-20T21:46:41.496517+0000","last_unstale":"2023-09-20T21:46:45.564591+0000","last_undegraded":"2023-09-20T21:46:45.564591+0000","last_fullsized":"2023-09-20T21:46:45.564591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:12.592040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502793+0000","last_change":"2023-09-20T21:46:45.587410+0000","last_active":"2023-09-20T21:46:47.502793+0000","last_peered":"2023-09-20T21:46:47.502793+0000","last_clean":"2023-09-20T21:46:47.502793+0000","last_became_active":"2023-09-20T21:46:45.586944+0000","last_became_peered":"2023-09-20T21:46:45.586944+0000","last_unstale":"2023-09-20T21:46:47.502793+0000","last_undegraded":"2023-09-20T21:46:47.502793+0000","last_fullsized":"2023-09-20T21:46:47.502793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:57.053765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568334+0000","last_change":"2023-09-20T21:46:41.511526+0000","last_active":"2023-09-20T21:46:45.568334+0000","last_peered":"2023-09-20T21:46:45.568334+0000","last_clean":"2023-09-20T21:46:45.568334+0000","last_became_active":"2023-09-20T21:46:41.511213+0000","last_became_peered":"2023-09-20T21:46:41.511213+0000","last_unstale":"2023-09-20T21:46:45.568334+0000","last_undegraded":"2023-09-20T21:46:45.568334+0000","last_fullsized":"2023-09-20T21:46:45.568334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:29:04.199190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569931+0000","last_change":"2023-09-20T21:46:40.514916+0000","last_active":"2023-09-20T21:46:45.569931+0000","last_peered":"2023-09-20T21:46:45.569931+0000","last_clean":"2023-09-20T21:46:45.569931+0000","last_became_active":"2023-09-20T21:46:40.514553+0000","last_became_peered":"2023-09-20T21:46:40.514553+0000","last_unstale":"2023-09-20T21:46:45.569931+0000","last_undegraded":"2023-09-20T21:46:45.569931+0000","last_fullsized":"2023-09-20T21:46:45.569931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:12:32.481852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.568779+0000","last_change":"2023-09-20T21:46:45.587489+0000","last_active":"2023-09-20T21:46:46.568779+0000","last_peered":"2023-09-20T21:46:46.568779+0000","last_clean":"2023-09-20T21:46:46.568779+0000","last_became_active":"2023-09-20T21:46:45.587022+0000","last_became_peered":"2023-09-20T21:46:45.587022+0000","last_unstale":"2023-09-20T21:46:46.568779+0000","last_undegraded":"2023-09-20T21:46:46.568779+0000","last_fullsized":"2023-09-20T21:46:46.568779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:09:05.870763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568768+0000","last_change":"2023-09-20T21:46:41.508886+0000","last_active":"2023-09-20T21:46:45.568768+0000","last_peered":"2023-09-20T21:46:45.568768+0000","last_clean":"2023-09-20T21:46:45.568768+0000","last_became_active":"2023-09-20T21:46:41.508470+0000","last_became_peered":"2023-09-20T21:46:41.508470+0000","last_unstale":"2023-09-20T21:46:45.568768+0000","last_undegraded":"2023-09-20T21:46:45.568768+0000","last_fullsized":"2023-09-20T21:46:45.568768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:45:04.888300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.576942+0000","last_change":"2023-09-20T21:46:40.503350+0000","last_active":"2023-09-20T21:47:08.576942+0000","last_peered":"2023-09-20T21:47:08.576942+0000","last_clean":"2023-09-20T21:47:08.576942+0000","last_became_active":"2023-09-20T21:46:40.503045+0000","last_became_peered":"2023-09-20T21:46:40.503045+0000","last_unstale":"2023-09-20T21:47:08.576942+0000","last_undegraded":"2023-09-20T21:47:08.576942+0000","last_fullsized":"2023-09-20T21:47:08.576942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:31:09.004342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.567932+0000","last_change":"2023-09-20T21:45:35.180242+0000","last_active":"2023-09-20T21:46:45.567932+0000","last_peered":"2023-09-20T21:46:45.567932+0000","last_clean":"2023-09-20T21:46:45.567932+0000","last_became_active":"2023-09-20T21:45:35.179705+0000","last_became_peered":"2023-09-20T21:45:35.179705+0000","last_unstale":"2023-09-20T21:46:45.567932+0000","last_undegraded":"2023-09-20T21:46:45.567932+0000","last_fullsized":"2023-09-20T21:46:45.567932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_clean_scrub_stamp":"2023-09-20T21:42:25.826683+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:02:03.448188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794591+0000","last_change":"2023-09-20T21:46:45.581396+0000","last_active":"2023-09-20T21:46:46.794591+0000","last_peered":"2023-09-20T21:46:46.794591+0000","last_clean":"2023-09-20T21:46:46.794591+0000","last_became_active":"2023-09-20T21:46:45.581018+0000","last_became_peered":"2023-09-20T21:46:45.581018+0000","last_unstale":"2023-09-20T21:46:46.794591+0000","last_undegraded":"2023-09-20T21:46:46.794591+0000","last_fullsized":"2023-09-20T21:46:46.794591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:16:35.934460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569837+0000","last_change":"2023-09-20T21:46:41.514392+0000","last_active":"2023-09-20T21:46:45.569837+0000","last_peered":"2023-09-20T21:46:45.569837+0000","last_clean":"2023-09-20T21:46:45.569837+0000","last_became_active":"2023-09-20T21:46:41.514090+0000","last_became_peered":"2023-09-20T21:46:41.514090+0000","last_unstale":"2023-09-20T21:46:45.569837+0000","last_undegraded":"2023-09-20T21:46:45.569837+0000","last_fullsized":"2023-09-20T21:46:45.569837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:05:47.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:06.596564+0000","last_change":"2023-09-20T21:46:40.507346+0000","last_active":"2023-09-20T21:47:06.596564+0000","last_peered":"2023-09-20T21:47:06.596564+0000","last_clean":"2023-09-20T21:47:06.596564+0000","last_became_active":"2023-09-20T21:46:40.506969+0000","last_became_peered":"2023-09-20T21:46:40.506969+0000","last_unstale":"2023-09-20T21:47:06.596564+0000","last_undegraded":"2023-09-20T21:47:06.596564+0000","last_fullsized":"2023-09-20T21:47:06.596564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:01:20.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600062+0000","last_change":"2023-09-20T21:46:45.578033+0000","last_active":"2023-09-20T21:47:07.600062+0000","last_peered":"2023-09-20T21:47:07.600062+0000","last_clean":"2023-09-20T21:47:07.600062+0000","last_became_active":"2023-09-20T21:46:45.577540+0000","last_became_peered":"2023-09-20T21:46:45.577540+0000","last_unstale":"2023-09-20T21:47:07.600062+0000","last_undegraded":"2023-09-20T21:47:07.600062+0000","last_fullsized":"2023-09-20T21:47:07.600062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:24:31.020660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.573672+0000","last_change":"2023-09-20T21:46:40.510634+0000","last_active":"2023-09-20T21:47:07.573672+0000","last_peered":"2023-09-20T21:47:07.573672+0000","last_clean":"2023-09-20T21:47:07.573672+0000","last_became_active":"2023-09-20T21:46:40.510390+0000","last_became_peered":"2023-09-20T21:46:40.510390+0000","last_unstale":"2023-09-20T21:47:07.573672+0000","last_undegraded":"2023-09-20T21:47:07.573672+0000","last_fullsized":"2023-09-20T21:47:07.573672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:44:49.272987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571610+0000","last_change":"2023-09-20T21:46:41.502981+0000","last_active":"2023-09-20T21:46:45.571610+0000","last_peered":"2023-09-20T21:46:45.571610+0000","last_clean":"2023-09-20T21:46:45.571610+0000","last_became_active":"2023-09-20T21:46:41.502594+0000","last_became_peered":"2023-09-20T21:46:41.502594+0000","last_unstale":"2023-09-20T21:46:45.571610+0000","last_undegraded":"2023-09-20T21:46:45.571610+0000","last_fullsized":"2023-09-20T21:46:45.571610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:41:15.261478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.862185+0000","last_change":"2023-09-20T21:46:45.582903+0000","last_active":"2023-09-20T21:46:46.862185+0000","last_peered":"2023-09-20T21:46:46.862185+0000","last_clean":"2023-09-20T21:46:46.862185+0000","last_became_active":"2023-09-20T21:46:45.582433+0000","last_became_peered":"2023-09-20T21:46:45.582433+0000","last_unstale":"2023-09-20T21:46:46.862185+0000","last_undegraded":"2023-09-20T21:46:46.862185+0000","last_fullsized":"2023-09-20T21:46:46.862185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:38:22.174903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563444+0000","last_change":"2023-09-20T21:46:40.507671+0000","last_active":"2023-09-20T21:46:45.563444+0000","last_peered":"2023-09-20T21:46:45.563444+0000","last_clean":"2023-09-20T21:46:45.563444+0000","last_became_active":"2023-09-20T21:46:40.507288+0000","last_became_peered":"2023-09-20T21:46:40.507288+0000","last_unstale":"2023-09-20T21:46:45.563444+0000","last_undegraded":"2023-09-20T21:46:45.563444+0000","last_fullsized":"2023-09-20T21:46:45.563444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:27:02.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570001+0000","last_change":"2023-09-20T21:46:41.507654+0000","last_active":"2023-09-20T21:46:45.570001+0000","last_peered":"2023-09-20T21:46:45.570001+0000","last_clean":"2023-09-20T21:46:45.570001+0000","last_became_active":"2023-09-20T21:46:41.507362+0000","last_became_peered":"2023-09-20T21:46:41.507362+0000","last_unstale":"2023-09-20T21:46:45.570001+0000","last_undegraded":"2023-09-20T21:46:45.570001+0000","last_fullsized":"2023-09-20T21:46:45.570001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:32:08.973379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565429+0000","last_change":"2023-09-20T21:46:45.578437+0000","last_active":"2023-09-20T21:46:46.565429+0000","last_peered":"2023-09-20T21:46:46.565429+0000","last_clean":"2023-09-20T21:46:46.565429+0000","last_became_active":"2023-09-20T21:46:45.578010+0000","last_became_peered":"2023-09-20T21:46:45.578010+0000","last_unstale":"2023-09-20T21:46:46.565429+0000","last_undegraded":"2023-09-20T21:46:46.565429+0000","last_fullsized":"2023-09-20T21:46:46.565429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:59:05.494861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577934+0000","last_change":"2023-09-20T21:46:40.508937+0000","last_active":"2023-09-20T21:47:08.577934+0000","last_peered":"2023-09-20T21:47:08.577934+0000","last_clean":"2023-09-20T21:47:08.577934+0000","last_became_active":"2023-09-20T21:46:40.508639+0000","last_became_peered":"2023-09-20T21:46:40.508639+0000","last_unstale":"2023-09-20T21:47:08.577934+0000","last_undegraded":"2023-09-20T21:47:08.577934+0000","last_fullsized":"2023-09-20T21:47:08.577934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:50:48.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571508+0000","last_change":"2023-09-20T21:46:41.510702+0000","last_active":"2023-09-20T21:46:45.571508+0000","last_peered":"2023-09-20T21:46:45.571508+0000","last_clean":"2023-09-20T21:46:45.571508+0000","last_became_active":"2023-09-20T21:46:41.510357+0000","last_became_peered":"2023-09-20T21:46:41.510357+0000","last_unstale":"2023-09-20T21:46:45.571508+0000","last_undegraded":"2023-09-20T21:46:45.571508+0000","last_fullsized":"2023-09-20T21:46:45.571508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:35:44.455981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596780+0000","last_change":"2023-09-20T21:46:45.581110+0000","last_active":"2023-09-20T21:47:07.596780+0000","last_peered":"2023-09-20T21:47:07.596780+0000","last_clean":"2023-09-20T21:47:07.596780+0000","last_became_active":"2023-09-20T21:46:45.580498+0000","last_became_peered":"2023-09-20T21:46:45.580498+0000","last_unstale":"2023-09-20T21:47:07.596780+0000","last_undegraded":"2023-09-20T21:47:07.596780+0000","last_fullsized":"2023-09-20T21:47:07.596780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:19:22.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571650+0000","last_change":"2023-09-20T21:46:40.497017+0000","last_active":"2023-09-20T21:46:45.571650+0000","last_peered":"2023-09-20T21:46:45.571650+0000","last_clean":"2023-09-20T21:46:45.571650+0000","last_became_active":"2023-09-20T21:46:40.496722+0000","last_became_peered":"2023-09-20T21:46:40.496722+0000","last_unstale":"2023-09-20T21:46:45.571650+0000","last_undegraded":"2023-09-20T21:46:45.571650+0000","last_fullsized":"2023-09-20T21:46:45.571650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:12:07.017318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501752+0000","last_change":"2023-09-20T21:46:41.508793+0000","last_active":"2023-09-20T21:46:47.501752+0000","last_peered":"2023-09-20T21:46:47.501752+0000","last_clean":"2023-09-20T21:46:47.501752+0000","last_became_active":"2023-09-20T21:46:41.508453+0000","last_became_peered":"2023-09-20T21:46:41.508453+0000","last_unstale":"2023-09-20T21:46:47.501752+0000","last_undegraded":"2023-09-20T21:46:47.501752+0000","last_fullsized":"2023-09-20T21:46:47.501752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:35:12.711570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565653+0000","last_change":"2023-09-20T21:46:45.583519+0000","last_active":"2023-09-20T21:46:46.565653+0000","last_peered":"2023-09-20T21:46:46.565653+0000","last_clean":"2023-09-20T21:46:46.565653+0000","last_became_active":"2023-09-20T21:46:45.582933+0000","last_became_peered":"2023-09-20T21:46:45.582933+0000","last_unstale":"2023-09-20T21:46:46.565653+0000","last_undegraded":"2023-09-20T21:46:46.565653+0000","last_fullsized":"2023-09-20T21:46:46.565653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:13:20.237865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.591317+0000","last_change":"2023-09-20T21:46:40.507384+0000","last_active":"2023-09-20T21:47:05.591317+0000","last_peered":"2023-09-20T21:47:05.591317+0000","last_clean":"2023-09-20T21:47:05.591317+0000","last_became_active":"2023-09-20T21:46:40.507021+0000","last_became_peered":"2023-09-20T21:46:40.507021+0000","last_unstale":"2023-09-20T21:47:05.591317+0000","last_undegraded":"2023-09-20T21:47:05.591317+0000","last_fullsized":"2023-09-20T21:47:05.591317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:48:04.360664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598093+0000","last_change":"2023-09-20T21:46:41.511537+0000","last_active":"2023-09-20T21:47:07.598093+0000","last_peered":"2023-09-20T21:47:07.598093+0000","last_clean":"2023-09-20T21:47:07.598093+0000","last_became_active":"2023-09-20T21:46:41.511238+0000","last_became_peered":"2023-09-20T21:46:41.511238+0000","last_unstale":"2023-09-20T21:47:07.598093+0000","last_undegraded":"2023-09-20T21:47:07.598093+0000","last_fullsized":"2023-09-20T21:47:07.598093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:01:41.492824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596769+0000","last_change":"2023-09-20T21:46:45.585750+0000","last_active":"2023-09-20T21:47:07.596769+0000","last_peered":"2023-09-20T21:47:07.596769+0000","last_clean":"2023-09-20T21:47:07.596769+0000","last_became_active":"2023-09-20T21:46:45.585213+0000","last_became_peered":"2023-09-20T21:46:45.585213+0000","last_unstale":"2023-09-20T21:47:07.596769+0000","last_undegraded":"2023-09-20T21:47:07.596769+0000","last_fullsized":"2023-09-20T21:47:07.596769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:31:15.650766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568158+0000","last_change":"2023-09-20T21:46:40.506394+0000","last_active":"2023-09-20T21:46:45.568158+0000","last_peered":"2023-09-20T21:46:45.568158+0000","last_clean":"2023-09-20T21:46:45.568158+0000","last_became_active":"2023-09-20T21:46:40.506085+0000","last_became_peered":"2023-09-20T21:46:40.506085+0000","last_unstale":"2023-09-20T21:46:45.568158+0000","last_undegraded":"2023-09-20T21:46:45.568158+0000","last_fullsized":"2023-09-20T21:46:45.568158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:02.738600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570438+0000","last_change":"2023-09-20T21:46:41.509491+0000","last_active":"2023-09-20T21:46:45.570438+0000","last_peered":"2023-09-20T21:46:45.570438+0000","last_clean":"2023-09-20T21:46:45.570438+0000","last_became_active":"2023-09-20T21:46:41.509039+0000","last_became_peered":"2023-09-20T21:46:41.509039+0000","last_unstale":"2023-09-20T21:46:45.570438+0000","last_undegraded":"2023-09-20T21:46:45.570438+0000","last_fullsized":"2023-09-20T21:46:45.570438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:57:39.003154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597192+0000","last_change":"2023-09-20T21:46:45.581651+0000","last_active":"2023-09-20T21:47:07.597192+0000","last_peered":"2023-09-20T21:47:07.597192+0000","last_clean":"2023-09-20T21:47:07.597192+0000","last_became_active":"2023-09-20T21:46:45.581224+0000","last_became_peered":"2023-09-20T21:46:45.581224+0000","last_unstale":"2023-09-20T21:47:07.597192+0000","last_undegraded":"2023-09-20T21:47:07.597192+0000","last_fullsized":"2023-09-20T21:47:07.597192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:32:17.778301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.610835+0000","last_change":"2023-09-20T21:47:08.610413+0000","last_active":"2023-09-20T21:47:08.610835+0000","last_peered":"2023-09-20T21:47:08.610835+0000","last_clean":"2023-09-20T21:47:08.610835+0000","last_became_active":"2023-09-20T21:47:08.603734+0000","last_became_peered":"2023-09-20T21:47:08.603734+0000","last_unstale":"2023-09-20T21:47:08.610835+0000","last_undegraded":"2023-09-20T21:47:08.610835+0000","last_fullsized":"2023-09-20T21:47:08.610835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:43:42.569371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568903+0000","last_change":"2023-09-20T21:46:41.507218+0000","last_active":"2023-09-20T21:46:45.568903+0000","last_peered":"2023-09-20T21:46:45.568903+0000","last_clean":"2023-09-20T21:46:45.568903+0000","last_became_active":"2023-09-20T21:46:41.506889+0000","last_became_peered":"2023-09-20T21:46:41.506889+0000","last_unstale":"2023-09-20T21:46:45.568903+0000","last_undegraded":"2023-09-20T21:46:45.568903+0000","last_fullsized":"2023-09-20T21:46:45.568903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:08:40.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.503610+0000","last_change":"2023-09-20T21:46:45.579269+0000","last_active":"2023-09-20T21:46:47.503610+0000","last_peered":"2023-09-20T21:46:47.503610+0000","last_clean":"2023-09-20T21:46:47.503610+0000","last_became_active":"2023-09-20T21:46:45.578541+0000","last_became_peered":"2023-09-20T21:46:45.578541+0000","last_unstale":"2023-09-20T21:46:47.503610+0000","last_undegraded":"2023-09-20T21:46:47.503610+0000","last_fullsized":"2023-09-20T21:46:47.503610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:07:50.241947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569525+0000","last_change":"2023-09-20T21:46:40.508774+0000","last_active":"2023-09-20T21:46:45.569525+0000","last_peered":"2023-09-20T21:46:45.569525+0000","last_clean":"2023-09-20T21:46:45.569525+0000","last_became_active":"2023-09-20T21:46:40.508480+0000","last_became_peered":"2023-09-20T21:46:40.508480+0000","last_unstale":"2023-09-20T21:46:45.569525+0000","last_undegraded":"2023-09-20T21:46:45.569525+0000","last_fullsized":"2023-09-20T21:46:45.569525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:22.643724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570698+0000","last_change":"2023-09-20T21:46:41.503763+0000","last_active":"2023-09-20T21:46:45.570698+0000","last_peered":"2023-09-20T21:46:45.570698+0000","last_clean":"2023-09-20T21:46:45.570698+0000","last_became_active":"2023-09-20T21:46:41.503360+0000","last_became_peered":"2023-09-20T21:46:41.503360+0000","last_unstale":"2023-09-20T21:46:45.570698+0000","last_undegraded":"2023-09-20T21:46:45.570698+0000","last_fullsized":"2023-09-20T21:46:45.570698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:03:00.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502032+0000","last_change":"2023-09-20T21:46:45.586702+0000","last_active":"2023-09-20T21:46:47.502032+0000","last_peered":"2023-09-20T21:46:47.502032+0000","last_clean":"2023-09-20T21:46:47.502032+0000","last_became_active":"2023-09-20T21:46:45.586177+0000","last_became_peered":"2023-09-20T21:46:45.586177+0000","last_unstale":"2023-09-20T21:46:47.502032+0000","last_undegraded":"2023-09-20T21:46:47.502032+0000","last_fullsized":"2023-09-20T21:46:47.502032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:19:03.883440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568892+0000","last_change":"2023-09-20T21:46:40.507270+0000","last_active":"2023-09-20T21:46:45.568892+0000","last_peered":"2023-09-20T21:46:45.568892+0000","last_clean":"2023-09-20T21:46:45.568892+0000","last_became_active":"2023-09-20T21:46:40.506933+0000","last_became_peered":"2023-09-20T21:46:40.506933+0000","last_unstale":"2023-09-20T21:46:45.568892+0000","last_undegraded":"2023-09-20T21:46:45.568892+0000","last_fullsized":"2023-09-20T21:46:45.568892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:59.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569676+0000","last_change":"2023-09-20T21:46:41.507868+0000","last_active":"2023-09-20T21:46:45.569676+0000","last_peered":"2023-09-20T21:46:45.569676+0000","last_clean":"2023-09-20T21:46:45.569676+0000","last_became_active":"2023-09-20T21:46:41.507568+0000","last_became_peered":"2023-09-20T21:46:41.507568+0000","last_unstale":"2023-09-20T21:46:45.569676+0000","last_undegraded":"2023-09-20T21:46:45.569676+0000","last_fullsized":"2023-09-20T21:46:45.569676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:44.615620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.793795+0000","last_change":"2023-09-20T21:46:45.578726+0000","last_active":"2023-09-20T21:46:46.793795+0000","last_peered":"2023-09-20T21:46:46.793795+0000","last_clean":"2023-09-20T21:46:46.793795+0000","last_became_active":"2023-09-20T21:46:45.578326+0000","last_became_peered":"2023-09-20T21:46:45.578326+0000","last_unstale":"2023-09-20T21:46:46.793795+0000","last_undegraded":"2023-09-20T21:46:46.793795+0000","last_fullsized":"2023-09-20T21:46:46.793795+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:21:34.564840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570141+0000","last_change":"2023-09-20T21:46:40.509024+0000","last_active":"2023-09-20T21:46:45.570141+0000","last_peered":"2023-09-20T21:46:45.570141+0000","last_clean":"2023-09-20T21:46:45.570141+0000","last_became_active":"2023-09-20T21:46:40.508659+0000","last_became_peered":"2023-09-20T21:46:40.508659+0000","last_unstale":"2023-09-20T21:46:45.570141+0000","last_undegraded":"2023-09-20T21:46:45.570141+0000","last_fullsized":"2023-09-20T21:46:45.570141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:12:08.302068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569100+0000","last_change":"2023-09-20T21:46:41.510307+0000","last_active":"2023-09-20T21:46:45.569100+0000","last_peered":"2023-09-20T21:46:45.569100+0000","last_clean":"2023-09-20T21:46:45.569100+0000","last_became_active":"2023-09-20T21:46:41.510014+0000","last_became_peered":"2023-09-20T21:46:41.510014+0000","last_unstale":"2023-09-20T21:46:45.569100+0000","last_undegraded":"2023-09-20T21:46:45.569100+0000","last_fullsized":"2023-09-20T21:46:45.569100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:05:55.062515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598156+0000","last_change":"2023-09-20T21:46:41.511525+0000","last_active":"2023-09-20T21:47:07.598156+0000","last_peered":"2023-09-20T21:47:07.598156+0000","last_clean":"2023-09-20T21:47:07.598156+0000","last_became_active":"2023-09-20T21:46:41.511203+0000","last_became_peered":"2023-09-20T21:46:41.511203+0000","last_unstale":"2023-09-20T21:47:07.598156+0000","last_undegraded":"2023-09-20T21:47:07.598156+0000","last_fullsized":"2023-09-20T21:47:07.598156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:42:15.757422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570189+0000","last_change":"2023-09-20T21:46:41.502936+0000","last_active":"2023-09-20T21:46:45.570189+0000","last_peered":"2023-09-20T21:46:45.570189+0000","last_clean":"2023-09-20T21:46:45.570189+0000","last_became_active":"2023-09-20T21:46:41.502617+0000","last_became_peered":"2023-09-20T21:46:41.502617+0000","last_unstale":"2023-09-20T21:46:45.570189+0000","last_undegraded":"2023-09-20T21:46:45.570189+0000","last_fullsized":"2023-09-20T21:46:45.570189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:45:21.197652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568057+0000","last_change":"2023-09-20T21:46:41.510766+0000","last_active":"2023-09-20T21:46:45.568057+0000","last_peered":"2023-09-20T21:46:45.568057+0000","last_clean":"2023-09-20T21:46:45.568057+0000","last_became_active":"2023-09-20T21:46:41.510433+0000","last_became_peered":"2023-09-20T21:46:41.510433+0000","last_unstale":"2023-09-20T21:46:45.568057+0000","last_undegraded":"2023-09-20T21:46:45.568057+0000","last_fullsized":"2023-09-20T21:46:45.568057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:56:12.085926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.572421+0000","last_change":"2023-09-20T21:46:41.508846+0000","last_active":"2023-09-20T21:46:45.572421+0000","last_peered":"2023-09-20T21:46:45.572421+0000","last_clean":"2023-09-20T21:46:45.572421+0000","last_became_active":"2023-09-20T21:46:41.508518+0000","last_became_peered":"2023-09-20T21:46:41.508518+0000","last_unstale":"2023-09-20T21:46:45.572421+0000","last_undegraded":"2023-09-20T21:46:45.572421+0000","last_fullsized":"2023-09-20T21:46:45.572421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:01:58.195327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571696+0000","last_change":"2023-09-20T21:46:41.505929+0000","last_active":"2023-09-20T21:46:45.571696+0000","last_peered":"2023-09-20T21:46:45.571696+0000","last_clean":"2023-09-20T21:46:45.571696+0000","last_became_active":"2023-09-20T21:46:41.505542+0000","last_became_peered":"2023-09-20T21:46:41.505542+0000","last_unstale":"2023-09-20T21:46:45.571696+0000","last_undegraded":"2023-09-20T21:46:45.571696+0000","last_fullsized":"2023-09-20T21:46:45.571696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:24:47.520344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598727+0000","last_change":"2023-09-20T21:46:41.507603+0000","last_active":"2023-09-20T21:47:07.598727+0000","last_peered":"2023-09-20T21:47:07.598727+0000","last_clean":"2023-09-20T21:47:07.598727+0000","last_became_active":"2023-09-20T21:46:41.507340+0000","last_became_peered":"2023-09-20T21:46:41.507340+0000","last_unstale":"2023-09-20T21:47:07.598727+0000","last_undegraded":"2023-09-20T21:47:07.598727+0000","last_fullsized":"2023-09-20T21:47:07.598727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:50:19.756491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568500+0000","last_change":"2023-09-20T21:46:41.509625+0000","last_active":"2023-09-20T21:46:45.568500+0000","last_peered":"2023-09-20T21:46:45.568500+0000","last_clean":"2023-09-20T21:46:45.568500+0000","last_became_active":"2023-09-20T21:46:41.509301+0000","last_became_peered":"2023-09-20T21:46:41.509301+0000","last_unstale":"2023-09-20T21:46:45.568500+0000","last_undegraded":"2023-09-20T21:46:45.568500+0000","last_fullsized":"2023-09-20T21:46:45.568500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:15:03.108060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570117+0000","last_change":"2023-09-20T21:46:41.516781+0000","last_active":"2023-09-20T21:46:45.570117+0000","last_peered":"2023-09-20T21:46:45.570117+0000","last_clean":"2023-09-20T21:46:45.570117+0000","last_became_active":"2023-09-20T21:46:41.516496+0000","last_became_peered":"2023-09-20T21:46:41.516496+0000","last_unstale":"2023-09-20T21:46:45.570117+0000","last_undegraded":"2023-09-20T21:46:45.570117+0000","last_fullsized":"2023-09-20T21:46:45.570117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:17:26.492851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568221+0000","last_change":"2023-09-20T21:46:41.499927+0000","last_active":"2023-09-20T21:46:45.568221+0000","last_peered":"2023-09-20T21:46:45.568221+0000","last_clean":"2023-09-20T21:46:45.568221+0000","last_became_active":"2023-09-20T21:46:41.499605+0000","last_became_peered":"2023-09-20T21:46:41.499605+0000","last_unstale":"2023-09-20T21:46:45.568221+0000","last_undegraded":"2023-09-20T21:46:45.568221+0000","last_fullsized":"2023-09-20T21:46:45.568221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:18:16.306349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570531+0000","last_change":"2023-09-20T21:46:41.515842+0000","last_active":"2023-09-20T21:46:45.570531+0000","last_peered":"2023-09-20T21:46:45.570531+0000","last_clean":"2023-09-20T21:46:45.570531+0000","last_became_active":"2023-09-20T21:46:41.515600+0000","last_became_peered":"2023-09-20T21:46:41.515600+0000","last_unstale":"2023-09-20T21:46:45.570531+0000","last_undegraded":"2023-09-20T21:46:45.570531+0000","last_fullsized":"2023-09-20T21:46:45.570531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:50:58.088103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569630+0000","last_change":"2023-09-20T21:46:41.510695+0000","last_active":"2023-09-20T21:46:45.569630+0000","last_peered":"2023-09-20T21:46:45.569630+0000","last_clean":"2023-09-20T21:46:45.569630+0000","last_became_active":"2023-09-20T21:46:41.510345+0000","last_became_peered":"2023-09-20T21:46:41.510345+0000","last_unstale":"2023-09-20T21:46:45.569630+0000","last_undegraded":"2023-09-20T21:46:45.569630+0000","last_fullsized":"2023-09-20T21:46:45.569630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:38:24.201382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575033+0000","last_change":"2023-09-20T21:46:41.507809+0000","last_active":"2023-09-20T21:46:45.575033+0000","last_peered":"2023-09-20T21:46:45.575033+0000","last_clean":"2023-09-20T21:46:45.575033+0000","last_became_active":"2023-09-20T21:46:41.507513+0000","last_became_peered":"2023-09-20T21:46:41.507513+0000","last_unstale":"2023-09-20T21:46:45.575033+0000","last_undegraded":"2023-09-20T21:46:45.575033+0000","last_fullsized":"2023-09-20T21:46:45.575033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:30:43.132728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598414+0000","last_change":"2023-09-20T21:46:41.503590+0000","last_active":"2023-09-20T21:47:07.598414+0000","last_peered":"2023-09-20T21:47:07.598414+0000","last_clean":"2023-09-20T21:47:07.598414+0000","last_became_active":"2023-09-20T21:46:41.503295+0000","last_became_peered":"2023-09-20T21:46:41.503295+0000","last_unstale":"2023-09-20T21:47:07.598414+0000","last_undegraded":"2023-09-20T21:47:07.598414+0000","last_fullsized":"2023-09-20T21:47:07.598414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:28:33.088873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5632,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":110,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":19996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":487223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.64600000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.85699999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.753}]}]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1071729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46100000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54700000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.59599999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.63200000000000001}]},{"osd":11,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71499999999999997}]}]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":477145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.68500000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.622},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.75}]},{"osd":4,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.79700000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.82899999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:46:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.7719999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.872}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":486769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77200000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.34100000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.371}]},{"osd":2,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65500000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64700000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":1.121}]},{"osd":3,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.64300000000000002,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.92600000000000005}]},{"osd":4,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79500000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.878,"15min":0.878},"last":0.84199999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.313,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78600000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.94599999999999995,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.71999999999999997}]},{"osd":7,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.876,"5min":1.018,"15min":1.018},"last":0.57499999999999996}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.96599999999999997}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.627},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1068723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41799999999999998},"min":{"1min":0.27500000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.438,"15min":0.433},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.623},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70799999999999996}]},{"osd":6,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.747},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.64000000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65300000000000002,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.53000000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.78400000000000003}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":1.018,"15min":1.018},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64500000000000002,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":1.2,"15min":1.2},"last":0.88700000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.57199999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.466,"15min":1.466},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":1.012,"15min":1.012},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.175,"5min":1.179,"15min":1.179},"last":0.79900000000000004}]},{"osd":4,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0640000000000001,"5min":1.093,"15min":1.093},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78700000000000003,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.022,"5min":1.407,"15min":1.407},"last":0.93400000000000005}]},{"osd":5,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0549999999999999,"5min":1.266,"15min":1.266},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.046,"5min":1.073,"15min":1.073},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.84799999999999998,"15min":0.83699999999999997},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.85899999999999999,"15min":0.84599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.95099999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:46:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.84499999999999997,"15min":0.85899999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0149999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.751},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.85599999999999998,"15min":0.86399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1240000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0509999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.085},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.91400000000000003}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":480405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.39400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86499999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.52000000000000002,"15min":0.5},"min":{"1min":0.40000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77700000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.41899999999999998,"15min":0.39300000000000002},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.83699999999999997}]},{"osd":6,"last update":"Wed Sep 20 21:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.55900000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.92200000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.372,"15min":0.372},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.81699999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.89200000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.85599999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":1},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":478041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.503,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92300000000000004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.44900000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66600000000000004,"15min":0.64200000000000002},"min":{"1min":0.52000000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.62},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.67400000000000004,"15min":0.627},"min":{"1min":0.64600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.752}]},{"osd":5,"last update":"Wed Sep 20 21:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86899999999999999,"5min":1.149,"15min":1.149},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.67200000000000004},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.94199999999999995,"5min":1.194,"15min":1.194},"last":0.92000000000000004}]},{"osd":6,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.69599999999999995,"15min":0.67000000000000004},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.026},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82399999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.012,"5min":1.214,"15min":1.214},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.90500000000000003,"15min":0.91600000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0940000000000001,"5min":1.232,"15min":1.232},"last":1.073}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.091},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.90000000000000002}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.111},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.96099999999999997}]},{"osd":11,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.129}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78900000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.39400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.59299999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85299999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.44,"15min":0.39900000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.55500000000000005,"15min":0.49399999999999999},"min":{"1min":0.38300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":7.1500000000000004,"5min":7.1500000000000004,"15min":7.1500000000000004},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.56999999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.65400000000000003,"15min":0.64100000000000001},"min":{"1min":0.53000000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73699999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.63}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.88}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.753},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":476667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.432,"15min":0.39000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81699999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":1,"15min":1},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.84899999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63300000000000001,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65300000000000002,"15min":0.626},"min":{"1min":0.53500000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68799999999999994,"15min":0.64500000000000002},"min":{"1min":0.60399999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed Sep 20 21:46:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":1.024,"15min":1.024},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75900000000000001,"15min":0.747},"min":{"1min":0.61099999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98399999999999999,"5min":1.167,"15min":1.167},"last":0.69599999999999995}]},{"osd":6,"last update":"Wed Sep 20 21:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.52500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.075,"5min":1.147,"15min":1.147},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.627}]},{"osd":7,"last update":"Wed Sep 20 21:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.55400000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.177,"5min":1.248,"15min":1.248},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0549999999999999,"5min":1.228,"15min":1.228},"last":0.874}]},{"osd":8,"last update":"Wed Sep 20 21:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.74299999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.85099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":486697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57699999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.38,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.40899999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.46400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68700000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.54500000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.48499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.51900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61799999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58599999999999997,"15min":0.56799999999999995},"min":{"1min":0.503,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:47:07 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67000000000000004,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.49399999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56699999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":1,"15min":1},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88200000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68500000000000005}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.69999999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.71599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1581056,"data_stored":1066832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85499999999999998,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.753,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.42699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81299999999999994,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.36699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.434},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40400000000000003,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.59899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79100000000000004,"5min":1.615,"15min":1.615},"last":0.496},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.008,"15min":1.008},"last":0.77300000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65900000000000003,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85999999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.75800000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.749,"5min":0.879,"15min":0.879},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59599999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.78900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-20T21:47:09.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.769+0000 7f8469dff700 1 -- 172.21.15.22:0/902352560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8464101fd0 msgr2=0x7f8464102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:09.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.769+0000 7f8469dff700 1 --2- 172.21.15.22:0/902352560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8464101fd0 0x7f8464102430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84580099c0 tx=0x7f8458009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.770+0000 7f8469dff700 1 -- 172.21.15.22:0/902352560 shutdown_connections 2023-09-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.770+0000 7f8469dff700 1 --2- 172.21.15.22:0/902352560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8464102970 0x7f8464109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.770+0000 7f8469dff700 1 --2- 172.21.15.22:0/902352560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8464100de0 0x7f8464101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.770+0000 7f8469dff700 1 --2- 172.21.15.22:0/902352560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8464101fd0 0x7f8464102430 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.770+0000 7f8469dff700 1 -- 172.21.15.22:0/902352560 >> 172.21.15.22:0/902352560 conn(0x7f84640fc310 msgr2=0x7f84640fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:09.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.770+0000 7f8469dff700 1 -- 172.21.15.22:0/902352560 shutdown_connections 2023-09-20T21:47:09.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.770+0000 7f8469dff700 1 -- 172.21.15.22:0/902352560 wait complete. 2023-09-20T21:47:09.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.771+0000 7f8469dff700 1 Processor -- start 2023-09-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.771+0000 7f8469dff700 1 -- start start 2023-09-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.771+0000 7f8469dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8464100de0 0x7f8464198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.771+0000 7f8469dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8464101fd0 0x7f8464199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.771+0000 7f8469dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8464102970 0x7f84641a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:09.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.771+0000 7f8469dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84641a0830 con 0x7f8464101fd0 2023-09-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.771+0000 7f8469dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84641a09a0 con 0x7f8464100de0 2023-09-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.771+0000 7f8469dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84641a0b60 con 0x7f8464102970 2023-09-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.771+0000 7f8462ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8464101fd0 0x7f8464199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:09.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8462ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8464101fd0 0x7f8464199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50812/0 (socket says 172.21.15.22:50812) 2023-09-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8462ffd700 1 -- 172.21.15.22:0/364406131 learned_addr learned my addr 172.21.15.22:0/364406131 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8463fff700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8464102970 0x7f84641a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f84637fe700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8464100de0 0x7f8464198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8462ffd700 1 -- 172.21.15.22:0/364406131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8464102970 msgr2=0x7f84641a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8462ffd700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8464102970 0x7f84641a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8462ffd700 1 -- 172.21.15.22:0/364406131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8464100de0 msgr2=0x7f8464198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8462ffd700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8464100de0 0x7f8464198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8462ffd700 1 -- 172.21.15.22:0/364406131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84580095f0 con 0x7f8464101fd0 2023-09-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8463fff700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8464102970 0x7f84641a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8462ffd700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8464101fd0 0x7f8464199230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8458009990 tx=0x7f845800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:09.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.772+0000 7f8460ff9700 1 -- 172.21.15.22:0/364406131 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845801d070 con 0x7f8464101fd0 2023-09-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.773+0000 7f8460ff9700 1 -- 172.21.15.22:0/364406131 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8458005030 con 0x7f8464101fd0 2023-09-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.773+0000 7f8469dff700 1 -- 172.21.15.22:0/364406131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84641a0de0 con 0x7f8464101fd0 2023-09-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.773+0000 7f8460ff9700 1 -- 172.21.15.22:0/364406131 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458004a70 con 0x7f8464101fd0 2023-09-20T21:47:09.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.773+0000 7f8469dff700 1 -- 172.21.15.22:0/364406131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84641a11b0 con 0x7f8464101fd0 2023-09-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.774+0000 7f8469dff700 1 -- 172.21.15.22:0/364406131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846410c640 con 0x7f8464101fd0 2023-09-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.775+0000 7f8460ff9700 1 -- 172.21.15.22:0/364406131 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f8458005410 con 0x7f8464101fd0 2023-09-20T21:47:09.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.777+0000 7f8460ff9700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f844c0a1350 0x7f844c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.777+0000 7f8460ff9700 1 -- 172.21.15.22:0/364406131 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f84580cdd50 con 0x7f8464101fd0 2023-09-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.777+0000 7f84637fe700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f844c0a1350 0x7f844c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:09.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.778+0000 7f84637fe700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f844c0a1350 0x7f844c0a3810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f846419a1c0 tx=0x7f8454009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:09.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.780+0000 7f8460ff9700 1 -- 172.21.15.22:0/364406131 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8458097150 con 0x7f8464101fd0 2023-09-20T21:47:09.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:09 smithi089 ceph-mon[134090]: osdmap e91: 12 total, 12 up, 12 in 2023-09-20T21:47:09.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:09 smithi089 ceph-mon[134090]: daemon mds.j is now active in filesystem cephfs as rank 4 2023-09-20T21:47:09.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:09 smithi089 ceph-mon[134090]: from='client.24805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:09.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:09 smithi089 ceph-mon[134090]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 582 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.1 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:47:09.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.986+0000 7f8469dff700 1 -- 172.21.15.22:0/364406131 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8464108920 con 0x7f844c0a1350 2023-09-20T21:47:09.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:09 smithi022 ceph-mon[97651]: osdmap e91: 12 total, 12 up, 12 in 2023-09-20T21:47:09.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:09 smithi022 ceph-mon[97651]: daemon mds.j is now active in filesystem cephfs as rank 4 2023-09-20T21:47:09.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:09 smithi022 ceph-mon[97651]: from='client.24805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:09.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:09 smithi022 ceph-mon[97651]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 582 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.1 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:47:09.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:09.999+0000 7f8460ff9700 1 -- 172.21.15.22:0/364406131 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+421925 (secure 0 0 0) 0x7f8464108920 con 0x7f844c0a1350 2023-09-20T21:47:10.004 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 -- 172.21.15.22:0/364406131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f844c0a1350 msgr2=0x7f844c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:10.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f844c0a1350 0x7f844c0a3810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f846419a1c0 tx=0x7f8454009260 comp rx=0 tx=0).stop 2023-09-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 -- 172.21.15.22:0/364406131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8464101fd0 msgr2=0x7f8464199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8464101fd0 0x7f8464199230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8458009990 tx=0x7f845800f690 comp rx=0 tx=0).stop 2023-09-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 -- 172.21.15.22:0/364406131 shutdown_connections 2023-09-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8464102970 0x7f84641a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f844c0a1350 0x7f844c0a3810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8464100de0 0x7f8464198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 --2- 172.21.15.22:0/364406131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8464101fd0 0x7f8464199230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 -- 172.21.15.22:0/364406131 >> 172.21.15.22:0/364406131 conn(0x7f84640fc310 msgr2=0x7f84640fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 -- 172.21.15.22:0/364406131 shutdown_connections 2023-09-20T21:47:10.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.004+0000 7f8469dff700 1 -- 172.21.15.22:0/364406131 wait complete. 2023-09-20T21:47:10.010 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-09-20T21:47:10.022 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-09-20T21:47:09.102716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596000,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":137,"num_read_kb":118,"num_write":291,"num_write_kb":2978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337664,"kb_used_data":13444,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604160,"statfs":{"total":1151940427776,"available":1151594659840,"internally_reserved":0,"allocated":13766656,"data_stored":7537855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2773,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":64,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002240"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598937+0000","last_change":"2023-09-20T21:46:41.506092+0000","last_active":"2023-09-20T21:47:07.598937+0000","last_peered":"2023-09-20T21:47:07.598937+0000","last_clean":"2023-09-20T21:47:07.598937+0000","last_became_active":"2023-09-20T21:46:41.505779+0000","last_became_peered":"2023-09-20T21:46:41.505779+0000","last_unstale":"2023-09-20T21:47:07.598937+0000","last_undegraded":"2023-09-20T21:47:07.598937+0000","last_fullsized":"2023-09-20T21:47:07.598937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:25.284068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571607+0000","last_change":"2023-09-20T21:46:41.505000+0000","last_active":"2023-09-20T21:46:45.571607+0000","last_peered":"2023-09-20T21:46:45.571607+0000","last_clean":"2023-09-20T21:46:45.571607+0000","last_became_active":"2023-09-20T21:46:41.504464+0000","last_became_peered":"2023-09-20T21:46:41.504464+0000","last_unstale":"2023-09-20T21:46:45.571607+0000","last_undegraded":"2023-09-20T21:46:45.571607+0000","last_fullsized":"2023-09-20T21:46:45.571607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:42:44.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569994+0000","last_change":"2023-09-20T21:46:41.513134+0000","last_active":"2023-09-20T21:46:45.569994+0000","last_peered":"2023-09-20T21:46:45.569994+0000","last_clean":"2023-09-20T21:46:45.569994+0000","last_became_active":"2023-09-20T21:46:41.512793+0000","last_became_peered":"2023-09-20T21:46:41.512793+0000","last_unstale":"2023-09-20T21:46:45.569994+0000","last_undegraded":"2023-09-20T21:46:45.569994+0000","last_fullsized":"2023-09-20T21:46:45.569994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:59.978630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597341+0000","last_change":"2023-09-20T21:46:41.500200+0000","last_active":"2023-09-20T21:47:07.597341+0000","last_peered":"2023-09-20T21:47:07.597341+0000","last_clean":"2023-09-20T21:47:07.597341+0000","last_became_active":"2023-09-20T21:46:41.499866+0000","last_became_peered":"2023-09-20T21:46:41.499866+0000","last_unstale":"2023-09-20T21:47:07.597341+0000","last_undegraded":"2023-09-20T21:47:07.597341+0000","last_fullsized":"2023-09-20T21:47:07.597341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:20:52.989008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501872+0000","last_change":"2023-09-20T21:46:41.507584+0000","last_active":"2023-09-20T21:46:47.501872+0000","last_peered":"2023-09-20T21:46:47.501872+0000","last_clean":"2023-09-20T21:46:47.501872+0000","last_became_active":"2023-09-20T21:46:41.507298+0000","last_became_peered":"2023-09-20T21:46:41.507298+0000","last_unstale":"2023-09-20T21:46:47.501872+0000","last_undegraded":"2023-09-20T21:46:47.501872+0000","last_fullsized":"2023-09-20T21:46:47.501872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:32:14.042414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564527+0000","last_change":"2023-09-20T21:46:41.497002+0000","last_active":"2023-09-20T21:46:45.564527+0000","last_peered":"2023-09-20T21:46:45.564527+0000","last_clean":"2023-09-20T21:46:45.564527+0000","last_became_active":"2023-09-20T21:46:41.496485+0000","last_became_peered":"2023-09-20T21:46:41.496485+0000","last_unstale":"2023-09-20T21:46:45.564527+0000","last_undegraded":"2023-09-20T21:46:45.564527+0000","last_fullsized":"2023-09-20T21:46:45.564527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:48:24.481575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.567850+0000","last_change":"2023-09-20T21:46:41.503303+0000","last_active":"2023-09-20T21:46:45.567850+0000","last_peered":"2023-09-20T21:46:45.567850+0000","last_clean":"2023-09-20T21:46:45.567850+0000","last_became_active":"2023-09-20T21:46:41.502982+0000","last_became_peered":"2023-09-20T21:46:41.502982+0000","last_unstale":"2023-09-20T21:46:45.567850+0000","last_undegraded":"2023-09-20T21:46:45.567850+0000","last_fullsized":"2023-09-20T21:46:45.567850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:23:27.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569351+0000","last_change":"2023-09-20T21:46:41.508243+0000","last_active":"2023-09-20T21:46:45.569351+0000","last_peered":"2023-09-20T21:46:45.569351+0000","last_clean":"2023-09-20T21:46:45.569351+0000","last_became_active":"2023-09-20T21:46:41.506407+0000","last_became_peered":"2023-09-20T21:46:41.506407+0000","last_unstale":"2023-09-20T21:46:45.569351+0000","last_undegraded":"2023-09-20T21:46:45.569351+0000","last_fullsized":"2023-09-20T21:46:45.569351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:39.388716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568936+0000","last_change":"2023-09-20T21:46:41.501513+0000","last_active":"2023-09-20T21:46:45.568936+0000","last_peered":"2023-09-20T21:46:45.568936+0000","last_clean":"2023-09-20T21:46:45.568936+0000","last_became_active":"2023-09-20T21:46:41.500051+0000","last_became_peered":"2023-09-20T21:46:41.500051+0000","last_unstale":"2023-09-20T21:46:45.568936+0000","last_undegraded":"2023-09-20T21:46:45.568936+0000","last_fullsized":"2023-09-20T21:46:45.568936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:40.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564777+0000","last_change":"2023-09-20T21:46:41.501045+0000","last_active":"2023-09-20T21:46:45.564777+0000","last_peered":"2023-09-20T21:46:45.564777+0000","last_clean":"2023-09-20T21:46:45.564777+0000","last_became_active":"2023-09-20T21:46:41.500744+0000","last_became_peered":"2023-09-20T21:46:41.500744+0000","last_unstale":"2023-09-20T21:46:45.564777+0000","last_undegraded":"2023-09-20T21:46:45.564777+0000","last_fullsized":"2023-09-20T21:46:45.564777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:57:29.120472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570586+0000","last_change":"2023-09-20T21:46:41.510235+0000","last_active":"2023-09-20T21:46:45.570586+0000","last_peered":"2023-09-20T21:46:45.570586+0000","last_clean":"2023-09-20T21:46:45.570586+0000","last_became_active":"2023-09-20T21:46:41.509943+0000","last_became_peered":"2023-09-20T21:46:41.509943+0000","last_unstale":"2023-09-20T21:46:45.570586+0000","last_undegraded":"2023-09-20T21:46:45.570586+0000","last_fullsized":"2023-09-20T21:46:45.570586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:09:19.383505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597247+0000","last_change":"2023-09-20T21:46:41.501194+0000","last_active":"2023-09-20T21:47:07.597247+0000","last_peered":"2023-09-20T21:47:07.597247+0000","last_clean":"2023-09-20T21:47:07.597247+0000","last_became_active":"2023-09-20T21:46:41.500095+0000","last_became_peered":"2023-09-20T21:46:41.500095+0000","last_unstale":"2023-09-20T21:47:07.597247+0000","last_undegraded":"2023-09-20T21:47:07.597247+0000","last_fullsized":"2023-09-20T21:47:07.597247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:46:51.343332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575166+0000","last_change":"2023-09-20T21:46:41.507562+0000","last_active":"2023-09-20T21:46:45.575166+0000","last_peered":"2023-09-20T21:46:45.575166+0000","last_clean":"2023-09-20T21:46:45.575166+0000","last_became_active":"2023-09-20T21:46:41.507094+0000","last_became_peered":"2023-09-20T21:46:41.507094+0000","last_unstale":"2023-09-20T21:46:45.575166+0000","last_undegraded":"2023-09-20T21:46:45.575166+0000","last_fullsized":"2023-09-20T21:46:45.575166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:34:32.401415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569178+0000","last_change":"2023-09-20T21:46:41.502864+0000","last_active":"2023-09-20T21:46:45.569178+0000","last_peered":"2023-09-20T21:46:45.569178+0000","last_clean":"2023-09-20T21:46:45.569178+0000","last_became_active":"2023-09-20T21:46:41.502543+0000","last_became_peered":"2023-09-20T21:46:41.502543+0000","last_unstale":"2023-09-20T21:46:45.569178+0000","last_undegraded":"2023-09-20T21:46:45.569178+0000","last_fullsized":"2023-09-20T21:46:45.569178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:02:04.631874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.572035+0000","last_change":"2023-09-20T21:46:41.512189+0000","last_active":"2023-09-20T21:46:45.572035+0000","last_peered":"2023-09-20T21:46:45.572035+0000","last_clean":"2023-09-20T21:46:45.572035+0000","last_became_active":"2023-09-20T21:46:41.511888+0000","last_became_peered":"2023-09-20T21:46:41.511888+0000","last_unstale":"2023-09-20T21:46:45.572035+0000","last_undegraded":"2023-09-20T21:46:45.572035+0000","last_fullsized":"2023-09-20T21:46:45.572035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:17:20.417468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571768+0000","last_change":"2023-09-20T21:46:41.502036+0000","last_active":"2023-09-20T21:46:45.571768+0000","last_peered":"2023-09-20T21:46:45.571768+0000","last_clean":"2023-09-20T21:46:45.571768+0000","last_became_active":"2023-09-20T21:46:41.501745+0000","last_became_peered":"2023-09-20T21:46:41.501745+0000","last_unstale":"2023-09-20T21:46:45.571768+0000","last_undegraded":"2023-09-20T21:46:45.571768+0000","last_fullsized":"2023-09-20T21:46:45.571768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:08:06.699543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598739+0000","last_change":"2023-09-20T21:46:41.506666+0000","last_active":"2023-09-20T21:47:07.598739+0000","last_peered":"2023-09-20T21:47:07.598739+0000","last_clean":"2023-09-20T21:47:07.598739+0000","last_became_active":"2023-09-20T21:46:41.506266+0000","last_became_peered":"2023-09-20T21:46:41.506266+0000","last_unstale":"2023-09-20T21:47:07.598739+0000","last_undegraded":"2023-09-20T21:47:07.598739+0000","last_fullsized":"2023-09-20T21:47:07.598739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:06:16.809609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563418+0000","last_change":"2023-09-20T21:46:41.497391+0000","last_active":"2023-09-20T21:46:45.563418+0000","last_peered":"2023-09-20T21:46:45.563418+0000","last_clean":"2023-09-20T21:46:45.563418+0000","last_became_active":"2023-09-20T21:46:41.497064+0000","last_became_peered":"2023-09-20T21:46:41.497064+0000","last_unstale":"2023-09-20T21:46:45.563418+0000","last_undegraded":"2023-09-20T21:46:45.563418+0000","last_fullsized":"2023-09-20T21:46:45.563418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:56:11.489585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569164+0000","last_change":"2023-09-20T21:46:41.510602+0000","last_active":"2023-09-20T21:46:45.569164+0000","last_peered":"2023-09-20T21:46:45.569164+0000","last_clean":"2023-09-20T21:46:45.569164+0000","last_became_active":"2023-09-20T21:46:41.510308+0000","last_became_peered":"2023-09-20T21:46:41.510308+0000","last_unstale":"2023-09-20T21:46:45.569164+0000","last_undegraded":"2023-09-20T21:46:45.569164+0000","last_fullsized":"2023-09-20T21:46:45.569164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:43:41.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565451+0000","last_change":"2023-09-20T21:46:45.581583+0000","last_active":"2023-09-20T21:46:46.565451+0000","last_peered":"2023-09-20T21:46:46.565451+0000","last_clean":"2023-09-20T21:46:46.565451+0000","last_became_active":"2023-09-20T21:46:45.581165+0000","last_became_peered":"2023-09-20T21:46:45.581165+0000","last_unstale":"2023-09-20T21:46:46.565451+0000","last_undegraded":"2023-09-20T21:46:46.565451+0000","last_fullsized":"2023-09-20T21:46:46.565451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:39:36.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564314+0000","last_change":"2023-09-20T21:46:40.500010+0000","last_active":"2023-09-20T21:46:45.564314+0000","last_peered":"2023-09-20T21:46:45.564314+0000","last_clean":"2023-09-20T21:46:45.564314+0000","last_became_active":"2023-09-20T21:46:40.499632+0000","last_became_peered":"2023-09-20T21:46:40.499632+0000","last_unstale":"2023-09-20T21:46:45.564314+0000","last_undegraded":"2023-09-20T21:46:45.564314+0000","last_fullsized":"2023-09-20T21:46:45.564314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:41:35.590982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569101+0000","last_change":"2023-09-20T21:46:41.516465+0000","last_active":"2023-09-20T21:46:45.569101+0000","last_peered":"2023-09-20T21:46:45.569101+0000","last_clean":"2023-09-20T21:46:45.569101+0000","last_became_active":"2023-09-20T21:46:41.516150+0000","last_became_peered":"2023-09-20T21:46:41.516150+0000","last_unstale":"2023-09-20T21:46:45.569101+0000","last_undegraded":"2023-09-20T21:46:45.569101+0000","last_fullsized":"2023-09-20T21:46:45.569101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:33:37.507049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596541+0000","last_change":"2023-09-20T21:46:45.582916+0000","last_active":"2023-09-20T21:47:07.596541+0000","last_peered":"2023-09-20T21:47:07.596541+0000","last_clean":"2023-09-20T21:47:07.596541+0000","last_became_active":"2023-09-20T21:46:45.582511+0000","last_became_peered":"2023-09-20T21:46:45.582511+0000","last_unstale":"2023-09-20T21:47:07.596541+0000","last_undegraded":"2023-09-20T21:47:07.596541+0000","last_fullsized":"2023-09-20T21:47:07.596541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:43:47.006768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569207+0000","last_change":"2023-09-20T21:46:40.516867+0000","last_active":"2023-09-20T21:46:45.569207+0000","last_peered":"2023-09-20T21:46:45.569207+0000","last_clean":"2023-09-20T21:46:45.569207+0000","last_became_active":"2023-09-20T21:46:40.516343+0000","last_became_peered":"2023-09-20T21:46:40.516343+0000","last_unstale":"2023-09-20T21:46:45.569207+0000","last_undegraded":"2023-09-20T21:46:45.569207+0000","last_fullsized":"2023-09-20T21:46:45.569207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:17:18.348921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573554+0000","last_change":"2023-09-20T21:46:41.507093+0000","last_active":"2023-09-20T21:46:45.573554+0000","last_peered":"2023-09-20T21:46:45.573554+0000","last_clean":"2023-09-20T21:46:45.573554+0000","last_became_active":"2023-09-20T21:46:41.506587+0000","last_became_peered":"2023-09-20T21:46:41.506587+0000","last_unstale":"2023-09-20T21:46:45.573554+0000","last_undegraded":"2023-09-20T21:46:45.573554+0000","last_fullsized":"2023-09-20T21:46:45.573554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:26:44.825499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597188+0000","last_change":"2023-09-20T21:46:45.583942+0000","last_active":"2023-09-20T21:47:07.597188+0000","last_peered":"2023-09-20T21:47:07.597188+0000","last_clean":"2023-09-20T21:47:07.597188+0000","last_became_active":"2023-09-20T21:46:45.583484+0000","last_became_peered":"2023-09-20T21:46:45.583484+0000","last_unstale":"2023-09-20T21:47:07.597188+0000","last_undegraded":"2023-09-20T21:47:07.597188+0000","last_fullsized":"2023-09-20T21:47:07.597188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:48.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"peering","last_fresh":"2023-09-20T21:47:07.573536+0000","last_change":"2023-09-20T21:47:07.596775+0000","last_active":"2023-09-20T21:47:07.596775+0000","last_peered":"2023-09-20T21:47:07.573536+0000","last_clean":"2023-09-20T21:47:07.573536+0000","last_became_active":"2023-09-20T21:46:40.501028+0000","last_became_peered":"2023-09-20T21:46:40.501028+0000","last_unstale":"2023-09-20T21:47:07.573536+0000","last_undegraded":"2023-09-20T21:47:07.573536+0000","last_fullsized":"2023-09-20T21:47:07.573536+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,7],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568146+0000","last_change":"2023-09-20T21:46:41.504449+0000","last_active":"2023-09-20T21:46:45.568146+0000","last_peered":"2023-09-20T21:46:45.568146+0000","last_clean":"2023-09-20T21:46:45.568146+0000","last_became_active":"2023-09-20T21:46:41.504169+0000","last_became_peered":"2023-09-20T21:46:41.504169+0000","last_unstale":"2023-09-20T21:46:45.568146+0000","last_undegraded":"2023-09-20T21:46:45.568146+0000","last_fullsized":"2023-09-20T21:46:45.568146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:36:43.980902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596618+0000","last_change":"2023-09-20T21:46:45.582269+0000","last_active":"2023-09-20T21:47:07.596618+0000","last_peered":"2023-09-20T21:47:07.596618+0000","last_clean":"2023-09-20T21:47:07.596618+0000","last_became_active":"2023-09-20T21:46:45.581782+0000","last_became_peered":"2023-09-20T21:46:45.581782+0000","last_unstale":"2023-09-20T21:47:07.596618+0000","last_undegraded":"2023-09-20T21:47:07.596618+0000","last_fullsized":"2023-09-20T21:47:07.596618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:32:09.518715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502079+0000","last_change":"2023-09-20T21:46:40.504099+0000","last_active":"2023-09-20T21:46:47.502079+0000","last_peered":"2023-09-20T21:46:47.502079+0000","last_clean":"2023-09-20T21:46:47.502079+0000","last_became_active":"2023-09-20T21:46:40.503774+0000","last_became_peered":"2023-09-20T21:46:40.503774+0000","last_unstale":"2023-09-20T21:46:47.502079+0000","last_undegraded":"2023-09-20T21:46:47.502079+0000","last_fullsized":"2023-09-20T21:46:47.502079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:17.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568776+0000","last_change":"2023-09-20T21:46:41.499802+0000","last_active":"2023-09-20T21:46:45.568776+0000","last_peered":"2023-09-20T21:46:45.568776+0000","last_clean":"2023-09-20T21:46:45.568776+0000","last_became_active":"2023-09-20T21:46:41.499491+0000","last_became_peered":"2023-09-20T21:46:41.499491+0000","last_unstale":"2023-09-20T21:46:45.568776+0000","last_undegraded":"2023-09-20T21:46:45.568776+0000","last_fullsized":"2023-09-20T21:46:45.568776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:44.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565986+0000","last_change":"2023-09-20T21:46:45.578753+0000","last_active":"2023-09-20T21:46:46.565986+0000","last_peered":"2023-09-20T21:46:46.565986+0000","last_clean":"2023-09-20T21:46:46.565986+0000","last_became_active":"2023-09-20T21:46:45.578297+0000","last_became_peered":"2023-09-20T21:46:45.578297+0000","last_unstale":"2023-09-20T21:46:46.565986+0000","last_undegraded":"2023-09-20T21:46:46.565986+0000","last_fullsized":"2023-09-20T21:46:46.565986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:18:22.596550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571690+0000","last_change":"2023-09-20T21:46:40.505675+0000","last_active":"2023-09-20T21:46:45.571690+0000","last_peered":"2023-09-20T21:46:45.571690+0000","last_clean":"2023-09-20T21:46:45.571690+0000","last_became_active":"2023-09-20T21:46:40.505432+0000","last_became_peered":"2023-09-20T21:46:40.505432+0000","last_unstale":"2023-09-20T21:46:45.571690+0000","last_undegraded":"2023-09-20T21:46:45.571690+0000","last_fullsized":"2023-09-20T21:46:45.571690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:14:53.652904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501949+0000","last_change":"2023-09-20T21:46:41.505924+0000","last_active":"2023-09-20T21:46:47.501949+0000","last_peered":"2023-09-20T21:46:47.501949+0000","last_clean":"2023-09-20T21:46:47.501949+0000","last_became_active":"2023-09-20T21:46:41.505609+0000","last_became_peered":"2023-09-20T21:46:41.505609+0000","last_unstale":"2023-09-20T21:46:47.501949+0000","last_undegraded":"2023-09-20T21:46:47.501949+0000","last_fullsized":"2023-09-20T21:46:47.501949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:13:46.506786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565886+0000","last_change":"2023-09-20T21:46:45.587818+0000","last_active":"2023-09-20T21:46:46.565886+0000","last_peered":"2023-09-20T21:46:46.565886+0000","last_clean":"2023-09-20T21:46:46.565886+0000","last_became_active":"2023-09-20T21:46:45.587376+0000","last_became_peered":"2023-09-20T21:46:45.587376+0000","last_unstale":"2023-09-20T21:46:46.565886+0000","last_undegraded":"2023-09-20T21:46:46.565886+0000","last_fullsized":"2023-09-20T21:46:46.565886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:28:53.605158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564415+0000","last_change":"2023-09-20T21:46:40.505610+0000","last_active":"2023-09-20T21:46:45.564415+0000","last_peered":"2023-09-20T21:46:45.564415+0000","last_clean":"2023-09-20T21:46:45.564415+0000","last_became_active":"2023-09-20T21:46:40.505279+0000","last_became_peered":"2023-09-20T21:46:40.505279+0000","last_unstale":"2023-09-20T21:46:45.564415+0000","last_undegraded":"2023-09-20T21:46:45.564415+0000","last_fullsized":"2023-09-20T21:46:45.564415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:41:19.575154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571794+0000","last_change":"2023-09-20T21:46:41.505337+0000","last_active":"2023-09-20T21:46:45.571794+0000","last_peered":"2023-09-20T21:46:45.571794+0000","last_clean":"2023-09-20T21:46:45.571794+0000","last_became_active":"2023-09-20T21:46:41.504959+0000","last_became_peered":"2023-09-20T21:46:41.504959+0000","last_unstale":"2023-09-20T21:46:45.571794+0000","last_undegraded":"2023-09-20T21:46:45.571794+0000","last_fullsized":"2023-09-20T21:46:45.571794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:53:37.797319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598682+0000","last_change":"2023-09-20T21:46:45.581738+0000","last_active":"2023-09-20T21:47:07.598682+0000","last_peered":"2023-09-20T21:47:07.598682+0000","last_clean":"2023-09-20T21:47:07.598682+0000","last_became_active":"2023-09-20T21:46:45.581279+0000","last_became_peered":"2023-09-20T21:46:45.581279+0000","last_unstale":"2023-09-20T21:47:07.598682+0000","last_undegraded":"2023-09-20T21:47:07.598682+0000","last_fullsized":"2023-09-20T21:47:07.598682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:41:41.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570018+0000","last_change":"2023-09-20T21:46:40.510539+0000","last_active":"2023-09-20T21:46:45.570018+0000","last_peered":"2023-09-20T21:46:45.570018+0000","last_clean":"2023-09-20T21:46:45.570018+0000","last_became_active":"2023-09-20T21:46:40.509880+0000","last_became_peered":"2023-09-20T21:46:40.509880+0000","last_unstale":"2023-09-20T21:46:45.570018+0000","last_undegraded":"2023-09-20T21:46:45.570018+0000","last_fullsized":"2023-09-20T21:46:45.570018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:52.877786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569259+0000","last_change":"2023-09-20T21:46:41.505105+0000","last_active":"2023-09-20T21:46:45.569259+0000","last_peered":"2023-09-20T21:46:45.569259+0000","last_clean":"2023-09-20T21:46:45.569259+0000","last_became_active":"2023-09-20T21:46:41.504785+0000","last_became_peered":"2023-09-20T21:46:41.504785+0000","last_unstale":"2023-09-20T21:46:45.569259+0000","last_undegraded":"2023-09-20T21:46:45.569259+0000","last_fullsized":"2023-09-20T21:46:45.569259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:07:57.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502213+0000","last_change":"2023-09-20T21:46:45.583160+0000","last_active":"2023-09-20T21:46:47.502213+0000","last_peered":"2023-09-20T21:46:47.502213+0000","last_clean":"2023-09-20T21:46:47.502213+0000","last_became_active":"2023-09-20T21:46:45.582709+0000","last_became_peered":"2023-09-20T21:46:45.582709+0000","last_unstale":"2023-09-20T21:46:47.502213+0000","last_undegraded":"2023-09-20T21:46:47.502213+0000","last_fullsized":"2023-09-20T21:46:47.502213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:24:57.563682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568131+0000","last_change":"2023-09-20T21:46:40.502591+0000","last_active":"2023-09-20T21:46:45.568131+0000","last_peered":"2023-09-20T21:46:45.568131+0000","last_clean":"2023-09-20T21:46:45.568131+0000","last_became_active":"2023-09-20T21:46:40.502288+0000","last_became_peered":"2023-09-20T21:46:40.502288+0000","last_unstale":"2023-09-20T21:46:45.568131+0000","last_undegraded":"2023-09-20T21:46:45.568131+0000","last_fullsized":"2023-09-20T21:46:45.568131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:09:37.983866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563192+0000","last_change":"2023-09-20T21:46:41.511367+0000","last_active":"2023-09-20T21:46:45.563192+0000","last_peered":"2023-09-20T21:46:45.563192+0000","last_clean":"2023-09-20T21:46:45.563192+0000","last_became_active":"2023-09-20T21:46:41.511005+0000","last_became_peered":"2023-09-20T21:46:41.511005+0000","last_unstale":"2023-09-20T21:46:45.563192+0000","last_undegraded":"2023-09-20T21:46:45.563192+0000","last_fullsized":"2023-09-20T21:46:45.563192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:37:59.684808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.862394+0000","last_change":"2023-09-20T21:46:45.580917+0000","last_active":"2023-09-20T21:46:46.862394+0000","last_peered":"2023-09-20T21:46:46.862394+0000","last_clean":"2023-09-20T21:46:46.862394+0000","last_became_active":"2023-09-20T21:46:45.580505+0000","last_became_peered":"2023-09-20T21:46:45.580505+0000","last_unstale":"2023-09-20T21:46:46.862394+0000","last_undegraded":"2023-09-20T21:46:46.862394+0000","last_fullsized":"2023-09-20T21:46:46.862394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:52:19.733919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.591401+0000","last_change":"2023-09-20T21:46:40.507040+0000","last_active":"2023-09-20T21:47:05.591401+0000","last_peered":"2023-09-20T21:47:05.591401+0000","last_clean":"2023-09-20T21:47:05.591401+0000","last_became_active":"2023-09-20T21:46:40.506712+0000","last_became_peered":"2023-09-20T21:46:40.506712+0000","last_unstale":"2023-09-20T21:47:05.591401+0000","last_undegraded":"2023-09-20T21:47:05.591401+0000","last_fullsized":"2023-09-20T21:47:05.591401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:02:56.040911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568722+0000","last_change":"2023-09-20T21:46:41.504794+0000","last_active":"2023-09-20T21:46:45.568722+0000","last_peered":"2023-09-20T21:46:45.568722+0000","last_clean":"2023-09-20T21:46:45.568722+0000","last_became_active":"2023-09-20T21:46:41.504372+0000","last_became_peered":"2023-09-20T21:46:41.504372+0000","last_unstale":"2023-09-20T21:46:45.568722+0000","last_undegraded":"2023-09-20T21:46:45.568722+0000","last_fullsized":"2023-09-20T21:46:45.568722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:42:20.527306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.564942+0000","last_change":"2023-09-20T21:46:45.586364+0000","last_active":"2023-09-20T21:46:46.564942+0000","last_peered":"2023-09-20T21:46:46.564942+0000","last_clean":"2023-09-20T21:46:46.564942+0000","last_became_active":"2023-09-20T21:46:45.584620+0000","last_became_peered":"2023-09-20T21:46:45.584620+0000","last_unstale":"2023-09-20T21:46:46.564942+0000","last_undegraded":"2023-09-20T21:46:46.564942+0000","last_fullsized":"2023-09-20T21:46:46.564942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:05:20.158118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.588953+0000","last_change":"2023-09-20T21:46:40.500797+0000","last_active":"2023-09-20T21:47:05.588953+0000","last_peered":"2023-09-20T21:47:05.588953+0000","last_clean":"2023-09-20T21:47:05.588953+0000","last_became_active":"2023-09-20T21:46:40.500337+0000","last_became_peered":"2023-09-20T21:46:40.500337+0000","last_unstale":"2023-09-20T21:47:05.588953+0000","last_undegraded":"2023-09-20T21:47:05.588953+0000","last_fullsized":"2023-09-20T21:47:05.588953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:19.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573645+0000","last_change":"2023-09-20T21:46:41.507805+0000","last_active":"2023-09-20T21:46:45.573645+0000","last_peered":"2023-09-20T21:46:45.573645+0000","last_clean":"2023-09-20T21:46:45.573645+0000","last_became_active":"2023-09-20T21:46:41.507441+0000","last_became_peered":"2023-09-20T21:46:41.507441+0000","last_unstale":"2023-09-20T21:46:45.573645+0000","last_undegraded":"2023-09-20T21:46:45.573645+0000","last_fullsized":"2023-09-20T21:46:45.573645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:51:33.240534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597442+0000","last_change":"2023-09-20T21:46:45.583081+0000","last_active":"2023-09-20T21:47:07.597442+0000","last_peered":"2023-09-20T21:47:07.597442+0000","last_clean":"2023-09-20T21:47:07.597442+0000","last_became_active":"2023-09-20T21:46:45.580496+0000","last_became_peered":"2023-09-20T21:46:45.580496+0000","last_unstale":"2023-09-20T21:47:07.597442+0000","last_undegraded":"2023-09-20T21:47:07.597442+0000","last_fullsized":"2023-09-20T21:47:07.597442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:42.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568676+0000","last_change":"2023-09-20T21:46:40.512739+0000","last_active":"2023-09-20T21:46:45.568676+0000","last_peered":"2023-09-20T21:46:45.568676+0000","last_clean":"2023-09-20T21:46:45.568676+0000","last_became_active":"2023-09-20T21:46:40.512485+0000","last_became_peered":"2023-09-20T21:46:40.512485+0000","last_unstale":"2023-09-20T21:46:45.568676+0000","last_undegraded":"2023-09-20T21:46:45.568676+0000","last_fullsized":"2023-09-20T21:46:45.568676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:02:49.392560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569091+0000","last_change":"2023-09-20T21:46:41.503093+0000","last_active":"2023-09-20T21:46:45.569091+0000","last_peered":"2023-09-20T21:46:45.569091+0000","last_clean":"2023-09-20T21:46:45.569091+0000","last_became_active":"2023-09-20T21:46:41.502806+0000","last_became_peered":"2023-09-20T21:46:41.502806+0000","last_unstale":"2023-09-20T21:46:45.569091+0000","last_undegraded":"2023-09-20T21:46:45.569091+0000","last_fullsized":"2023-09-20T21:46:45.569091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:53:21.143258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565158+0000","last_change":"2023-09-20T21:46:45.579478+0000","last_active":"2023-09-20T21:46:46.565158+0000","last_peered":"2023-09-20T21:46:46.565158+0000","last_clean":"2023-09-20T21:46:46.565158+0000","last_became_active":"2023-09-20T21:46:45.579055+0000","last_became_peered":"2023-09-20T21:46:45.579055+0000","last_unstale":"2023-09-20T21:46:46.565158+0000","last_undegraded":"2023-09-20T21:46:46.565158+0000","last_fullsized":"2023-09-20T21:46:46.565158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:17:32.013280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.590075+0000","last_change":"2023-09-20T21:46:40.507321+0000","last_active":"2023-09-20T21:47:05.590075+0000","last_peered":"2023-09-20T21:47:05.590075+0000","last_clean":"2023-09-20T21:47:05.590075+0000","last_became_active":"2023-09-20T21:46:40.506983+0000","last_became_peered":"2023-09-20T21:46:40.506983+0000","last_unstale":"2023-09-20T21:47:05.590075+0000","last_undegraded":"2023-09-20T21:47:05.590075+0000","last_fullsized":"2023-09-20T21:47:05.590075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:46:40.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502157+0000","last_change":"2023-09-20T21:46:41.509646+0000","last_active":"2023-09-20T21:46:47.502157+0000","last_peered":"2023-09-20T21:46:47.502157+0000","last_clean":"2023-09-20T21:46:47.502157+0000","last_became_active":"2023-09-20T21:46:41.509183+0000","last_became_peered":"2023-09-20T21:46:41.509183+0000","last_unstale":"2023-09-20T21:46:47.502157+0000","last_undegraded":"2023-09-20T21:46:47.502157+0000","last_fullsized":"2023-09-20T21:46:47.502157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:17:21.084830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794668+0000","last_change":"2023-09-20T21:46:45.583709+0000","last_active":"2023-09-20T21:46:46.794668+0000","last_peered":"2023-09-20T21:46:46.794668+0000","last_clean":"2023-09-20T21:46:46.794668+0000","last_became_active":"2023-09-20T21:46:45.583302+0000","last_became_peered":"2023-09-20T21:46:45.583302+0000","last_unstale":"2023-09-20T21:46:46.794668+0000","last_undegraded":"2023-09-20T21:46:46.794668+0000","last_fullsized":"2023-09-20T21:46:46.794668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:31:35.472750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.590965+0000","last_change":"2023-09-20T21:46:40.506411+0000","last_active":"2023-09-20T21:47:05.590965+0000","last_peered":"2023-09-20T21:47:05.590965+0000","last_clean":"2023-09-20T21:47:05.590965+0000","last_became_active":"2023-09-20T21:46:40.506116+0000","last_became_peered":"2023-09-20T21:46:40.506116+0000","last_unstale":"2023-09-20T21:47:05.590965+0000","last_undegraded":"2023-09-20T21:47:05.590965+0000","last_fullsized":"2023-09-20T21:47:05.590965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:23:19.175007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575844+0000","last_change":"2023-09-20T21:46:41.513793+0000","last_active":"2023-09-20T21:46:45.575844+0000","last_peered":"2023-09-20T21:46:45.575844+0000","last_clean":"2023-09-20T21:46:45.575844+0000","last_became_active":"2023-09-20T21:46:41.513434+0000","last_became_peered":"2023-09-20T21:46:41.513434+0000","last_unstale":"2023-09-20T21:46:45.575844+0000","last_undegraded":"2023-09-20T21:46:45.575844+0000","last_fullsized":"2023-09-20T21:46:45.575844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:00:57.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598091+0000","last_change":"2023-09-20T21:46:45.578188+0000","last_active":"2023-09-20T21:47:07.598091+0000","last_peered":"2023-09-20T21:47:07.598091+0000","last_clean":"2023-09-20T21:47:07.598091+0000","last_became_active":"2023-09-20T21:46:45.577548+0000","last_became_peered":"2023-09-20T21:46:45.577548+0000","last_unstale":"2023-09-20T21:47:07.598091+0000","last_undegraded":"2023-09-20T21:47:07.598091+0000","last_fullsized":"2023-09-20T21:47:07.598091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:54:58.894269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573689+0000","last_change":"2023-09-20T21:46:40.505465+0000","last_active":"2023-09-20T21:46:45.573689+0000","last_peered":"2023-09-20T21:46:45.573689+0000","last_clean":"2023-09-20T21:46:45.573689+0000","last_became_active":"2023-09-20T21:46:40.505174+0000","last_became_peered":"2023-09-20T21:46:40.505174+0000","last_unstale":"2023-09-20T21:46:45.573689+0000","last_undegraded":"2023-09-20T21:46:45.573689+0000","last_fullsized":"2023-09-20T21:46:45.573689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:50:53.191963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568918+0000","last_change":"2023-09-20T21:46:41.507370+0000","last_active":"2023-09-20T21:46:45.568918+0000","last_peered":"2023-09-20T21:46:45.568918+0000","last_clean":"2023-09-20T21:46:45.568918+0000","last_became_active":"2023-09-20T21:46:41.507035+0000","last_became_peered":"2023-09-20T21:46:41.507035+0000","last_unstale":"2023-09-20T21:46:45.568918+0000","last_undegraded":"2023-09-20T21:46:45.568918+0000","last_fullsized":"2023-09-20T21:46:45.568918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:50.298899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565547+0000","last_change":"2023-09-20T21:46:45.584458+0000","last_active":"2023-09-20T21:46:46.565547+0000","last_peered":"2023-09-20T21:46:46.565547+0000","last_clean":"2023-09-20T21:46:46.565547+0000","last_became_active":"2023-09-20T21:46:45.584033+0000","last_became_peered":"2023-09-20T21:46:45.584033+0000","last_unstale":"2023-09-20T21:46:46.565547+0000","last_undegraded":"2023-09-20T21:46:46.565547+0000","last_fullsized":"2023-09-20T21:46:46.565547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:48:44.653880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.567221+0000","last_change":"2023-09-20T21:46:40.501329+0000","last_active":"2023-09-20T21:47:08.567221+0000","last_peered":"2023-09-20T21:47:08.567221+0000","last_clean":"2023-09-20T21:47:08.567221+0000","last_became_active":"2023-09-20T21:46:40.500999+0000","last_became_peered":"2023-09-20T21:46:40.500999+0000","last_unstale":"2023-09-20T21:47:08.567221+0000","last_undegraded":"2023-09-20T21:47:08.567221+0000","last_fullsized":"2023-09-20T21:47:08.567221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:38.028398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568141+0000","last_change":"2023-09-20T21:46:41.504583+0000","last_active":"2023-09-20T21:46:45.568141+0000","last_peered":"2023-09-20T21:46:45.568141+0000","last_clean":"2023-09-20T21:46:45.568141+0000","last_became_active":"2023-09-20T21:46:41.504273+0000","last_became_peered":"2023-09-20T21:46:41.504273+0000","last_unstale":"2023-09-20T21:46:45.568141+0000","last_undegraded":"2023-09-20T21:46:45.568141+0000","last_fullsized":"2023-09-20T21:46:45.568141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:37.100382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502476+0000","last_change":"2023-09-20T21:46:45.587116+0000","last_active":"2023-09-20T21:46:47.502476+0000","last_peered":"2023-09-20T21:46:47.502476+0000","last_clean":"2023-09-20T21:46:47.502476+0000","last_became_active":"2023-09-20T21:46:45.586629+0000","last_became_peered":"2023-09-20T21:46:45.586629+0000","last_unstale":"2023-09-20T21:46:47.502476+0000","last_undegraded":"2023-09-20T21:46:47.502476+0000","last_fullsized":"2023-09-20T21:46:47.502476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:10:33.024235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'16","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.572218+0000","last_change":"2023-09-20T21:46:40.499985+0000","last_active":"2023-09-20T21:47:07.572218+0000","last_peered":"2023-09-20T21:47:07.572218+0000","last_clean":"2023-09-20T21:47:07.572218+0000","last_became_active":"2023-09-20T21:46:40.499646+0000","last_became_peered":"2023-09-20T21:46:40.499646+0000","last_unstale":"2023-09-20T21:47:07.572218+0000","last_undegraded":"2023-09-20T21:47:07.572218+0000","last_fullsized":"2023-09-20T21:47:07.572218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:52:02.758556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568813+0000","last_change":"2023-09-20T21:46:41.501241+0000","last_active":"2023-09-20T21:46:45.568813+0000","last_peered":"2023-09-20T21:46:45.568813+0000","last_clean":"2023-09-20T21:46:45.568813+0000","last_became_active":"2023-09-20T21:46:41.500902+0000","last_became_peered":"2023-09-20T21:46:41.500902+0000","last_unstale":"2023-09-20T21:46:45.568813+0000","last_undegraded":"2023-09-20T21:46:45.568813+0000","last_fullsized":"2023-09-20T21:46:45.568813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:21:55.210065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794066+0000","last_change":"2023-09-20T21:46:45.583405+0000","last_active":"2023-09-20T21:46:46.794066+0000","last_peered":"2023-09-20T21:46:46.794066+0000","last_clean":"2023-09-20T21:46:46.794066+0000","last_became_active":"2023-09-20T21:46:45.582945+0000","last_became_peered":"2023-09-20T21:46:45.582945+0000","last_unstale":"2023-09-20T21:46:46.794066+0000","last_undegraded":"2023-09-20T21:46:46.794066+0000","last_fullsized":"2023-09-20T21:46:46.794066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:27:14.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.561084+0000","last_change":"2023-09-20T21:46:40.507744+0000","last_active":"2023-09-20T21:47:07.561084+0000","last_peered":"2023-09-20T21:47:07.561084+0000","last_clean":"2023-09-20T21:47:07.561084+0000","last_became_active":"2023-09-20T21:46:40.507454+0000","last_became_peered":"2023-09-20T21:46:40.507454+0000","last_unstale":"2023-09-20T21:47:07.561084+0000","last_undegraded":"2023-09-20T21:47:07.561084+0000","last_fullsized":"2023-09-20T21:47:07.561084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:00:24.492789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598829+0000","last_change":"2023-09-20T21:46:41.504263+0000","last_active":"2023-09-20T21:47:07.598829+0000","last_peered":"2023-09-20T21:47:07.598829+0000","last_clean":"2023-09-20T21:47:07.598829+0000","last_became_active":"2023-09-20T21:46:41.503961+0000","last_became_peered":"2023-09-20T21:46:41.503961+0000","last_unstale":"2023-09-20T21:47:07.598829+0000","last_undegraded":"2023-09-20T21:47:07.598829+0000","last_fullsized":"2023-09-20T21:47:07.598829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:31:58.035517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.944063+0000","last_change":"2023-09-20T21:46:45.581343+0000","last_active":"2023-09-20T21:46:47.944063+0000","last_peered":"2023-09-20T21:46:47.944063+0000","last_clean":"2023-09-20T21:46:47.944063+0000","last_became_active":"2023-09-20T21:46:45.580830+0000","last_became_peered":"2023-09-20T21:46:45.580830+0000","last_unstale":"2023-09-20T21:46:47.944063+0000","last_undegraded":"2023-09-20T21:46:47.944063+0000","last_fullsized":"2023-09-20T21:46:47.944063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:20:49.865825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563359+0000","last_change":"2023-09-20T21:46:40.501850+0000","last_active":"2023-09-20T21:46:45.563359+0000","last_peered":"2023-09-20T21:46:45.563359+0000","last_clean":"2023-09-20T21:46:45.563359+0000","last_became_active":"2023-09-20T21:46:40.501500+0000","last_became_peered":"2023-09-20T21:46:40.501500+0000","last_unstale":"2023-09-20T21:46:45.563359+0000","last_undegraded":"2023-09-20T21:46:45.563359+0000","last_fullsized":"2023-09-20T21:46:45.563359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:14:15.907335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564629+0000","last_change":"2023-09-20T21:46:41.507432+0000","last_active":"2023-09-20T21:46:45.564629+0000","last_peered":"2023-09-20T21:46:45.564629+0000","last_clean":"2023-09-20T21:46:45.564629+0000","last_became_active":"2023-09-20T21:46:41.507119+0000","last_became_peered":"2023-09-20T21:46:41.507119+0000","last_unstale":"2023-09-20T21:46:45.564629+0000","last_undegraded":"2023-09-20T21:46:45.564629+0000","last_fullsized":"2023-09-20T21:46:45.564629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:14:10.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502815+0000","last_change":"2023-09-20T21:46:45.588455+0000","last_active":"2023-09-20T21:46:47.502815+0000","last_peered":"2023-09-20T21:46:47.502815+0000","last_clean":"2023-09-20T21:46:47.502815+0000","last_became_active":"2023-09-20T21:46:45.587980+0000","last_became_peered":"2023-09-20T21:46:45.587980+0000","last_unstale":"2023-09-20T21:46:47.502815+0000","last_undegraded":"2023-09-20T21:46:47.502815+0000","last_fullsized":"2023-09-20T21:46:47.502815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:07:35.697330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.568603+0000","last_change":"2023-09-20T21:46:40.506260+0000","last_active":"2023-09-20T21:46:46.568603+0000","last_peered":"2023-09-20T21:46:46.568603+0000","last_clean":"2023-09-20T21:46:46.568603+0000","last_became_active":"2023-09-20T21:46:40.505999+0000","last_became_peered":"2023-09-20T21:46:40.505999+0000","last_unstale":"2023-09-20T21:46:46.568603+0000","last_undegraded":"2023-09-20T21:46:46.568603+0000","last_fullsized":"2023-09-20T21:46:46.568603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:39:01.748038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564591+0000","last_change":"2023-09-20T21:46:41.496822+0000","last_active":"2023-09-20T21:46:45.564591+0000","last_peered":"2023-09-20T21:46:45.564591+0000","last_clean":"2023-09-20T21:46:45.564591+0000","last_became_active":"2023-09-20T21:46:41.496517+0000","last_became_peered":"2023-09-20T21:46:41.496517+0000","last_unstale":"2023-09-20T21:46:45.564591+0000","last_undegraded":"2023-09-20T21:46:45.564591+0000","last_fullsized":"2023-09-20T21:46:45.564591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:12.592040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502793+0000","last_change":"2023-09-20T21:46:45.587410+0000","last_active":"2023-09-20T21:46:47.502793+0000","last_peered":"2023-09-20T21:46:47.502793+0000","last_clean":"2023-09-20T21:46:47.502793+0000","last_became_active":"2023-09-20T21:46:45.586944+0000","last_became_peered":"2023-09-20T21:46:45.586944+0000","last_unstale":"2023-09-20T21:46:47.502793+0000","last_undegraded":"2023-09-20T21:46:47.502793+0000","last_fullsized":"2023-09-20T21:46:47.502793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:57.053765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568334+0000","last_change":"2023-09-20T21:46:41.511526+0000","last_active":"2023-09-20T21:46:45.568334+0000","last_peered":"2023-09-20T21:46:45.568334+0000","last_clean":"2023-09-20T21:46:45.568334+0000","last_became_active":"2023-09-20T21:46:41.511213+0000","last_became_peered":"2023-09-20T21:46:41.511213+0000","last_unstale":"2023-09-20T21:46:45.568334+0000","last_undegraded":"2023-09-20T21:46:45.568334+0000","last_fullsized":"2023-09-20T21:46:45.568334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:29:04.199190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569931+0000","last_change":"2023-09-20T21:46:40.514916+0000","last_active":"2023-09-20T21:46:45.569931+0000","last_peered":"2023-09-20T21:46:45.569931+0000","last_clean":"2023-09-20T21:46:45.569931+0000","last_became_active":"2023-09-20T21:46:40.514553+0000","last_became_peered":"2023-09-20T21:46:40.514553+0000","last_unstale":"2023-09-20T21:46:45.569931+0000","last_undegraded":"2023-09-20T21:46:45.569931+0000","last_fullsized":"2023-09-20T21:46:45.569931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:12:32.481852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.568779+0000","last_change":"2023-09-20T21:46:45.587489+0000","last_active":"2023-09-20T21:46:46.568779+0000","last_peered":"2023-09-20T21:46:46.568779+0000","last_clean":"2023-09-20T21:46:46.568779+0000","last_became_active":"2023-09-20T21:46:45.587022+0000","last_became_peered":"2023-09-20T21:46:45.587022+0000","last_unstale":"2023-09-20T21:46:46.568779+0000","last_undegraded":"2023-09-20T21:46:46.568779+0000","last_fullsized":"2023-09-20T21:46:46.568779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:09:05.870763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568768+0000","last_change":"2023-09-20T21:46:41.508886+0000","last_active":"2023-09-20T21:46:45.568768+0000","last_peered":"2023-09-20T21:46:45.568768+0000","last_clean":"2023-09-20T21:46:45.568768+0000","last_became_active":"2023-09-20T21:46:41.508470+0000","last_became_peered":"2023-09-20T21:46:41.508470+0000","last_unstale":"2023-09-20T21:46:45.568768+0000","last_undegraded":"2023-09-20T21:46:45.568768+0000","last_fullsized":"2023-09-20T21:46:45.568768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:45:04.888300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.576942+0000","last_change":"2023-09-20T21:46:40.503350+0000","last_active":"2023-09-20T21:47:08.576942+0000","last_peered":"2023-09-20T21:47:08.576942+0000","last_clean":"2023-09-20T21:47:08.576942+0000","last_became_active":"2023-09-20T21:46:40.503045+0000","last_became_peered":"2023-09-20T21:46:40.503045+0000","last_unstale":"2023-09-20T21:47:08.576942+0000","last_undegraded":"2023-09-20T21:47:08.576942+0000","last_fullsized":"2023-09-20T21:47:08.576942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:31:09.004342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.567932+0000","last_change":"2023-09-20T21:45:35.180242+0000","last_active":"2023-09-20T21:46:45.567932+0000","last_peered":"2023-09-20T21:46:45.567932+0000","last_clean":"2023-09-20T21:46:45.567932+0000","last_became_active":"2023-09-20T21:45:35.179705+0000","last_became_peered":"2023-09-20T21:45:35.179705+0000","last_unstale":"2023-09-20T21:46:45.567932+0000","last_undegraded":"2023-09-20T21:46:45.567932+0000","last_fullsized":"2023-09-20T21:46:45.567932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_clean_scrub_stamp":"2023-09-20T21:42:25.826683+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:02:03.448188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794591+0000","last_change":"2023-09-20T21:46:45.581396+0000","last_active":"2023-09-20T21:46:46.794591+0000","last_peered":"2023-09-20T21:46:46.794591+0000","last_clean":"2023-09-20T21:46:46.794591+0000","last_became_active":"2023-09-20T21:46:45.581018+0000","last_became_peered":"2023-09-20T21:46:45.581018+0000","last_unstale":"2023-09-20T21:46:46.794591+0000","last_undegraded":"2023-09-20T21:46:46.794591+0000","last_fullsized":"2023-09-20T21:46:46.794591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:16:35.934460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569837+0000","last_change":"2023-09-20T21:46:41.514392+0000","last_active":"2023-09-20T21:46:45.569837+0000","last_peered":"2023-09-20T21:46:45.569837+0000","last_clean":"2023-09-20T21:46:45.569837+0000","last_became_active":"2023-09-20T21:46:41.514090+0000","last_became_peered":"2023-09-20T21:46:41.514090+0000","last_unstale":"2023-09-20T21:46:45.569837+0000","last_undegraded":"2023-09-20T21:46:45.569837+0000","last_fullsized":"2023-09-20T21:46:45.569837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:05:47.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:06.596564+0000","last_change":"2023-09-20T21:46:40.507346+0000","last_active":"2023-09-20T21:47:06.596564+0000","last_peered":"2023-09-20T21:47:06.596564+0000","last_clean":"2023-09-20T21:47:06.596564+0000","last_became_active":"2023-09-20T21:46:40.506969+0000","last_became_peered":"2023-09-20T21:46:40.506969+0000","last_unstale":"2023-09-20T21:47:06.596564+0000","last_undegraded":"2023-09-20T21:47:06.596564+0000","last_fullsized":"2023-09-20T21:47:06.596564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:01:20.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600062+0000","last_change":"2023-09-20T21:46:45.578033+0000","last_active":"2023-09-20T21:47:07.600062+0000","last_peered":"2023-09-20T21:47:07.600062+0000","last_clean":"2023-09-20T21:47:07.600062+0000","last_became_active":"2023-09-20T21:46:45.577540+0000","last_became_peered":"2023-09-20T21:46:45.577540+0000","last_unstale":"2023-09-20T21:47:07.600062+0000","last_undegraded":"2023-09-20T21:47:07.600062+0000","last_fullsized":"2023-09-20T21:47:07.600062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:24:31.020660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.573672+0000","last_change":"2023-09-20T21:46:40.510634+0000","last_active":"2023-09-20T21:47:07.573672+0000","last_peered":"2023-09-20T21:47:07.573672+0000","last_clean":"2023-09-20T21:47:07.573672+0000","last_became_active":"2023-09-20T21:46:40.510390+0000","last_became_peered":"2023-09-20T21:46:40.510390+0000","last_unstale":"2023-09-20T21:47:07.573672+0000","last_undegraded":"2023-09-20T21:47:07.573672+0000","last_fullsized":"2023-09-20T21:47:07.573672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:44:49.272987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571610+0000","last_change":"2023-09-20T21:46:41.502981+0000","last_active":"2023-09-20T21:46:45.571610+0000","last_peered":"2023-09-20T21:46:45.571610+0000","last_clean":"2023-09-20T21:46:45.571610+0000","last_became_active":"2023-09-20T21:46:41.502594+0000","last_became_peered":"2023-09-20T21:46:41.502594+0000","last_unstale":"2023-09-20T21:46:45.571610+0000","last_undegraded":"2023-09-20T21:46:45.571610+0000","last_fullsized":"2023-09-20T21:46:45.571610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:41:15.261478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.862185+0000","last_change":"2023-09-20T21:46:45.582903+0000","last_active":"2023-09-20T21:46:46.862185+0000","last_peered":"2023-09-20T21:46:46.862185+0000","last_clean":"2023-09-20T21:46:46.862185+0000","last_became_active":"2023-09-20T21:46:45.582433+0000","last_became_peered":"2023-09-20T21:46:45.582433+0000","last_unstale":"2023-09-20T21:46:46.862185+0000","last_undegraded":"2023-09-20T21:46:46.862185+0000","last_fullsized":"2023-09-20T21:46:46.862185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:38:22.174903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563444+0000","last_change":"2023-09-20T21:46:40.507671+0000","last_active":"2023-09-20T21:46:45.563444+0000","last_peered":"2023-09-20T21:46:45.563444+0000","last_clean":"2023-09-20T21:46:45.563444+0000","last_became_active":"2023-09-20T21:46:40.507288+0000","last_became_peered":"2023-09-20T21:46:40.507288+0000","last_unstale":"2023-09-20T21:46:45.563444+0000","last_undegraded":"2023-09-20T21:46:45.563444+0000","last_fullsized":"2023-09-20T21:46:45.563444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:27:02.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570001+0000","last_change":"2023-09-20T21:46:41.507654+0000","last_active":"2023-09-20T21:46:45.570001+0000","last_peered":"2023-09-20T21:46:45.570001+0000","last_clean":"2023-09-20T21:46:45.570001+0000","last_became_active":"2023-09-20T21:46:41.507362+0000","last_became_peered":"2023-09-20T21:46:41.507362+0000","last_unstale":"2023-09-20T21:46:45.570001+0000","last_undegraded":"2023-09-20T21:46:45.570001+0000","last_fullsized":"2023-09-20T21:46:45.570001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:32:08.973379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565429+0000","last_change":"2023-09-20T21:46:45.578437+0000","last_active":"2023-09-20T21:46:46.565429+0000","last_peered":"2023-09-20T21:46:46.565429+0000","last_clean":"2023-09-20T21:46:46.565429+0000","last_became_active":"2023-09-20T21:46:45.578010+0000","last_became_peered":"2023-09-20T21:46:45.578010+0000","last_unstale":"2023-09-20T21:46:46.565429+0000","last_undegraded":"2023-09-20T21:46:46.565429+0000","last_fullsized":"2023-09-20T21:46:46.565429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:59:05.494861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577934+0000","last_change":"2023-09-20T21:46:40.508937+0000","last_active":"2023-09-20T21:47:08.577934+0000","last_peered":"2023-09-20T21:47:08.577934+0000","last_clean":"2023-09-20T21:47:08.577934+0000","last_became_active":"2023-09-20T21:46:40.508639+0000","last_became_peered":"2023-09-20T21:46:40.508639+0000","last_unstale":"2023-09-20T21:47:08.577934+0000","last_undegraded":"2023-09-20T21:47:08.577934+0000","last_fullsized":"2023-09-20T21:47:08.577934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:50:48.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571508+0000","last_change":"2023-09-20T21:46:41.510702+0000","last_active":"2023-09-20T21:46:45.571508+0000","last_peered":"2023-09-20T21:46:45.571508+0000","last_clean":"2023-09-20T21:46:45.571508+0000","last_became_active":"2023-09-20T21:46:41.510357+0000","last_became_peered":"2023-09-20T21:46:41.510357+0000","last_unstale":"2023-09-20T21:46:45.571508+0000","last_undegraded":"2023-09-20T21:46:45.571508+0000","last_fullsized":"2023-09-20T21:46:45.571508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:35:44.455981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596780+0000","last_change":"2023-09-20T21:46:45.581110+0000","last_active":"2023-09-20T21:47:07.596780+0000","last_peered":"2023-09-20T21:47:07.596780+0000","last_clean":"2023-09-20T21:47:07.596780+0000","last_became_active":"2023-09-20T21:46:45.580498+0000","last_became_peered":"2023-09-20T21:46:45.580498+0000","last_unstale":"2023-09-20T21:47:07.596780+0000","last_undegraded":"2023-09-20T21:47:07.596780+0000","last_fullsized":"2023-09-20T21:47:07.596780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:19:22.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571650+0000","last_change":"2023-09-20T21:46:40.497017+0000","last_active":"2023-09-20T21:46:45.571650+0000","last_peered":"2023-09-20T21:46:45.571650+0000","last_clean":"2023-09-20T21:46:45.571650+0000","last_became_active":"2023-09-20T21:46:40.496722+0000","last_became_peered":"2023-09-20T21:46:40.496722+0000","last_unstale":"2023-09-20T21:46:45.571650+0000","last_undegraded":"2023-09-20T21:46:45.571650+0000","last_fullsized":"2023-09-20T21:46:45.571650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:12:07.017318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501752+0000","last_change":"2023-09-20T21:46:41.508793+0000","last_active":"2023-09-20T21:46:47.501752+0000","last_peered":"2023-09-20T21:46:47.501752+0000","last_clean":"2023-09-20T21:46:47.501752+0000","last_became_active":"2023-09-20T21:46:41.508453+0000","last_became_peered":"2023-09-20T21:46:41.508453+0000","last_unstale":"2023-09-20T21:46:47.501752+0000","last_undegraded":"2023-09-20T21:46:47.501752+0000","last_fullsized":"2023-09-20T21:46:47.501752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:35:12.711570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565653+0000","last_change":"2023-09-20T21:46:45.583519+0000","last_active":"2023-09-20T21:46:46.565653+0000","last_peered":"2023-09-20T21:46:46.565653+0000","last_clean":"2023-09-20T21:46:46.565653+0000","last_became_active":"2023-09-20T21:46:45.582933+0000","last_became_peered":"2023-09-20T21:46:45.582933+0000","last_unstale":"2023-09-20T21:46:46.565653+0000","last_undegraded":"2023-09-20T21:46:46.565653+0000","last_fullsized":"2023-09-20T21:46:46.565653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:13:20.237865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.591317+0000","last_change":"2023-09-20T21:46:40.507384+0000","last_active":"2023-09-20T21:47:05.591317+0000","last_peered":"2023-09-20T21:47:05.591317+0000","last_clean":"2023-09-20T21:47:05.591317+0000","last_became_active":"2023-09-20T21:46:40.507021+0000","last_became_peered":"2023-09-20T21:46:40.507021+0000","last_unstale":"2023-09-20T21:47:05.591317+0000","last_undegraded":"2023-09-20T21:47:05.591317+0000","last_fullsized":"2023-09-20T21:47:05.591317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:48:04.360664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598093+0000","last_change":"2023-09-20T21:46:41.511537+0000","last_active":"2023-09-20T21:47:07.598093+0000","last_peered":"2023-09-20T21:47:07.598093+0000","last_clean":"2023-09-20T21:47:07.598093+0000","last_became_active":"2023-09-20T21:46:41.511238+0000","last_became_peered":"2023-09-20T21:46:41.511238+0000","last_unstale":"2023-09-20T21:47:07.598093+0000","last_undegraded":"2023-09-20T21:47:07.598093+0000","last_fullsized":"2023-09-20T21:47:07.598093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:01:41.492824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596769+0000","last_change":"2023-09-20T21:46:45.585750+0000","last_active":"2023-09-20T21:47:07.596769+0000","last_peered":"2023-09-20T21:47:07.596769+0000","last_clean":"2023-09-20T21:47:07.596769+0000","last_became_active":"2023-09-20T21:46:45.585213+0000","last_became_peered":"2023-09-20T21:46:45.585213+0000","last_unstale":"2023-09-20T21:47:07.596769+0000","last_undegraded":"2023-09-20T21:47:07.596769+0000","last_fullsized":"2023-09-20T21:47:07.596769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:31:15.650766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568158+0000","last_change":"2023-09-20T21:46:40.506394+0000","last_active":"2023-09-20T21:46:45.568158+0000","last_peered":"2023-09-20T21:46:45.568158+0000","last_clean":"2023-09-20T21:46:45.568158+0000","last_became_active":"2023-09-20T21:46:40.506085+0000","last_became_peered":"2023-09-20T21:46:40.506085+0000","last_unstale":"2023-09-20T21:46:45.568158+0000","last_undegraded":"2023-09-20T21:46:45.568158+0000","last_fullsized":"2023-09-20T21:46:45.568158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:02.738600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570438+0000","last_change":"2023-09-20T21:46:41.509491+0000","last_active":"2023-09-20T21:46:45.570438+0000","last_peered":"2023-09-20T21:46:45.570438+0000","last_clean":"2023-09-20T21:46:45.570438+0000","last_became_active":"2023-09-20T21:46:41.509039+0000","last_became_peered":"2023-09-20T21:46:41.509039+0000","last_unstale":"2023-09-20T21:46:45.570438+0000","last_undegraded":"2023-09-20T21:46:45.570438+0000","last_fullsized":"2023-09-20T21:46:45.570438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:57:39.003154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597192+0000","last_change":"2023-09-20T21:46:45.581651+0000","last_active":"2023-09-20T21:47:07.597192+0000","last_peered":"2023-09-20T21:47:07.597192+0000","last_clean":"2023-09-20T21:47:07.597192+0000","last_became_active":"2023-09-20T21:46:45.581224+0000","last_became_peered":"2023-09-20T21:46:45.581224+0000","last_unstale":"2023-09-20T21:47:07.597192+0000","last_undegraded":"2023-09-20T21:47:07.597192+0000","last_fullsized":"2023-09-20T21:47:07.597192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:32:17.778301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.610835+0000","last_change":"2023-09-20T21:47:08.610413+0000","last_active":"2023-09-20T21:47:08.610835+0000","last_peered":"2023-09-20T21:47:08.610835+0000","last_clean":"2023-09-20T21:47:08.610835+0000","last_became_active":"2023-09-20T21:47:08.603734+0000","last_became_peered":"2023-09-20T21:47:08.603734+0000","last_unstale":"2023-09-20T21:47:08.610835+0000","last_undegraded":"2023-09-20T21:47:08.610835+0000","last_fullsized":"2023-09-20T21:47:08.610835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:43:42.569371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568903+0000","last_change":"2023-09-20T21:46:41.507218+0000","last_active":"2023-09-20T21:46:45.568903+0000","last_peered":"2023-09-20T21:46:45.568903+0000","last_clean":"2023-09-20T21:46:45.568903+0000","last_became_active":"2023-09-20T21:46:41.506889+0000","last_became_peered":"2023-09-20T21:46:41.506889+0000","last_unstale":"2023-09-20T21:46:45.568903+0000","last_undegraded":"2023-09-20T21:46:45.568903+0000","last_fullsized":"2023-09-20T21:46:45.568903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:08:40.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.503610+0000","last_change":"2023-09-20T21:46:45.579269+0000","last_active":"2023-09-20T21:46:47.503610+0000","last_peered":"2023-09-20T21:46:47.503610+0000","last_clean":"2023-09-20T21:46:47.503610+0000","last_became_active":"2023-09-20T21:46:45.578541+0000","last_became_peered":"2023-09-20T21:46:45.578541+0000","last_unstale":"2023-09-20T21:46:47.503610+0000","last_undegraded":"2023-09-20T21:46:47.503610+0000","last_fullsized":"2023-09-20T21:46:47.503610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:07:50.241947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569525+0000","last_change":"2023-09-20T21:46:40.508774+0000","last_active":"2023-09-20T21:46:45.569525+0000","last_peered":"2023-09-20T21:46:45.569525+0000","last_clean":"2023-09-20T21:46:45.569525+0000","last_became_active":"2023-09-20T21:46:40.508480+0000","last_became_peered":"2023-09-20T21:46:40.508480+0000","last_unstale":"2023-09-20T21:46:45.569525+0000","last_undegraded":"2023-09-20T21:46:45.569525+0000","last_fullsized":"2023-09-20T21:46:45.569525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:22.643724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570698+0000","last_change":"2023-09-20T21:46:41.503763+0000","last_active":"2023-09-20T21:46:45.570698+0000","last_peered":"2023-09-20T21:46:45.570698+0000","last_clean":"2023-09-20T21:46:45.570698+0000","last_became_active":"2023-09-20T21:46:41.503360+0000","last_became_peered":"2023-09-20T21:46:41.503360+0000","last_unstale":"2023-09-20T21:46:45.570698+0000","last_undegraded":"2023-09-20T21:46:45.570698+0000","last_fullsized":"2023-09-20T21:46:45.570698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:03:00.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502032+0000","last_change":"2023-09-20T21:46:45.586702+0000","last_active":"2023-09-20T21:46:47.502032+0000","last_peered":"2023-09-20T21:46:47.502032+0000","last_clean":"2023-09-20T21:46:47.502032+0000","last_became_active":"2023-09-20T21:46:45.586177+0000","last_became_peered":"2023-09-20T21:46:45.586177+0000","last_unstale":"2023-09-20T21:46:47.502032+0000","last_undegraded":"2023-09-20T21:46:47.502032+0000","last_fullsized":"2023-09-20T21:46:47.502032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:19:03.883440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568892+0000","last_change":"2023-09-20T21:46:40.507270+0000","last_active":"2023-09-20T21:46:45.568892+0000","last_peered":"2023-09-20T21:46:45.568892+0000","last_clean":"2023-09-20T21:46:45.568892+0000","last_became_active":"2023-09-20T21:46:40.506933+0000","last_became_peered":"2023-09-20T21:46:40.506933+0000","last_unstale":"2023-09-20T21:46:45.568892+0000","last_undegraded":"2023-09-20T21:46:45.568892+0000","last_fullsized":"2023-09-20T21:46:45.568892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:59.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569676+0000","last_change":"2023-09-20T21:46:41.507868+0000","last_active":"2023-09-20T21:46:45.569676+0000","last_peered":"2023-09-20T21:46:45.569676+0000","last_clean":"2023-09-20T21:46:45.569676+0000","last_became_active":"2023-09-20T21:46:41.507568+0000","last_became_peered":"2023-09-20T21:46:41.507568+0000","last_unstale":"2023-09-20T21:46:45.569676+0000","last_undegraded":"2023-09-20T21:46:45.569676+0000","last_fullsized":"2023-09-20T21:46:45.569676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:44.615620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.793795+0000","last_change":"2023-09-20T21:46:45.578726+0000","last_active":"2023-09-20T21:46:46.793795+0000","last_peered":"2023-09-20T21:46:46.793795+0000","last_clean":"2023-09-20T21:46:46.793795+0000","last_became_active":"2023-09-20T21:46:45.578326+0000","last_became_peered":"2023-09-20T21:46:45.578326+0000","last_unstale":"2023-09-20T21:46:46.793795+0000","last_undegraded":"2023-09-20T21:46:46.793795+0000","last_fullsized":"2023-09-20T21:46:46.793795+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:21:34.564840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570141+0000","last_change":"2023-09-20T21:46:40.509024+0000","last_active":"2023-09-20T21:46:45.570141+0000","last_peered":"2023-09-20T21:46:45.570141+0000","last_clean":"2023-09-20T21:46:45.570141+0000","last_became_active":"2023-09-20T21:46:40.508659+0000","last_became_peered":"2023-09-20T21:46:40.508659+0000","last_unstale":"2023-09-20T21:46:45.570141+0000","last_undegraded":"2023-09-20T21:46:45.570141+0000","last_fullsized":"2023-09-20T21:46:45.570141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:12:08.302068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569100+0000","last_change":"2023-09-20T21:46:41.510307+0000","last_active":"2023-09-20T21:46:45.569100+0000","last_peered":"2023-09-20T21:46:45.569100+0000","last_clean":"2023-09-20T21:46:45.569100+0000","last_became_active":"2023-09-20T21:46:41.510014+0000","last_became_peered":"2023-09-20T21:46:41.510014+0000","last_unstale":"2023-09-20T21:46:45.569100+0000","last_undegraded":"2023-09-20T21:46:45.569100+0000","last_fullsized":"2023-09-20T21:46:45.569100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:05:55.062515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598156+0000","last_change":"2023-09-20T21:46:41.511525+0000","last_active":"2023-09-20T21:47:07.598156+0000","last_peered":"2023-09-20T21:47:07.598156+0000","last_clean":"2023-09-20T21:47:07.598156+0000","last_became_active":"2023-09-20T21:46:41.511203+0000","last_became_peered":"2023-09-20T21:46:41.511203+0000","last_unstale":"2023-09-20T21:47:07.598156+0000","last_undegraded":"2023-09-20T21:47:07.598156+0000","last_fullsized":"2023-09-20T21:47:07.598156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:42:15.757422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570189+0000","last_change":"2023-09-20T21:46:41.502936+0000","last_active":"2023-09-20T21:46:45.570189+0000","last_peered":"2023-09-20T21:46:45.570189+0000","last_clean":"2023-09-20T21:46:45.570189+0000","last_became_active":"2023-09-20T21:46:41.502617+0000","last_became_peered":"2023-09-20T21:46:41.502617+0000","last_unstale":"2023-09-20T21:46:45.570189+0000","last_undegraded":"2023-09-20T21:46:45.570189+0000","last_fullsized":"2023-09-20T21:46:45.570189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:45:21.197652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568057+0000","last_change":"2023-09-20T21:46:41.510766+0000","last_active":"2023-09-20T21:46:45.568057+0000","last_peered":"2023-09-20T21:46:45.568057+0000","last_clean":"2023-09-20T21:46:45.568057+0000","last_became_active":"2023-09-20T21:46:41.510433+0000","last_became_peered":"2023-09-20T21:46:41.510433+0000","last_unstale":"2023-09-20T21:46:45.568057+0000","last_undegraded":"2023-09-20T21:46:45.568057+0000","last_fullsized":"2023-09-20T21:46:45.568057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:56:12.085926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.572421+0000","last_change":"2023-09-20T21:46:41.508846+0000","last_active":"2023-09-20T21:46:45.572421+0000","last_peered":"2023-09-20T21:46:45.572421+0000","last_clean":"2023-09-20T21:46:45.572421+0000","last_became_active":"2023-09-20T21:46:41.508518+0000","last_became_peered":"2023-09-20T21:46:41.508518+0000","last_unstale":"2023-09-20T21:46:45.572421+0000","last_undegraded":"2023-09-20T21:46:45.572421+0000","last_fullsized":"2023-09-20T21:46:45.572421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:01:58.195327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571696+0000","last_change":"2023-09-20T21:46:41.505929+0000","last_active":"2023-09-20T21:46:45.571696+0000","last_peered":"2023-09-20T21:46:45.571696+0000","last_clean":"2023-09-20T21:46:45.571696+0000","last_became_active":"2023-09-20T21:46:41.505542+0000","last_became_peered":"2023-09-20T21:46:41.505542+0000","last_unstale":"2023-09-20T21:46:45.571696+0000","last_undegraded":"2023-09-20T21:46:45.571696+0000","last_fullsized":"2023-09-20T21:46:45.571696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:24:47.520344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598727+0000","last_change":"2023-09-20T21:46:41.507603+0000","last_active":"2023-09-20T21:47:07.598727+0000","last_peered":"2023-09-20T21:47:07.598727+0000","last_clean":"2023-09-20T21:47:07.598727+0000","last_became_active":"2023-09-20T21:46:41.507340+0000","last_became_peered":"2023-09-20T21:46:41.507340+0000","last_unstale":"2023-09-20T21:47:07.598727+0000","last_undegraded":"2023-09-20T21:47:07.598727+0000","last_fullsized":"2023-09-20T21:47:07.598727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:50:19.756491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568500+0000","last_change":"2023-09-20T21:46:41.509625+0000","last_active":"2023-09-20T21:46:45.568500+0000","last_peered":"2023-09-20T21:46:45.568500+0000","last_clean":"2023-09-20T21:46:45.568500+0000","last_became_active":"2023-09-20T21:46:41.509301+0000","last_became_peered":"2023-09-20T21:46:41.509301+0000","last_unstale":"2023-09-20T21:46:45.568500+0000","last_undegraded":"2023-09-20T21:46:45.568500+0000","last_fullsized":"2023-09-20T21:46:45.568500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:15:03.108060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570117+0000","last_change":"2023-09-20T21:46:41.516781+0000","last_active":"2023-09-20T21:46:45.570117+0000","last_peered":"2023-09-20T21:46:45.570117+0000","last_clean":"2023-09-20T21:46:45.570117+0000","last_became_active":"2023-09-20T21:46:41.516496+0000","last_became_peered":"2023-09-20T21:46:41.516496+0000","last_unstale":"2023-09-20T21:46:45.570117+0000","last_undegraded":"2023-09-20T21:46:45.570117+0000","last_fullsized":"2023-09-20T21:46:45.570117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:17:26.492851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568221+0000","last_change":"2023-09-20T21:46:41.499927+0000","last_active":"2023-09-20T21:46:45.568221+0000","last_peered":"2023-09-20T21:46:45.568221+0000","last_clean":"2023-09-20T21:46:45.568221+0000","last_became_active":"2023-09-20T21:46:41.499605+0000","last_became_peered":"2023-09-20T21:46:41.499605+0000","last_unstale":"2023-09-20T21:46:45.568221+0000","last_undegraded":"2023-09-20T21:46:45.568221+0000","last_fullsized":"2023-09-20T21:46:45.568221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:18:16.306349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570531+0000","last_change":"2023-09-20T21:46:41.515842+0000","last_active":"2023-09-20T21:46:45.570531+0000","last_peered":"2023-09-20T21:46:45.570531+0000","last_clean":"2023-09-20T21:46:45.570531+0000","last_became_active":"2023-09-20T21:46:41.515600+0000","last_became_peered":"2023-09-20T21:46:41.515600+0000","last_unstale":"2023-09-20T21:46:45.570531+0000","last_undegraded":"2023-09-20T21:46:45.570531+0000","last_fullsized":"2023-09-20T21:46:45.570531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:50:58.088103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569630+0000","last_change":"2023-09-20T21:46:41.510695+0000","last_active":"2023-09-20T21:46:45.569630+0000","last_peered":"2023-09-20T21:46:45.569630+0000","last_clean":"2023-09-20T21:46:45.569630+0000","last_became_active":"2023-09-20T21:46:41.510345+0000","last_became_peered":"2023-09-20T21:46:41.510345+0000","last_unstale":"2023-09-20T21:46:45.569630+0000","last_undegraded":"2023-09-20T21:46:45.569630+0000","last_fullsized":"2023-09-20T21:46:45.569630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:38:24.201382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575033+0000","last_change":"2023-09-20T21:46:41.507809+0000","last_active":"2023-09-20T21:46:45.575033+0000","last_peered":"2023-09-20T21:46:45.575033+0000","last_clean":"2023-09-20T21:46:45.575033+0000","last_became_active":"2023-09-20T21:46:41.507513+0000","last_became_peered":"2023-09-20T21:46:41.507513+0000","last_unstale":"2023-09-20T21:46:45.575033+0000","last_undegraded":"2023-09-20T21:46:45.575033+0000","last_fullsized":"2023-09-20T21:46:45.575033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:30:43.132728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598414+0000","last_change":"2023-09-20T21:46:41.503590+0000","last_active":"2023-09-20T21:47:07.598414+0000","last_peered":"2023-09-20T21:47:07.598414+0000","last_clean":"2023-09-20T21:47:07.598414+0000","last_became_active":"2023-09-20T21:46:41.503295+0000","last_became_peered":"2023-09-20T21:46:41.503295+0000","last_unstale":"2023-09-20T21:47:07.598414+0000","last_undegraded":"2023-09-20T21:47:07.598414+0000","last_fullsized":"2023-09-20T21:47:07.598414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:28:33.088873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5632,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":110,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":19996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":487223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.64600000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.85699999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.753}]}]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1071729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46100000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54700000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.59599999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.63200000000000001}]},{"osd":11,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71499999999999997}]}]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":477145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.68500000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.622},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.75}]},{"osd":4,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.79700000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.82899999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:46:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.7719999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.872}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":486769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77200000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.34100000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.371}]},{"osd":2,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65500000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64700000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":1.121}]},{"osd":3,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.64300000000000002,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.92600000000000005}]},{"osd":4,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79500000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.878,"15min":0.878},"last":0.84199999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.313,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78600000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.94599999999999995,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.71999999999999997}]},{"osd":7,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.876,"5min":1.018,"15min":1.018},"last":0.57499999999999996}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.96599999999999997}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.627},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1068723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41799999999999998},"min":{"1min":0.27500000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.438,"15min":0.433},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.623},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70799999999999996}]},{"osd":6,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.747},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.64000000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65300000000000002,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.53000000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.78400000000000003}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":1.018,"15min":1.018},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64500000000000002,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":1.2,"15min":1.2},"last":0.88700000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.57199999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.466,"15min":1.466},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":1.012,"15min":1.012},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.175,"5min":1.179,"15min":1.179},"last":0.79900000000000004}]},{"osd":4,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0640000000000001,"5min":1.093,"15min":1.093},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78700000000000003,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.022,"5min":1.407,"15min":1.407},"last":0.93400000000000005}]},{"osd":5,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0549999999999999,"5min":1.266,"15min":1.266},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.046,"5min":1.073,"15min":1.073},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.84799999999999998,"15min":0.83699999999999997},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.85899999999999999,"15min":0.84599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.95099999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:46:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.84499999999999997,"15min":0.85899999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0149999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.751},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.85599999999999998,"15min":0.86399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1240000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0509999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.085},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.91400000000000003}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":480405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.39400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86499999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.52000000000000002,"15min":0.5},"min":{"1min":0.40000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77700000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.41899999999999998,"15min":0.39300000000000002},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.83699999999999997}]},{"osd":6,"last update":"Wed Sep 20 21:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.55900000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.92200000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.372,"15min":0.372},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.81699999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.89200000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.85599999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":1},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":478041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.503,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92300000000000004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.44900000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66600000000000004,"15min":0.64200000000000002},"min":{"1min":0.52000000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.62},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.67400000000000004,"15min":0.627},"min":{"1min":0.64600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.752}]},{"osd":5,"last update":"Wed Sep 20 21:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86899999999999999,"5min":1.149,"15min":1.149},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.67200000000000004},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.94199999999999995,"5min":1.194,"15min":1.194},"last":0.92000000000000004}]},{"osd":6,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.69599999999999995,"15min":0.67000000000000004},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.026},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82399999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.012,"5min":1.214,"15min":1.214},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.90500000000000003,"15min":0.91600000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0940000000000001,"5min":1.232,"15min":1.232},"last":1.073}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.091},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.90000000000000002}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.111},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.96099999999999997}]},{"osd":11,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.129}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78900000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.39400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.59299999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85299999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.44,"15min":0.39900000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.55500000000000005,"15min":0.49399999999999999},"min":{"1min":0.38300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":7.1500000000000004,"5min":7.1500000000000004,"15min":7.1500000000000004},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.56999999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.65400000000000003,"15min":0.64100000000000001},"min":{"1min":0.53000000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73699999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.63}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.88}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.753},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":476667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.432,"15min":0.39000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81699999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":1,"15min":1},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.84899999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63300000000000001,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65300000000000002,"15min":0.626},"min":{"1min":0.53500000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68799999999999994,"15min":0.64500000000000002},"min":{"1min":0.60399999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed Sep 20 21:46:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":1.024,"15min":1.024},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75900000000000001,"15min":0.747},"min":{"1min":0.61099999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98399999999999999,"5min":1.167,"15min":1.167},"last":0.69599999999999995}]},{"osd":6,"last update":"Wed Sep 20 21:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.52500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.075,"5min":1.147,"15min":1.147},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.627}]},{"osd":7,"last update":"Wed Sep 20 21:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.55400000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.177,"5min":1.248,"15min":1.248},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0549999999999999,"5min":1.228,"15min":1.228},"last":0.874}]},{"osd":8,"last update":"Wed Sep 20 21:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.74299999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.85099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":486697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57699999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.38,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.40899999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.46400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68700000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.54500000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.48499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.51900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61799999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58599999999999997,"15min":0.56799999999999995},"min":{"1min":0.503,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:47:07 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67000000000000004,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.49399999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56699999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":1,"15min":1},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88200000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68500000000000005}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.69999999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.71599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1581056,"data_stored":1066832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85499999999999998,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.753,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.42699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81299999999999994,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.36699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.434},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40400000000000003,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.59899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79100000000000004,"5min":1.615,"15min":1.615},"last":0.496},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.008,"15min":1.008},"last":0.77300000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65900000000000003,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85999999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.75800000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.749,"5min":0.879,"15min":0.879},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59599999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.78900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-20T21:47:10.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2023-09-20T21:47:10.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "89'7", 'reported_seq': 24, 'reported_epoch': 89, 'state': 'peering', 'last_fresh': '2023-09-20T21:47:07.573536+0000', 'last_change': '2023-09-20T21:47:07.596775+0000', 'last_active': '2023-09-20T21:47:07.596775+0000', 'last_peered': '2023-09-20T21:47:07.573536+0000', 'last_clean': '2023-09-20T21:47:07.573536+0000', 'last_became_active': '2023-09-20T21:46:40.501028+0000', 'last_became_peered': '2023-09-20T21:46:40.501028+0000', 'last_unstale': '2023-09-20T21:47:07.573536+0000', 'last_undegraded': '2023-09-20T21:47:07.573536+0000', 'last_fullsized': '2023-09-20T21:47:07.573536+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 82, 'last_epoch_clean': 83, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-20T21:46:39.413748+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-20T21:46:39.413748+0000', 'last_clean_scrub_stamp': '2023-09-20T21:46:39.413748+0000', 'objects_scrubbed': 0, 'log_size': 7, 'log_dups_size': 0, 'ondisk_log_size': 7, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 4, 'num_object_clones': 0, 'num_object_copies': 12, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9, 'num_write_kb': 6, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 4, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [8, 4, 7], 'acting': [8, 4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2, 4, 7], 'up_primary': 8, 'acting_primary': 8, 'purged_snaps': []} 2023-09-20T21:47:10.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-20T21:47:10.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:09 smithi123 ceph-mon[134433]: osdmap e91: 12 total, 12 up, 12 in 2023-09-20T21:47:10.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:09 smithi123 ceph-mon[134433]: daemon mds.j is now active in filesystem cephfs as rank 4 2023-09-20T21:47:10.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:09 smithi123 ceph-mon[134433]: from='client.24805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:10.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:09 smithi123 ceph-mon[134433]: pgmap v292: 129 pgs: 1 peering, 128 active+clean; 582 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.1 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:47:10.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.268+0000 7fa16c5ff700 1 -- 172.21.15.22:0/845242624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164102970 msgr2=0x7fa164109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.268+0000 7fa16c5ff700 1 --2- 172.21.15.22:0/845242624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164102970 0x7fa164109220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009f90 tx=0x7fa15c00b330 comp rx=0 tx=0).stop 2023-09-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.269+0000 7fa16c5ff700 1 -- 172.21.15.22:0/845242624 shutdown_connections 2023-09-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.269+0000 7fa16c5ff700 1 --2- 172.21.15.22:0/845242624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164102970 0x7fa164109220 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.269+0000 7fa16c5ff700 1 --2- 172.21.15.22:0/845242624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa164100de0 0x7fa164101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.269+0000 7fa16c5ff700 1 --2- 172.21.15.22:0/845242624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa164101fd0 0x7fa164102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.269+0000 7fa16c5ff700 1 -- 172.21.15.22:0/845242624 >> 172.21.15.22:0/845242624 conn(0x7fa1640fc310 msgr2=0x7fa1640fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.269+0000 7fa16c5ff700 1 -- 172.21.15.22:0/845242624 shutdown_connections 2023-09-20T21:47:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.269+0000 7fa16c5ff700 1 -- 172.21.15.22:0/845242624 wait complete. 2023-09-20T21:47:10.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.270+0000 7fa16c5ff700 1 Processor -- start 2023-09-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.270+0000 7fa16c5ff700 1 -- start start 2023-09-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.270+0000 7fa16c5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa164100de0 0x7fa164196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.270+0000 7fa16c5ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164101fd0 0x7fa164197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.270+0000 7fa16c5ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa164102970 0x7fa16419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.270+0000 7fa16c5ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16419e740 con 0x7fa164102970 2023-09-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.270+0000 7fa16c5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16419e880 con 0x7fa164100de0 2023-09-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.270+0000 7fa16c5ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16419ea10 con 0x7fa164101fd0 2023-09-20T21:47:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.270+0000 7fa16ab9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa164102970 0x7fa16419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa16ab9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa164102970 0x7fa16419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50822/0 (socket says 172.21.15.22:50822) 2023-09-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa16ab9c700 1 -- 172.21.15.22:0/4093996108 learned_addr learned my addr 172.21.15.22:0/4093996108 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa16a39b700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa164100de0 0x7fa164196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa169b9a700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164101fd0 0x7fa164197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa16ab9c700 1 -- 172.21.15.22:0/4093996108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164101fd0 msgr2=0x7fa164197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa16ab9c700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164101fd0 0x7fa164197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa16ab9c700 1 -- 172.21.15.22:0/4093996108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa164100de0 msgr2=0x7fa164196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa16ab9c700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa164100de0 0x7fa164196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa16ab9c700 1 -- 172.21.15.22:0/4093996108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1600095f0 con 0x7fa164102970 2023-09-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa16ab9c700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa164102970 0x7fa16419e170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009f60 tx=0x7fa15c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.271+0000 7fa15b7fe700 1 -- 172.21.15.22:0/4093996108 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00e040 con 0x7fa164102970 2023-09-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.272+0000 7fa15b7fe700 1 -- 172.21.15.22:0/4093996108 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa15c008e80 con 0x7fa164102970 2023-09-20T21:47:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.272+0000 7fa16c5ff700 1 -- 172.21.15.22:0/4093996108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c009c40 con 0x7fa164102970 2023-09-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.272+0000 7fa15b7fe700 1 -- 172.21.15.22:0/4093996108 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0052a0 con 0x7fa164102970 2023-09-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.272+0000 7fa16c5ff700 1 -- 172.21.15.22:0/4093996108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16419ef30 con 0x7fa164102970 2023-09-20T21:47:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.273+0000 7fa16c5ff700 1 -- 172.21.15.22:0/4093996108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa164107c70 con 0x7fa164102970 2023-09-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.275+0000 7fa15b7fe700 1 -- 172.21.15.22:0/4093996108 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa15c01d470 con 0x7fa164102970 2023-09-20T21:47:10.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.276+0000 7fa15b7fe700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa1500a1350 0x7fa1500a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.276+0000 7fa15b7fe700 1 -- 172.21.15.22:0/4093996108 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa15c0ce620 con 0x7fa164102970 2023-09-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.277+0000 7fa16a39b700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa1500a1350 0x7fa1500a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.277+0000 7fa16a39b700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa1500a1350 0x7fa1500a3810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa164101e30 tx=0x7fa160009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:10.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.280+0000 7fa15b7fe700 1 -- 172.21.15.22:0/4093996108 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa15c097a20 con 0x7fa164102970 2023-09-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.577+0000 7fa16c5ff700 1 -- 172.21.15.22:0/4093996108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa16402d0f0 con 0x7fa164102970 2023-09-20T21:47:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.577+0000 7fa15b7fe700 1 -- 172.21.15.22:0/4093996108 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3493 (secure 0 0 0) 0x7fa15c021dc0 con 0x7fa164102970 2023-09-20T21:47:10.577 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:10.577 INFO:teuthology.orchestra.run.smithi022.stdout:{"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":366,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":91,"num_osds":12,"num_up_osds":12,"osd_up_since":1695246352,"num_in_osds":12,"osd_in_since":1695246335,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":128},{"state_name":"peering","count":1}],"num_pgs":129,"num_pools":4,"num_objects":53,"data_bytes":596000,"bytes_used":345767936,"bytes_avail":1151594659840,"bytes_total":1151940427776,"inactive_pgs_ratio":0.0077519379556179047,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"read_bytes_sec":1151,"write_bytes_sec":3199,"read_op_per_sec":1,"write_op_per_sec":7},"fsmap":{"epoch":28,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"k","status":"up:standby-replay","gid":24461},{"filesystem_id":1,"rank":3,"name":"d","status":"up:standby-replay","gid":24466},{"filesystem_id":1,"rank":4,"name":"j","status":"up:active","gid":24473},{"filesystem_id":1,"rank":3,"name":"f","status":"up:active","gid":24487},{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24497},{"filesystem_id":1,"rank":1,"name":"c","status":"up:standby-replay","gid":24499},{"filesystem_id":1,"rank":0,"name":"i","status":"up:standby-replay","gid":24520},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24530},{"filesystem_id":1,"rank":1,"name":"a","status":"up:active","gid":24566},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24578}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.22:8443/"}},"servicemap":{"epoch":19,"modified":"2023-09-20T21:47:07.102418+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"j":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-09-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.579+0000 7fa16c5ff700 1 -- 172.21.15.22:0/4093996108 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa1500a1350 msgr2=0x7fa1500a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.579+0000 7fa16c5ff700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa1500a1350 0x7fa1500a3810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa164101e30 tx=0x7fa160009310 comp rx=0 tx=0).stop 2023-09-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.579+0000 7fa16c5ff700 1 -- 172.21.15.22:0/4093996108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa164102970 msgr2=0x7fa16419e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.579+0000 7fa16c5ff700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa164102970 0x7fa16419e170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009f60 tx=0x7fa15c004c80 comp rx=0 tx=0).stop 2023-09-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.580+0000 7fa16c5ff700 1 -- 172.21.15.22:0/4093996108 shutdown_connections 2023-09-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.580+0000 7fa16c5ff700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa164102970 0x7fa16419e170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.580+0000 7fa16c5ff700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa1500a1350 0x7fa1500a3810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.580+0000 7fa16c5ff700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa164100de0 0x7fa164196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.580+0000 7fa16c5ff700 1 --2- 172.21.15.22:0/4093996108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164101fd0 0x7fa164197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.580+0000 7fa16c5ff700 1 -- 172.21.15.22:0/4093996108 >> 172.21.15.22:0/4093996108 conn(0x7fa1640fc310 msgr2=0x7fa1640fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.580+0000 7fa16c5ff700 1 -- 172.21.15.22:0/4093996108 shutdown_connections 2023-09-20T21:47:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.580+0000 7fa16c5ff700 1 -- 172.21.15.22:0/4093996108 wait complete. 2023-09-20T21:47:10.597 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-20T21:47:10.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-20T21:47:10.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.804+0000 7f2af5fff700 1 -- 172.21.15.22:0/2283455456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2af0074dc0 msgr2=0x7f2af0073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:10.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.804+0000 7f2af5fff700 1 --2- 172.21.15.22:0/2283455456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2af0074dc0 0x7f2af0073210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009990 tx=0x7f2ae0009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.804+0000 7f2af5fff700 1 -- 172.21.15.22:0/2283455456 shutdown_connections 2023-09-20T21:47:10.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.804+0000 7f2af5fff700 1 --2- 172.21.15.22:0/2283455456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af0108cf0 0x7f2af010b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.804+0000 7f2af5fff700 1 --2- 172.21.15.22:0/2283455456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2af0074dc0 0x7f2af0073210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.804+0000 7f2af5fff700 1 --2- 172.21.15.22:0/2283455456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af00737e0 0x7f2af0073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.804+0000 7f2af5fff700 1 -- 172.21.15.22:0/2283455456 >> 172.21.15.22:0/2283455456 conn(0x7f2af00fc390 msgr2=0x7f2af00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.804+0000 7f2af5fff700 1 -- 172.21.15.22:0/2283455456 shutdown_connections 2023-09-20T21:47:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.804+0000 7f2af5fff700 1 -- 172.21.15.22:0/2283455456 wait complete. 2023-09-20T21:47:10.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.805+0000 7f2af5fff700 1 Processor -- start 2023-09-20T21:47:10.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.805+0000 7f2af5fff700 1 -- start start 2023-09-20T21:47:10.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.805+0000 7f2af5fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af00737e0 0x7f2af0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:10.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.805+0000 7f2af5fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2af0074dc0 0x7f2af01991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:10.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.805+0000 7f2af5fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af0108cf0 0x7f2af01a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:10.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.805+0000 7f2af5fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af01a07a0 con 0x7f2af0074dc0 2023-09-20T21:47:10.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.805+0000 7f2af5fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af01a0910 con 0x7f2af00737e0 2023-09-20T21:47:10.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.805+0000 7f2af5fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af01a0ad0 con 0x7f2af0108cf0 2023-09-20T21:47:10.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aef7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af00737e0 0x7f2af0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:10.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aeffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af0108cf0 0x7f2af01a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:10.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aef7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af00737e0 0x7f2af0198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43642/0 (socket says 172.21.15.22:43642) 2023-09-20T21:47:10.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aeeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2af0074dc0 0x7f2af01991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:10.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aef7fe700 1 -- 172.21.15.22:0/2180242634 learned_addr learned my addr 172.21.15.22:0/2180242634 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:10.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aef7fe700 1 -- 172.21.15.22:0/2180242634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af0108cf0 msgr2=0x7f2af01a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:10.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aef7fe700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af0108cf0 0x7f2af01a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aef7fe700 1 -- 172.21.15.22:0/2180242634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2af0074dc0 msgr2=0x7f2af01991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:10.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aef7fe700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2af0074dc0 0x7f2af01991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:10.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aef7fe700 1 -- 172.21.15.22:0/2180242634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae00095f0 con 0x7f2af00737e0 2023-09-20T21:47:10.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aeffff700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af0108cf0 0x7f2af01a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:10.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.806+0000 7f2aef7fe700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af00737e0 0x7f2af0198c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0004730 tx=0x7f2ae0004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:10.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.807+0000 7f2aecff9700 1 -- 172.21.15.22:0/2180242634 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae001d070 con 0x7f2af00737e0 2023-09-20T21:47:10.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.807+0000 7f2af5fff700 1 -- 172.21.15.22:0/2180242634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af01a0d50 con 0x7f2af00737e0 2023-09-20T21:47:10.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.807+0000 7f2af5fff700 1 -- 172.21.15.22:0/2180242634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af01a1170 con 0x7f2af00737e0 2023-09-20T21:47:10.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.807+0000 7f2aecff9700 1 -- 172.21.15.22:0/2180242634 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ae0004c90 con 0x7f2af00737e0 2023-09-20T21:47:10.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.807+0000 7f2aecff9700 1 -- 172.21.15.22:0/2180242634 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae00054d0 con 0x7f2af00737e0 2023-09-20T21:47:10.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.808+0000 7f2af5fff700 1 -- 172.21.15.22:0/2180242634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2adc005320 con 0x7f2af00737e0 2023-09-20T21:47:10.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.810+0000 7f2aecff9700 1 -- 172.21.15.22:0/2180242634 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2ae000fbc0 con 0x7f2af00737e0 2023-09-20T21:47:10.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.811+0000 7f2aecff9700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2ad80a13d0 0x7f2ad80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:10.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.811+0000 7f2aecff9700 1 -- 172.21.15.22:0/2180242634 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f2ae00ce0c0 con 0x7f2af00737e0 2023-09-20T21:47:10.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.812+0000 7f2aeeffd700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2ad80a13d0 0x7f2ad80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:10.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.812+0000 7f2aeeffd700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2ad80a13d0 0x7f2ad80a3890 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400ac50 tx=0x7f2ae400a380 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:10.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:10.813+0000 7f2aecff9700 1 -- 172.21.15.22:0/2180242634 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2ae00974c0 con 0x7f2af00737e0 2023-09-20T21:47:10.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:10 smithi089 ceph-mon[134090]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:10.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:10 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] up:active 2023-09-20T21:47:10.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:10 smithi089 ceph-mon[134090]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-09-20T21:47:10.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:10 smithi089 ceph-mon[134090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:47:10.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:10 smithi089 ceph-mon[134090]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:10.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:10 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4093996108' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-09-20T21:47:10.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:10 smithi022 ceph-mon[97651]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:10.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:10 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] up:active 2023-09-20T21:47:10.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:10 smithi022 ceph-mon[97651]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-09-20T21:47:10.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:10 smithi022 ceph-mon[97651]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:47:10.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:10 smithi022 ceph-mon[97651]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:10.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:10 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4093996108' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-09-20T21:47:11.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.022+0000 7f2af5fff700 1 -- 172.21.15.22:0/2180242634 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2adc000bf0 con 0x7f2ad80a13d0 2023-09-20T21:47:11.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.034+0000 7f2aecff9700 1 -- 172.21.15.22:0/2180242634 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+421925 (secure 0 0 0) 0x7f2adc000bf0 con 0x7f2ad80a13d0 2023-09-20T21:47:11.039 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:11.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 -- 172.21.15.22:0/2180242634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2ad80a13d0 msgr2=0x7f2ad80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:11.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2ad80a13d0 0x7f2ad80a3890 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400ac50 tx=0x7f2ae400a380 comp rx=0 tx=0).stop 2023-09-20T21:47:11.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 -- 172.21.15.22:0/2180242634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af00737e0 msgr2=0x7f2af0198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af00737e0 0x7f2af0198c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0004730 tx=0x7f2ae0004760 comp rx=0 tx=0).stop 2023-09-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 -- 172.21.15.22:0/2180242634 shutdown_connections 2023-09-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af0108cf0 0x7f2af01a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2ad80a13d0 0x7f2ad80a3890 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2af00737e0 0x7f2af0198c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:11.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 --2- 172.21.15.22:0/2180242634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2af0074dc0 0x7f2af01991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 -- 172.21.15.22:0/2180242634 >> 172.21.15.22:0/2180242634 conn(0x7f2af00fc390 msgr2=0x7f2af010b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 -- 172.21.15.22:0/2180242634 shutdown_connections 2023-09-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:11.040+0000 7f2af5fff700 1 -- 172.21.15.22:0/2180242634 wait complete. 2023-09-20T21:47:11.045 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-09-20T21:47:11.057 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-09-20T21:47:09.102716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596000,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":137,"num_read_kb":118,"num_write":291,"num_write_kb":2978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337664,"kb_used_data":13444,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604160,"statfs":{"total":1151940427776,"available":1151594659840,"internally_reserved":0,"allocated":13766656,"data_stored":7537855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2773,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":64,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002240"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598937+0000","last_change":"2023-09-20T21:46:41.506092+0000","last_active":"2023-09-20T21:47:07.598937+0000","last_peered":"2023-09-20T21:47:07.598937+0000","last_clean":"2023-09-20T21:47:07.598937+0000","last_became_active":"2023-09-20T21:46:41.505779+0000","last_became_peered":"2023-09-20T21:46:41.505779+0000","last_unstale":"2023-09-20T21:47:07.598937+0000","last_undegraded":"2023-09-20T21:47:07.598937+0000","last_fullsized":"2023-09-20T21:47:07.598937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:25.284068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571607+0000","last_change":"2023-09-20T21:46:41.505000+0000","last_active":"2023-09-20T21:46:45.571607+0000","last_peered":"2023-09-20T21:46:45.571607+0000","last_clean":"2023-09-20T21:46:45.571607+0000","last_became_active":"2023-09-20T21:46:41.504464+0000","last_became_peered":"2023-09-20T21:46:41.504464+0000","last_unstale":"2023-09-20T21:46:45.571607+0000","last_undegraded":"2023-09-20T21:46:45.571607+0000","last_fullsized":"2023-09-20T21:46:45.571607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:42:44.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569994+0000","last_change":"2023-09-20T21:46:41.513134+0000","last_active":"2023-09-20T21:46:45.569994+0000","last_peered":"2023-09-20T21:46:45.569994+0000","last_clean":"2023-09-20T21:46:45.569994+0000","last_became_active":"2023-09-20T21:46:41.512793+0000","last_became_peered":"2023-09-20T21:46:41.512793+0000","last_unstale":"2023-09-20T21:46:45.569994+0000","last_undegraded":"2023-09-20T21:46:45.569994+0000","last_fullsized":"2023-09-20T21:46:45.569994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:59.978630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597341+0000","last_change":"2023-09-20T21:46:41.500200+0000","last_active":"2023-09-20T21:47:07.597341+0000","last_peered":"2023-09-20T21:47:07.597341+0000","last_clean":"2023-09-20T21:47:07.597341+0000","last_became_active":"2023-09-20T21:46:41.499866+0000","last_became_peered":"2023-09-20T21:46:41.499866+0000","last_unstale":"2023-09-20T21:47:07.597341+0000","last_undegraded":"2023-09-20T21:47:07.597341+0000","last_fullsized":"2023-09-20T21:47:07.597341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:20:52.989008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501872+0000","last_change":"2023-09-20T21:46:41.507584+0000","last_active":"2023-09-20T21:46:47.501872+0000","last_peered":"2023-09-20T21:46:47.501872+0000","last_clean":"2023-09-20T21:46:47.501872+0000","last_became_active":"2023-09-20T21:46:41.507298+0000","last_became_peered":"2023-09-20T21:46:41.507298+0000","last_unstale":"2023-09-20T21:46:47.501872+0000","last_undegraded":"2023-09-20T21:46:47.501872+0000","last_fullsized":"2023-09-20T21:46:47.501872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:32:14.042414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564527+0000","last_change":"2023-09-20T21:46:41.497002+0000","last_active":"2023-09-20T21:46:45.564527+0000","last_peered":"2023-09-20T21:46:45.564527+0000","last_clean":"2023-09-20T21:46:45.564527+0000","last_became_active":"2023-09-20T21:46:41.496485+0000","last_became_peered":"2023-09-20T21:46:41.496485+0000","last_unstale":"2023-09-20T21:46:45.564527+0000","last_undegraded":"2023-09-20T21:46:45.564527+0000","last_fullsized":"2023-09-20T21:46:45.564527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:48:24.481575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.567850+0000","last_change":"2023-09-20T21:46:41.503303+0000","last_active":"2023-09-20T21:46:45.567850+0000","last_peered":"2023-09-20T21:46:45.567850+0000","last_clean":"2023-09-20T21:46:45.567850+0000","last_became_active":"2023-09-20T21:46:41.502982+0000","last_became_peered":"2023-09-20T21:46:41.502982+0000","last_unstale":"2023-09-20T21:46:45.567850+0000","last_undegraded":"2023-09-20T21:46:45.567850+0000","last_fullsized":"2023-09-20T21:46:45.567850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:23:27.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569351+0000","last_change":"2023-09-20T21:46:41.508243+0000","last_active":"2023-09-20T21:46:45.569351+0000","last_peered":"2023-09-20T21:46:45.569351+0000","last_clean":"2023-09-20T21:46:45.569351+0000","last_became_active":"2023-09-20T21:46:41.506407+0000","last_became_peered":"2023-09-20T21:46:41.506407+0000","last_unstale":"2023-09-20T21:46:45.569351+0000","last_undegraded":"2023-09-20T21:46:45.569351+0000","last_fullsized":"2023-09-20T21:46:45.569351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:39.388716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568936+0000","last_change":"2023-09-20T21:46:41.501513+0000","last_active":"2023-09-20T21:46:45.568936+0000","last_peered":"2023-09-20T21:46:45.568936+0000","last_clean":"2023-09-20T21:46:45.568936+0000","last_became_active":"2023-09-20T21:46:41.500051+0000","last_became_peered":"2023-09-20T21:46:41.500051+0000","last_unstale":"2023-09-20T21:46:45.568936+0000","last_undegraded":"2023-09-20T21:46:45.568936+0000","last_fullsized":"2023-09-20T21:46:45.568936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:40.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564777+0000","last_change":"2023-09-20T21:46:41.501045+0000","last_active":"2023-09-20T21:46:45.564777+0000","last_peered":"2023-09-20T21:46:45.564777+0000","last_clean":"2023-09-20T21:46:45.564777+0000","last_became_active":"2023-09-20T21:46:41.500744+0000","last_became_peered":"2023-09-20T21:46:41.500744+0000","last_unstale":"2023-09-20T21:46:45.564777+0000","last_undegraded":"2023-09-20T21:46:45.564777+0000","last_fullsized":"2023-09-20T21:46:45.564777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:57:29.120472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570586+0000","last_change":"2023-09-20T21:46:41.510235+0000","last_active":"2023-09-20T21:46:45.570586+0000","last_peered":"2023-09-20T21:46:45.570586+0000","last_clean":"2023-09-20T21:46:45.570586+0000","last_became_active":"2023-09-20T21:46:41.509943+0000","last_became_peered":"2023-09-20T21:46:41.509943+0000","last_unstale":"2023-09-20T21:46:45.570586+0000","last_undegraded":"2023-09-20T21:46:45.570586+0000","last_fullsized":"2023-09-20T21:46:45.570586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:09:19.383505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597247+0000","last_change":"2023-09-20T21:46:41.501194+0000","last_active":"2023-09-20T21:47:07.597247+0000","last_peered":"2023-09-20T21:47:07.597247+0000","last_clean":"2023-09-20T21:47:07.597247+0000","last_became_active":"2023-09-20T21:46:41.500095+0000","last_became_peered":"2023-09-20T21:46:41.500095+0000","last_unstale":"2023-09-20T21:47:07.597247+0000","last_undegraded":"2023-09-20T21:47:07.597247+0000","last_fullsized":"2023-09-20T21:47:07.597247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:46:51.343332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575166+0000","last_change":"2023-09-20T21:46:41.507562+0000","last_active":"2023-09-20T21:46:45.575166+0000","last_peered":"2023-09-20T21:46:45.575166+0000","last_clean":"2023-09-20T21:46:45.575166+0000","last_became_active":"2023-09-20T21:46:41.507094+0000","last_became_peered":"2023-09-20T21:46:41.507094+0000","last_unstale":"2023-09-20T21:46:45.575166+0000","last_undegraded":"2023-09-20T21:46:45.575166+0000","last_fullsized":"2023-09-20T21:46:45.575166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:34:32.401415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569178+0000","last_change":"2023-09-20T21:46:41.502864+0000","last_active":"2023-09-20T21:46:45.569178+0000","last_peered":"2023-09-20T21:46:45.569178+0000","last_clean":"2023-09-20T21:46:45.569178+0000","last_became_active":"2023-09-20T21:46:41.502543+0000","last_became_peered":"2023-09-20T21:46:41.502543+0000","last_unstale":"2023-09-20T21:46:45.569178+0000","last_undegraded":"2023-09-20T21:46:45.569178+0000","last_fullsized":"2023-09-20T21:46:45.569178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:02:04.631874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.572035+0000","last_change":"2023-09-20T21:46:41.512189+0000","last_active":"2023-09-20T21:46:45.572035+0000","last_peered":"2023-09-20T21:46:45.572035+0000","last_clean":"2023-09-20T21:46:45.572035+0000","last_became_active":"2023-09-20T21:46:41.511888+0000","last_became_peered":"2023-09-20T21:46:41.511888+0000","last_unstale":"2023-09-20T21:46:45.572035+0000","last_undegraded":"2023-09-20T21:46:45.572035+0000","last_fullsized":"2023-09-20T21:46:45.572035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:17:20.417468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571768+0000","last_change":"2023-09-20T21:46:41.502036+0000","last_active":"2023-09-20T21:46:45.571768+0000","last_peered":"2023-09-20T21:46:45.571768+0000","last_clean":"2023-09-20T21:46:45.571768+0000","last_became_active":"2023-09-20T21:46:41.501745+0000","last_became_peered":"2023-09-20T21:46:41.501745+0000","last_unstale":"2023-09-20T21:46:45.571768+0000","last_undegraded":"2023-09-20T21:46:45.571768+0000","last_fullsized":"2023-09-20T21:46:45.571768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:08:06.699543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598739+0000","last_change":"2023-09-20T21:46:41.506666+0000","last_active":"2023-09-20T21:47:07.598739+0000","last_peered":"2023-09-20T21:47:07.598739+0000","last_clean":"2023-09-20T21:47:07.598739+0000","last_became_active":"2023-09-20T21:46:41.506266+0000","last_became_peered":"2023-09-20T21:46:41.506266+0000","last_unstale":"2023-09-20T21:47:07.598739+0000","last_undegraded":"2023-09-20T21:47:07.598739+0000","last_fullsized":"2023-09-20T21:47:07.598739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:06:16.809609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563418+0000","last_change":"2023-09-20T21:46:41.497391+0000","last_active":"2023-09-20T21:46:45.563418+0000","last_peered":"2023-09-20T21:46:45.563418+0000","last_clean":"2023-09-20T21:46:45.563418+0000","last_became_active":"2023-09-20T21:46:41.497064+0000","last_became_peered":"2023-09-20T21:46:41.497064+0000","last_unstale":"2023-09-20T21:46:45.563418+0000","last_undegraded":"2023-09-20T21:46:45.563418+0000","last_fullsized":"2023-09-20T21:46:45.563418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:56:11.489585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569164+0000","last_change":"2023-09-20T21:46:41.510602+0000","last_active":"2023-09-20T21:46:45.569164+0000","last_peered":"2023-09-20T21:46:45.569164+0000","last_clean":"2023-09-20T21:46:45.569164+0000","last_became_active":"2023-09-20T21:46:41.510308+0000","last_became_peered":"2023-09-20T21:46:41.510308+0000","last_unstale":"2023-09-20T21:46:45.569164+0000","last_undegraded":"2023-09-20T21:46:45.569164+0000","last_fullsized":"2023-09-20T21:46:45.569164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:43:41.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565451+0000","last_change":"2023-09-20T21:46:45.581583+0000","last_active":"2023-09-20T21:46:46.565451+0000","last_peered":"2023-09-20T21:46:46.565451+0000","last_clean":"2023-09-20T21:46:46.565451+0000","last_became_active":"2023-09-20T21:46:45.581165+0000","last_became_peered":"2023-09-20T21:46:45.581165+0000","last_unstale":"2023-09-20T21:46:46.565451+0000","last_undegraded":"2023-09-20T21:46:46.565451+0000","last_fullsized":"2023-09-20T21:46:46.565451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:39:36.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564314+0000","last_change":"2023-09-20T21:46:40.500010+0000","last_active":"2023-09-20T21:46:45.564314+0000","last_peered":"2023-09-20T21:46:45.564314+0000","last_clean":"2023-09-20T21:46:45.564314+0000","last_became_active":"2023-09-20T21:46:40.499632+0000","last_became_peered":"2023-09-20T21:46:40.499632+0000","last_unstale":"2023-09-20T21:46:45.564314+0000","last_undegraded":"2023-09-20T21:46:45.564314+0000","last_fullsized":"2023-09-20T21:46:45.564314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:41:35.590982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569101+0000","last_change":"2023-09-20T21:46:41.516465+0000","last_active":"2023-09-20T21:46:45.569101+0000","last_peered":"2023-09-20T21:46:45.569101+0000","last_clean":"2023-09-20T21:46:45.569101+0000","last_became_active":"2023-09-20T21:46:41.516150+0000","last_became_peered":"2023-09-20T21:46:41.516150+0000","last_unstale":"2023-09-20T21:46:45.569101+0000","last_undegraded":"2023-09-20T21:46:45.569101+0000","last_fullsized":"2023-09-20T21:46:45.569101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:33:37.507049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596541+0000","last_change":"2023-09-20T21:46:45.582916+0000","last_active":"2023-09-20T21:47:07.596541+0000","last_peered":"2023-09-20T21:47:07.596541+0000","last_clean":"2023-09-20T21:47:07.596541+0000","last_became_active":"2023-09-20T21:46:45.582511+0000","last_became_peered":"2023-09-20T21:46:45.582511+0000","last_unstale":"2023-09-20T21:47:07.596541+0000","last_undegraded":"2023-09-20T21:47:07.596541+0000","last_fullsized":"2023-09-20T21:47:07.596541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:43:47.006768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569207+0000","last_change":"2023-09-20T21:46:40.516867+0000","last_active":"2023-09-20T21:46:45.569207+0000","last_peered":"2023-09-20T21:46:45.569207+0000","last_clean":"2023-09-20T21:46:45.569207+0000","last_became_active":"2023-09-20T21:46:40.516343+0000","last_became_peered":"2023-09-20T21:46:40.516343+0000","last_unstale":"2023-09-20T21:46:45.569207+0000","last_undegraded":"2023-09-20T21:46:45.569207+0000","last_fullsized":"2023-09-20T21:46:45.569207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:17:18.348921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573554+0000","last_change":"2023-09-20T21:46:41.507093+0000","last_active":"2023-09-20T21:46:45.573554+0000","last_peered":"2023-09-20T21:46:45.573554+0000","last_clean":"2023-09-20T21:46:45.573554+0000","last_became_active":"2023-09-20T21:46:41.506587+0000","last_became_peered":"2023-09-20T21:46:41.506587+0000","last_unstale":"2023-09-20T21:46:45.573554+0000","last_undegraded":"2023-09-20T21:46:45.573554+0000","last_fullsized":"2023-09-20T21:46:45.573554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:26:44.825499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597188+0000","last_change":"2023-09-20T21:46:45.583942+0000","last_active":"2023-09-20T21:47:07.597188+0000","last_peered":"2023-09-20T21:47:07.597188+0000","last_clean":"2023-09-20T21:47:07.597188+0000","last_became_active":"2023-09-20T21:46:45.583484+0000","last_became_peered":"2023-09-20T21:46:45.583484+0000","last_unstale":"2023-09-20T21:47:07.597188+0000","last_undegraded":"2023-09-20T21:47:07.597188+0000","last_fullsized":"2023-09-20T21:47:07.597188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:48.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'7","reported_seq":24,"reported_epoch":89,"state":"peering","last_fresh":"2023-09-20T21:47:07.573536+0000","last_change":"2023-09-20T21:47:07.596775+0000","last_active":"2023-09-20T21:47:07.596775+0000","last_peered":"2023-09-20T21:47:07.573536+0000","last_clean":"2023-09-20T21:47:07.573536+0000","last_became_active":"2023-09-20T21:46:40.501028+0000","last_became_peered":"2023-09-20T21:46:40.501028+0000","last_unstale":"2023-09-20T21:47:07.573536+0000","last_undegraded":"2023-09-20T21:47:07.573536+0000","last_fullsized":"2023-09-20T21:47:07.573536+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,7],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568146+0000","last_change":"2023-09-20T21:46:41.504449+0000","last_active":"2023-09-20T21:46:45.568146+0000","last_peered":"2023-09-20T21:46:45.568146+0000","last_clean":"2023-09-20T21:46:45.568146+0000","last_became_active":"2023-09-20T21:46:41.504169+0000","last_became_peered":"2023-09-20T21:46:41.504169+0000","last_unstale":"2023-09-20T21:46:45.568146+0000","last_undegraded":"2023-09-20T21:46:45.568146+0000","last_fullsized":"2023-09-20T21:46:45.568146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:36:43.980902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596618+0000","last_change":"2023-09-20T21:46:45.582269+0000","last_active":"2023-09-20T21:47:07.596618+0000","last_peered":"2023-09-20T21:47:07.596618+0000","last_clean":"2023-09-20T21:47:07.596618+0000","last_became_active":"2023-09-20T21:46:45.581782+0000","last_became_peered":"2023-09-20T21:46:45.581782+0000","last_unstale":"2023-09-20T21:47:07.596618+0000","last_undegraded":"2023-09-20T21:47:07.596618+0000","last_fullsized":"2023-09-20T21:47:07.596618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:32:09.518715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502079+0000","last_change":"2023-09-20T21:46:40.504099+0000","last_active":"2023-09-20T21:46:47.502079+0000","last_peered":"2023-09-20T21:46:47.502079+0000","last_clean":"2023-09-20T21:46:47.502079+0000","last_became_active":"2023-09-20T21:46:40.503774+0000","last_became_peered":"2023-09-20T21:46:40.503774+0000","last_unstale":"2023-09-20T21:46:47.502079+0000","last_undegraded":"2023-09-20T21:46:47.502079+0000","last_fullsized":"2023-09-20T21:46:47.502079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:17.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568776+0000","last_change":"2023-09-20T21:46:41.499802+0000","last_active":"2023-09-20T21:46:45.568776+0000","last_peered":"2023-09-20T21:46:45.568776+0000","last_clean":"2023-09-20T21:46:45.568776+0000","last_became_active":"2023-09-20T21:46:41.499491+0000","last_became_peered":"2023-09-20T21:46:41.499491+0000","last_unstale":"2023-09-20T21:46:45.568776+0000","last_undegraded":"2023-09-20T21:46:45.568776+0000","last_fullsized":"2023-09-20T21:46:45.568776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:44.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565986+0000","last_change":"2023-09-20T21:46:45.578753+0000","last_active":"2023-09-20T21:46:46.565986+0000","last_peered":"2023-09-20T21:46:46.565986+0000","last_clean":"2023-09-20T21:46:46.565986+0000","last_became_active":"2023-09-20T21:46:45.578297+0000","last_became_peered":"2023-09-20T21:46:45.578297+0000","last_unstale":"2023-09-20T21:46:46.565986+0000","last_undegraded":"2023-09-20T21:46:46.565986+0000","last_fullsized":"2023-09-20T21:46:46.565986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:18:22.596550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571690+0000","last_change":"2023-09-20T21:46:40.505675+0000","last_active":"2023-09-20T21:46:45.571690+0000","last_peered":"2023-09-20T21:46:45.571690+0000","last_clean":"2023-09-20T21:46:45.571690+0000","last_became_active":"2023-09-20T21:46:40.505432+0000","last_became_peered":"2023-09-20T21:46:40.505432+0000","last_unstale":"2023-09-20T21:46:45.571690+0000","last_undegraded":"2023-09-20T21:46:45.571690+0000","last_fullsized":"2023-09-20T21:46:45.571690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:14:53.652904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501949+0000","last_change":"2023-09-20T21:46:41.505924+0000","last_active":"2023-09-20T21:46:47.501949+0000","last_peered":"2023-09-20T21:46:47.501949+0000","last_clean":"2023-09-20T21:46:47.501949+0000","last_became_active":"2023-09-20T21:46:41.505609+0000","last_became_peered":"2023-09-20T21:46:41.505609+0000","last_unstale":"2023-09-20T21:46:47.501949+0000","last_undegraded":"2023-09-20T21:46:47.501949+0000","last_fullsized":"2023-09-20T21:46:47.501949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:13:46.506786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565886+0000","last_change":"2023-09-20T21:46:45.587818+0000","last_active":"2023-09-20T21:46:46.565886+0000","last_peered":"2023-09-20T21:46:46.565886+0000","last_clean":"2023-09-20T21:46:46.565886+0000","last_became_active":"2023-09-20T21:46:45.587376+0000","last_became_peered":"2023-09-20T21:46:45.587376+0000","last_unstale":"2023-09-20T21:46:46.565886+0000","last_undegraded":"2023-09-20T21:46:46.565886+0000","last_fullsized":"2023-09-20T21:46:46.565886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:28:53.605158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564415+0000","last_change":"2023-09-20T21:46:40.505610+0000","last_active":"2023-09-20T21:46:45.564415+0000","last_peered":"2023-09-20T21:46:45.564415+0000","last_clean":"2023-09-20T21:46:45.564415+0000","last_became_active":"2023-09-20T21:46:40.505279+0000","last_became_peered":"2023-09-20T21:46:40.505279+0000","last_unstale":"2023-09-20T21:46:45.564415+0000","last_undegraded":"2023-09-20T21:46:45.564415+0000","last_fullsized":"2023-09-20T21:46:45.564415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:41:19.575154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571794+0000","last_change":"2023-09-20T21:46:41.505337+0000","last_active":"2023-09-20T21:46:45.571794+0000","last_peered":"2023-09-20T21:46:45.571794+0000","last_clean":"2023-09-20T21:46:45.571794+0000","last_became_active":"2023-09-20T21:46:41.504959+0000","last_became_peered":"2023-09-20T21:46:41.504959+0000","last_unstale":"2023-09-20T21:46:45.571794+0000","last_undegraded":"2023-09-20T21:46:45.571794+0000","last_fullsized":"2023-09-20T21:46:45.571794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:53:37.797319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598682+0000","last_change":"2023-09-20T21:46:45.581738+0000","last_active":"2023-09-20T21:47:07.598682+0000","last_peered":"2023-09-20T21:47:07.598682+0000","last_clean":"2023-09-20T21:47:07.598682+0000","last_became_active":"2023-09-20T21:46:45.581279+0000","last_became_peered":"2023-09-20T21:46:45.581279+0000","last_unstale":"2023-09-20T21:47:07.598682+0000","last_undegraded":"2023-09-20T21:47:07.598682+0000","last_fullsized":"2023-09-20T21:47:07.598682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:41:41.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570018+0000","last_change":"2023-09-20T21:46:40.510539+0000","last_active":"2023-09-20T21:46:45.570018+0000","last_peered":"2023-09-20T21:46:45.570018+0000","last_clean":"2023-09-20T21:46:45.570018+0000","last_became_active":"2023-09-20T21:46:40.509880+0000","last_became_peered":"2023-09-20T21:46:40.509880+0000","last_unstale":"2023-09-20T21:46:45.570018+0000","last_undegraded":"2023-09-20T21:46:45.570018+0000","last_fullsized":"2023-09-20T21:46:45.570018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:52.877786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569259+0000","last_change":"2023-09-20T21:46:41.505105+0000","last_active":"2023-09-20T21:46:45.569259+0000","last_peered":"2023-09-20T21:46:45.569259+0000","last_clean":"2023-09-20T21:46:45.569259+0000","last_became_active":"2023-09-20T21:46:41.504785+0000","last_became_peered":"2023-09-20T21:46:41.504785+0000","last_unstale":"2023-09-20T21:46:45.569259+0000","last_undegraded":"2023-09-20T21:46:45.569259+0000","last_fullsized":"2023-09-20T21:46:45.569259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:07:57.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502213+0000","last_change":"2023-09-20T21:46:45.583160+0000","last_active":"2023-09-20T21:46:47.502213+0000","last_peered":"2023-09-20T21:46:47.502213+0000","last_clean":"2023-09-20T21:46:47.502213+0000","last_became_active":"2023-09-20T21:46:45.582709+0000","last_became_peered":"2023-09-20T21:46:45.582709+0000","last_unstale":"2023-09-20T21:46:47.502213+0000","last_undegraded":"2023-09-20T21:46:47.502213+0000","last_fullsized":"2023-09-20T21:46:47.502213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:24:57.563682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568131+0000","last_change":"2023-09-20T21:46:40.502591+0000","last_active":"2023-09-20T21:46:45.568131+0000","last_peered":"2023-09-20T21:46:45.568131+0000","last_clean":"2023-09-20T21:46:45.568131+0000","last_became_active":"2023-09-20T21:46:40.502288+0000","last_became_peered":"2023-09-20T21:46:40.502288+0000","last_unstale":"2023-09-20T21:46:45.568131+0000","last_undegraded":"2023-09-20T21:46:45.568131+0000","last_fullsized":"2023-09-20T21:46:45.568131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:09:37.983866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563192+0000","last_change":"2023-09-20T21:46:41.511367+0000","last_active":"2023-09-20T21:46:45.563192+0000","last_peered":"2023-09-20T21:46:45.563192+0000","last_clean":"2023-09-20T21:46:45.563192+0000","last_became_active":"2023-09-20T21:46:41.511005+0000","last_became_peered":"2023-09-20T21:46:41.511005+0000","last_unstale":"2023-09-20T21:46:45.563192+0000","last_undegraded":"2023-09-20T21:46:45.563192+0000","last_fullsized":"2023-09-20T21:46:45.563192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:37:59.684808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.862394+0000","last_change":"2023-09-20T21:46:45.580917+0000","last_active":"2023-09-20T21:46:46.862394+0000","last_peered":"2023-09-20T21:46:46.862394+0000","last_clean":"2023-09-20T21:46:46.862394+0000","last_became_active":"2023-09-20T21:46:45.580505+0000","last_became_peered":"2023-09-20T21:46:45.580505+0000","last_unstale":"2023-09-20T21:46:46.862394+0000","last_undegraded":"2023-09-20T21:46:46.862394+0000","last_fullsized":"2023-09-20T21:46:46.862394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:52:19.733919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.591401+0000","last_change":"2023-09-20T21:46:40.507040+0000","last_active":"2023-09-20T21:47:05.591401+0000","last_peered":"2023-09-20T21:47:05.591401+0000","last_clean":"2023-09-20T21:47:05.591401+0000","last_became_active":"2023-09-20T21:46:40.506712+0000","last_became_peered":"2023-09-20T21:46:40.506712+0000","last_unstale":"2023-09-20T21:47:05.591401+0000","last_undegraded":"2023-09-20T21:47:05.591401+0000","last_fullsized":"2023-09-20T21:47:05.591401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:02:56.040911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568722+0000","last_change":"2023-09-20T21:46:41.504794+0000","last_active":"2023-09-20T21:46:45.568722+0000","last_peered":"2023-09-20T21:46:45.568722+0000","last_clean":"2023-09-20T21:46:45.568722+0000","last_became_active":"2023-09-20T21:46:41.504372+0000","last_became_peered":"2023-09-20T21:46:41.504372+0000","last_unstale":"2023-09-20T21:46:45.568722+0000","last_undegraded":"2023-09-20T21:46:45.568722+0000","last_fullsized":"2023-09-20T21:46:45.568722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:42:20.527306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.564942+0000","last_change":"2023-09-20T21:46:45.586364+0000","last_active":"2023-09-20T21:46:46.564942+0000","last_peered":"2023-09-20T21:46:46.564942+0000","last_clean":"2023-09-20T21:46:46.564942+0000","last_became_active":"2023-09-20T21:46:45.584620+0000","last_became_peered":"2023-09-20T21:46:45.584620+0000","last_unstale":"2023-09-20T21:46:46.564942+0000","last_undegraded":"2023-09-20T21:46:46.564942+0000","last_fullsized":"2023-09-20T21:46:46.564942+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:05:20.158118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.588953+0000","last_change":"2023-09-20T21:46:40.500797+0000","last_active":"2023-09-20T21:47:05.588953+0000","last_peered":"2023-09-20T21:47:05.588953+0000","last_clean":"2023-09-20T21:47:05.588953+0000","last_became_active":"2023-09-20T21:46:40.500337+0000","last_became_peered":"2023-09-20T21:46:40.500337+0000","last_unstale":"2023-09-20T21:47:05.588953+0000","last_undegraded":"2023-09-20T21:47:05.588953+0000","last_fullsized":"2023-09-20T21:47:05.588953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:19.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573645+0000","last_change":"2023-09-20T21:46:41.507805+0000","last_active":"2023-09-20T21:46:45.573645+0000","last_peered":"2023-09-20T21:46:45.573645+0000","last_clean":"2023-09-20T21:46:45.573645+0000","last_became_active":"2023-09-20T21:46:41.507441+0000","last_became_peered":"2023-09-20T21:46:41.507441+0000","last_unstale":"2023-09-20T21:46:45.573645+0000","last_undegraded":"2023-09-20T21:46:45.573645+0000","last_fullsized":"2023-09-20T21:46:45.573645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:51:33.240534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597442+0000","last_change":"2023-09-20T21:46:45.583081+0000","last_active":"2023-09-20T21:47:07.597442+0000","last_peered":"2023-09-20T21:47:07.597442+0000","last_clean":"2023-09-20T21:47:07.597442+0000","last_became_active":"2023-09-20T21:46:45.580496+0000","last_became_peered":"2023-09-20T21:46:45.580496+0000","last_unstale":"2023-09-20T21:47:07.597442+0000","last_undegraded":"2023-09-20T21:47:07.597442+0000","last_fullsized":"2023-09-20T21:47:07.597442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:42.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568676+0000","last_change":"2023-09-20T21:46:40.512739+0000","last_active":"2023-09-20T21:46:45.568676+0000","last_peered":"2023-09-20T21:46:45.568676+0000","last_clean":"2023-09-20T21:46:45.568676+0000","last_became_active":"2023-09-20T21:46:40.512485+0000","last_became_peered":"2023-09-20T21:46:40.512485+0000","last_unstale":"2023-09-20T21:46:45.568676+0000","last_undegraded":"2023-09-20T21:46:45.568676+0000","last_fullsized":"2023-09-20T21:46:45.568676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:02:49.392560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569091+0000","last_change":"2023-09-20T21:46:41.503093+0000","last_active":"2023-09-20T21:46:45.569091+0000","last_peered":"2023-09-20T21:46:45.569091+0000","last_clean":"2023-09-20T21:46:45.569091+0000","last_became_active":"2023-09-20T21:46:41.502806+0000","last_became_peered":"2023-09-20T21:46:41.502806+0000","last_unstale":"2023-09-20T21:46:45.569091+0000","last_undegraded":"2023-09-20T21:46:45.569091+0000","last_fullsized":"2023-09-20T21:46:45.569091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:53:21.143258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565158+0000","last_change":"2023-09-20T21:46:45.579478+0000","last_active":"2023-09-20T21:46:46.565158+0000","last_peered":"2023-09-20T21:46:46.565158+0000","last_clean":"2023-09-20T21:46:46.565158+0000","last_became_active":"2023-09-20T21:46:45.579055+0000","last_became_peered":"2023-09-20T21:46:45.579055+0000","last_unstale":"2023-09-20T21:46:46.565158+0000","last_undegraded":"2023-09-20T21:46:46.565158+0000","last_fullsized":"2023-09-20T21:46:46.565158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:17:32.013280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.590075+0000","last_change":"2023-09-20T21:46:40.507321+0000","last_active":"2023-09-20T21:47:05.590075+0000","last_peered":"2023-09-20T21:47:05.590075+0000","last_clean":"2023-09-20T21:47:05.590075+0000","last_became_active":"2023-09-20T21:46:40.506983+0000","last_became_peered":"2023-09-20T21:46:40.506983+0000","last_unstale":"2023-09-20T21:47:05.590075+0000","last_undegraded":"2023-09-20T21:47:05.590075+0000","last_fullsized":"2023-09-20T21:47:05.590075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:46:40.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502157+0000","last_change":"2023-09-20T21:46:41.509646+0000","last_active":"2023-09-20T21:46:47.502157+0000","last_peered":"2023-09-20T21:46:47.502157+0000","last_clean":"2023-09-20T21:46:47.502157+0000","last_became_active":"2023-09-20T21:46:41.509183+0000","last_became_peered":"2023-09-20T21:46:41.509183+0000","last_unstale":"2023-09-20T21:46:47.502157+0000","last_undegraded":"2023-09-20T21:46:47.502157+0000","last_fullsized":"2023-09-20T21:46:47.502157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:17:21.084830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794668+0000","last_change":"2023-09-20T21:46:45.583709+0000","last_active":"2023-09-20T21:46:46.794668+0000","last_peered":"2023-09-20T21:46:46.794668+0000","last_clean":"2023-09-20T21:46:46.794668+0000","last_became_active":"2023-09-20T21:46:45.583302+0000","last_became_peered":"2023-09-20T21:46:45.583302+0000","last_unstale":"2023-09-20T21:46:46.794668+0000","last_undegraded":"2023-09-20T21:46:46.794668+0000","last_fullsized":"2023-09-20T21:46:46.794668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:31:35.472750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.590965+0000","last_change":"2023-09-20T21:46:40.506411+0000","last_active":"2023-09-20T21:47:05.590965+0000","last_peered":"2023-09-20T21:47:05.590965+0000","last_clean":"2023-09-20T21:47:05.590965+0000","last_became_active":"2023-09-20T21:46:40.506116+0000","last_became_peered":"2023-09-20T21:46:40.506116+0000","last_unstale":"2023-09-20T21:47:05.590965+0000","last_undegraded":"2023-09-20T21:47:05.590965+0000","last_fullsized":"2023-09-20T21:47:05.590965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:23:19.175007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575844+0000","last_change":"2023-09-20T21:46:41.513793+0000","last_active":"2023-09-20T21:46:45.575844+0000","last_peered":"2023-09-20T21:46:45.575844+0000","last_clean":"2023-09-20T21:46:45.575844+0000","last_became_active":"2023-09-20T21:46:41.513434+0000","last_became_peered":"2023-09-20T21:46:41.513434+0000","last_unstale":"2023-09-20T21:46:45.575844+0000","last_undegraded":"2023-09-20T21:46:45.575844+0000","last_fullsized":"2023-09-20T21:46:45.575844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:00:57.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598091+0000","last_change":"2023-09-20T21:46:45.578188+0000","last_active":"2023-09-20T21:47:07.598091+0000","last_peered":"2023-09-20T21:47:07.598091+0000","last_clean":"2023-09-20T21:47:07.598091+0000","last_became_active":"2023-09-20T21:46:45.577548+0000","last_became_peered":"2023-09-20T21:46:45.577548+0000","last_unstale":"2023-09-20T21:47:07.598091+0000","last_undegraded":"2023-09-20T21:47:07.598091+0000","last_fullsized":"2023-09-20T21:47:07.598091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:54:58.894269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.573689+0000","last_change":"2023-09-20T21:46:40.505465+0000","last_active":"2023-09-20T21:46:45.573689+0000","last_peered":"2023-09-20T21:46:45.573689+0000","last_clean":"2023-09-20T21:46:45.573689+0000","last_became_active":"2023-09-20T21:46:40.505174+0000","last_became_peered":"2023-09-20T21:46:40.505174+0000","last_unstale":"2023-09-20T21:46:45.573689+0000","last_undegraded":"2023-09-20T21:46:45.573689+0000","last_fullsized":"2023-09-20T21:46:45.573689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:50:53.191963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568918+0000","last_change":"2023-09-20T21:46:41.507370+0000","last_active":"2023-09-20T21:46:45.568918+0000","last_peered":"2023-09-20T21:46:45.568918+0000","last_clean":"2023-09-20T21:46:45.568918+0000","last_became_active":"2023-09-20T21:46:41.507035+0000","last_became_peered":"2023-09-20T21:46:41.507035+0000","last_unstale":"2023-09-20T21:46:45.568918+0000","last_undegraded":"2023-09-20T21:46:45.568918+0000","last_fullsized":"2023-09-20T21:46:45.568918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:50.298899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565547+0000","last_change":"2023-09-20T21:46:45.584458+0000","last_active":"2023-09-20T21:46:46.565547+0000","last_peered":"2023-09-20T21:46:46.565547+0000","last_clean":"2023-09-20T21:46:46.565547+0000","last_became_active":"2023-09-20T21:46:45.584033+0000","last_became_peered":"2023-09-20T21:46:45.584033+0000","last_unstale":"2023-09-20T21:46:46.565547+0000","last_undegraded":"2023-09-20T21:46:46.565547+0000","last_fullsized":"2023-09-20T21:46:46.565547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:48:44.653880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.567221+0000","last_change":"2023-09-20T21:46:40.501329+0000","last_active":"2023-09-20T21:47:08.567221+0000","last_peered":"2023-09-20T21:47:08.567221+0000","last_clean":"2023-09-20T21:47:08.567221+0000","last_became_active":"2023-09-20T21:46:40.500999+0000","last_became_peered":"2023-09-20T21:46:40.500999+0000","last_unstale":"2023-09-20T21:47:08.567221+0000","last_undegraded":"2023-09-20T21:47:08.567221+0000","last_fullsized":"2023-09-20T21:47:08.567221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:38.028398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568141+0000","last_change":"2023-09-20T21:46:41.504583+0000","last_active":"2023-09-20T21:46:45.568141+0000","last_peered":"2023-09-20T21:46:45.568141+0000","last_clean":"2023-09-20T21:46:45.568141+0000","last_became_active":"2023-09-20T21:46:41.504273+0000","last_became_peered":"2023-09-20T21:46:41.504273+0000","last_unstale":"2023-09-20T21:46:45.568141+0000","last_undegraded":"2023-09-20T21:46:45.568141+0000","last_fullsized":"2023-09-20T21:46:45.568141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:37.100382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502476+0000","last_change":"2023-09-20T21:46:45.587116+0000","last_active":"2023-09-20T21:46:47.502476+0000","last_peered":"2023-09-20T21:46:47.502476+0000","last_clean":"2023-09-20T21:46:47.502476+0000","last_became_active":"2023-09-20T21:46:45.586629+0000","last_became_peered":"2023-09-20T21:46:45.586629+0000","last_unstale":"2023-09-20T21:46:47.502476+0000","last_undegraded":"2023-09-20T21:46:47.502476+0000","last_fullsized":"2023-09-20T21:46:47.502476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:10:33.024235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'16","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.572218+0000","last_change":"2023-09-20T21:46:40.499985+0000","last_active":"2023-09-20T21:47:07.572218+0000","last_peered":"2023-09-20T21:47:07.572218+0000","last_clean":"2023-09-20T21:47:07.572218+0000","last_became_active":"2023-09-20T21:46:40.499646+0000","last_became_peered":"2023-09-20T21:46:40.499646+0000","last_unstale":"2023-09-20T21:47:07.572218+0000","last_undegraded":"2023-09-20T21:47:07.572218+0000","last_fullsized":"2023-09-20T21:47:07.572218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:52:02.758556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568813+0000","last_change":"2023-09-20T21:46:41.501241+0000","last_active":"2023-09-20T21:46:45.568813+0000","last_peered":"2023-09-20T21:46:45.568813+0000","last_clean":"2023-09-20T21:46:45.568813+0000","last_became_active":"2023-09-20T21:46:41.500902+0000","last_became_peered":"2023-09-20T21:46:41.500902+0000","last_unstale":"2023-09-20T21:46:45.568813+0000","last_undegraded":"2023-09-20T21:46:45.568813+0000","last_fullsized":"2023-09-20T21:46:45.568813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:21:55.210065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794066+0000","last_change":"2023-09-20T21:46:45.583405+0000","last_active":"2023-09-20T21:46:46.794066+0000","last_peered":"2023-09-20T21:46:46.794066+0000","last_clean":"2023-09-20T21:46:46.794066+0000","last_became_active":"2023-09-20T21:46:45.582945+0000","last_became_peered":"2023-09-20T21:46:45.582945+0000","last_unstale":"2023-09-20T21:46:46.794066+0000","last_undegraded":"2023-09-20T21:46:46.794066+0000","last_fullsized":"2023-09-20T21:46:46.794066+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:27:14.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.561084+0000","last_change":"2023-09-20T21:46:40.507744+0000","last_active":"2023-09-20T21:47:07.561084+0000","last_peered":"2023-09-20T21:47:07.561084+0000","last_clean":"2023-09-20T21:47:07.561084+0000","last_became_active":"2023-09-20T21:46:40.507454+0000","last_became_peered":"2023-09-20T21:46:40.507454+0000","last_unstale":"2023-09-20T21:47:07.561084+0000","last_undegraded":"2023-09-20T21:47:07.561084+0000","last_fullsized":"2023-09-20T21:47:07.561084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:00:24.492789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598829+0000","last_change":"2023-09-20T21:46:41.504263+0000","last_active":"2023-09-20T21:47:07.598829+0000","last_peered":"2023-09-20T21:47:07.598829+0000","last_clean":"2023-09-20T21:47:07.598829+0000","last_became_active":"2023-09-20T21:46:41.503961+0000","last_became_peered":"2023-09-20T21:46:41.503961+0000","last_unstale":"2023-09-20T21:47:07.598829+0000","last_undegraded":"2023-09-20T21:47:07.598829+0000","last_fullsized":"2023-09-20T21:47:07.598829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:31:58.035517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.944063+0000","last_change":"2023-09-20T21:46:45.581343+0000","last_active":"2023-09-20T21:46:47.944063+0000","last_peered":"2023-09-20T21:46:47.944063+0000","last_clean":"2023-09-20T21:46:47.944063+0000","last_became_active":"2023-09-20T21:46:45.580830+0000","last_became_peered":"2023-09-20T21:46:45.580830+0000","last_unstale":"2023-09-20T21:46:47.944063+0000","last_undegraded":"2023-09-20T21:46:47.944063+0000","last_fullsized":"2023-09-20T21:46:47.944063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:20:49.865825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563359+0000","last_change":"2023-09-20T21:46:40.501850+0000","last_active":"2023-09-20T21:46:45.563359+0000","last_peered":"2023-09-20T21:46:45.563359+0000","last_clean":"2023-09-20T21:46:45.563359+0000","last_became_active":"2023-09-20T21:46:40.501500+0000","last_became_peered":"2023-09-20T21:46:40.501500+0000","last_unstale":"2023-09-20T21:46:45.563359+0000","last_undegraded":"2023-09-20T21:46:45.563359+0000","last_fullsized":"2023-09-20T21:46:45.563359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:14:15.907335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564629+0000","last_change":"2023-09-20T21:46:41.507432+0000","last_active":"2023-09-20T21:46:45.564629+0000","last_peered":"2023-09-20T21:46:45.564629+0000","last_clean":"2023-09-20T21:46:45.564629+0000","last_became_active":"2023-09-20T21:46:41.507119+0000","last_became_peered":"2023-09-20T21:46:41.507119+0000","last_unstale":"2023-09-20T21:46:45.564629+0000","last_undegraded":"2023-09-20T21:46:45.564629+0000","last_fullsized":"2023-09-20T21:46:45.564629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:14:10.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502815+0000","last_change":"2023-09-20T21:46:45.588455+0000","last_active":"2023-09-20T21:46:47.502815+0000","last_peered":"2023-09-20T21:46:47.502815+0000","last_clean":"2023-09-20T21:46:47.502815+0000","last_became_active":"2023-09-20T21:46:45.587980+0000","last_became_peered":"2023-09-20T21:46:45.587980+0000","last_unstale":"2023-09-20T21:46:47.502815+0000","last_undegraded":"2023-09-20T21:46:47.502815+0000","last_fullsized":"2023-09-20T21:46:47.502815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:07:35.697330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.568603+0000","last_change":"2023-09-20T21:46:40.506260+0000","last_active":"2023-09-20T21:46:46.568603+0000","last_peered":"2023-09-20T21:46:46.568603+0000","last_clean":"2023-09-20T21:46:46.568603+0000","last_became_active":"2023-09-20T21:46:40.505999+0000","last_became_peered":"2023-09-20T21:46:40.505999+0000","last_unstale":"2023-09-20T21:46:46.568603+0000","last_undegraded":"2023-09-20T21:46:46.568603+0000","last_fullsized":"2023-09-20T21:46:46.568603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:39:01.748038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.564591+0000","last_change":"2023-09-20T21:46:41.496822+0000","last_active":"2023-09-20T21:46:45.564591+0000","last_peered":"2023-09-20T21:46:45.564591+0000","last_clean":"2023-09-20T21:46:45.564591+0000","last_became_active":"2023-09-20T21:46:41.496517+0000","last_became_peered":"2023-09-20T21:46:41.496517+0000","last_unstale":"2023-09-20T21:46:45.564591+0000","last_undegraded":"2023-09-20T21:46:45.564591+0000","last_fullsized":"2023-09-20T21:46:45.564591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:12.592040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502793+0000","last_change":"2023-09-20T21:46:45.587410+0000","last_active":"2023-09-20T21:46:47.502793+0000","last_peered":"2023-09-20T21:46:47.502793+0000","last_clean":"2023-09-20T21:46:47.502793+0000","last_became_active":"2023-09-20T21:46:45.586944+0000","last_became_peered":"2023-09-20T21:46:45.586944+0000","last_unstale":"2023-09-20T21:46:47.502793+0000","last_undegraded":"2023-09-20T21:46:47.502793+0000","last_fullsized":"2023-09-20T21:46:47.502793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:57.053765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568334+0000","last_change":"2023-09-20T21:46:41.511526+0000","last_active":"2023-09-20T21:46:45.568334+0000","last_peered":"2023-09-20T21:46:45.568334+0000","last_clean":"2023-09-20T21:46:45.568334+0000","last_became_active":"2023-09-20T21:46:41.511213+0000","last_became_peered":"2023-09-20T21:46:41.511213+0000","last_unstale":"2023-09-20T21:46:45.568334+0000","last_undegraded":"2023-09-20T21:46:45.568334+0000","last_fullsized":"2023-09-20T21:46:45.568334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:29:04.199190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569931+0000","last_change":"2023-09-20T21:46:40.514916+0000","last_active":"2023-09-20T21:46:45.569931+0000","last_peered":"2023-09-20T21:46:45.569931+0000","last_clean":"2023-09-20T21:46:45.569931+0000","last_became_active":"2023-09-20T21:46:40.514553+0000","last_became_peered":"2023-09-20T21:46:40.514553+0000","last_unstale":"2023-09-20T21:46:45.569931+0000","last_undegraded":"2023-09-20T21:46:45.569931+0000","last_fullsized":"2023-09-20T21:46:45.569931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:12:32.481852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.568779+0000","last_change":"2023-09-20T21:46:45.587489+0000","last_active":"2023-09-20T21:46:46.568779+0000","last_peered":"2023-09-20T21:46:46.568779+0000","last_clean":"2023-09-20T21:46:46.568779+0000","last_became_active":"2023-09-20T21:46:45.587022+0000","last_became_peered":"2023-09-20T21:46:45.587022+0000","last_unstale":"2023-09-20T21:46:46.568779+0000","last_undegraded":"2023-09-20T21:46:46.568779+0000","last_fullsized":"2023-09-20T21:46:46.568779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:09:05.870763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568768+0000","last_change":"2023-09-20T21:46:41.508886+0000","last_active":"2023-09-20T21:46:45.568768+0000","last_peered":"2023-09-20T21:46:45.568768+0000","last_clean":"2023-09-20T21:46:45.568768+0000","last_became_active":"2023-09-20T21:46:41.508470+0000","last_became_peered":"2023-09-20T21:46:41.508470+0000","last_unstale":"2023-09-20T21:46:45.568768+0000","last_undegraded":"2023-09-20T21:46:45.568768+0000","last_fullsized":"2023-09-20T21:46:45.568768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:45:04.888300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.576942+0000","last_change":"2023-09-20T21:46:40.503350+0000","last_active":"2023-09-20T21:47:08.576942+0000","last_peered":"2023-09-20T21:47:08.576942+0000","last_clean":"2023-09-20T21:47:08.576942+0000","last_became_active":"2023-09-20T21:46:40.503045+0000","last_became_peered":"2023-09-20T21:46:40.503045+0000","last_unstale":"2023-09-20T21:47:08.576942+0000","last_undegraded":"2023-09-20T21:47:08.576942+0000","last_fullsized":"2023-09-20T21:47:08.576942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:31:09.004342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":313,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.567932+0000","last_change":"2023-09-20T21:45:35.180242+0000","last_active":"2023-09-20T21:46:45.567932+0000","last_peered":"2023-09-20T21:46:45.567932+0000","last_clean":"2023-09-20T21:46:45.567932+0000","last_became_active":"2023-09-20T21:45:35.179705+0000","last_became_peered":"2023-09-20T21:45:35.179705+0000","last_unstale":"2023-09-20T21:46:45.567932+0000","last_undegraded":"2023-09-20T21:46:45.567932+0000","last_fullsized":"2023-09-20T21:46:45.567932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_clean_scrub_stamp":"2023-09-20T21:42:25.826683+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:02:03.448188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.794591+0000","last_change":"2023-09-20T21:46:45.581396+0000","last_active":"2023-09-20T21:46:46.794591+0000","last_peered":"2023-09-20T21:46:46.794591+0000","last_clean":"2023-09-20T21:46:46.794591+0000","last_became_active":"2023-09-20T21:46:45.581018+0000","last_became_peered":"2023-09-20T21:46:45.581018+0000","last_unstale":"2023-09-20T21:46:46.794591+0000","last_undegraded":"2023-09-20T21:46:46.794591+0000","last_fullsized":"2023-09-20T21:46:46.794591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:16:35.934460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569837+0000","last_change":"2023-09-20T21:46:41.514392+0000","last_active":"2023-09-20T21:46:45.569837+0000","last_peered":"2023-09-20T21:46:45.569837+0000","last_clean":"2023-09-20T21:46:45.569837+0000","last_became_active":"2023-09-20T21:46:41.514090+0000","last_became_peered":"2023-09-20T21:46:41.514090+0000","last_unstale":"2023-09-20T21:46:45.569837+0000","last_undegraded":"2023-09-20T21:46:45.569837+0000","last_fullsized":"2023-09-20T21:46:45.569837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:05:47.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:06.596564+0000","last_change":"2023-09-20T21:46:40.507346+0000","last_active":"2023-09-20T21:47:06.596564+0000","last_peered":"2023-09-20T21:47:06.596564+0000","last_clean":"2023-09-20T21:47:06.596564+0000","last_became_active":"2023-09-20T21:46:40.506969+0000","last_became_peered":"2023-09-20T21:46:40.506969+0000","last_unstale":"2023-09-20T21:47:06.596564+0000","last_undegraded":"2023-09-20T21:47:06.596564+0000","last_fullsized":"2023-09-20T21:47:06.596564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:01:20.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600062+0000","last_change":"2023-09-20T21:46:45.578033+0000","last_active":"2023-09-20T21:47:07.600062+0000","last_peered":"2023-09-20T21:47:07.600062+0000","last_clean":"2023-09-20T21:47:07.600062+0000","last_became_active":"2023-09-20T21:46:45.577540+0000","last_became_peered":"2023-09-20T21:46:45.577540+0000","last_unstale":"2023-09-20T21:47:07.600062+0000","last_undegraded":"2023-09-20T21:47:07.600062+0000","last_fullsized":"2023-09-20T21:47:07.600062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:24:31.020660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.573672+0000","last_change":"2023-09-20T21:46:40.510634+0000","last_active":"2023-09-20T21:47:07.573672+0000","last_peered":"2023-09-20T21:47:07.573672+0000","last_clean":"2023-09-20T21:47:07.573672+0000","last_became_active":"2023-09-20T21:46:40.510390+0000","last_became_peered":"2023-09-20T21:46:40.510390+0000","last_unstale":"2023-09-20T21:47:07.573672+0000","last_undegraded":"2023-09-20T21:47:07.573672+0000","last_fullsized":"2023-09-20T21:47:07.573672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:44:49.272987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571610+0000","last_change":"2023-09-20T21:46:41.502981+0000","last_active":"2023-09-20T21:46:45.571610+0000","last_peered":"2023-09-20T21:46:45.571610+0000","last_clean":"2023-09-20T21:46:45.571610+0000","last_became_active":"2023-09-20T21:46:41.502594+0000","last_became_peered":"2023-09-20T21:46:41.502594+0000","last_unstale":"2023-09-20T21:46:45.571610+0000","last_undegraded":"2023-09-20T21:46:45.571610+0000","last_fullsized":"2023-09-20T21:46:45.571610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:41:15.261478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.862185+0000","last_change":"2023-09-20T21:46:45.582903+0000","last_active":"2023-09-20T21:46:46.862185+0000","last_peered":"2023-09-20T21:46:46.862185+0000","last_clean":"2023-09-20T21:46:46.862185+0000","last_became_active":"2023-09-20T21:46:45.582433+0000","last_became_peered":"2023-09-20T21:46:45.582433+0000","last_unstale":"2023-09-20T21:46:46.862185+0000","last_undegraded":"2023-09-20T21:46:46.862185+0000","last_fullsized":"2023-09-20T21:46:46.862185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:38:22.174903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.563444+0000","last_change":"2023-09-20T21:46:40.507671+0000","last_active":"2023-09-20T21:46:45.563444+0000","last_peered":"2023-09-20T21:46:45.563444+0000","last_clean":"2023-09-20T21:46:45.563444+0000","last_became_active":"2023-09-20T21:46:40.507288+0000","last_became_peered":"2023-09-20T21:46:40.507288+0000","last_unstale":"2023-09-20T21:46:45.563444+0000","last_undegraded":"2023-09-20T21:46:45.563444+0000","last_fullsized":"2023-09-20T21:46:45.563444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:27:02.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570001+0000","last_change":"2023-09-20T21:46:41.507654+0000","last_active":"2023-09-20T21:46:45.570001+0000","last_peered":"2023-09-20T21:46:45.570001+0000","last_clean":"2023-09-20T21:46:45.570001+0000","last_became_active":"2023-09-20T21:46:41.507362+0000","last_became_peered":"2023-09-20T21:46:41.507362+0000","last_unstale":"2023-09-20T21:46:45.570001+0000","last_undegraded":"2023-09-20T21:46:45.570001+0000","last_fullsized":"2023-09-20T21:46:45.570001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:32:08.973379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565429+0000","last_change":"2023-09-20T21:46:45.578437+0000","last_active":"2023-09-20T21:46:46.565429+0000","last_peered":"2023-09-20T21:46:46.565429+0000","last_clean":"2023-09-20T21:46:46.565429+0000","last_became_active":"2023-09-20T21:46:45.578010+0000","last_became_peered":"2023-09-20T21:46:45.578010+0000","last_unstale":"2023-09-20T21:46:46.565429+0000","last_undegraded":"2023-09-20T21:46:46.565429+0000","last_fullsized":"2023-09-20T21:46:46.565429+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:59:05.494861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577934+0000","last_change":"2023-09-20T21:46:40.508937+0000","last_active":"2023-09-20T21:47:08.577934+0000","last_peered":"2023-09-20T21:47:08.577934+0000","last_clean":"2023-09-20T21:47:08.577934+0000","last_became_active":"2023-09-20T21:46:40.508639+0000","last_became_peered":"2023-09-20T21:46:40.508639+0000","last_unstale":"2023-09-20T21:47:08.577934+0000","last_undegraded":"2023-09-20T21:47:08.577934+0000","last_fullsized":"2023-09-20T21:47:08.577934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:50:48.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571508+0000","last_change":"2023-09-20T21:46:41.510702+0000","last_active":"2023-09-20T21:46:45.571508+0000","last_peered":"2023-09-20T21:46:45.571508+0000","last_clean":"2023-09-20T21:46:45.571508+0000","last_became_active":"2023-09-20T21:46:41.510357+0000","last_became_peered":"2023-09-20T21:46:41.510357+0000","last_unstale":"2023-09-20T21:46:45.571508+0000","last_undegraded":"2023-09-20T21:46:45.571508+0000","last_fullsized":"2023-09-20T21:46:45.571508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:35:44.455981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596780+0000","last_change":"2023-09-20T21:46:45.581110+0000","last_active":"2023-09-20T21:47:07.596780+0000","last_peered":"2023-09-20T21:47:07.596780+0000","last_clean":"2023-09-20T21:47:07.596780+0000","last_became_active":"2023-09-20T21:46:45.580498+0000","last_became_peered":"2023-09-20T21:46:45.580498+0000","last_unstale":"2023-09-20T21:47:07.596780+0000","last_undegraded":"2023-09-20T21:47:07.596780+0000","last_fullsized":"2023-09-20T21:47:07.596780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:19:22.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571650+0000","last_change":"2023-09-20T21:46:40.497017+0000","last_active":"2023-09-20T21:46:45.571650+0000","last_peered":"2023-09-20T21:46:45.571650+0000","last_clean":"2023-09-20T21:46:45.571650+0000","last_became_active":"2023-09-20T21:46:40.496722+0000","last_became_peered":"2023-09-20T21:46:40.496722+0000","last_unstale":"2023-09-20T21:46:45.571650+0000","last_undegraded":"2023-09-20T21:46:45.571650+0000","last_fullsized":"2023-09-20T21:46:45.571650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:12:07.017318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.501752+0000","last_change":"2023-09-20T21:46:41.508793+0000","last_active":"2023-09-20T21:46:47.501752+0000","last_peered":"2023-09-20T21:46:47.501752+0000","last_clean":"2023-09-20T21:46:47.501752+0000","last_became_active":"2023-09-20T21:46:41.508453+0000","last_became_peered":"2023-09-20T21:46:41.508453+0000","last_unstale":"2023-09-20T21:46:47.501752+0000","last_undegraded":"2023-09-20T21:46:47.501752+0000","last_fullsized":"2023-09-20T21:46:47.501752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:35:12.711570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.565653+0000","last_change":"2023-09-20T21:46:45.583519+0000","last_active":"2023-09-20T21:46:46.565653+0000","last_peered":"2023-09-20T21:46:46.565653+0000","last_clean":"2023-09-20T21:46:46.565653+0000","last_became_active":"2023-09-20T21:46:45.582933+0000","last_became_peered":"2023-09-20T21:46:45.582933+0000","last_unstale":"2023-09-20T21:46:46.565653+0000","last_undegraded":"2023-09-20T21:46:46.565653+0000","last_fullsized":"2023-09-20T21:46:46.565653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:13:20.237865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:47:05.591317+0000","last_change":"2023-09-20T21:46:40.507384+0000","last_active":"2023-09-20T21:47:05.591317+0000","last_peered":"2023-09-20T21:47:05.591317+0000","last_clean":"2023-09-20T21:47:05.591317+0000","last_became_active":"2023-09-20T21:46:40.507021+0000","last_became_peered":"2023-09-20T21:46:40.507021+0000","last_unstale":"2023-09-20T21:47:05.591317+0000","last_undegraded":"2023-09-20T21:47:05.591317+0000","last_fullsized":"2023-09-20T21:47:05.591317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:48:04.360664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598093+0000","last_change":"2023-09-20T21:46:41.511537+0000","last_active":"2023-09-20T21:47:07.598093+0000","last_peered":"2023-09-20T21:47:07.598093+0000","last_clean":"2023-09-20T21:47:07.598093+0000","last_became_active":"2023-09-20T21:46:41.511238+0000","last_became_peered":"2023-09-20T21:46:41.511238+0000","last_unstale":"2023-09-20T21:47:07.598093+0000","last_undegraded":"2023-09-20T21:47:07.598093+0000","last_fullsized":"2023-09-20T21:47:07.598093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:01:41.492824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596769+0000","last_change":"2023-09-20T21:46:45.585750+0000","last_active":"2023-09-20T21:47:07.596769+0000","last_peered":"2023-09-20T21:47:07.596769+0000","last_clean":"2023-09-20T21:47:07.596769+0000","last_became_active":"2023-09-20T21:46:45.585213+0000","last_became_peered":"2023-09-20T21:46:45.585213+0000","last_unstale":"2023-09-20T21:47:07.596769+0000","last_undegraded":"2023-09-20T21:47:07.596769+0000","last_fullsized":"2023-09-20T21:47:07.596769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:31:15.650766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568158+0000","last_change":"2023-09-20T21:46:40.506394+0000","last_active":"2023-09-20T21:46:45.568158+0000","last_peered":"2023-09-20T21:46:45.568158+0000","last_clean":"2023-09-20T21:46:45.568158+0000","last_became_active":"2023-09-20T21:46:40.506085+0000","last_became_peered":"2023-09-20T21:46:40.506085+0000","last_unstale":"2023-09-20T21:46:45.568158+0000","last_undegraded":"2023-09-20T21:46:45.568158+0000","last_fullsized":"2023-09-20T21:46:45.568158+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:02.738600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570438+0000","last_change":"2023-09-20T21:46:41.509491+0000","last_active":"2023-09-20T21:46:45.570438+0000","last_peered":"2023-09-20T21:46:45.570438+0000","last_clean":"2023-09-20T21:46:45.570438+0000","last_became_active":"2023-09-20T21:46:41.509039+0000","last_became_peered":"2023-09-20T21:46:41.509039+0000","last_unstale":"2023-09-20T21:46:45.570438+0000","last_undegraded":"2023-09-20T21:46:45.570438+0000","last_fullsized":"2023-09-20T21:46:45.570438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:57:39.003154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597192+0000","last_change":"2023-09-20T21:46:45.581651+0000","last_active":"2023-09-20T21:47:07.597192+0000","last_peered":"2023-09-20T21:47:07.597192+0000","last_clean":"2023-09-20T21:47:07.597192+0000","last_became_active":"2023-09-20T21:46:45.581224+0000","last_became_peered":"2023-09-20T21:46:45.581224+0000","last_unstale":"2023-09-20T21:47:07.597192+0000","last_undegraded":"2023-09-20T21:47:07.597192+0000","last_fullsized":"2023-09-20T21:47:07.597192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:32:17.778301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.610835+0000","last_change":"2023-09-20T21:47:08.610413+0000","last_active":"2023-09-20T21:47:08.610835+0000","last_peered":"2023-09-20T21:47:08.610835+0000","last_clean":"2023-09-20T21:47:08.610835+0000","last_became_active":"2023-09-20T21:47:08.603734+0000","last_became_peered":"2023-09-20T21:47:08.603734+0000","last_unstale":"2023-09-20T21:47:08.610835+0000","last_undegraded":"2023-09-20T21:47:08.610835+0000","last_fullsized":"2023-09-20T21:47:08.610835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:43:42.569371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568903+0000","last_change":"2023-09-20T21:46:41.507218+0000","last_active":"2023-09-20T21:46:45.568903+0000","last_peered":"2023-09-20T21:46:45.568903+0000","last_clean":"2023-09-20T21:46:45.568903+0000","last_became_active":"2023-09-20T21:46:41.506889+0000","last_became_peered":"2023-09-20T21:46:41.506889+0000","last_unstale":"2023-09-20T21:46:45.568903+0000","last_undegraded":"2023-09-20T21:46:45.568903+0000","last_fullsized":"2023-09-20T21:46:45.568903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:08:40.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.503610+0000","last_change":"2023-09-20T21:46:45.579269+0000","last_active":"2023-09-20T21:46:47.503610+0000","last_peered":"2023-09-20T21:46:47.503610+0000","last_clean":"2023-09-20T21:46:47.503610+0000","last_became_active":"2023-09-20T21:46:45.578541+0000","last_became_peered":"2023-09-20T21:46:45.578541+0000","last_unstale":"2023-09-20T21:46:47.503610+0000","last_undegraded":"2023-09-20T21:46:47.503610+0000","last_fullsized":"2023-09-20T21:46:47.503610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:07:50.241947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569525+0000","last_change":"2023-09-20T21:46:40.508774+0000","last_active":"2023-09-20T21:46:45.569525+0000","last_peered":"2023-09-20T21:46:45.569525+0000","last_clean":"2023-09-20T21:46:45.569525+0000","last_became_active":"2023-09-20T21:46:40.508480+0000","last_became_peered":"2023-09-20T21:46:40.508480+0000","last_unstale":"2023-09-20T21:46:45.569525+0000","last_undegraded":"2023-09-20T21:46:45.569525+0000","last_fullsized":"2023-09-20T21:46:45.569525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:22.643724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570698+0000","last_change":"2023-09-20T21:46:41.503763+0000","last_active":"2023-09-20T21:46:45.570698+0000","last_peered":"2023-09-20T21:46:45.570698+0000","last_clean":"2023-09-20T21:46:45.570698+0000","last_became_active":"2023-09-20T21:46:41.503360+0000","last_became_peered":"2023-09-20T21:46:41.503360+0000","last_unstale":"2023-09-20T21:46:45.570698+0000","last_undegraded":"2023-09-20T21:46:45.570698+0000","last_fullsized":"2023-09-20T21:46:45.570698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:03:00.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:47.502032+0000","last_change":"2023-09-20T21:46:45.586702+0000","last_active":"2023-09-20T21:46:47.502032+0000","last_peered":"2023-09-20T21:46:47.502032+0000","last_clean":"2023-09-20T21:46:47.502032+0000","last_became_active":"2023-09-20T21:46:45.586177+0000","last_became_peered":"2023-09-20T21:46:45.586177+0000","last_unstale":"2023-09-20T21:46:47.502032+0000","last_undegraded":"2023-09-20T21:46:47.502032+0000","last_fullsized":"2023-09-20T21:46:47.502032+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:19:03.883440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568892+0000","last_change":"2023-09-20T21:46:40.507270+0000","last_active":"2023-09-20T21:46:45.568892+0000","last_peered":"2023-09-20T21:46:45.568892+0000","last_clean":"2023-09-20T21:46:45.568892+0000","last_became_active":"2023-09-20T21:46:40.506933+0000","last_became_peered":"2023-09-20T21:46:40.506933+0000","last_unstale":"2023-09-20T21:46:45.568892+0000","last_undegraded":"2023-09-20T21:46:45.568892+0000","last_fullsized":"2023-09-20T21:46:45.568892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:59.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569676+0000","last_change":"2023-09-20T21:46:41.507868+0000","last_active":"2023-09-20T21:46:45.569676+0000","last_peered":"2023-09-20T21:46:45.569676+0000","last_clean":"2023-09-20T21:46:45.569676+0000","last_became_active":"2023-09-20T21:46:41.507568+0000","last_became_peered":"2023-09-20T21:46:41.507568+0000","last_unstale":"2023-09-20T21:46:45.569676+0000","last_undegraded":"2023-09-20T21:46:45.569676+0000","last_fullsized":"2023-09-20T21:46:45.569676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:44.615620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-09-20T21:46:46.793795+0000","last_change":"2023-09-20T21:46:45.578726+0000","last_active":"2023-09-20T21:46:46.793795+0000","last_peered":"2023-09-20T21:46:46.793795+0000","last_clean":"2023-09-20T21:46:46.793795+0000","last_became_active":"2023-09-20T21:46:45.578326+0000","last_became_peered":"2023-09-20T21:46:45.578326+0000","last_unstale":"2023-09-20T21:46:46.793795+0000","last_undegraded":"2023-09-20T21:46:46.793795+0000","last_fullsized":"2023-09-20T21:46:46.793795+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:21:34.564840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570141+0000","last_change":"2023-09-20T21:46:40.509024+0000","last_active":"2023-09-20T21:46:45.570141+0000","last_peered":"2023-09-20T21:46:45.570141+0000","last_clean":"2023-09-20T21:46:45.570141+0000","last_became_active":"2023-09-20T21:46:40.508659+0000","last_became_peered":"2023-09-20T21:46:40.508659+0000","last_unstale":"2023-09-20T21:46:45.570141+0000","last_undegraded":"2023-09-20T21:46:45.570141+0000","last_fullsized":"2023-09-20T21:46:45.570141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:12:08.302068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569100+0000","last_change":"2023-09-20T21:46:41.510307+0000","last_active":"2023-09-20T21:46:45.569100+0000","last_peered":"2023-09-20T21:46:45.569100+0000","last_clean":"2023-09-20T21:46:45.569100+0000","last_became_active":"2023-09-20T21:46:41.510014+0000","last_became_peered":"2023-09-20T21:46:41.510014+0000","last_unstale":"2023-09-20T21:46:45.569100+0000","last_undegraded":"2023-09-20T21:46:45.569100+0000","last_fullsized":"2023-09-20T21:46:45.569100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:05:55.062515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598156+0000","last_change":"2023-09-20T21:46:41.511525+0000","last_active":"2023-09-20T21:47:07.598156+0000","last_peered":"2023-09-20T21:47:07.598156+0000","last_clean":"2023-09-20T21:47:07.598156+0000","last_became_active":"2023-09-20T21:46:41.511203+0000","last_became_peered":"2023-09-20T21:46:41.511203+0000","last_unstale":"2023-09-20T21:47:07.598156+0000","last_undegraded":"2023-09-20T21:47:07.598156+0000","last_fullsized":"2023-09-20T21:47:07.598156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:42:15.757422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570189+0000","last_change":"2023-09-20T21:46:41.502936+0000","last_active":"2023-09-20T21:46:45.570189+0000","last_peered":"2023-09-20T21:46:45.570189+0000","last_clean":"2023-09-20T21:46:45.570189+0000","last_became_active":"2023-09-20T21:46:41.502617+0000","last_became_peered":"2023-09-20T21:46:41.502617+0000","last_unstale":"2023-09-20T21:46:45.570189+0000","last_undegraded":"2023-09-20T21:46:45.570189+0000","last_fullsized":"2023-09-20T21:46:45.570189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:45:21.197652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568057+0000","last_change":"2023-09-20T21:46:41.510766+0000","last_active":"2023-09-20T21:46:45.568057+0000","last_peered":"2023-09-20T21:46:45.568057+0000","last_clean":"2023-09-20T21:46:45.568057+0000","last_became_active":"2023-09-20T21:46:41.510433+0000","last_became_peered":"2023-09-20T21:46:41.510433+0000","last_unstale":"2023-09-20T21:46:45.568057+0000","last_undegraded":"2023-09-20T21:46:45.568057+0000","last_fullsized":"2023-09-20T21:46:45.568057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:56:12.085926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.572421+0000","last_change":"2023-09-20T21:46:41.508846+0000","last_active":"2023-09-20T21:46:45.572421+0000","last_peered":"2023-09-20T21:46:45.572421+0000","last_clean":"2023-09-20T21:46:45.572421+0000","last_became_active":"2023-09-20T21:46:41.508518+0000","last_became_peered":"2023-09-20T21:46:41.508518+0000","last_unstale":"2023-09-20T21:46:45.572421+0000","last_undegraded":"2023-09-20T21:46:45.572421+0000","last_fullsized":"2023-09-20T21:46:45.572421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:01:58.195327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.571696+0000","last_change":"2023-09-20T21:46:41.505929+0000","last_active":"2023-09-20T21:46:45.571696+0000","last_peered":"2023-09-20T21:46:45.571696+0000","last_clean":"2023-09-20T21:46:45.571696+0000","last_became_active":"2023-09-20T21:46:41.505542+0000","last_became_peered":"2023-09-20T21:46:41.505542+0000","last_unstale":"2023-09-20T21:46:45.571696+0000","last_undegraded":"2023-09-20T21:46:45.571696+0000","last_fullsized":"2023-09-20T21:46:45.571696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:24:47.520344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598727+0000","last_change":"2023-09-20T21:46:41.507603+0000","last_active":"2023-09-20T21:47:07.598727+0000","last_peered":"2023-09-20T21:47:07.598727+0000","last_clean":"2023-09-20T21:47:07.598727+0000","last_became_active":"2023-09-20T21:46:41.507340+0000","last_became_peered":"2023-09-20T21:46:41.507340+0000","last_unstale":"2023-09-20T21:47:07.598727+0000","last_undegraded":"2023-09-20T21:47:07.598727+0000","last_fullsized":"2023-09-20T21:47:07.598727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:50:19.756491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568500+0000","last_change":"2023-09-20T21:46:41.509625+0000","last_active":"2023-09-20T21:46:45.568500+0000","last_peered":"2023-09-20T21:46:45.568500+0000","last_clean":"2023-09-20T21:46:45.568500+0000","last_became_active":"2023-09-20T21:46:41.509301+0000","last_became_peered":"2023-09-20T21:46:41.509301+0000","last_unstale":"2023-09-20T21:46:45.568500+0000","last_undegraded":"2023-09-20T21:46:45.568500+0000","last_fullsized":"2023-09-20T21:46:45.568500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:15:03.108060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570117+0000","last_change":"2023-09-20T21:46:41.516781+0000","last_active":"2023-09-20T21:46:45.570117+0000","last_peered":"2023-09-20T21:46:45.570117+0000","last_clean":"2023-09-20T21:46:45.570117+0000","last_became_active":"2023-09-20T21:46:41.516496+0000","last_became_peered":"2023-09-20T21:46:41.516496+0000","last_unstale":"2023-09-20T21:46:45.570117+0000","last_undegraded":"2023-09-20T21:46:45.570117+0000","last_fullsized":"2023-09-20T21:46:45.570117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:17:26.492851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.568221+0000","last_change":"2023-09-20T21:46:41.499927+0000","last_active":"2023-09-20T21:46:45.568221+0000","last_peered":"2023-09-20T21:46:45.568221+0000","last_clean":"2023-09-20T21:46:45.568221+0000","last_became_active":"2023-09-20T21:46:41.499605+0000","last_became_peered":"2023-09-20T21:46:41.499605+0000","last_unstale":"2023-09-20T21:46:45.568221+0000","last_undegraded":"2023-09-20T21:46:45.568221+0000","last_fullsized":"2023-09-20T21:46:45.568221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:18:16.306349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.570531+0000","last_change":"2023-09-20T21:46:41.515842+0000","last_active":"2023-09-20T21:46:45.570531+0000","last_peered":"2023-09-20T21:46:45.570531+0000","last_clean":"2023-09-20T21:46:45.570531+0000","last_became_active":"2023-09-20T21:46:41.515600+0000","last_became_peered":"2023-09-20T21:46:41.515600+0000","last_unstale":"2023-09-20T21:46:45.570531+0000","last_undegraded":"2023-09-20T21:46:45.570531+0000","last_fullsized":"2023-09-20T21:46:45.570531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:50:58.088103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.569630+0000","last_change":"2023-09-20T21:46:41.510695+0000","last_active":"2023-09-20T21:46:45.569630+0000","last_peered":"2023-09-20T21:46:45.569630+0000","last_clean":"2023-09-20T21:46:45.569630+0000","last_became_active":"2023-09-20T21:46:41.510345+0000","last_became_peered":"2023-09-20T21:46:41.510345+0000","last_unstale":"2023-09-20T21:46:45.569630+0000","last_undegraded":"2023-09-20T21:46:45.569630+0000","last_fullsized":"2023-09-20T21:46:45.569630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:38:24.201382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-09-20T21:46:45.575033+0000","last_change":"2023-09-20T21:46:41.507809+0000","last_active":"2023-09-20T21:46:45.575033+0000","last_peered":"2023-09-20T21:46:45.575033+0000","last_clean":"2023-09-20T21:46:45.575033+0000","last_became_active":"2023-09-20T21:46:41.507513+0000","last_became_peered":"2023-09-20T21:46:41.507513+0000","last_unstale":"2023-09-20T21:46:45.575033+0000","last_undegraded":"2023-09-20T21:46:45.575033+0000","last_fullsized":"2023-09-20T21:46:45.575033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:30:43.132728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598414+0000","last_change":"2023-09-20T21:46:41.503590+0000","last_active":"2023-09-20T21:47:07.598414+0000","last_peered":"2023-09-20T21:47:07.598414+0000","last_clean":"2023-09-20T21:47:07.598414+0000","last_became_active":"2023-09-20T21:46:41.503295+0000","last_became_peered":"2023-09-20T21:46:41.503295+0000","last_unstale":"2023-09-20T21:47:07.598414+0000","last_undegraded":"2023-09-20T21:47:07.598414+0000","last_fullsized":"2023-09-20T21:47:07.598414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:28:33.088873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5632,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":110,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200704,"data_stored":19996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93,"ondisk_log_size":93,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":487223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.64600000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.85699999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.753}]}]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1071729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46100000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54700000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.72899999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.59599999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.63200000000000001}]},{"osd":11,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71499999999999997}]}]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":477145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.68500000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.622},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.75}]},{"osd":4,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.79700000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.51300000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.82899999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:46:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.7719999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.872}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":486769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77200000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.34100000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.371}]},{"osd":2,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65500000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64700000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":1.121}]},{"osd":3,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.64300000000000002,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.92600000000000005}]},{"osd":4,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79500000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.878,"15min":0.878},"last":0.84199999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.313,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78600000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.94599999999999995,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.71999999999999997}]},{"osd":7,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.876,"5min":1.018,"15min":1.018},"last":0.57499999999999996}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.96599999999999997}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.627},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1589248,"data_stored":1068723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41799999999999998},"min":{"1min":0.27500000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.438,"15min":0.433},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65500000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68700000000000006}]},{"osd":4,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.623},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70799999999999996}]},{"osd":6,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.747},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.64000000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65300000000000002,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.53000000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.78400000000000003}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":1.018,"15min":1.018},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64500000000000002,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":1.2,"15min":1.2},"last":0.88700000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.57199999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.466,"15min":1.466},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":1.012,"15min":1.012},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.175,"5min":1.179,"15min":1.179},"last":0.79900000000000004}]},{"osd":4,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0640000000000001,"5min":1.093,"15min":1.093},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78700000000000003,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.022,"5min":1.407,"15min":1.407},"last":0.93400000000000005}]},{"osd":5,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0549999999999999,"5min":1.266,"15min":1.266},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.046,"5min":1.073,"15min":1.073},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.84799999999999998,"15min":0.83699999999999997},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.85899999999999999,"15min":0.84599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.95099999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:46:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.84499999999999997,"15min":0.85899999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0149999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.751},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.85599999999999998,"15min":0.86399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1240000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0509999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.085},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.91400000000000003}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":480405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.39400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86499999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.52000000000000002,"15min":0.5},"min":{"1min":0.40000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77700000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.41899999999999998,"15min":0.39300000000000002},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71299999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.83699999999999997}]},{"osd":6,"last update":"Wed Sep 20 21:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.55900000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.92200000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.372,"15min":0.372},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.81699999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.89200000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.85599999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":1},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.88}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":478041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.503,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.66200000000000003,"15min":0.63900000000000001},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92300000000000004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.44900000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.74199999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66600000000000004,"15min":0.64200000000000002},"min":{"1min":0.52000000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.62},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.67400000000000004,"15min":0.627},"min":{"1min":0.64600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.752}]},{"osd":5,"last update":"Wed Sep 20 21:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86899999999999999,"5min":1.149,"15min":1.149},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.67200000000000004},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.94199999999999995,"5min":1.194,"15min":1.194},"last":0.92000000000000004}]},{"osd":6,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.69599999999999995,"15min":0.67000000000000004},"min":{"1min":0.51700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.026},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82399999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.012,"5min":1.214,"15min":1.214},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.90500000000000003,"15min":0.91600000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0940000000000001,"5min":1.232,"15min":1.232},"last":1.073}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.091},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.90000000000000002}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.111},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.96099999999999997}]},{"osd":11,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.129}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78900000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.39400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.59299999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.48299999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85299999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.44,"15min":0.39900000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.55500000000000005,"15min":0.49399999999999999},"min":{"1min":0.38300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":7.1500000000000004,"5min":7.1500000000000004,"15min":7.1500000000000004},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.56999999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.65400000000000003,"15min":0.64100000000000001},"min":{"1min":0.53000000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73699999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.63}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.88}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.753},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":476667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.432,"15min":0.39000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81699999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":1,"15min":1},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.84899999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63300000000000001,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65300000000000002,"15min":0.626},"min":{"1min":0.53500000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68799999999999994,"15min":0.64500000000000002},"min":{"1min":0.60399999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed Sep 20 21:46:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":1.024,"15min":1.024},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75900000000000001,"15min":0.747},"min":{"1min":0.61099999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98399999999999999,"5min":1.167,"15min":1.167},"last":0.69599999999999995}]},{"osd":6,"last update":"Wed Sep 20 21:46:04 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.52500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.075,"5min":1.147,"15min":1.147},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.627}]},{"osd":7,"last update":"Wed Sep 20 21:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.55400000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.177,"5min":1.248,"15min":1.248},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0549999999999999,"5min":1.228,"15min":1.228},"last":0.874}]},{"osd":8,"last update":"Wed Sep 20 21:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.74299999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.85099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":486697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57699999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.38,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.40899999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.46400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68700000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.54500000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.48499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.51900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61799999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58599999999999997,"15min":0.56799999999999995},"min":{"1min":0.503,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:47:07 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67000000000000004,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.49399999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56699999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":1,"15min":1},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88200000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68500000000000005}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.69999999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.71599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1581056,"data_stored":1066832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85499999999999998,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.753,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.42699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81299999999999994,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.36699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.434},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40400000000000003,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.59899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79100000000000004,"5min":1.615,"15min":1.615},"last":0.496},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.008,"15min":1.008},"last":0.77300000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:46:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.46999999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65900000000000003,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85999999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.75800000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.749,"5min":0.879,"15min":0.879},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59599999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.78900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-20T21:47:11.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:10 smithi123 ceph-mon[134433]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:11.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:10 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] up:active 2023-09-20T21:47:11.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:10 smithi123 ceph-mon[134433]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-09-20T21:47:11.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:10 smithi123 ceph-mon[134433]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:47:11.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:10 smithi123 ceph-mon[134433]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:11.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:10 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4093996108' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-09-20T21:47:11.360 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 21:47:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b[171416]: 2023-09-20T21:47:10.992+0000 7f3b247a9700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-09-20T21:47:11.360 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 21:47:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b[171416]: 2023-09-20T21:47:10.992+0000 7f3b247a9700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-09-20T21:47:11.361 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 21:47:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b[171416]: 2023-09-20T21:47:10.992+0000 7f3b247a9700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-09-20T21:47:11.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:11 smithi022 ceph-mon[97651]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:11.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:11 smithi022 ceph-mon[97651]: pgmap v293: 129 pgs: 1 peering, 128 active+clean; 585 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5.6 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:47:12.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:11 smithi123 ceph-mon[134433]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:12.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:11 smithi123 ceph-mon[134433]: pgmap v293: 129 pgs: 1 peering, 128 active+clean; 585 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5.6 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:47:12.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:11 smithi089 ceph-mon[134090]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:12.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:11 smithi089 ceph-mon[134090]: pgmap v293: 129 pgs: 1 peering, 128 active+clean; 585 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5.6 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:47:13.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:13 smithi123 ceph-mon[134433]: pgmap v294: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.1 KiB/s wr, 26 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-20T21:47:13.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:13 smithi089 ceph-mon[134090]: pgmap v294: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.1 KiB/s wr, 26 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-20T21:47:13.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:13 smithi022 ceph-mon[97651]: pgmap v294: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.1 KiB/s wr, 26 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-20T21:47:14.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-20T21:47:14.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.282+0000 7fc210fff700 1 -- 172.21.15.22:0/2348779830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c101ff0 msgr2=0x7fc20c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.282+0000 7fc210fff700 1 --2- 172.21.15.22:0/2348779830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c101ff0 0x7fc20c102450 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc2000099c0 tx=0x7fc200009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.282+0000 7fc210fff700 1 -- 172.21.15.22:0/2348779830 shutdown_connections 2023-09-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.282+0000 7fc210fff700 1 --2- 172.21.15.22:0/2348779830 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c102990 0x7fc20c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.282+0000 7fc210fff700 1 --2- 172.21.15.22:0/2348779830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c100e00 0x7fc20c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.282+0000 7fc210fff700 1 --2- 172.21.15.22:0/2348779830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c101ff0 0x7fc20c102450 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.282+0000 7fc210fff700 1 -- 172.21.15.22:0/2348779830 >> 172.21.15.22:0/2348779830 conn(0x7fc20c0fc350 msgr2=0x7fc20c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.282+0000 7fc210fff700 1 -- 172.21.15.22:0/2348779830 shutdown_connections 2023-09-20T21:47:14.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.282+0000 7fc210fff700 1 -- 172.21.15.22:0/2348779830 wait complete. 2023-09-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.283+0000 7fc210fff700 1 Processor -- start 2023-09-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc210fff700 1 -- start start 2023-09-20T21:47:14.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc210fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c100e00 0x7fc20c074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc210fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c101ff0 0x7fc20c0731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc210fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c102990 0x7fc20c073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc210fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c073eb0 con 0x7fc20c101ff0 2023-09-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc210fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c074020 con 0x7fc20c102990 2023-09-20T21:47:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc210fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c1a0b00 con 0x7fc20c100e00 2023-09-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc20a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c100e00 0x7fc20c074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc209d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c101ff0 0x7fc20c0731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc20a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c100e00 0x7fc20c074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:56648/0 (socket says 172.21.15.22:56648) 2023-09-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc20a59c700 1 -- 172.21.15.22:0/3605009657 learned_addr learned my addr 172.21.15.22:0/3605009657 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.284+0000 7fc20ad9d700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c102990 0x7fc20c073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:14.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc20a59c700 1 -- 172.21.15.22:0/3605009657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c102990 msgr2=0x7fc20c073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc20a59c700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c102990 0x7fc20c073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc20a59c700 1 -- 172.21.15.22:0/3605009657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c101ff0 msgr2=0x7fc20c0731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc20a59c700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c101ff0 0x7fc20c0731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc20a59c700 1 -- 172.21.15.22:0/3605009657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2000095f0 con 0x7fc20c100e00 2023-09-20T21:47:14.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc209d9b700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c101ff0 0x7fc20c0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc20a59c700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c100e00 0x7fc20c074b90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e970 tx=0x7fc1fc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc1fb7fe700 1 -- 172.21.15.22:0/3605009657 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc009790 con 0x7fc20c100e00 2023-09-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc210fff700 1 -- 172.21.15.22:0/3605009657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c1a0de0 con 0x7fc20c100e00 2023-09-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc1fb7fe700 1 -- 172.21.15.22:0/3605009657 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1fc00c7e0 con 0x7fc20c100e00 2023-09-20T21:47:14.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc1fb7fe700 1 -- 172.21.15.22:0/3605009657 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc003c70 con 0x7fc20c100e00 2023-09-20T21:47:14.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.285+0000 7fc210fff700 1 -- 172.21.15.22:0/3605009657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20c1a1210 con 0x7fc20c100e00 2023-09-20T21:47:14.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.286+0000 7fc210fff700 1 -- 172.21.15.22:0/3605009657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20c108370 con 0x7fc20c100e00 2023-09-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.289+0000 7fc1fb7fe700 1 -- 172.21.15.22:0/3605009657 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fc1fc00cc50 con 0x7fc20c100e00 2023-09-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.290+0000 7fc1fb7fe700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc1f40a1350 0x7fc1f40a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.290+0000 7fc1fb7fe700 1 -- 172.21.15.22:0/3605009657 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc1fc014070 con 0x7fc20c100e00 2023-09-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.290+0000 7fc209d9b700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc1f40a1350 0x7fc1f40a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:14.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.291+0000 7fc209d9b700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc1f40a1350 0x7fc1f40a3810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc20000b580 tx=0x7fc200005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:14.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.292+0000 7fc1fb7fe700 1 -- 172.21.15.22:0/3605009657 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc1fc096620 con 0x7fc20c100e00 2023-09-20T21:47:14.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.496+0000 7fc210fff700 1 -- 172.21.15.22:0/3605009657 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc20c10c660 con 0x7fc1f40a1350 2023-09-20T21:47:14.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.510+0000 7fc1fb7fe700 1 -- 172.21.15.22:0/3605009657 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422077 (secure 0 0 0) 0x7fc20c10c660 con 0x7fc1f40a1350 2023-09-20T21:47:14.516 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:14.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 -- 172.21.15.22:0/3605009657 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc1f40a1350 msgr2=0x7fc1f40a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc1f40a1350 0x7fc1f40a3810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc20000b580 tx=0x7fc200005fb0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 -- 172.21.15.22:0/3605009657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c100e00 msgr2=0x7fc20c074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c100e00 0x7fc20c074b90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00e970 tx=0x7fc1fc00ec80 comp rx=0 tx=0).stop 2023-09-20T21:47:14.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 -- 172.21.15.22:0/3605009657 shutdown_connections 2023-09-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc1f40a1350 0x7fc1f40a3810 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c102990 0x7fc20c073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c100e00 0x7fc20c074b90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 --2- 172.21.15.22:0/3605009657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc20c101ff0 0x7fc20c0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 -- 172.21.15.22:0/3605009657 >> 172.21.15.22:0/3605009657 conn(0x7fc20c0fc350 msgr2=0x7fc20c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 -- 172.21.15.22:0/3605009657 shutdown_connections 2023-09-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:14.515+0000 7fc210fff700 1 -- 172.21.15.22:0/3605009657 wait complete. 2023-09-20T21:47:14.522 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-09-20T21:47:14.534 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-09-20T21:47:13.104049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600632,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":181,"num_read_kb":153,"num_write":390,"num_write_kb":3005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338192,"kb_used_data":13940,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603632,"statfs":{"total":1151940427776,"available":1151594119168,"internally_reserved":0,"allocated":14274560,"data_stored":7729193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5765,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":161,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002488"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598937+0000","last_change":"2023-09-20T21:46:41.506092+0000","last_active":"2023-09-20T21:47:07.598937+0000","last_peered":"2023-09-20T21:47:07.598937+0000","last_clean":"2023-09-20T21:47:07.598937+0000","last_became_active":"2023-09-20T21:46:41.505779+0000","last_became_peered":"2023-09-20T21:46:41.505779+0000","last_unstale":"2023-09-20T21:47:07.598937+0000","last_undegraded":"2023-09-20T21:47:07.598937+0000","last_fullsized":"2023-09-20T21:47:07.598937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:25.284068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599485+0000","last_change":"2023-09-20T21:46:41.505000+0000","last_active":"2023-09-20T21:47:07.599485+0000","last_peered":"2023-09-20T21:47:07.599485+0000","last_clean":"2023-09-20T21:47:07.599485+0000","last_became_active":"2023-09-20T21:46:41.504464+0000","last_became_peered":"2023-09-20T21:46:41.504464+0000","last_unstale":"2023-09-20T21:47:07.599485+0000","last_undegraded":"2023-09-20T21:47:07.599485+0000","last_fullsized":"2023-09-20T21:47:07.599485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:42:44.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599342+0000","last_change":"2023-09-20T21:46:41.513134+0000","last_active":"2023-09-20T21:47:07.599342+0000","last_peered":"2023-09-20T21:47:07.599342+0000","last_clean":"2023-09-20T21:47:07.599342+0000","last_became_active":"2023-09-20T21:46:41.512793+0000","last_became_peered":"2023-09-20T21:46:41.512793+0000","last_unstale":"2023-09-20T21:47:07.599342+0000","last_undegraded":"2023-09-20T21:47:07.599342+0000","last_fullsized":"2023-09-20T21:47:07.599342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:59.978630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597341+0000","last_change":"2023-09-20T21:46:41.500200+0000","last_active":"2023-09-20T21:47:07.597341+0000","last_peered":"2023-09-20T21:47:07.597341+0000","last_clean":"2023-09-20T21:47:07.597341+0000","last_became_active":"2023-09-20T21:46:41.499866+0000","last_became_peered":"2023-09-20T21:46:41.499866+0000","last_unstale":"2023-09-20T21:47:07.597341+0000","last_undegraded":"2023-09-20T21:47:07.597341+0000","last_fullsized":"2023-09-20T21:47:07.597341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:20:52.989008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594880+0000","last_change":"2023-09-20T21:46:41.507584+0000","last_active":"2023-09-20T21:47:07.594880+0000","last_peered":"2023-09-20T21:47:07.594880+0000","last_clean":"2023-09-20T21:47:07.594880+0000","last_became_active":"2023-09-20T21:46:41.507298+0000","last_became_peered":"2023-09-20T21:46:41.507298+0000","last_unstale":"2023-09-20T21:47:07.594880+0000","last_undegraded":"2023-09-20T21:47:07.594880+0000","last_fullsized":"2023-09-20T21:47:07.594880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:32:14.042414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595455+0000","last_change":"2023-09-20T21:46:41.497002+0000","last_active":"2023-09-20T21:47:07.595455+0000","last_peered":"2023-09-20T21:47:07.595455+0000","last_clean":"2023-09-20T21:47:07.595455+0000","last_became_active":"2023-09-20T21:46:41.496485+0000","last_became_peered":"2023-09-20T21:46:41.496485+0000","last_unstale":"2023-09-20T21:47:07.595455+0000","last_undegraded":"2023-09-20T21:47:07.595455+0000","last_fullsized":"2023-09-20T21:47:07.595455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:48:24.481575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598725+0000","last_change":"2023-09-20T21:46:41.503303+0000","last_active":"2023-09-20T21:47:07.598725+0000","last_peered":"2023-09-20T21:47:07.598725+0000","last_clean":"2023-09-20T21:47:07.598725+0000","last_became_active":"2023-09-20T21:46:41.502982+0000","last_became_peered":"2023-09-20T21:46:41.502982+0000","last_unstale":"2023-09-20T21:47:07.598725+0000","last_undegraded":"2023-09-20T21:47:07.598725+0000","last_fullsized":"2023-09-20T21:47:07.598725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:23:27.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595388+0000","last_change":"2023-09-20T21:46:41.508243+0000","last_active":"2023-09-20T21:47:07.595388+0000","last_peered":"2023-09-20T21:47:07.595388+0000","last_clean":"2023-09-20T21:47:07.595388+0000","last_became_active":"2023-09-20T21:46:41.506407+0000","last_became_peered":"2023-09-20T21:46:41.506407+0000","last_unstale":"2023-09-20T21:47:07.595388+0000","last_undegraded":"2023-09-20T21:47:07.595388+0000","last_fullsized":"2023-09-20T21:47:07.595388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:39.388716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594461+0000","last_change":"2023-09-20T21:46:41.501513+0000","last_active":"2023-09-20T21:47:07.594461+0000","last_peered":"2023-09-20T21:47:07.594461+0000","last_clean":"2023-09-20T21:47:07.594461+0000","last_became_active":"2023-09-20T21:46:41.500051+0000","last_became_peered":"2023-09-20T21:46:41.500051+0000","last_unstale":"2023-09-20T21:47:07.594461+0000","last_undegraded":"2023-09-20T21:47:07.594461+0000","last_fullsized":"2023-09-20T21:47:07.594461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:40.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596488+0000","last_change":"2023-09-20T21:46:41.501045+0000","last_active":"2023-09-20T21:47:07.596488+0000","last_peered":"2023-09-20T21:47:07.596488+0000","last_clean":"2023-09-20T21:47:07.596488+0000","last_became_active":"2023-09-20T21:46:41.500744+0000","last_became_peered":"2023-09-20T21:46:41.500744+0000","last_unstale":"2023-09-20T21:47:07.596488+0000","last_undegraded":"2023-09-20T21:47:07.596488+0000","last_fullsized":"2023-09-20T21:47:07.596488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:57:29.120472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599791+0000","last_change":"2023-09-20T21:46:41.510235+0000","last_active":"2023-09-20T21:47:07.599791+0000","last_peered":"2023-09-20T21:47:07.599791+0000","last_clean":"2023-09-20T21:47:07.599791+0000","last_became_active":"2023-09-20T21:46:41.509943+0000","last_became_peered":"2023-09-20T21:46:41.509943+0000","last_unstale":"2023-09-20T21:47:07.599791+0000","last_undegraded":"2023-09-20T21:47:07.599791+0000","last_fullsized":"2023-09-20T21:47:07.599791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:09:19.383505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597247+0000","last_change":"2023-09-20T21:46:41.501194+0000","last_active":"2023-09-20T21:47:07.597247+0000","last_peered":"2023-09-20T21:47:07.597247+0000","last_clean":"2023-09-20T21:47:07.597247+0000","last_became_active":"2023-09-20T21:46:41.500095+0000","last_became_peered":"2023-09-20T21:46:41.500095+0000","last_unstale":"2023-09-20T21:47:07.597247+0000","last_undegraded":"2023-09-20T21:47:07.597247+0000","last_fullsized":"2023-09-20T21:47:07.597247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:46:51.343332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602350+0000","last_change":"2023-09-20T21:46:41.507562+0000","last_active":"2023-09-20T21:47:07.602350+0000","last_peered":"2023-09-20T21:47:07.602350+0000","last_clean":"2023-09-20T21:47:07.602350+0000","last_became_active":"2023-09-20T21:46:41.507094+0000","last_became_peered":"2023-09-20T21:46:41.507094+0000","last_unstale":"2023-09-20T21:47:07.602350+0000","last_undegraded":"2023-09-20T21:47:07.602350+0000","last_fullsized":"2023-09-20T21:47:07.602350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:34:32.401415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598795+0000","last_change":"2023-09-20T21:46:41.502864+0000","last_active":"2023-09-20T21:47:07.598795+0000","last_peered":"2023-09-20T21:47:07.598795+0000","last_clean":"2023-09-20T21:47:07.598795+0000","last_became_active":"2023-09-20T21:46:41.502543+0000","last_became_peered":"2023-09-20T21:46:41.502543+0000","last_unstale":"2023-09-20T21:47:07.598795+0000","last_undegraded":"2023-09-20T21:47:07.598795+0000","last_fullsized":"2023-09-20T21:47:07.598795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:02:04.631874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.600445+0000","last_change":"2023-09-20T21:47:08.600445+0000","last_active":"2023-09-20T21:47:08.600445+0000","last_peered":"2023-09-20T21:47:08.600445+0000","last_clean":"2023-09-20T21:47:08.600445+0000","last_became_active":"2023-09-20T21:47:08.600054+0000","last_became_peered":"2023-09-20T21:47:08.600054+0000","last_unstale":"2023-09-20T21:47:08.600445+0000","last_undegraded":"2023-09-20T21:47:08.600445+0000","last_fullsized":"2023-09-20T21:47:08.600445+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:43:18.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593608+0000","last_change":"2023-09-20T21:46:41.502036+0000","last_active":"2023-09-20T21:47:07.593608+0000","last_peered":"2023-09-20T21:47:07.593608+0000","last_clean":"2023-09-20T21:47:07.593608+0000","last_became_active":"2023-09-20T21:46:41.501745+0000","last_became_peered":"2023-09-20T21:46:41.501745+0000","last_unstale":"2023-09-20T21:47:07.593608+0000","last_undegraded":"2023-09-20T21:47:07.593608+0000","last_fullsized":"2023-09-20T21:47:07.593608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:08:06.699543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598739+0000","last_change":"2023-09-20T21:46:41.506666+0000","last_active":"2023-09-20T21:47:07.598739+0000","last_peered":"2023-09-20T21:47:07.598739+0000","last_clean":"2023-09-20T21:47:07.598739+0000","last_became_active":"2023-09-20T21:46:41.506266+0000","last_became_peered":"2023-09-20T21:46:41.506266+0000","last_unstale":"2023-09-20T21:47:07.598739+0000","last_undegraded":"2023-09-20T21:47:07.598739+0000","last_fullsized":"2023-09-20T21:47:07.598739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:06:16.809609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592904+0000","last_change":"2023-09-20T21:46:41.497391+0000","last_active":"2023-09-20T21:47:07.592904+0000","last_peered":"2023-09-20T21:47:07.592904+0000","last_clean":"2023-09-20T21:47:07.592904+0000","last_became_active":"2023-09-20T21:46:41.497064+0000","last_became_peered":"2023-09-20T21:46:41.497064+0000","last_unstale":"2023-09-20T21:47:07.592904+0000","last_undegraded":"2023-09-20T21:47:07.592904+0000","last_fullsized":"2023-09-20T21:47:07.592904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:56:11.489585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599538+0000","last_change":"2023-09-20T21:46:41.510602+0000","last_active":"2023-09-20T21:47:07.599538+0000","last_peered":"2023-09-20T21:47:07.599538+0000","last_clean":"2023-09-20T21:47:07.599538+0000","last_became_active":"2023-09-20T21:46:41.510308+0000","last_became_peered":"2023-09-20T21:46:41.510308+0000","last_unstale":"2023-09-20T21:47:07.599538+0000","last_undegraded":"2023-09-20T21:47:07.599538+0000","last_fullsized":"2023-09-20T21:47:07.599538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:43:41.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598788+0000","last_change":"2023-09-20T21:46:45.581583+0000","last_active":"2023-09-20T21:47:07.598788+0000","last_peered":"2023-09-20T21:47:07.598788+0000","last_clean":"2023-09-20T21:47:07.598788+0000","last_became_active":"2023-09-20T21:46:45.581165+0000","last_became_peered":"2023-09-20T21:46:45.581165+0000","last_unstale":"2023-09-20T21:47:07.598788+0000","last_undegraded":"2023-09-20T21:47:07.598788+0000","last_fullsized":"2023-09-20T21:47:07.598788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:39:36.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577791+0000","last_change":"2023-09-20T21:46:40.500010+0000","last_active":"2023-09-20T21:47:08.577791+0000","last_peered":"2023-09-20T21:47:08.577791+0000","last_clean":"2023-09-20T21:47:08.577791+0000","last_became_active":"2023-09-20T21:46:40.499632+0000","last_became_peered":"2023-09-20T21:46:40.499632+0000","last_unstale":"2023-09-20T21:47:08.577791+0000","last_undegraded":"2023-09-20T21:47:08.577791+0000","last_fullsized":"2023-09-20T21:47:08.577791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:41:35.590982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595846+0000","last_change":"2023-09-20T21:46:41.516465+0000","last_active":"2023-09-20T21:47:07.595846+0000","last_peered":"2023-09-20T21:47:07.595846+0000","last_clean":"2023-09-20T21:47:07.595846+0000","last_became_active":"2023-09-20T21:46:41.516150+0000","last_became_peered":"2023-09-20T21:46:41.516150+0000","last_unstale":"2023-09-20T21:47:07.595846+0000","last_undegraded":"2023-09-20T21:47:07.595846+0000","last_fullsized":"2023-09-20T21:47:07.595846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:33:37.507049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596541+0000","last_change":"2023-09-20T21:46:45.582916+0000","last_active":"2023-09-20T21:47:07.596541+0000","last_peered":"2023-09-20T21:47:07.596541+0000","last_clean":"2023-09-20T21:47:07.596541+0000","last_became_active":"2023-09-20T21:46:45.582511+0000","last_became_peered":"2023-09-20T21:46:45.582511+0000","last_unstale":"2023-09-20T21:47:07.596541+0000","last_undegraded":"2023-09-20T21:47:07.596541+0000","last_fullsized":"2023-09-20T21:47:07.596541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:43:47.006768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596015+0000","last_change":"2023-09-20T21:46:40.516867+0000","last_active":"2023-09-20T21:47:07.596015+0000","last_peered":"2023-09-20T21:47:07.596015+0000","last_clean":"2023-09-20T21:47:07.596015+0000","last_became_active":"2023-09-20T21:46:40.516343+0000","last_became_peered":"2023-09-20T21:46:40.516343+0000","last_unstale":"2023-09-20T21:47:07.596015+0000","last_undegraded":"2023-09-20T21:47:07.596015+0000","last_fullsized":"2023-09-20T21:47:07.596015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:17:18.348921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599149+0000","last_change":"2023-09-20T21:46:41.507093+0000","last_active":"2023-09-20T21:47:07.599149+0000","last_peered":"2023-09-20T21:47:07.599149+0000","last_clean":"2023-09-20T21:47:07.599149+0000","last_became_active":"2023-09-20T21:46:41.506587+0000","last_became_peered":"2023-09-20T21:46:41.506587+0000","last_unstale":"2023-09-20T21:47:07.599149+0000","last_undegraded":"2023-09-20T21:47:07.599149+0000","last_fullsized":"2023-09-20T21:47:07.599149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:26:44.825499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597188+0000","last_change":"2023-09-20T21:46:45.583942+0000","last_active":"2023-09-20T21:47:07.597188+0000","last_peered":"2023-09-20T21:47:07.597188+0000","last_clean":"2023-09-20T21:47:07.597188+0000","last_became_active":"2023-09-20T21:46:45.583484+0000","last_became_peered":"2023-09-20T21:46:45.583484+0000","last_unstale":"2023-09-20T21:47:07.597188+0000","last_undegraded":"2023-09-20T21:47:07.597188+0000","last_fullsized":"2023-09-20T21:47:07.597188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:48.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:09.612757+0000","last_change":"2023-09-20T21:47:08.615780+0000","last_active":"2023-09-20T21:47:09.612757+0000","last_peered":"2023-09-20T21:47:09.612757+0000","last_clean":"2023-09-20T21:47:09.612757+0000","last_became_active":"2023-09-20T21:47:08.603974+0000","last_became_peered":"2023-09-20T21:47:08.603974+0000","last_unstale":"2023-09-20T21:47:09.612757+0000","last_undegraded":"2023-09-20T21:47:09.612757+0000","last_fullsized":"2023-09-20T21:47:09.612757+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:53:23.172401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594055+0000","last_change":"2023-09-20T21:46:41.504449+0000","last_active":"2023-09-20T21:47:07.594055+0000","last_peered":"2023-09-20T21:47:07.594055+0000","last_clean":"2023-09-20T21:47:07.594055+0000","last_became_active":"2023-09-20T21:46:41.504169+0000","last_became_peered":"2023-09-20T21:46:41.504169+0000","last_unstale":"2023-09-20T21:47:07.594055+0000","last_undegraded":"2023-09-20T21:47:07.594055+0000","last_fullsized":"2023-09-20T21:47:07.594055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:36:43.980902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596618+0000","last_change":"2023-09-20T21:46:45.582269+0000","last_active":"2023-09-20T21:47:07.596618+0000","last_peered":"2023-09-20T21:47:07.596618+0000","last_clean":"2023-09-20T21:47:07.596618+0000","last_became_active":"2023-09-20T21:46:45.581782+0000","last_became_peered":"2023-09-20T21:46:45.581782+0000","last_unstale":"2023-09-20T21:47:07.596618+0000","last_undegraded":"2023-09-20T21:47:07.596618+0000","last_fullsized":"2023-09-20T21:47:07.596618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:32:09.518715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.598645+0000","last_change":"2023-09-20T21:46:40.504099+0000","last_active":"2023-09-20T21:47:08.598645+0000","last_peered":"2023-09-20T21:47:08.598645+0000","last_clean":"2023-09-20T21:47:08.598645+0000","last_became_active":"2023-09-20T21:46:40.503774+0000","last_became_peered":"2023-09-20T21:46:40.503774+0000","last_unstale":"2023-09-20T21:47:08.598645+0000","last_undegraded":"2023-09-20T21:47:08.598645+0000","last_fullsized":"2023-09-20T21:47:08.598645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:17.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595338+0000","last_change":"2023-09-20T21:46:41.499802+0000","last_active":"2023-09-20T21:47:07.595338+0000","last_peered":"2023-09-20T21:47:07.595338+0000","last_clean":"2023-09-20T21:47:07.595338+0000","last_became_active":"2023-09-20T21:46:41.499491+0000","last_became_peered":"2023-09-20T21:46:41.499491+0000","last_unstale":"2023-09-20T21:47:07.595338+0000","last_undegraded":"2023-09-20T21:47:07.595338+0000","last_fullsized":"2023-09-20T21:47:07.595338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:44.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599395+0000","last_change":"2023-09-20T21:46:45.578753+0000","last_active":"2023-09-20T21:47:07.599395+0000","last_peered":"2023-09-20T21:47:07.599395+0000","last_clean":"2023-09-20T21:47:07.599395+0000","last_became_active":"2023-09-20T21:46:45.578297+0000","last_became_peered":"2023-09-20T21:46:45.578297+0000","last_unstale":"2023-09-20T21:47:07.599395+0000","last_undegraded":"2023-09-20T21:47:07.599395+0000","last_fullsized":"2023-09-20T21:47:07.599395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:18:22.596550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.907247+0000","last_change":"2023-09-20T21:46:40.505675+0000","last_active":"2023-09-20T21:47:08.907247+0000","last_peered":"2023-09-20T21:47:08.907247+0000","last_clean":"2023-09-20T21:47:08.907247+0000","last_became_active":"2023-09-20T21:46:40.505432+0000","last_became_peered":"2023-09-20T21:46:40.505432+0000","last_unstale":"2023-09-20T21:47:08.907247+0000","last_undegraded":"2023-09-20T21:47:08.907247+0000","last_fullsized":"2023-09-20T21:47:08.907247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:14:53.652904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595519+0000","last_change":"2023-09-20T21:46:41.505924+0000","last_active":"2023-09-20T21:47:07.595519+0000","last_peered":"2023-09-20T21:47:07.595519+0000","last_clean":"2023-09-20T21:47:07.595519+0000","last_became_active":"2023-09-20T21:46:41.505609+0000","last_became_peered":"2023-09-20T21:46:41.505609+0000","last_unstale":"2023-09-20T21:47:07.595519+0000","last_undegraded":"2023-09-20T21:47:07.595519+0000","last_fullsized":"2023-09-20T21:47:07.595519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:13:46.506786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600999+0000","last_change":"2023-09-20T21:46:45.587818+0000","last_active":"2023-09-20T21:47:07.600999+0000","last_peered":"2023-09-20T21:47:07.600999+0000","last_clean":"2023-09-20T21:47:07.600999+0000","last_became_active":"2023-09-20T21:46:45.587376+0000","last_became_peered":"2023-09-20T21:46:45.587376+0000","last_unstale":"2023-09-20T21:47:07.600999+0000","last_undegraded":"2023-09-20T21:47:07.600999+0000","last_fullsized":"2023-09-20T21:47:07.600999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:28:53.605158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595338+0000","last_change":"2023-09-20T21:46:40.505610+0000","last_active":"2023-09-20T21:47:07.595338+0000","last_peered":"2023-09-20T21:47:07.595338+0000","last_clean":"2023-09-20T21:47:07.595338+0000","last_became_active":"2023-09-20T21:46:40.505279+0000","last_became_peered":"2023-09-20T21:46:40.505279+0000","last_unstale":"2023-09-20T21:47:07.595338+0000","last_undegraded":"2023-09-20T21:47:07.595338+0000","last_fullsized":"2023-09-20T21:47:07.595338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:41:19.575154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.909510+0000","last_change":"2023-09-20T21:47:08.908513+0000","last_active":"2023-09-20T21:47:08.909510+0000","last_peered":"2023-09-20T21:47:08.909510+0000","last_clean":"2023-09-20T21:47:08.909510+0000","last_became_active":"2023-09-20T21:47:08.908142+0000","last_became_peered":"2023-09-20T21:47:08.908142+0000","last_unstale":"2023-09-20T21:47:08.909510+0000","last_undegraded":"2023-09-20T21:47:08.909510+0000","last_fullsized":"2023-09-20T21:47:08.909510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:31:33.992202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598682+0000","last_change":"2023-09-20T21:46:45.581738+0000","last_active":"2023-09-20T21:47:07.598682+0000","last_peered":"2023-09-20T21:47:07.598682+0000","last_clean":"2023-09-20T21:47:07.598682+0000","last_became_active":"2023-09-20T21:46:45.581279+0000","last_became_peered":"2023-09-20T21:46:45.581279+0000","last_unstale":"2023-09-20T21:47:07.598682+0000","last_undegraded":"2023-09-20T21:47:07.598682+0000","last_fullsized":"2023-09-20T21:47:07.598682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:41:41.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577335+0000","last_change":"2023-09-20T21:46:40.510539+0000","last_active":"2023-09-20T21:47:08.577335+0000","last_peered":"2023-09-20T21:47:08.577335+0000","last_clean":"2023-09-20T21:47:08.577335+0000","last_became_active":"2023-09-20T21:46:40.509880+0000","last_became_peered":"2023-09-20T21:46:40.509880+0000","last_unstale":"2023-09-20T21:47:08.577335+0000","last_undegraded":"2023-09-20T21:47:08.577335+0000","last_fullsized":"2023-09-20T21:47:08.577335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:52.877786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595248+0000","last_change":"2023-09-20T21:46:41.505105+0000","last_active":"2023-09-20T21:47:07.595248+0000","last_peered":"2023-09-20T21:47:07.595248+0000","last_clean":"2023-09-20T21:47:07.595248+0000","last_became_active":"2023-09-20T21:46:41.504785+0000","last_became_peered":"2023-09-20T21:46:41.504785+0000","last_unstale":"2023-09-20T21:47:07.595248+0000","last_undegraded":"2023-09-20T21:47:07.595248+0000","last_fullsized":"2023-09-20T21:47:07.595248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:07:57.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599298+0000","last_change":"2023-09-20T21:46:45.583160+0000","last_active":"2023-09-20T21:47:07.599298+0000","last_peered":"2023-09-20T21:47:07.599298+0000","last_clean":"2023-09-20T21:47:07.599298+0000","last_became_active":"2023-09-20T21:46:45.582709+0000","last_became_peered":"2023-09-20T21:46:45.582709+0000","last_unstale":"2023-09-20T21:47:07.599298+0000","last_undegraded":"2023-09-20T21:47:07.599298+0000","last_fullsized":"2023-09-20T21:47:07.599298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:24:57.563682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.906996+0000","last_change":"2023-09-20T21:46:40.502591+0000","last_active":"2023-09-20T21:47:08.906996+0000","last_peered":"2023-09-20T21:47:08.906996+0000","last_clean":"2023-09-20T21:47:08.906996+0000","last_became_active":"2023-09-20T21:46:40.502288+0000","last_became_peered":"2023-09-20T21:46:40.502288+0000","last_unstale":"2023-09-20T21:47:08.906996+0000","last_undegraded":"2023-09-20T21:47:08.906996+0000","last_fullsized":"2023-09-20T21:47:08.906996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:09:37.983866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.599912+0000","last_change":"2023-09-20T21:47:08.599912+0000","last_active":"2023-09-20T21:47:08.599912+0000","last_peered":"2023-09-20T21:47:08.599912+0000","last_clean":"2023-09-20T21:47:08.599912+0000","last_became_active":"2023-09-20T21:47:08.599434+0000","last_became_peered":"2023-09-20T21:47:08.599434+0000","last_unstale":"2023-09-20T21:47:08.599912+0000","last_undegraded":"2023-09-20T21:47:08.599912+0000","last_fullsized":"2023-09-20T21:47:08.599912+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:37:01.125731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595936+0000","last_change":"2023-09-20T21:46:45.580917+0000","last_active":"2023-09-20T21:47:07.595936+0000","last_peered":"2023-09-20T21:47:07.595936+0000","last_clean":"2023-09-20T21:47:07.595936+0000","last_became_active":"2023-09-20T21:46:45.580505+0000","last_became_peered":"2023-09-20T21:46:45.580505+0000","last_unstale":"2023-09-20T21:47:07.595936+0000","last_undegraded":"2023-09-20T21:47:07.595936+0000","last_fullsized":"2023-09-20T21:47:07.595936+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:52:19.733919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:10.571608+0000","last_change":"2023-09-20T21:46:40.507040+0000","last_active":"2023-09-20T21:47:10.571608+0000","last_peered":"2023-09-20T21:47:10.571608+0000","last_clean":"2023-09-20T21:47:10.571608+0000","last_became_active":"2023-09-20T21:46:40.506712+0000","last_became_peered":"2023-09-20T21:46:40.506712+0000","last_unstale":"2023-09-20T21:47:10.571608+0000","last_undegraded":"2023-09-20T21:47:10.571608+0000","last_fullsized":"2023-09-20T21:47:10.571608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:02:56.040911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594395+0000","last_change":"2023-09-20T21:46:41.504794+0000","last_active":"2023-09-20T21:47:07.594395+0000","last_peered":"2023-09-20T21:47:07.594395+0000","last_clean":"2023-09-20T21:47:07.594395+0000","last_became_active":"2023-09-20T21:46:41.504372+0000","last_became_peered":"2023-09-20T21:46:41.504372+0000","last_unstale":"2023-09-20T21:47:07.594395+0000","last_undegraded":"2023-09-20T21:47:07.594395+0000","last_fullsized":"2023-09-20T21:47:07.594395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:42:20.527306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592403+0000","last_change":"2023-09-20T21:46:45.586364+0000","last_active":"2023-09-20T21:47:07.592403+0000","last_peered":"2023-09-20T21:47:07.592403+0000","last_clean":"2023-09-20T21:47:07.592403+0000","last_became_active":"2023-09-20T21:46:45.584620+0000","last_became_peered":"2023-09-20T21:46:45.584620+0000","last_unstale":"2023-09-20T21:47:07.592403+0000","last_undegraded":"2023-09-20T21:47:07.592403+0000","last_fullsized":"2023-09-20T21:47:07.592403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:05:20.158118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577704+0000","last_change":"2023-09-20T21:46:40.500797+0000","last_active":"2023-09-20T21:47:08.577704+0000","last_peered":"2023-09-20T21:47:08.577704+0000","last_clean":"2023-09-20T21:47:08.577704+0000","last_became_active":"2023-09-20T21:46:40.500337+0000","last_became_peered":"2023-09-20T21:46:40.500337+0000","last_unstale":"2023-09-20T21:47:08.577704+0000","last_undegraded":"2023-09-20T21:47:08.577704+0000","last_fullsized":"2023-09-20T21:47:08.577704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:19.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.908615+0000","last_change":"2023-09-20T21:47:08.908615+0000","last_active":"2023-09-20T21:47:08.908615+0000","last_peered":"2023-09-20T21:47:08.908615+0000","last_clean":"2023-09-20T21:47:08.908615+0000","last_became_active":"2023-09-20T21:47:08.908228+0000","last_became_peered":"2023-09-20T21:47:08.908228+0000","last_unstale":"2023-09-20T21:47:08.908615+0000","last_undegraded":"2023-09-20T21:47:08.908615+0000","last_fullsized":"2023-09-20T21:47:08.908615+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:15:28.772129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597442+0000","last_change":"2023-09-20T21:46:45.583081+0000","last_active":"2023-09-20T21:47:07.597442+0000","last_peered":"2023-09-20T21:47:07.597442+0000","last_clean":"2023-09-20T21:47:07.597442+0000","last_became_active":"2023-09-20T21:46:45.580496+0000","last_became_peered":"2023-09-20T21:46:45.580496+0000","last_unstale":"2023-09-20T21:47:07.597442+0000","last_undegraded":"2023-09-20T21:47:07.597442+0000","last_fullsized":"2023-09-20T21:47:07.597442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:42.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595250+0000","last_change":"2023-09-20T21:46:40.512739+0000","last_active":"2023-09-20T21:47:07.595250+0000","last_peered":"2023-09-20T21:47:07.595250+0000","last_clean":"2023-09-20T21:47:07.595250+0000","last_became_active":"2023-09-20T21:46:40.512485+0000","last_became_peered":"2023-09-20T21:46:40.512485+0000","last_unstale":"2023-09-20T21:47:07.595250+0000","last_undegraded":"2023-09-20T21:47:07.595250+0000","last_fullsized":"2023-09-20T21:47:07.595250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:02:49.392560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.604682+0000","last_change":"2023-09-20T21:47:08.604682+0000","last_active":"2023-09-20T21:47:08.604682+0000","last_peered":"2023-09-20T21:47:08.604682+0000","last_clean":"2023-09-20T21:47:08.604682+0000","last_became_active":"2023-09-20T21:47:08.604271+0000","last_became_peered":"2023-09-20T21:47:08.604271+0000","last_unstale":"2023-09-20T21:47:08.604682+0000","last_undegraded":"2023-09-20T21:47:08.604682+0000","last_fullsized":"2023-09-20T21:47:08.604682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:40:51.542781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592702+0000","last_change":"2023-09-20T21:46:45.579478+0000","last_active":"2023-09-20T21:47:07.592702+0000","last_peered":"2023-09-20T21:47:07.592702+0000","last_clean":"2023-09-20T21:47:07.592702+0000","last_became_active":"2023-09-20T21:46:45.579055+0000","last_became_peered":"2023-09-20T21:46:45.579055+0000","last_unstale":"2023-09-20T21:47:07.592702+0000","last_undegraded":"2023-09-20T21:47:07.592702+0000","last_fullsized":"2023-09-20T21:47:07.592702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:17:32.013280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'14","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:10.580044+0000","last_change":"2023-09-20T21:46:40.507321+0000","last_active":"2023-09-20T21:47:10.580044+0000","last_peered":"2023-09-20T21:47:10.580044+0000","last_clean":"2023-09-20T21:47:10.580044+0000","last_became_active":"2023-09-20T21:46:40.506983+0000","last_became_peered":"2023-09-20T21:46:40.506983+0000","last_unstale":"2023-09-20T21:47:10.580044+0000","last_undegraded":"2023-09-20T21:47:10.580044+0000","last_fullsized":"2023-09-20T21:47:10.580044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:46:40.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596051+0000","last_change":"2023-09-20T21:46:41.509646+0000","last_active":"2023-09-20T21:47:07.596051+0000","last_peered":"2023-09-20T21:47:07.596051+0000","last_clean":"2023-09-20T21:47:07.596051+0000","last_became_active":"2023-09-20T21:46:41.509183+0000","last_became_peered":"2023-09-20T21:46:41.509183+0000","last_unstale":"2023-09-20T21:47:07.596051+0000","last_undegraded":"2023-09-20T21:47:07.596051+0000","last_fullsized":"2023-09-20T21:47:07.596051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:17:21.084830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595627+0000","last_change":"2023-09-20T21:46:45.583709+0000","last_active":"2023-09-20T21:47:07.595627+0000","last_peered":"2023-09-20T21:47:07.595627+0000","last_clean":"2023-09-20T21:47:07.595627+0000","last_became_active":"2023-09-20T21:46:45.583302+0000","last_became_peered":"2023-09-20T21:46:45.583302+0000","last_unstale":"2023-09-20T21:47:07.595627+0000","last_undegraded":"2023-09-20T21:47:07.595627+0000","last_fullsized":"2023-09-20T21:47:07.595627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:31:35.472750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.594685+0000","last_change":"2023-09-20T21:46:40.506411+0000","last_active":"2023-09-20T21:47:08.594685+0000","last_peered":"2023-09-20T21:47:08.594685+0000","last_clean":"2023-09-20T21:47:08.594685+0000","last_became_active":"2023-09-20T21:46:40.506116+0000","last_became_peered":"2023-09-20T21:46:40.506116+0000","last_unstale":"2023-09-20T21:47:08.594685+0000","last_undegraded":"2023-09-20T21:47:08.594685+0000","last_fullsized":"2023-09-20T21:47:08.594685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:23:19.175007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602083+0000","last_change":"2023-09-20T21:46:41.513793+0000","last_active":"2023-09-20T21:47:07.602083+0000","last_peered":"2023-09-20T21:47:07.602083+0000","last_clean":"2023-09-20T21:47:07.602083+0000","last_became_active":"2023-09-20T21:46:41.513434+0000","last_became_peered":"2023-09-20T21:46:41.513434+0000","last_unstale":"2023-09-20T21:47:07.602083+0000","last_undegraded":"2023-09-20T21:47:07.602083+0000","last_fullsized":"2023-09-20T21:47:07.602083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:00:57.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598091+0000","last_change":"2023-09-20T21:46:45.578188+0000","last_active":"2023-09-20T21:47:07.598091+0000","last_peered":"2023-09-20T21:47:07.598091+0000","last_clean":"2023-09-20T21:47:07.598091+0000","last_became_active":"2023-09-20T21:46:45.577548+0000","last_became_peered":"2023-09-20T21:46:45.577548+0000","last_unstale":"2023-09-20T21:47:07.598091+0000","last_undegraded":"2023-09-20T21:47:07.598091+0000","last_fullsized":"2023-09-20T21:47:07.598091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:54:58.894269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.907407+0000","last_change":"2023-09-20T21:46:40.505465+0000","last_active":"2023-09-20T21:47:08.907407+0000","last_peered":"2023-09-20T21:47:08.907407+0000","last_clean":"2023-09-20T21:47:08.907407+0000","last_became_active":"2023-09-20T21:46:40.505174+0000","last_became_peered":"2023-09-20T21:46:40.505174+0000","last_unstale":"2023-09-20T21:47:08.907407+0000","last_undegraded":"2023-09-20T21:47:08.907407+0000","last_fullsized":"2023-09-20T21:47:08.907407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:50:53.191963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598850+0000","last_change":"2023-09-20T21:46:41.507370+0000","last_active":"2023-09-20T21:47:07.598850+0000","last_peered":"2023-09-20T21:47:07.598850+0000","last_clean":"2023-09-20T21:47:07.598850+0000","last_became_active":"2023-09-20T21:46:41.507035+0000","last_became_peered":"2023-09-20T21:46:41.507035+0000","last_unstale":"2023-09-20T21:47:07.598850+0000","last_undegraded":"2023-09-20T21:47:07.598850+0000","last_fullsized":"2023-09-20T21:47:07.598850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:50.298899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592822+0000","last_change":"2023-09-20T21:46:45.584458+0000","last_active":"2023-09-20T21:47:07.592822+0000","last_peered":"2023-09-20T21:47:07.592822+0000","last_clean":"2023-09-20T21:47:07.592822+0000","last_became_active":"2023-09-20T21:46:45.584033+0000","last_became_peered":"2023-09-20T21:46:45.584033+0000","last_unstale":"2023-09-20T21:47:07.592822+0000","last_undegraded":"2023-09-20T21:47:07.592822+0000","last_fullsized":"2023-09-20T21:47:07.592822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:48:44.653880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.567221+0000","last_change":"2023-09-20T21:46:40.501329+0000","last_active":"2023-09-20T21:47:08.567221+0000","last_peered":"2023-09-20T21:47:08.567221+0000","last_clean":"2023-09-20T21:47:08.567221+0000","last_became_active":"2023-09-20T21:46:40.500999+0000","last_became_peered":"2023-09-20T21:46:40.500999+0000","last_unstale":"2023-09-20T21:47:08.567221+0000","last_undegraded":"2023-09-20T21:47:08.567221+0000","last_fullsized":"2023-09-20T21:47:08.567221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:38.028398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.602123+0000","last_change":"2023-09-20T21:47:08.602123+0000","last_active":"2023-09-20T21:47:08.602123+0000","last_peered":"2023-09-20T21:47:08.602123+0000","last_clean":"2023-09-20T21:47:08.602123+0000","last_became_active":"2023-09-20T21:47:08.601738+0000","last_became_peered":"2023-09-20T21:47:08.601738+0000","last_unstale":"2023-09-20T21:47:08.602123+0000","last_undegraded":"2023-09-20T21:47:08.602123+0000","last_fullsized":"2023-09-20T21:47:08.602123+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:26:36.710230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602591+0000","last_change":"2023-09-20T21:46:45.587116+0000","last_active":"2023-09-20T21:47:07.602591+0000","last_peered":"2023-09-20T21:47:07.602591+0000","last_clean":"2023-09-20T21:47:07.602591+0000","last_became_active":"2023-09-20T21:46:45.586629+0000","last_became_peered":"2023-09-20T21:46:45.586629+0000","last_unstale":"2023-09-20T21:47:07.602591+0000","last_undegraded":"2023-09-20T21:47:07.602591+0000","last_fullsized":"2023-09-20T21:47:07.602591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:10:33.024235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577220+0000","last_change":"2023-09-20T21:46:40.499985+0000","last_active":"2023-09-20T21:47:08.577220+0000","last_peered":"2023-09-20T21:47:08.577220+0000","last_clean":"2023-09-20T21:47:08.577220+0000","last_became_active":"2023-09-20T21:46:40.499646+0000","last_became_peered":"2023-09-20T21:46:40.499646+0000","last_unstale":"2023-09-20T21:47:08.577220+0000","last_undegraded":"2023-09-20T21:47:08.577220+0000","last_fullsized":"2023-09-20T21:47:08.577220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:52:02.758556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594909+0000","last_change":"2023-09-20T21:46:41.501241+0000","last_active":"2023-09-20T21:47:07.594909+0000","last_peered":"2023-09-20T21:47:07.594909+0000","last_clean":"2023-09-20T21:47:07.594909+0000","last_became_active":"2023-09-20T21:46:41.500902+0000","last_became_peered":"2023-09-20T21:46:41.500902+0000","last_unstale":"2023-09-20T21:47:07.594909+0000","last_undegraded":"2023-09-20T21:47:07.594909+0000","last_fullsized":"2023-09-20T21:47:07.594909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:21:55.210065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594812+0000","last_change":"2023-09-20T21:46:45.583405+0000","last_active":"2023-09-20T21:47:07.594812+0000","last_peered":"2023-09-20T21:47:07.594812+0000","last_clean":"2023-09-20T21:47:07.594812+0000","last_became_active":"2023-09-20T21:46:45.582945+0000","last_became_peered":"2023-09-20T21:46:45.582945+0000","last_unstale":"2023-09-20T21:47:07.594812+0000","last_undegraded":"2023-09-20T21:47:07.594812+0000","last_fullsized":"2023-09-20T21:47:07.594812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:27:14.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.598738+0000","last_change":"2023-09-20T21:46:40.507744+0000","last_active":"2023-09-20T21:47:08.598738+0000","last_peered":"2023-09-20T21:47:08.598738+0000","last_clean":"2023-09-20T21:47:08.598738+0000","last_became_active":"2023-09-20T21:46:40.507454+0000","last_became_peered":"2023-09-20T21:46:40.507454+0000","last_unstale":"2023-09-20T21:47:08.598738+0000","last_undegraded":"2023-09-20T21:47:08.598738+0000","last_fullsized":"2023-09-20T21:47:08.598738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:00:24.492789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598829+0000","last_change":"2023-09-20T21:46:41.504263+0000","last_active":"2023-09-20T21:47:07.598829+0000","last_peered":"2023-09-20T21:47:07.598829+0000","last_clean":"2023-09-20T21:47:07.598829+0000","last_became_active":"2023-09-20T21:46:41.503961+0000","last_became_peered":"2023-09-20T21:46:41.503961+0000","last_unstale":"2023-09-20T21:47:07.598829+0000","last_undegraded":"2023-09-20T21:47:07.598829+0000","last_fullsized":"2023-09-20T21:47:07.598829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:31:58.035517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595426+0000","last_change":"2023-09-20T21:46:45.581343+0000","last_active":"2023-09-20T21:47:07.595426+0000","last_peered":"2023-09-20T21:47:07.595426+0000","last_clean":"2023-09-20T21:47:07.595426+0000","last_became_active":"2023-09-20T21:46:45.580830+0000","last_became_peered":"2023-09-20T21:46:45.580830+0000","last_unstale":"2023-09-20T21:47:07.595426+0000","last_undegraded":"2023-09-20T21:47:07.595426+0000","last_fullsized":"2023-09-20T21:47:07.595426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:20:49.865825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592677+0000","last_change":"2023-09-20T21:46:40.501850+0000","last_active":"2023-09-20T21:47:07.592677+0000","last_peered":"2023-09-20T21:47:07.592677+0000","last_clean":"2023-09-20T21:47:07.592677+0000","last_became_active":"2023-09-20T21:46:40.501500+0000","last_became_peered":"2023-09-20T21:46:40.501500+0000","last_unstale":"2023-09-20T21:47:07.592677+0000","last_undegraded":"2023-09-20T21:47:07.592677+0000","last_fullsized":"2023-09-20T21:47:07.592677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:14:15.907335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595648+0000","last_change":"2023-09-20T21:46:41.507432+0000","last_active":"2023-09-20T21:47:07.595648+0000","last_peered":"2023-09-20T21:47:07.595648+0000","last_clean":"2023-09-20T21:47:07.595648+0000","last_became_active":"2023-09-20T21:46:41.507119+0000","last_became_peered":"2023-09-20T21:46:41.507119+0000","last_unstale":"2023-09-20T21:47:07.595648+0000","last_undegraded":"2023-09-20T21:47:07.595648+0000","last_fullsized":"2023-09-20T21:47:07.595648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:14:10.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602550+0000","last_change":"2023-09-20T21:46:45.588455+0000","last_active":"2023-09-20T21:47:07.602550+0000","last_peered":"2023-09-20T21:47:07.602550+0000","last_clean":"2023-09-20T21:47:07.602550+0000","last_became_active":"2023-09-20T21:46:45.587980+0000","last_became_peered":"2023-09-20T21:46:45.587980+0000","last_unstale":"2023-09-20T21:47:07.602550+0000","last_undegraded":"2023-09-20T21:47:07.602550+0000","last_fullsized":"2023-09-20T21:47:07.602550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:07:35.697330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:10.618744+0000","last_change":"2023-09-20T21:46:40.506260+0000","last_active":"2023-09-20T21:47:10.618744+0000","last_peered":"2023-09-20T21:47:10.618744+0000","last_clean":"2023-09-20T21:47:10.618744+0000","last_became_active":"2023-09-20T21:46:40.505999+0000","last_became_peered":"2023-09-20T21:46:40.505999+0000","last_unstale":"2023-09-20T21:47:10.618744+0000","last_undegraded":"2023-09-20T21:47:10.618744+0000","last_fullsized":"2023-09-20T21:47:10.618744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:39:01.748038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.599481+0000","last_change":"2023-09-20T21:47:08.599481+0000","last_active":"2023-09-20T21:47:08.599481+0000","last_peered":"2023-09-20T21:47:08.599481+0000","last_clean":"2023-09-20T21:47:08.599481+0000","last_became_active":"2023-09-20T21:47:08.599104+0000","last_became_peered":"2023-09-20T21:47:08.599104+0000","last_unstale":"2023-09-20T21:47:08.599481+0000","last_undegraded":"2023-09-20T21:47:08.599481+0000","last_fullsized":"2023-09-20T21:47:08.599481+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:33:56.845446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602710+0000","last_change":"2023-09-20T21:46:45.587410+0000","last_active":"2023-09-20T21:47:07.602710+0000","last_peered":"2023-09-20T21:47:07.602710+0000","last_clean":"2023-09-20T21:47:07.602710+0000","last_became_active":"2023-09-20T21:46:45.586944+0000","last_became_peered":"2023-09-20T21:46:45.586944+0000","last_unstale":"2023-09-20T21:47:07.602710+0000","last_undegraded":"2023-09-20T21:47:07.602710+0000","last_fullsized":"2023-09-20T21:47:07.602710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:57.053765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594876+0000","last_change":"2023-09-20T21:46:41.511526+0000","last_active":"2023-09-20T21:47:07.594876+0000","last_peered":"2023-09-20T21:47:07.594876+0000","last_clean":"2023-09-20T21:47:07.594876+0000","last_became_active":"2023-09-20T21:46:41.511213+0000","last_became_peered":"2023-09-20T21:46:41.511213+0000","last_unstale":"2023-09-20T21:47:07.594876+0000","last_undegraded":"2023-09-20T21:47:07.594876+0000","last_fullsized":"2023-09-20T21:47:07.594876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:29:04.199190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:09.567879+0000","last_change":"2023-09-20T21:46:40.514916+0000","last_active":"2023-09-20T21:47:09.567879+0000","last_peered":"2023-09-20T21:47:09.567879+0000","last_clean":"2023-09-20T21:47:09.567879+0000","last_became_active":"2023-09-20T21:46:40.514553+0000","last_became_peered":"2023-09-20T21:46:40.514553+0000","last_unstale":"2023-09-20T21:47:09.567879+0000","last_undegraded":"2023-09-20T21:47:09.567879+0000","last_fullsized":"2023-09-20T21:47:09.567879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:12:32.481852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599672+0000","last_change":"2023-09-20T21:46:45.587489+0000","last_active":"2023-09-20T21:47:07.599672+0000","last_peered":"2023-09-20T21:47:07.599672+0000","last_clean":"2023-09-20T21:47:07.599672+0000","last_became_active":"2023-09-20T21:46:45.587022+0000","last_became_peered":"2023-09-20T21:46:45.587022+0000","last_unstale":"2023-09-20T21:47:07.599672+0000","last_undegraded":"2023-09-20T21:47:07.599672+0000","last_fullsized":"2023-09-20T21:47:07.599672+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:09:05.870763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595937+0000","last_change":"2023-09-20T21:46:41.508886+0000","last_active":"2023-09-20T21:47:07.595937+0000","last_peered":"2023-09-20T21:47:07.595937+0000","last_clean":"2023-09-20T21:47:07.595937+0000","last_became_active":"2023-09-20T21:46:41.508470+0000","last_became_peered":"2023-09-20T21:46:41.508470+0000","last_unstale":"2023-09-20T21:47:07.595937+0000","last_undegraded":"2023-09-20T21:47:07.595937+0000","last_fullsized":"2023-09-20T21:47:07.595937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:45:04.888300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.576942+0000","last_change":"2023-09-20T21:46:40.503350+0000","last_active":"2023-09-20T21:47:08.576942+0000","last_peered":"2023-09-20T21:47:08.576942+0000","last_clean":"2023-09-20T21:47:08.576942+0000","last_became_active":"2023-09-20T21:46:40.503045+0000","last_became_peered":"2023-09-20T21:46:40.503045+0000","last_unstale":"2023-09-20T21:47:08.576942+0000","last_undegraded":"2023-09-20T21:47:08.576942+0000","last_fullsized":"2023-09-20T21:47:08.576942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:31:09.004342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":315,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599438+0000","last_change":"2023-09-20T21:45:35.180242+0000","last_active":"2023-09-20T21:47:07.599438+0000","last_peered":"2023-09-20T21:47:07.599438+0000","last_clean":"2023-09-20T21:47:07.599438+0000","last_became_active":"2023-09-20T21:45:35.179705+0000","last_became_peered":"2023-09-20T21:45:35.179705+0000","last_unstale":"2023-09-20T21:47:07.599438+0000","last_undegraded":"2023-09-20T21:47:07.599438+0000","last_fullsized":"2023-09-20T21:47:07.599438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_clean_scrub_stamp":"2023-09-20T21:42:25.826683+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:02:03.448188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594840+0000","last_change":"2023-09-20T21:46:45.581396+0000","last_active":"2023-09-20T21:47:07.594840+0000","last_peered":"2023-09-20T21:47:07.594840+0000","last_clean":"2023-09-20T21:47:07.594840+0000","last_became_active":"2023-09-20T21:46:45.581018+0000","last_became_peered":"2023-09-20T21:46:45.581018+0000","last_unstale":"2023-09-20T21:47:07.594840+0000","last_undegraded":"2023-09-20T21:47:07.594840+0000","last_fullsized":"2023-09-20T21:47:07.594840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:16:35.934460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.601645+0000","last_change":"2023-09-20T21:46:41.514392+0000","last_active":"2023-09-20T21:47:07.601645+0000","last_peered":"2023-09-20T21:47:07.601645+0000","last_clean":"2023-09-20T21:47:07.601645+0000","last_became_active":"2023-09-20T21:46:41.514090+0000","last_became_peered":"2023-09-20T21:46:41.514090+0000","last_unstale":"2023-09-20T21:47:07.601645+0000","last_undegraded":"2023-09-20T21:47:07.601645+0000","last_fullsized":"2023-09-20T21:47:07.601645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:05:47.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577137+0000","last_change":"2023-09-20T21:46:40.507346+0000","last_active":"2023-09-20T21:47:08.577137+0000","last_peered":"2023-09-20T21:47:08.577137+0000","last_clean":"2023-09-20T21:47:08.577137+0000","last_became_active":"2023-09-20T21:46:40.506969+0000","last_became_peered":"2023-09-20T21:46:40.506969+0000","last_unstale":"2023-09-20T21:47:08.577137+0000","last_undegraded":"2023-09-20T21:47:08.577137+0000","last_fullsized":"2023-09-20T21:47:08.577137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:01:20.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600062+0000","last_change":"2023-09-20T21:46:45.578033+0000","last_active":"2023-09-20T21:47:07.600062+0000","last_peered":"2023-09-20T21:47:07.600062+0000","last_clean":"2023-09-20T21:47:07.600062+0000","last_became_active":"2023-09-20T21:46:45.577540+0000","last_became_peered":"2023-09-20T21:46:45.577540+0000","last_unstale":"2023-09-20T21:47:07.600062+0000","last_undegraded":"2023-09-20T21:47:07.600062+0000","last_fullsized":"2023-09-20T21:47:07.600062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:24:31.020660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577276+0000","last_change":"2023-09-20T21:46:40.510634+0000","last_active":"2023-09-20T21:47:08.577276+0000","last_peered":"2023-09-20T21:47:08.577276+0000","last_clean":"2023-09-20T21:47:08.577276+0000","last_became_active":"2023-09-20T21:46:40.510390+0000","last_became_peered":"2023-09-20T21:46:40.510390+0000","last_unstale":"2023-09-20T21:47:08.577276+0000","last_undegraded":"2023-09-20T21:47:08.577276+0000","last_fullsized":"2023-09-20T21:47:08.577276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:44:49.272987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602533+0000","last_change":"2023-09-20T21:46:41.502981+0000","last_active":"2023-09-20T21:47:07.602533+0000","last_peered":"2023-09-20T21:47:07.602533+0000","last_clean":"2023-09-20T21:47:07.602533+0000","last_became_active":"2023-09-20T21:46:41.502594+0000","last_became_peered":"2023-09-20T21:46:41.502594+0000","last_unstale":"2023-09-20T21:47:07.602533+0000","last_undegraded":"2023-09-20T21:47:07.602533+0000","last_fullsized":"2023-09-20T21:47:07.602533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:41:15.261478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595222+0000","last_change":"2023-09-20T21:46:45.582903+0000","last_active":"2023-09-20T21:47:07.595222+0000","last_peered":"2023-09-20T21:47:07.595222+0000","last_clean":"2023-09-20T21:47:07.595222+0000","last_became_active":"2023-09-20T21:46:45.582433+0000","last_became_peered":"2023-09-20T21:46:45.582433+0000","last_unstale":"2023-09-20T21:47:07.595222+0000","last_undegraded":"2023-09-20T21:47:07.595222+0000","last_fullsized":"2023-09-20T21:47:07.595222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:38:22.174903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592961+0000","last_change":"2023-09-20T21:46:40.507671+0000","last_active":"2023-09-20T21:47:07.592961+0000","last_peered":"2023-09-20T21:47:07.592961+0000","last_clean":"2023-09-20T21:47:07.592961+0000","last_became_active":"2023-09-20T21:46:40.507288+0000","last_became_peered":"2023-09-20T21:46:40.507288+0000","last_unstale":"2023-09-20T21:47:07.592961+0000","last_undegraded":"2023-09-20T21:47:07.592961+0000","last_fullsized":"2023-09-20T21:47:07.592961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:27:02.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596349+0000","last_change":"2023-09-20T21:46:41.507654+0000","last_active":"2023-09-20T21:47:07.596349+0000","last_peered":"2023-09-20T21:47:07.596349+0000","last_clean":"2023-09-20T21:47:07.596349+0000","last_became_active":"2023-09-20T21:46:41.507362+0000","last_became_peered":"2023-09-20T21:46:41.507362+0000","last_unstale":"2023-09-20T21:47:07.596349+0000","last_undegraded":"2023-09-20T21:47:07.596349+0000","last_fullsized":"2023-09-20T21:47:07.596349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:32:08.973379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598688+0000","last_change":"2023-09-20T21:46:45.578437+0000","last_active":"2023-09-20T21:47:07.598688+0000","last_peered":"2023-09-20T21:47:07.598688+0000","last_clean":"2023-09-20T21:47:07.598688+0000","last_became_active":"2023-09-20T21:46:45.578010+0000","last_became_peered":"2023-09-20T21:46:45.578010+0000","last_unstale":"2023-09-20T21:47:07.598688+0000","last_undegraded":"2023-09-20T21:47:07.598688+0000","last_fullsized":"2023-09-20T21:47:07.598688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:59:05.494861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577934+0000","last_change":"2023-09-20T21:46:40.508937+0000","last_active":"2023-09-20T21:47:08.577934+0000","last_peered":"2023-09-20T21:47:08.577934+0000","last_clean":"2023-09-20T21:47:08.577934+0000","last_became_active":"2023-09-20T21:46:40.508639+0000","last_became_peered":"2023-09-20T21:46:40.508639+0000","last_unstale":"2023-09-20T21:47:08.577934+0000","last_undegraded":"2023-09-20T21:47:08.577934+0000","last_fullsized":"2023-09-20T21:47:08.577934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:50:48.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599421+0000","last_change":"2023-09-20T21:46:41.510702+0000","last_active":"2023-09-20T21:47:07.599421+0000","last_peered":"2023-09-20T21:47:07.599421+0000","last_clean":"2023-09-20T21:47:07.599421+0000","last_became_active":"2023-09-20T21:46:41.510357+0000","last_became_peered":"2023-09-20T21:46:41.510357+0000","last_unstale":"2023-09-20T21:47:07.599421+0000","last_undegraded":"2023-09-20T21:47:07.599421+0000","last_fullsized":"2023-09-20T21:47:07.599421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:35:44.455981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596780+0000","last_change":"2023-09-20T21:46:45.581110+0000","last_active":"2023-09-20T21:47:07.596780+0000","last_peered":"2023-09-20T21:47:07.596780+0000","last_clean":"2023-09-20T21:47:07.596780+0000","last_became_active":"2023-09-20T21:46:45.580498+0000","last_became_peered":"2023-09-20T21:46:45.580498+0000","last_unstale":"2023-09-20T21:47:07.596780+0000","last_undegraded":"2023-09-20T21:47:07.596780+0000","last_fullsized":"2023-09-20T21:47:07.596780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:19:22.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593487+0000","last_change":"2023-09-20T21:46:40.497017+0000","last_active":"2023-09-20T21:47:07.593487+0000","last_peered":"2023-09-20T21:47:07.593487+0000","last_clean":"2023-09-20T21:47:07.593487+0000","last_became_active":"2023-09-20T21:46:40.496722+0000","last_became_peered":"2023-09-20T21:46:40.496722+0000","last_unstale":"2023-09-20T21:47:07.593487+0000","last_undegraded":"2023-09-20T21:47:07.593487+0000","last_fullsized":"2023-09-20T21:47:07.593487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:12:07.017318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596148+0000","last_change":"2023-09-20T21:46:41.508793+0000","last_active":"2023-09-20T21:47:07.596148+0000","last_peered":"2023-09-20T21:47:07.596148+0000","last_clean":"2023-09-20T21:47:07.596148+0000","last_became_active":"2023-09-20T21:46:41.508453+0000","last_became_peered":"2023-09-20T21:46:41.508453+0000","last_unstale":"2023-09-20T21:47:07.596148+0000","last_undegraded":"2023-09-20T21:47:07.596148+0000","last_fullsized":"2023-09-20T21:47:07.596148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:35:12.711570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592993+0000","last_change":"2023-09-20T21:46:45.583519+0000","last_active":"2023-09-20T21:47:07.592993+0000","last_peered":"2023-09-20T21:47:07.592993+0000","last_clean":"2023-09-20T21:47:07.592993+0000","last_became_active":"2023-09-20T21:46:45.582933+0000","last_became_peered":"2023-09-20T21:46:45.582933+0000","last_unstale":"2023-09-20T21:47:07.592993+0000","last_undegraded":"2023-09-20T21:47:07.592993+0000","last_fullsized":"2023-09-20T21:47:07.592993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:13:20.237865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.578270+0000","last_change":"2023-09-20T21:46:40.507384+0000","last_active":"2023-09-20T21:47:08.578270+0000","last_peered":"2023-09-20T21:47:08.578270+0000","last_clean":"2023-09-20T21:47:08.578270+0000","last_became_active":"2023-09-20T21:46:40.507021+0000","last_became_peered":"2023-09-20T21:46:40.507021+0000","last_unstale":"2023-09-20T21:47:08.578270+0000","last_undegraded":"2023-09-20T21:47:08.578270+0000","last_fullsized":"2023-09-20T21:47:08.578270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:48:04.360664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598093+0000","last_change":"2023-09-20T21:46:41.511537+0000","last_active":"2023-09-20T21:47:07.598093+0000","last_peered":"2023-09-20T21:47:07.598093+0000","last_clean":"2023-09-20T21:47:07.598093+0000","last_became_active":"2023-09-20T21:46:41.511238+0000","last_became_peered":"2023-09-20T21:46:41.511238+0000","last_unstale":"2023-09-20T21:47:07.598093+0000","last_undegraded":"2023-09-20T21:47:07.598093+0000","last_fullsized":"2023-09-20T21:47:07.598093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:01:41.492824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596769+0000","last_change":"2023-09-20T21:46:45.585750+0000","last_active":"2023-09-20T21:47:07.596769+0000","last_peered":"2023-09-20T21:47:07.596769+0000","last_clean":"2023-09-20T21:47:07.596769+0000","last_became_active":"2023-09-20T21:46:45.585213+0000","last_became_peered":"2023-09-20T21:46:45.585213+0000","last_unstale":"2023-09-20T21:47:07.596769+0000","last_undegraded":"2023-09-20T21:47:07.596769+0000","last_fullsized":"2023-09-20T21:47:07.596769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:31:15.650766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.594299+0000","last_change":"2023-09-20T21:46:40.506394+0000","last_active":"2023-09-20T21:47:08.594299+0000","last_peered":"2023-09-20T21:47:08.594299+0000","last_clean":"2023-09-20T21:47:08.594299+0000","last_became_active":"2023-09-20T21:46:40.506085+0000","last_became_peered":"2023-09-20T21:46:40.506085+0000","last_unstale":"2023-09-20T21:47:08.594299+0000","last_undegraded":"2023-09-20T21:47:08.594299+0000","last_fullsized":"2023-09-20T21:47:08.594299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:02.738600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596347+0000","last_change":"2023-09-20T21:46:41.509491+0000","last_active":"2023-09-20T21:47:07.596347+0000","last_peered":"2023-09-20T21:47:07.596347+0000","last_clean":"2023-09-20T21:47:07.596347+0000","last_became_active":"2023-09-20T21:46:41.509039+0000","last_became_peered":"2023-09-20T21:46:41.509039+0000","last_unstale":"2023-09-20T21:47:07.596347+0000","last_undegraded":"2023-09-20T21:47:07.596347+0000","last_fullsized":"2023-09-20T21:47:07.596347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:57:39.003154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597192+0000","last_change":"2023-09-20T21:46:45.581651+0000","last_active":"2023-09-20T21:47:07.597192+0000","last_peered":"2023-09-20T21:47:07.597192+0000","last_clean":"2023-09-20T21:47:07.597192+0000","last_became_active":"2023-09-20T21:46:45.581224+0000","last_became_peered":"2023-09-20T21:46:45.581224+0000","last_unstale":"2023-09-20T21:47:07.597192+0000","last_undegraded":"2023-09-20T21:47:07.597192+0000","last_fullsized":"2023-09-20T21:47:07.597192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:32:17.778301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.610835+0000","last_change":"2023-09-20T21:47:08.610413+0000","last_active":"2023-09-20T21:47:08.610835+0000","last_peered":"2023-09-20T21:47:08.610835+0000","last_clean":"2023-09-20T21:47:08.610835+0000","last_became_active":"2023-09-20T21:47:08.603734+0000","last_became_peered":"2023-09-20T21:47:08.603734+0000","last_unstale":"2023-09-20T21:47:08.610835+0000","last_undegraded":"2023-09-20T21:47:08.610835+0000","last_fullsized":"2023-09-20T21:47:08.610835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:43:42.569371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598787+0000","last_change":"2023-09-20T21:46:41.507218+0000","last_active":"2023-09-20T21:47:07.598787+0000","last_peered":"2023-09-20T21:47:07.598787+0000","last_clean":"2023-09-20T21:47:07.598787+0000","last_became_active":"2023-09-20T21:46:41.506889+0000","last_became_peered":"2023-09-20T21:46:41.506889+0000","last_unstale":"2023-09-20T21:47:07.598787+0000","last_undegraded":"2023-09-20T21:47:07.598787+0000","last_fullsized":"2023-09-20T21:47:07.598787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:08:40.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593769+0000","last_change":"2023-09-20T21:46:45.579269+0000","last_active":"2023-09-20T21:47:07.593769+0000","last_peered":"2023-09-20T21:47:07.593769+0000","last_clean":"2023-09-20T21:47:07.593769+0000","last_became_active":"2023-09-20T21:46:45.578541+0000","last_became_peered":"2023-09-20T21:46:45.578541+0000","last_unstale":"2023-09-20T21:47:07.593769+0000","last_undegraded":"2023-09-20T21:47:07.593769+0000","last_fullsized":"2023-09-20T21:47:07.593769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:07:50.241947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:09.616784+0000","last_change":"2023-09-20T21:46:40.508774+0000","last_active":"2023-09-20T21:47:09.616784+0000","last_peered":"2023-09-20T21:47:09.616784+0000","last_clean":"2023-09-20T21:47:09.616784+0000","last_became_active":"2023-09-20T21:46:40.508480+0000","last_became_peered":"2023-09-20T21:46:40.508480+0000","last_unstale":"2023-09-20T21:47:09.616784+0000","last_undegraded":"2023-09-20T21:47:09.616784+0000","last_fullsized":"2023-09-20T21:47:09.616784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:22.643724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599875+0000","last_change":"2023-09-20T21:46:41.503763+0000","last_active":"2023-09-20T21:47:07.599875+0000","last_peered":"2023-09-20T21:47:07.599875+0000","last_clean":"2023-09-20T21:47:07.599875+0000","last_became_active":"2023-09-20T21:46:41.503360+0000","last_became_peered":"2023-09-20T21:46:41.503360+0000","last_unstale":"2023-09-20T21:47:07.599875+0000","last_undegraded":"2023-09-20T21:47:07.599875+0000","last_fullsized":"2023-09-20T21:47:07.599875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:03:00.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598933+0000","last_change":"2023-09-20T21:46:45.586702+0000","last_active":"2023-09-20T21:47:07.598933+0000","last_peered":"2023-09-20T21:47:07.598933+0000","last_clean":"2023-09-20T21:47:07.598933+0000","last_became_active":"2023-09-20T21:46:45.586177+0000","last_became_peered":"2023-09-20T21:46:45.586177+0000","last_unstale":"2023-09-20T21:47:07.598933+0000","last_undegraded":"2023-09-20T21:47:07.598933+0000","last_fullsized":"2023-09-20T21:47:07.598933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:19:03.883440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.576396+0000","last_change":"2023-09-20T21:46:40.507270+0000","last_active":"2023-09-20T21:47:08.576396+0000","last_peered":"2023-09-20T21:47:08.576396+0000","last_clean":"2023-09-20T21:47:08.576396+0000","last_became_active":"2023-09-20T21:46:40.506933+0000","last_became_peered":"2023-09-20T21:46:40.506933+0000","last_unstale":"2023-09-20T21:47:08.576396+0000","last_undegraded":"2023-09-20T21:47:08.576396+0000","last_fullsized":"2023-09-20T21:47:08.576396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:59.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599660+0000","last_change":"2023-09-20T21:46:41.507868+0000","last_active":"2023-09-20T21:47:07.599660+0000","last_peered":"2023-09-20T21:47:07.599660+0000","last_clean":"2023-09-20T21:47:07.599660+0000","last_became_active":"2023-09-20T21:46:41.507568+0000","last_became_peered":"2023-09-20T21:46:41.507568+0000","last_unstale":"2023-09-20T21:47:07.599660+0000","last_undegraded":"2023-09-20T21:47:07.599660+0000","last_fullsized":"2023-09-20T21:47:07.599660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:44.615620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594494+0000","last_change":"2023-09-20T21:46:45.578726+0000","last_active":"2023-09-20T21:47:07.594494+0000","last_peered":"2023-09-20T21:47:07.594494+0000","last_clean":"2023-09-20T21:47:07.594494+0000","last_became_active":"2023-09-20T21:46:45.578326+0000","last_became_peered":"2023-09-20T21:46:45.578326+0000","last_unstale":"2023-09-20T21:47:07.594494+0000","last_undegraded":"2023-09-20T21:47:07.594494+0000","last_fullsized":"2023-09-20T21:47:07.594494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:21:34.564840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577629+0000","last_change":"2023-09-20T21:46:40.509024+0000","last_active":"2023-09-20T21:47:08.577629+0000","last_peered":"2023-09-20T21:47:08.577629+0000","last_clean":"2023-09-20T21:47:08.577629+0000","last_became_active":"2023-09-20T21:46:40.508659+0000","last_became_peered":"2023-09-20T21:46:40.508659+0000","last_unstale":"2023-09-20T21:47:08.577629+0000","last_undegraded":"2023-09-20T21:47:08.577629+0000","last_fullsized":"2023-09-20T21:47:08.577629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:12:08.302068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599976+0000","last_change":"2023-09-20T21:46:41.510307+0000","last_active":"2023-09-20T21:47:07.599976+0000","last_peered":"2023-09-20T21:47:07.599976+0000","last_clean":"2023-09-20T21:47:07.599976+0000","last_became_active":"2023-09-20T21:46:41.510014+0000","last_became_peered":"2023-09-20T21:46:41.510014+0000","last_unstale":"2023-09-20T21:47:07.599976+0000","last_undegraded":"2023-09-20T21:47:07.599976+0000","last_fullsized":"2023-09-20T21:47:07.599976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:05:55.062515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598156+0000","last_change":"2023-09-20T21:46:41.511525+0000","last_active":"2023-09-20T21:47:07.598156+0000","last_peered":"2023-09-20T21:47:07.598156+0000","last_clean":"2023-09-20T21:47:07.598156+0000","last_became_active":"2023-09-20T21:46:41.511203+0000","last_became_peered":"2023-09-20T21:46:41.511203+0000","last_unstale":"2023-09-20T21:47:07.598156+0000","last_undegraded":"2023-09-20T21:47:07.598156+0000","last_fullsized":"2023-09-20T21:47:07.598156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:42:15.757422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599580+0000","last_change":"2023-09-20T21:46:41.502936+0000","last_active":"2023-09-20T21:47:07.599580+0000","last_peered":"2023-09-20T21:47:07.599580+0000","last_clean":"2023-09-20T21:47:07.599580+0000","last_became_active":"2023-09-20T21:46:41.502617+0000","last_became_peered":"2023-09-20T21:46:41.502617+0000","last_unstale":"2023-09-20T21:47:07.599580+0000","last_undegraded":"2023-09-20T21:47:07.599580+0000","last_fullsized":"2023-09-20T21:47:07.599580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:45:21.197652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599689+0000","last_change":"2023-09-20T21:46:41.510766+0000","last_active":"2023-09-20T21:47:07.599689+0000","last_peered":"2023-09-20T21:47:07.599689+0000","last_clean":"2023-09-20T21:47:07.599689+0000","last_became_active":"2023-09-20T21:46:41.510433+0000","last_became_peered":"2023-09-20T21:46:41.510433+0000","last_unstale":"2023-09-20T21:47:07.599689+0000","last_undegraded":"2023-09-20T21:47:07.599689+0000","last_fullsized":"2023-09-20T21:47:07.599689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:56:12.085926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599196+0000","last_change":"2023-09-20T21:46:41.508846+0000","last_active":"2023-09-20T21:47:07.599196+0000","last_peered":"2023-09-20T21:47:07.599196+0000","last_clean":"2023-09-20T21:47:07.599196+0000","last_became_active":"2023-09-20T21:46:41.508518+0000","last_became_peered":"2023-09-20T21:46:41.508518+0000","last_unstale":"2023-09-20T21:47:07.599196+0000","last_undegraded":"2023-09-20T21:47:07.599196+0000","last_fullsized":"2023-09-20T21:47:07.599196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:01:58.195327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602625+0000","last_change":"2023-09-20T21:46:41.505929+0000","last_active":"2023-09-20T21:47:07.602625+0000","last_peered":"2023-09-20T21:47:07.602625+0000","last_clean":"2023-09-20T21:47:07.602625+0000","last_became_active":"2023-09-20T21:46:41.505542+0000","last_became_peered":"2023-09-20T21:46:41.505542+0000","last_unstale":"2023-09-20T21:47:07.602625+0000","last_undegraded":"2023-09-20T21:47:07.602625+0000","last_fullsized":"2023-09-20T21:47:07.602625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:24:47.520344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598727+0000","last_change":"2023-09-20T21:46:41.507603+0000","last_active":"2023-09-20T21:47:07.598727+0000","last_peered":"2023-09-20T21:47:07.598727+0000","last_clean":"2023-09-20T21:47:07.598727+0000","last_became_active":"2023-09-20T21:46:41.507340+0000","last_became_peered":"2023-09-20T21:46:41.507340+0000","last_unstale":"2023-09-20T21:47:07.598727+0000","last_undegraded":"2023-09-20T21:47:07.598727+0000","last_fullsized":"2023-09-20T21:47:07.598727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:50:19.756491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600180+0000","last_change":"2023-09-20T21:46:41.509625+0000","last_active":"2023-09-20T21:47:07.600180+0000","last_peered":"2023-09-20T21:47:07.600180+0000","last_clean":"2023-09-20T21:47:07.600180+0000","last_became_active":"2023-09-20T21:46:41.509301+0000","last_became_peered":"2023-09-20T21:46:41.509301+0000","last_unstale":"2023-09-20T21:47:07.600180+0000","last_undegraded":"2023-09-20T21:47:07.600180+0000","last_fullsized":"2023-09-20T21:47:07.600180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:15:03.108060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596444+0000","last_change":"2023-09-20T21:46:41.516781+0000","last_active":"2023-09-20T21:47:07.596444+0000","last_peered":"2023-09-20T21:47:07.596444+0000","last_clean":"2023-09-20T21:47:07.596444+0000","last_became_active":"2023-09-20T21:46:41.516496+0000","last_became_peered":"2023-09-20T21:46:41.516496+0000","last_unstale":"2023-09-20T21:47:07.596444+0000","last_undegraded":"2023-09-20T21:47:07.596444+0000","last_fullsized":"2023-09-20T21:47:07.596444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:17:26.492851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594775+0000","last_change":"2023-09-20T21:46:41.499927+0000","last_active":"2023-09-20T21:47:07.594775+0000","last_peered":"2023-09-20T21:47:07.594775+0000","last_clean":"2023-09-20T21:47:07.594775+0000","last_became_active":"2023-09-20T21:46:41.499605+0000","last_became_peered":"2023-09-20T21:46:41.499605+0000","last_unstale":"2023-09-20T21:47:07.594775+0000","last_undegraded":"2023-09-20T21:47:07.594775+0000","last_fullsized":"2023-09-20T21:47:07.594775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:18:16.306349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595650+0000","last_change":"2023-09-20T21:46:41.515842+0000","last_active":"2023-09-20T21:47:07.595650+0000","last_peered":"2023-09-20T21:47:07.595650+0000","last_clean":"2023-09-20T21:47:07.595650+0000","last_became_active":"2023-09-20T21:46:41.515600+0000","last_became_peered":"2023-09-20T21:46:41.515600+0000","last_unstale":"2023-09-20T21:47:07.595650+0000","last_undegraded":"2023-09-20T21:47:07.595650+0000","last_fullsized":"2023-09-20T21:47:07.595650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:50:58.088103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593931+0000","last_change":"2023-09-20T21:46:41.510695+0000","last_active":"2023-09-20T21:47:07.593931+0000","last_peered":"2023-09-20T21:47:07.593931+0000","last_clean":"2023-09-20T21:47:07.593931+0000","last_became_active":"2023-09-20T21:46:41.510345+0000","last_became_peered":"2023-09-20T21:46:41.510345+0000","last_unstale":"2023-09-20T21:47:07.593931+0000","last_undegraded":"2023-09-20T21:47:07.593931+0000","last_fullsized":"2023-09-20T21:47:07.593931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:38:24.201382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602278+0000","last_change":"2023-09-20T21:46:41.507809+0000","last_active":"2023-09-20T21:47:07.602278+0000","last_peered":"2023-09-20T21:47:07.602278+0000","last_clean":"2023-09-20T21:47:07.602278+0000","last_became_active":"2023-09-20T21:46:41.507513+0000","last_became_peered":"2023-09-20T21:46:41.507513+0000","last_unstale":"2023-09-20T21:47:07.602278+0000","last_undegraded":"2023-09-20T21:47:07.602278+0000","last_fullsized":"2023-09-20T21:47:07.602278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:30:43.132728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598414+0000","last_change":"2023-09-20T21:46:41.503590+0000","last_active":"2023-09-20T21:47:07.598414+0000","last_peered":"2023-09-20T21:47:07.598414+0000","last_clean":"2023-09-20T21:47:07.598414+0000","last_became_active":"2023-09-20T21:46:41.503295+0000","last_became_peered":"2023-09-20T21:46:41.503295+0000","last_unstale":"2023-09-20T21:47:07.598414+0000","last_undegraded":"2023-09-20T21:47:07.598414+0000","last_fullsized":"2023-09-20T21:47:07.598414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:28:33.088873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10264,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":55,"num_read_kb":44,"num_write":209,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":30792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":487223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.47999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.64600000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.85699999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.753}]}]},{"osd":10,"up_from":72,"seq":309237645333,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1662976,"data_stored":1091501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":1,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.77300000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.79200000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.71099999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.88800000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.84399999999999997}]},{"osd":6,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.747},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.90300000000000002}]},{"osd":7,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.873},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.94499999999999995}]},{"osd":9,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.98699999999999999}]},{"osd":11,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.96699999999999997}]}]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":496677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.625}]},{"osd":1,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.89300000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.72399999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.755}]},{"osd":8,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.70199999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.7719999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.85999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070495,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":497055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77200000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.77400000000000002}]},{"osd":1,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49399999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65500000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64700000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.58499999999999996}]},{"osd":3,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.64300000000000002,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83199999999999996,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.68999999999999995}]},{"osd":4,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79500000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.878,"15min":0.878},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.313,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78600000000000003,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.46300000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.94599999999999995,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.73199999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:46:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.73099999999999998,"15min":0.73599999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.876,"5min":1.018,"15min":1.018},"last":0.67500000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:47:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76100000000000001,"15min":0.75700000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.03,"5min":1.073,"15min":1.073},"last":0.749},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78000000000000003,"15min":0.77300000000000002},"min":{"1min":0.621,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82699999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1638400,"data_stored":1089065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41799999999999998},"min":{"1min":0.27500000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.434},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.438,"15min":0.433},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61499999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.47899999999999998},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.48499999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.67900000000000005}]},{"osd":6,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67300000000000004,"15min":0.66100000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65000000000000002}]},{"osd":8,"last update":"Wed Sep 20 21:46:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65300000000000002,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.53000000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73899999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.79200000000000004}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":496277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":1.018,"15min":1.018},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64500000000000002,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.83799999999999997}]},{"osd":1,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":1.2,"15min":1.2},"last":0.497}]},{"osd":2,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.57199999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.466,"15min":1.466},"last":0.63400000000000001}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":1.012,"15min":1.012},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.175,"5min":1.179,"15min":1.179},"last":0.60199999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0640000000000001,"5min":1.093,"15min":1.093},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78700000000000003,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.022,"5min":1.407,"15min":1.407},"last":0.68500000000000005}]},{"osd":5,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0549999999999999,"5min":1.266,"15min":1.266},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.046,"5min":1.073,"15min":1.073},"last":0.65800000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.84799999999999998,"15min":0.83699999999999997},"min":{"1min":0.56399999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.046},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.85899999999999999,"15min":0.84599999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.98099999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:46:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.84499999999999997,"15min":0.85899999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0149999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.621},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.85599999999999998,"15min":0.86399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.1240000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.77300000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.94399999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.0129999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":501149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.39400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86499999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.52000000000000002,"15min":0.5},"min":{"1min":0.40000000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65200000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.54500000000000004}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.41899999999999998,"15min":0.39300000000000002},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:46:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60599999999999998,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.70299999999999996}]},{"osd":6,"last update":"Wed Sep 20 21:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67900000000000005,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70799999999999996,"15min":0.68400000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.68500000000000005}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.434,"5min":0.372,"15min":0.372},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61099999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.67600000000000005,"15min":0.66400000000000003},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.69599999999999995,"15min":0.68799999999999994},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.752}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.77000000000000002}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.72399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":497733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 20 21:47:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77800000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.64600000000000002},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.78900000000000003}]},{"osd":2,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61799999999999999},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45200000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92300000000000004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Sep 20 21:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.44900000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.504}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66600000000000004,"15min":0.64200000000000002},"min":{"1min":0.52000000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.67400000000000004,"15min":0.627},"min":{"1min":0.64600000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Wed Sep 20 21:46:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86899999999999999,"5min":1.149,"15min":1.149},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.67200000000000004},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.94199999999999995,"5min":1.194,"15min":1.194},"last":0.67200000000000004}]},{"osd":6,"last update":"Wed Sep 20 21:47:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.72199999999999998,"15min":0.67900000000000005},"min":{"1min":0.39700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94499999999999995,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.94499999999999995}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.80000000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.752},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.85899999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:46:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.012,"5min":1.214,"15min":1.214},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.90500000000000003,"15min":0.91600000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0940000000000001,"5min":1.232,"15min":1.232},"last":0.84099999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.82499999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.878}]},{"osd":11,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.95999999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":501369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76500000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.378},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78900000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40500000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.51100000000000001},"min":{"1min":0.39400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.63900000000000001}]},{"osd":3,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.58799999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:46:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.91900000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:46:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.44,"15min":0.39900000000000002},"min":{"1min":0.31,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.55500000000000005,"15min":0.49399999999999999},"min":{"1min":0.38300000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":7.1500000000000004,"5min":7.1500000000000004,"15min":7.1500000000000004},"last":0.48599999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.69499999999999995,"15min":0.63600000000000001},"min":{"1min":0.497,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":7.367,"5min":7.367,"15min":7.367},"last":0.622},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.72699999999999998,"15min":0.66500000000000004},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":6.4649999999999999,"5min":6.4649999999999999,"15min":6.4649999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.71799999999999997}]},{"osd":8,"last update":"Wed Sep 20 21:46:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.66100000000000003,"15min":0.621},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.69899999999999995}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.97099999999999997}]},{"osd":10,"last update":"Wed Sep 20 21:46:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.86899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":495985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.432,"15min":0.39000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81699999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.497}]},{"osd":1,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":1,"15min":1},"last":0.625},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.84899999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63300000000000001,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.44900000000000001}]},{"osd":4,"last update":"Wed Sep 20 21:46:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.65300000000000002,"15min":0.626},"min":{"1min":0.53500000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68799999999999994,"15min":0.64500000000000002},"min":{"1min":0.60399999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:46:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.91400000000000003,"5min":1.024,"15min":1.024},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75900000000000001,"15min":0.747},"min":{"1min":0.61099999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98399999999999999,"5min":1.167,"15min":1.167},"last":0.73999999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.51200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.018,"5min":1.147,"15min":1.147},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.018,"5min":1.296,"15min":1.296},"last":0.71499999999999997}]},{"osd":7,"last update":"Wed Sep 20 21:46:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.55400000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.177,"5min":1.248,"15min":1.248},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.52800000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0549999999999999,"5min":1.228,"15min":1.228},"last":0.76100000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.79100000000000004}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.82699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":486697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.372,"15min":0.35599999999999998},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57699999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.38,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.40899999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54000000000000004,"15min":0.52700000000000002},"min":{"1min":0.46400000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68700000000000006,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.54500000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.48499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.51900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61799999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:46:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58599999999999997,"15min":0.56799999999999995},"min":{"1min":0.503,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:47:07 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67000000000000004,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.49399999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:46:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.57599999999999996,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56699999999999995}]},{"osd":8,"last update":"Wed Sep 20 21:46:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.83599999999999997,"5min":1,"15min":1},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88200000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68500000000000005}]},{"osd":9,"last update":"Wed Sep 20 21:46:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.69999999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.71599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1646592,"data_stored":1088462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85499999999999998,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.42999999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.753,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.42699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81299999999999994,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67600000000000005}]},{"osd":2,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.36699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.40400000000000003,"15min":0.39000000000000001},"min":{"1min":0.28000000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.59899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79100000000000004,"5min":1.615,"15min":1.615},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.64600000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95599999999999996,"5min":1.008,"15min":1.008},"last":0.66200000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65800000000000003,"15min":0.65200000000000002},"min":{"1min":0.53900000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80400000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.628},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.76100000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.57599999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.86499999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65900000000000003,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85999999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76100000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:46:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.61799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.749,"5min":0.879,"15min":0.879},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.69499999999999995}]},{"osd":9,"last update":"Wed Sep 20 21:46:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.73399999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:46:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.745},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-20T21:47:14.544 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-20T21:47:14.545 INFO:tasks.cephadm:Setting up client nodes... 2023-09-20T21:47:14.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-09-20T21:47:15.312 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:16.412 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:16 smithi022 ceph-mon[97651]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:16.413 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:16 smithi022 ceph-mon[97651]: pgmap v295: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.7 KiB/s wr, 26 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-20T21:47:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:16 smithi123 ceph-mon[134433]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:16 smithi123 ceph-mon[134433]: pgmap v295: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.7 KiB/s wr, 26 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-20T21:47:16.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:16 smithi089 ceph-mon[134090]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:47:16.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:16 smithi089 ceph-mon[134090]: pgmap v295: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.7 KiB/s wr, 26 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-20T21:47:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.047+0000 7faba1f09700 1 -- 172.21.15.22:0/1759907524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c073840 msgr2=0x7fab9c073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.047+0000 7faba1f09700 1 --2- 172.21.15.22:0/1759907524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c073840 0x7fab9c073cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab90009990 tx=0x7fab90009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.047+0000 7faba1f09700 1 -- 172.21.15.22:0/1759907524 shutdown_connections 2023-09-20T21:47:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.047+0000 7faba1f09700 1 --2- 172.21.15.22:0/1759907524 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab9c10d480 0x7fab9c10f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.047+0000 7faba1f09700 1 --2- 172.21.15.22:0/1759907524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab9c074e10 0x7fab9c073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.047+0000 7faba1f09700 1 --2- 172.21.15.22:0/1759907524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c073840 0x7fab9c073cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.047+0000 7faba1f09700 1 -- 172.21.15.22:0/1759907524 >> 172.21.15.22:0/1759907524 conn(0x7fab9c0fc460 msgr2=0x7fab9c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:17.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.047+0000 7faba1f09700 1 -- 172.21.15.22:0/1759907524 shutdown_connections 2023-09-20T21:47:17.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.047+0000 7faba1f09700 1 -- 172.21.15.22:0/1759907524 wait complete. 2023-09-20T21:47:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.048+0000 7faba1f09700 1 Processor -- start 2023-09-20T21:47:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7faba1f09700 1 -- start start 2023-09-20T21:47:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7faba1f09700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c073840 0x7fab9c19d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7faba1f09700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab9c074e10 0x7fab9c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7faba1f09700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab9c10d480 0x7fab9c1a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7faba1f09700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c1a4ce0 con 0x7fab9c073840 2023-09-20T21:47:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7faba1f09700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c1a4e20 con 0x7fab9c074e10 2023-09-20T21:47:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7faba1f09700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c1a4fb0 con 0x7fab9c10d480 2023-09-20T21:47:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7fab9b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c073840 0x7fab9c19d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:17.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7fab9b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c073840 0x7fab9c19d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49138/0 (socket says 172.21.15.22:49138) 2023-09-20T21:47:17.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7fab9b7fe700 1 -- 172.21.15.22:0/2945066256 learned_addr learned my addr 172.21.15.22:0/2945066256 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:17.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7fab9affd700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab9c074e10 0x7fab9c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:17.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.049+0000 7fab9bfff700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab9c10d480 0x7fab9c1a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:17.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7fab9b7fe700 1 -- 172.21.15.22:0/2945066256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab9c10d480 msgr2=0x7fab9c1a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7fab9b7fe700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab9c10d480 0x7fab9c1a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7fab9b7fe700 1 -- 172.21.15.22:0/2945066256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab9c074e10 msgr2=0x7fab9c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7fab9b7fe700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab9c074e10 0x7fab9c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7fab9b7fe700 1 -- 172.21.15.22:0/2945066256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900095f0 con 0x7fab9c073840 2023-09-20T21:47:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7fab9b7fe700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c073840 0x7fab9c19d150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00e9a0 tx=0x7fab8c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7fab98ff9700 1 -- 172.21.15.22:0/2945066256 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c009790 con 0x7fab9c073840 2023-09-20T21:47:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7fab98ff9700 1 -- 172.21.15.22:0/2945066256 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab8c004cf0 con 0x7fab9c073840 2023-09-20T21:47:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7faba1f09700 1 -- 172.21.15.22:0/2945066256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c1a5290 con 0x7fab9c073840 2023-09-20T21:47:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7fab98ff9700 1 -- 172.21.15.22:0/2945066256 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0055c0 con 0x7fab9c073840 2023-09-20T21:47:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.050+0000 7faba1f09700 1 -- 172.21.15.22:0/2945066256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c1a57e0 con 0x7fab9c073840 2023-09-20T21:47:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.052+0000 7faba1f09700 1 -- 172.21.15.22:0/2945066256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c102fd0 con 0x7fab9c073840 2023-09-20T21:47:17.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.054+0000 7fab98ff9700 1 -- 172.21.15.22:0/2945066256 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fab8c00cc60 con 0x7fab9c073840 2023-09-20T21:47:17.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.055+0000 7fab98ff9700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fab840a1380 0x7fab840a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.055+0000 7fab98ff9700 1 -- 172.21.15.22:0/2945066256 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fab8c014070 con 0x7fab9c073840 2023-09-20T21:47:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.055+0000 7fab9affd700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fab840a1380 0x7fab840a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:17.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.056+0000 7fab9affd700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fab840a1380 0x7fab840a3840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab9000b2e0 tx=0x7fab90005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.060+0000 7fab98ff9700 1 -- 172.21.15.22:0/2945066256 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fab8c096650 con 0x7fab9c073840 2023-09-20T21:47:17.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.352+0000 7faba1f09700 1 -- 172.21.15.22:0/2945066256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fab9c112cc0 con 0x7fab9c073840 2023-09-20T21:47:17.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.356+0000 7fab98ff9700 1 -- 172.21.15.22:0/2945066256 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7fab8c099c70 con 0x7fab9c073840 2023-09-20T21:47:17.357 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2023-09-20T21:47:17.357 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBlaAtlYGIWFRAA2UfzaNRxPLUlhHikeqcx5Q== 2023-09-20T21:47:17.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 -- 172.21.15.22:0/2945066256 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fab840a1380 msgr2=0x7fab840a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:17.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fab840a1380 0x7fab840a3840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab9000b2e0 tx=0x7fab90005fb0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 -- 172.21.15.22:0/2945066256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c073840 msgr2=0x7fab9c19d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:17.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c073840 0x7fab9c19d150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00e9a0 tx=0x7fab8c00ecb0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 -- 172.21.15.22:0/2945066256 shutdown_connections 2023-09-20T21:47:17.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab9c10d480 0x7fab9c1a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab9c073840 0x7fab9c19d150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fab840a1380 0x7fab840a3840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 --2- 172.21.15.22:0/2945066256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab9c074e10 0x7fab9c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:17.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 -- 172.21.15.22:0/2945066256 >> 172.21.15.22:0/2945066256 conn(0x7fab9c0fc460 msgr2=0x7fab9c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:17.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 -- 172.21.15.22:0/2945066256 shutdown_connections 2023-09-20T21:47:17.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:17.359+0000 7faba1f09700 1 -- 172.21.15.22:0/2945066256 wait complete. 2023-09-20T21:47:17.808 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:47:17.808 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-09-20T21:47:17.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-20T21:47:17.912 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-20T21:47:17.913 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-09-20T21:47:17.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph mgr dump --format=json 2023-09-20T21:47:18.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:18 smithi022 ceph-mon[97651]: pgmap v296: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.8 KiB/s wr, 25 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-20T21:47:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:18 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2945066256' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-20T21:47:18.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:18 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2945066256' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-20T21:47:18.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:18 smithi123 ceph-mon[134433]: pgmap v296: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.8 KiB/s wr, 25 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-20T21:47:18.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:18 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2945066256' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-20T21:47:18.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:18 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2945066256' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-20T21:47:18.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:18 smithi089 ceph-mon[134090]: pgmap v296: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.8 KiB/s wr, 25 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-09-20T21:47:18.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:18 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2945066256' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-09-20T21:47:18.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:18 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2945066256' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-09-20T21:47:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:20.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.202+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3786139913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd8410d640 msgr2=0x7fbd8410fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:20.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.202+0000 7fbd8b5ac700 1 --2- 172.21.15.22:0/3786139913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd8410d640 0x7fbd8410fab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00b0a0 tx=0x7fbd7c00b3b0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.202+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3786139913 shutdown_connections 2023-09-20T21:47:20.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.202+0000 7fbd8b5ac700 1 --2- 172.21.15.22:0/3786139913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd8410d640 0x7fbd8410fab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.202+0000 7fbd8b5ac700 1 --2- 172.21.15.22:0/3786139913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd84073210 0x7fbd840735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.202+0000 7fbd8b5ac700 1 --2- 172.21.15.22:0/3786139913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd84073b30 0x7fbd8410d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.202+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3786139913 >> 172.21.15.22:0/3786139913 conn(0x7fbd840fc8d0 msgr2=0x7fbd840fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:20.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.202+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3786139913 shutdown_connections 2023-09-20T21:47:20.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.202+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3786139913 wait complete. 2023-09-20T21:47:20.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.203+0000 7fbd8b5ac700 1 Processor -- start 2023-09-20T21:47:20.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.203+0000 7fbd8b5ac700 1 -- start start 2023-09-20T21:47:20.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd8b5ac700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd84073210 0x7fbd841991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd8b5ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd84073b30 0x7fbd84199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd8b5ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8410d640 0x7fbd8419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd8b5ac700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8419e1a0 con 0x7fbd84073210 2023-09-20T21:47:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd8b5ac700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8419e310 con 0x7fbd8410d640 2023-09-20T21:47:20.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd8b5ac700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8419e4d0 con 0x7fbd84073b30 2023-09-20T21:47:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd89348700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd84073210 0x7fbd841991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd89348700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd84073210 0x7fbd841991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49164/0 (socket says 172.21.15.22:49164) 2023-09-20T21:47:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd89348700 1 -- 172.21.15.22:0/3386361587 learned_addr learned my addr 172.21.15.22:0/3386361587 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd88b47700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd84073b30 0x7fbd84199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.204+0000 7fbd89b49700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8410d640 0x7fbd8419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd89348700 1 -- 172.21.15.22:0/3386361587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd84073b30 msgr2=0x7fbd84199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd89348700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd84073b30 0x7fbd84199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd89348700 1 -- 172.21.15.22:0/3386361587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8410d640 msgr2=0x7fbd8419db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd89348700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8410d640 0x7fbd8419db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd89348700 1 -- 172.21.15.22:0/3386361587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c009d00 con 0x7fbd84073210 2023-09-20T21:47:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd89348700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd84073210 0x7fbd841991e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000e9a0 tx=0x7fbd8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:20.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd7a7fc700 1 -- 172.21.15.22:0/3386361587 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80009790 con 0x7fbd84073210 2023-09-20T21:47:20.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd7a7fc700 1 -- 172.21.15.22:0/3386361587 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd80010430 con 0x7fbd84073210 2023-09-20T21:47:20.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd7a7fc700 1 -- 172.21.15.22:0/3386361587 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80018680 con 0x7fbd84073210 2023-09-20T21:47:20.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3386361587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8419e7b0 con 0x7fbd84073210 2023-09-20T21:47:20.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.205+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3386361587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd841a6010 con 0x7fbd84073210 2023-09-20T21:47:20.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.207+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3386361587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd84107f60 con 0x7fbd84073210 2023-09-20T21:47:20.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.208+0000 7fbd7a7fc700 1 -- 172.21.15.22:0/3386361587 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fbd800105a0 con 0x7fbd84073210 2023-09-20T21:47:20.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.210+0000 7fbd7a7fc700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbd700a1380 0x7fbd700a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:20.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.210+0000 7fbd7a7fc700 1 -- 172.21.15.22:0/3386361587 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbd80014070 con 0x7fbd84073210 2023-09-20T21:47:20.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.210+0000 7fbd88b47700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbd700a1380 0x7fbd700a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:20.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.211+0000 7fbd88b47700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbd700a1380 0x7fbd700a3840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbd8419a800 tx=0x7fbd74009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:20.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.214+0000 7fbd7a7fc700 1 -- 172.21.15.22:0/3386361587 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbd80051e10 con 0x7fbd84073210 2023-09-20T21:47:20.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:20 smithi022 ceph-mon[97651]: pgmap v297: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.5 KiB/s wr, 21 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-20T21:47:20.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.484+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3386361587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbd8404eae0 con 0x7fbd84073210 2023-09-20T21:47:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.489+0000 7fbd7a7fc700 1 -- 172.21.15.22:0/3386361587 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+260448 (secure 0 0 0) 0x7fbd80065970 con 0x7fbd84073210 2023-09-20T21:47:20.494 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:20.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.494+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3386361587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbd700a1380 msgr2=0x7fbd700a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:20.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.494+0000 7fbd8b5ac700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbd700a1380 0x7fbd700a3840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbd8419a800 tx=0x7fbd74009260 comp rx=0 tx=0).stop 2023-09-20T21:47:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.494+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3386361587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd84073210 msgr2=0x7fbd841991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.494+0000 7fbd8b5ac700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd84073210 0x7fbd841991e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000e9a0 tx=0x7fbd8000ecb0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.494+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3386361587 shutdown_connections 2023-09-20T21:47:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.494+0000 7fbd8b5ac700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbd700a1380 0x7fbd700a3840 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.494+0000 7fbd8b5ac700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd8410d640 0x7fbd8419db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.494+0000 7fbd8b5ac700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd84073210 0x7fbd841991e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.494+0000 7fbd8b5ac700 1 --2- 172.21.15.22:0/3386361587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd84073b30 0x7fbd84199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:20.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.494+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3386361587 >> 172.21.15.22:0/3386361587 conn(0x7fbd840fc8d0 msgr2=0x7fbd841050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:20.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.495+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3386361587 shutdown_connections 2023-09-20T21:47:20.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:20.495+0000 7fbd8b5ac700 1 -- 172.21.15.22:0/3386361587 wait complete. 2023-09-20T21:47:20.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:20 smithi123 ceph-mon[134433]: pgmap v297: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.5 KiB/s wr, 21 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-20T21:47:20.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:20 smithi089 ceph-mon[134090]: pgmap v297: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.5 KiB/s wr, 21 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-20T21:47:21.080 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"active_gid":14154,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":718010003},{"type":"v1","addr":"172.21.15.22:6801","nonce":718010003}]},"active_addr":"172.21.15.22:6801/718010003","active_change":"2023-09-20T21:40:07.017297+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.22:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":3109490890}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2546572633}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1576470946}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":627665965}]}]} 2023-09-20T21:47:21.086 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-09-20T21:47:21.086 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-09-20T21:47:21.087 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd dump --format=json 2023-09-20T21:47:21.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:21 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3386361587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-20T21:47:21.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:21 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3386361587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-20T21:47:21.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:21 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3386361587' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-09-20T21:47:21.834 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:22.454 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:22 smithi022 ceph-mon[97651]: pgmap v298: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 18 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-20T21:47:22.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:22 smithi123 ceph-mon[134433]: pgmap v298: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 18 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-20T21:47:22.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:22 smithi089 ceph-mon[134090]: pgmap v298: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.2 KiB/s wr, 18 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-20T21:47:23.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.492+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/3642935230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c8101a10 msgr2=0x7fb9c8105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:23.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.492+0000 7fb9cf9dc700 1 --2- 172.21.15.22:0/3642935230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c8101a10 0x7fb9c8105a60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009990 tx=0x7fb9b8009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:23.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.492+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/3642935230 shutdown_connections 2023-09-20T21:47:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.492+0000 7fb9cf9dc700 1 --2- 172.21.15.22:0/3642935230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8106140 0x7fb9c810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.492+0000 7fb9cf9dc700 1 --2- 172.21.15.22:0/3642935230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8101060 0x7fb9c8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.492+0000 7fb9cf9dc700 1 --2- 172.21.15.22:0/3642935230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c8101a10 0x7fb9c8105a60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.492+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/3642935230 >> 172.21.15.22:0/3642935230 conn(0x7fb9c80fc8d0 msgr2=0x7fb9c80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.492+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/3642935230 shutdown_connections 2023-09-20T21:47:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.492+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/3642935230 wait complete. 2023-09-20T21:47:23.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.494+0000 7fb9cf9dc700 1 Processor -- start 2023-09-20T21:47:23.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.494+0000 7fb9cf9dc700 1 -- start start 2023-09-20T21:47:23.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9cf9dc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8101060 0x7fb9c80730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9cf9dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8101a10 0x7fb9c8073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9cf9dc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c8106140 0x7fb9c8073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9cf9dc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c81a53f0 con 0x7fb9c8106140 2023-09-20T21:47:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9cf9dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c81a5560 con 0x7fb9c8101a10 2023-09-20T21:47:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9cf9dc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c81a5720 con 0x7fb9c8101060 2023-09-20T21:47:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9cdf79700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c8106140 0x7fb9c8073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9ccf77700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8101a10 0x7fb9c8073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9cd778700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8101060 0x7fb9c80730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9ccf77700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8101a10 0x7fb9c8073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60858/0 (socket says 172.21.15.22:60858) 2023-09-20T21:47:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.495+0000 7fb9ccf77700 1 -- 172.21.15.22:0/4006576665 learned_addr learned my addr 172.21.15.22:0/4006576665 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.496+0000 7fb9ccf77700 1 -- 172.21.15.22:0/4006576665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8101060 msgr2=0x7fb9c80730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.496+0000 7fb9ccf77700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8101060 0x7fb9c80730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.496+0000 7fb9ccf77700 1 -- 172.21.15.22:0/4006576665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c8106140 msgr2=0x7fb9c8073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.496+0000 7fb9ccf77700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c8106140 0x7fb9c8073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.496+0000 7fb9ccf77700 1 -- 172.21.15.22:0/4006576665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b80095f0 con 0x7fb9c8101a10 2023-09-20T21:47:23.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.496+0000 7fb9ccf77700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8101a10 0x7fb9c8073620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8000c00 tx=0x7fb9b8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.496+0000 7fb9be7fc700 1 -- 172.21.15.22:0/4006576665 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b801d070 con 0x7fb9c8101a10 2023-09-20T21:47:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.496+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/4006576665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c81a59a0 con 0x7fb9c8101a10 2023-09-20T21:47:23.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.496+0000 7fb9be7fc700 1 -- 172.21.15.22:0/4006576665 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b8021410 con 0x7fb9c8101a10 2023-09-20T21:47:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.497+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/4006576665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c81a5e10 con 0x7fb9c8101a10 2023-09-20T21:47:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.497+0000 7fb9be7fc700 1 -- 172.21.15.22:0/4006576665 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8017650 con 0x7fb9c8101a10 2023-09-20T21:47:23.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.498+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/4006576665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c804eae0 con 0x7fb9c8101a10 2023-09-20T21:47:23.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.500+0000 7fb9be7fc700 1 -- 172.21.15.22:0/4006576665 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fb9b800fb00 con 0x7fb9c8101a10 2023-09-20T21:47:23.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.501+0000 7fb9be7fc700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb9b40a1380 0x7fb9b40a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:23.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.501+0000 7fb9be7fc700 1 -- 172.21.15.22:0/4006576665 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb9b80cdff0 con 0x7fb9c8101a10 2023-09-20T21:47:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.501+0000 7fb9cd778700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb9b40a1380 0x7fb9b40a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.502+0000 7fb9cd778700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb9b40a1380 0x7fb9b40a3840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009c60 tx=0x7fb9c4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.503+0000 7fb9be7fc700 1 -- 172.21.15.22:0/4006576665 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb9b80973f0 con 0x7fb9c8101a10 2023-09-20T21:47:23.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.713+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/4006576665 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9c8110fd0 con 0x7fb9c8101a10 2023-09-20T21:47:23.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.715+0000 7fb9be7fc700 1 -- 172.21.15.22:0/4006576665 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24411 (secure 0 0 0) 0x7fb9b809aa10 con 0x7fb9c8101a10 2023-09-20T21:47:23.716 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:23.716 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":91,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:47:08.590323+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:47:23.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/4006576665 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb9b40a1380 msgr2=0x7fb9b40a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:23.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb9b40a1380 0x7fb9b40a3840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009c60 tx=0x7fb9c4009260 comp rx=0 tx=0).stop 2023-09-20T21:47:23.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/4006576665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8101a10 msgr2=0x7fb9c8073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:23.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8101a10 0x7fb9c8073620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8000c00 tx=0x7fb9b8005740 comp rx=0 tx=0).stop 2023-09-20T21:47:23.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/4006576665 shutdown_connections 2023-09-20T21:47:23.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c8106140 0x7fb9c8073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:23.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8101060 0x7fb9c80730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:23.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb9b40a1380 0x7fb9b40a3840 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:23.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 --2- 172.21.15.22:0/4006576665 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb9c8101a10 0x7fb9c8073620 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:23.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/4006576665 >> 172.21.15.22:0/4006576665 conn(0x7fb9c80fc8d0 msgr2=0x7fb9c80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:23.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/4006576665 shutdown_connections 2023-09-20T21:47:23.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:23.718+0000 7fb9cf9dc700 1 -- 172.21.15.22:0/4006576665 wait complete. 2023-09-20T21:47:23.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:23 smithi123 ceph-mon[134433]: pgmap v299: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 13 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-20T21:47:23.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:23 smithi022 ceph-mon[97651]: pgmap v299: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 13 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-20T21:47:23.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:23 smithi089 ceph-mon[134090]: pgmap v299: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 13 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-09-20T21:47:24.130 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-09-20T21:47:24.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd dump --format=json 2023-09-20T21:47:24.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:24 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4006576665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:24.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:24 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4006576665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:24.870 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:24.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:24 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4006576665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:25.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:25 smithi022 ceph-mon[97651]: pgmap v300: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-09-20T21:47:25.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:25 smithi123 ceph-mon[134433]: pgmap v300: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-09-20T21:47:25.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:25 smithi089 ceph-mon[134090]: pgmap v300: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-09-20T21:47:26.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.476+0000 7fe5eec23700 1 -- 172.21.15.22:0/2515259647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e8106140 msgr2=0x7fe5e810dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:26.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.476+0000 7fe5eec23700 1 --2- 172.21.15.22:0/2515259647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e8106140 0x7fe5e810dd10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b0a0 tx=0x7fe5e000b3b0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.476+0000 7fe5eec23700 1 -- 172.21.15.22:0/2515259647 shutdown_connections 2023-09-20T21:47:26.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.476+0000 7fe5eec23700 1 --2- 172.21.15.22:0/2515259647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e8106140 0x7fe5e810dd10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.476+0000 7fe5eec23700 1 --2- 172.21.15.22:0/2515259647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e8101060 0x7fe5e8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.476+0000 7fe5eec23700 1 --2- 172.21.15.22:0/2515259647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e8101a10 0x7fe5e8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.476+0000 7fe5eec23700 1 -- 172.21.15.22:0/2515259647 >> 172.21.15.22:0/2515259647 conn(0x7fe5e80fc8d0 msgr2=0x7fe5e80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:26.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.476+0000 7fe5eec23700 1 -- 172.21.15.22:0/2515259647 shutdown_connections 2023-09-20T21:47:26.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.476+0000 7fe5eec23700 1 -- 172.21.15.22:0/2515259647 wait complete. 2023-09-20T21:47:26.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.477+0000 7fe5eec23700 1 Processor -- start 2023-09-20T21:47:26.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.477+0000 7fe5eec23700 1 -- start start 2023-09-20T21:47:26.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5eec23700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e8101060 0x7fe5e8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:26.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5eec23700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e8101a10 0x7fe5e8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:26.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5eec23700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e8106140 0x7fe5e819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:26.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5eec23700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e819e1f0 con 0x7fe5e8101060 2023-09-20T21:47:26.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5eec23700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e819e360 con 0x7fe5e8101a10 2023-09-20T21:47:26.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5eec23700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e819e520 con 0x7fe5e8106140 2023-09-20T21:47:26.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5ec9bf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e8101060 0x7fe5e8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:26.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5ec9bf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e8101060 0x7fe5e8199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60062/0 (socket says 172.21.15.22:60062) 2023-09-20T21:47:26.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5ec9bf700 1 -- 172.21.15.22:0/1473953940 learned_addr learned my addr 172.21.15.22:0/1473953940 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:26.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5ed1c0700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e8106140 0x7fe5e819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:26.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.478+0000 7fe5e7fff700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e8101a10 0x7fe5e8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:26.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5ec9bf700 1 -- 172.21.15.22:0/1473953940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e8106140 msgr2=0x7fe5e819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:26.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5ec9bf700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e8106140 0x7fe5e819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5ec9bf700 1 -- 172.21.15.22:0/1473953940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e8101a10 msgr2=0x7fe5e8199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:26.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5ec9bf700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e8101a10 0x7fe5e8199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5ec9bf700 1 -- 172.21.15.22:0/1473953940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0009d00 con 0x7fe5e8101060 2023-09-20T21:47:26.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5ec9bf700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e8101060 0x7fe5e8199230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00e9a0 tx=0x7fe5dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:26.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5e5ffb700 1 -- 172.21.15.22:0/1473953940 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc009790 con 0x7fe5e8101060 2023-09-20T21:47:26.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5eec23700 1 -- 172.21.15.22:0/1473953940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e819e800 con 0x7fe5e8101060 2023-09-20T21:47:26.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5e5ffb700 1 -- 172.21.15.22:0/1473953940 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5dc010430 con 0x7fe5e8101060 2023-09-20T21:47:26.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5e5ffb700 1 -- 172.21.15.22:0/1473953940 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc018680 con 0x7fe5e8101060 2023-09-20T21:47:26.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.479+0000 7fe5eec23700 1 -- 172.21.15.22:0/1473953940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e81a6060 con 0x7fe5e8101060 2023-09-20T21:47:26.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.480+0000 7fe5eec23700 1 -- 172.21.15.22:0/1473953940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e804eae0 con 0x7fe5e8101060 2023-09-20T21:47:26.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.482+0000 7fe5e5ffb700 1 -- 172.21.15.22:0/1473953940 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe5dc0105a0 con 0x7fe5e8101060 2023-09-20T21:47:26.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.484+0000 7fe5e5ffb700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5d80a1300 0x7fe5d80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:26.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.484+0000 7fe5e5ffb700 1 -- 172.21.15.22:0/1473953940 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe5dc014070 con 0x7fe5e8101060 2023-09-20T21:47:26.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.484+0000 7fe5e7fff700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5d80a1300 0x7fe5d80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:26.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.485+0000 7fe5e7fff700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5d80a1300 0x7fe5d80a37c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5e819a850 tx=0x7fe5d4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:26.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.486+0000 7fe5e5ffb700 1 -- 172.21.15.22:0/1473953940 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe5dc0966a0 con 0x7fe5e8101060 2023-09-20T21:47:26.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.699+0000 7fe5eec23700 1 -- 172.21.15.22:0/1473953940 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5e819a590 con 0x7fe5e8101060 2023-09-20T21:47:26.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.701+0000 7fe5e5ffb700 1 -- 172.21.15.22:0/1473953940 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24411 (secure 0 0 0) 0x7fe5dc099cc0 con 0x7fe5e8101060 2023-09-20T21:47:26.702 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:26.702 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":91,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:47:08.590323+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:47:26.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 -- 172.21.15.22:0/1473953940 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5d80a1300 msgr2=0x7fe5d80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:26.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5d80a1300 0x7fe5d80a37c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5e819a850 tx=0x7fe5d4009260 comp rx=0 tx=0).stop 2023-09-20T21:47:26.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 -- 172.21.15.22:0/1473953940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e8101060 msgr2=0x7fe5e8199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:26.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e8101060 0x7fe5e8199230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00e9a0 tx=0x7fe5dc00ecb0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 -- 172.21.15.22:0/1473953940 shutdown_connections 2023-09-20T21:47:26.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5e8106140 0x7fe5e819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5e8101060 0x7fe5e8199230 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe5d80a1300 0x7fe5d80a37c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 --2- 172.21.15.22:0/1473953940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe5e8101a10 0x7fe5e8199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:26.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 -- 172.21.15.22:0/1473953940 >> 172.21.15.22:0/1473953940 conn(0x7fe5e80fc8d0 msgr2=0x7fe5e80fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:26.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 -- 172.21.15.22:0/1473953940 shutdown_connections 2023-09-20T21:47:26.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:26.704+0000 7fe5eec23700 1 -- 172.21.15.22:0/1473953940 wait complete. 2023-09-20T21:47:26.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:26 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1473953940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:27.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:26 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1473953940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:27.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:26 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1473953940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:47:27.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.0 flush_pg_stats 2023-09-20T21:47:27.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.1 flush_pg_stats 2023-09-20T21:47:27.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.2 flush_pg_stats 2023-09-20T21:47:27.168 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.3 flush_pg_stats 2023-09-20T21:47:27.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.4 flush_pg_stats 2023-09-20T21:47:27.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.5 flush_pg_stats 2023-09-20T21:47:27.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.6 flush_pg_stats 2023-09-20T21:47:27.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.7 flush_pg_stats 2023-09-20T21:47:27.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.8 flush_pg_stats 2023-09-20T21:47:27.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.9 flush_pg_stats 2023-09-20T21:47:27.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.10 flush_pg_stats 2023-09-20T21:47:27.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph tell osd.11 flush_pg_stats 2023-09-20T21:47:28.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:27 smithi123 ceph-mon[134433]: pgmap v301: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:28.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:27 smithi089 ceph-mon[134090]: pgmap v301: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:28.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:27 smithi022 ceph-mon[97651]: pgmap v301: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:28.848 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:28.876 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:29.038 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:29.105 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:29.264 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:29.971 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:29.972 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:30.452 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:30 smithi022 ceph-mon[97651]: pgmap v302: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:47:30.593 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:30.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:30 smithi123 ceph-mon[134433]: pgmap v302: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:47:30.595 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:30.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:30 smithi089 ceph-mon[134090]: pgmap v302: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:47:30.892 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:31.922 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:31.949 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:32.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:32 smithi022 ceph-mon[97651]: pgmap v303: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:47:32.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.359+0000 7f8f6d422700 1 -- 172.21.15.22:0/2402954586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f68106140 msgr2=0x7f8f6810dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:32.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.359+0000 7f8f6d422700 1 --2- 172.21.15.22:0/2402954586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f68106140 0x7f8f6810dd10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000b0a0 tx=0x7f8f6000b3b0 comp rx=0 tx=0).stop 2023-09-20T21:47:32.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.359+0000 7f8f6d422700 1 -- 172.21.15.22:0/2402954586 shutdown_connections 2023-09-20T21:47:32.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.359+0000 7f8f6d422700 1 --2- 172.21.15.22:0/2402954586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f68106140 0x7f8f6810dd10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:32.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.359+0000 7f8f6d422700 1 --2- 172.21.15.22:0/2402954586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f68101060 0x7f8f68101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:32.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.359+0000 7f8f6d422700 1 --2- 172.21.15.22:0/2402954586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68101a10 0x7f8f68105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:32.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.359+0000 7f8f6d422700 1 -- 172.21.15.22:0/2402954586 >> 172.21.15.22:0/2402954586 conn(0x7f8f680fc8d0 msgr2=0x7f8f680fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:32.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.359+0000 7f8f6d422700 1 -- 172.21.15.22:0/2402954586 shutdown_connections 2023-09-20T21:47:32.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.359+0000 7f8f6d422700 1 -- 172.21.15.22:0/2402954586 wait complete. 2023-09-20T21:47:32.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.360+0000 7f8f6d422700 1 Processor -- start 2023-09-20T21:47:32.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f6d422700 1 -- start start 2023-09-20T21:47:32.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f6d422700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68101060 0x7f8f68199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:32.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f6d422700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f68101a10 0x7f8f681997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:32.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f6d422700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f68106140 0x7f8f6819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:32.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f6d422700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819e220 con 0x7f8f68101a10 2023-09-20T21:47:32.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f6d422700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819e390 con 0x7f8f68106140 2023-09-20T21:47:32.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f6d422700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6819e550 con 0x7f8f68101060 2023-09-20T21:47:32.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f667fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f68101a10 0x7f8f681997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:32.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f667fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f68101a10 0x7f8f681997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60086/0 (socket says 172.21.15.22:60086) 2023-09-20T21:47:32.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f667fc700 1 -- 172.21.15.22:0/1376388302 learned_addr learned my addr 172.21.15.22:0/1376388302 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:32.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f66ffd700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68101060 0x7f8f68199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:32.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.361+0000 7f8f677fe700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f68106140 0x7f8f6819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:32.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.362+0000 7f8f667fc700 1 -- 172.21.15.22:0/1376388302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68101060 msgr2=0x7f8f68199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:32.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.362+0000 7f8f667fc700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f68101060 0x7f8f68199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:32.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.362+0000 7f8f667fc700 1 -- 172.21.15.22:0/1376388302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f68106140 msgr2=0x7f8f6819dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:32.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.362+0000 7f8f667fc700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f68106140 0x7f8f6819dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:32.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.362+0000 7f8f667fc700 1 -- 172.21.15.22:0/1376388302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f60009d00 con 0x7f8f68101a10 2023-09-20T21:47:32.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.362+0000 7f8f667fc700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f68101a10 0x7f8f681997a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00d980 tx=0x7f8f5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:32.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.362+0000 7f8f4ffff700 1 -- 172.21.15.22:0/1376388302 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c00f890 con 0x7f8f68101a10 2023-09-20T21:47:32.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.362+0000 7f8f4ffff700 1 -- 172.21.15.22:0/1376388302 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f5c010460 con 0x7f8f68101a10 2023-09-20T21:47:32.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.362+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6819e830 con 0x7f8f68101a10 2023-09-20T21:47:32.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.362+0000 7f8f4ffff700 1 -- 172.21.15.22:0/1376388302 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c00e610 con 0x7f8f68101a10 2023-09-20T21:47:32.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.363+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f681a6090 con 0x7f8f68101a10 2023-09-20T21:47:32.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.364+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f68111090 con 0x7f8f68101a10 2023-09-20T21:47:32.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.366+0000 7f8f4ffff700 1 -- 172.21.15.22:0/1376388302 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f8f5c0105d0 con 0x7f8f68101a10 2023-09-20T21:47:32.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.367+0000 7f8f4ffff700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8f500a1380 0x7f8f500a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:32.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.367+0000 7f8f4ffff700 1 -- 172.21.15.22:0/1376388302 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8f5c0ccc50 con 0x7f8f68101a10 2023-09-20T21:47:32.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.367+0000 7f8f66ffd700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8f500a1380 0x7f8f500a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:32.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.368+0000 7f8f4ffff700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] conn(0x7f8f500ac320 0x7f8f500ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:32.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.368+0000 7f8f4ffff700 1 -- 172.21.15.22:0/1376388302 --> [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f500aee90 con 0x7f8f500ac320 2023-09-20T21:47:32.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.368+0000 7f8f4ffff700 1 -- 172.21.15.22:0/1376388302 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f5c0ccfc0 con 0x7f8f68101a10 2023-09-20T21:47:32.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.368+0000 7f8f677fe700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] conn(0x7f8f500ac320 0x7f8f500ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:32.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.368+0000 7f8f66ffd700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8f500a1380 0x7f8f500a3840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009c60 tx=0x7f8f58009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:32.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.368+0000 7f8f677fe700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] conn(0x7f8f500ac320 0x7f8f500ae7c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:32.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.369+0000 7f8f4ffff700 1 -- 172.21.15.22:0/1376388302 <== osd.4 v2:172.21.15.89:6808/1802541493 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f8f500aee90 con 0x7f8f500ac320 2023-09-20T21:47:32.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.386+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 --> [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f6804eae0 con 0x7f8f500ac320 2023-09-20T21:47:32.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.387+0000 7f8f4ffff700 1 -- 172.21.15.22:0/1376388302 <== osd.4 v2:172.21.15.89:6808/1802541493 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8f6804eae0 con 0x7f8f500ac320 2023-09-20T21:47:32.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.387+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 >> [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] conn(0x7f8f500ac320 msgr2=0x7f8f500ae7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:32.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.387+0000 7f8f6d422700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] conn(0x7f8f500ac320 0x7f8f500ae7c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:32.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.388+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8f500a1380 msgr2=0x7f8f500a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:32.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.388+0000 7f8f6d422700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8f500a1380 0x7f8f500a3840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009c60 tx=0x7f8f58009260 comp rx=0 tx=0).stop 2023-09-20T21:47:32.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.388+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f68101a10 msgr2=0x7f8f681997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:32.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.388+0000 7f8f6d422700 1 --2- 172.21.15.22:0/1376388302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f68101a10 0x7f8f681997a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00d980 tx=0x7f8f5c00dc90 comp rx=0 tx=0).stop 2023-09-20T21:47:32.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.388+0000 7f8f677fe700 1 -- 172.21.15.22:0/1376388302 reap_dead start 2023-09-20T21:47:32.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.388+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 shutdown_connections 2023-09-20T21:47:32.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.388+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 >> 172.21.15.22:0/1376388302 conn(0x7f8f680fc8d0 msgr2=0x7f8f680fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:32.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.388+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 shutdown_connections 2023-09-20T21:47:32.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:32.388+0000 7f8f6d422700 1 -- 172.21.15.22:0/1376388302 wait complete. 2023-09-20T21:47:32.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:32 smithi123 ceph-mon[134433]: pgmap v303: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:47:32.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:32 smithi089 ceph-mon[134090]: pgmap v303: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:47:33.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:33 smithi123 ceph-mon[134433]: pgmap v304: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:47:33.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:33 smithi022 ceph-mon[97651]: pgmap v304: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:47:33.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:33 smithi089 ceph-mon[134090]: pgmap v304: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:47:34.326 INFO:teuthology.orchestra.run.smithi022.stdout:141733920822 2023-09-20T21:47:34.326 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.4 2023-09-20T21:47:34.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.629+0000 7f448deab700 1 -- 172.21.15.22:0/1441389968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4488101a10 msgr2=0x7f4488105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:34.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.629+0000 7f448deab700 1 --2- 172.21.15.22:0/1441389968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4488101a10 0x7f4488105a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4470009990 tx=0x7f4470009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:34.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.629+0000 7f448deab700 1 -- 172.21.15.22:0/1441389968 shutdown_connections 2023-09-20T21:47:34.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.629+0000 7f448deab700 1 --2- 172.21.15.22:0/1441389968 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4488106140 0x7f448810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:34.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.629+0000 7f448deab700 1 --2- 172.21.15.22:0/1441389968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4488101060 0x7f4488101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:34.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.629+0000 7f448deab700 1 --2- 172.21.15.22:0/1441389968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4488101a10 0x7f4488105a60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:34.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.629+0000 7f448deab700 1 -- 172.21.15.22:0/1441389968 >> 172.21.15.22:0/1441389968 conn(0x7f44880fc8d0 msgr2=0x7f44880fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:34.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.629+0000 7f448deab700 1 -- 172.21.15.22:0/1441389968 shutdown_connections 2023-09-20T21:47:34.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.629+0000 7f448deab700 1 -- 172.21.15.22:0/1441389968 wait complete. 2023-09-20T21:47:34.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.631+0000 7f448deab700 1 Processor -- start 2023-09-20T21:47:34.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.631+0000 7f448deab700 1 -- start start 2023-09-20T21:47:34.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f448deab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4488101060 0x7f44880730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:34.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f448deab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4488101a10 0x7f4488073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:34.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f448deab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4488106140 0x7f4488073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:34.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f448deab700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44881a5400 con 0x7f4488101a10 2023-09-20T21:47:34.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f448deab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44881a5570 con 0x7f4488106140 2023-09-20T21:47:34.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f448deab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44881a5730 con 0x7f4488101060 2023-09-20T21:47:34.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f4486ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4488101a10 0x7f4488073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:34.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f4486ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4488101a10 0x7f4488073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60136/0 (socket says 172.21.15.22:60136) 2023-09-20T21:47:34.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f4486ffd700 1 -- 172.21.15.22:0/765490831 learned_addr learned my addr 172.21.15.22:0/765490831 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:34.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f44877fe700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4488101060 0x7f44880730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:34.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f4487fff700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4488106140 0x7f4488073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:34.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f4486ffd700 1 -- 172.21.15.22:0/765490831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4488101060 msgr2=0x7f44880730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:34.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f4486ffd700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4488101060 0x7f44880730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:34.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f4486ffd700 1 -- 172.21.15.22:0/765490831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4488106140 msgr2=0x7f4488073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:34.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f4486ffd700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4488106140 0x7f4488073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:34.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f4486ffd700 1 -- 172.21.15.22:0/765490831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44700095f0 con 0x7f4488101a10 2023-09-20T21:47:34.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.632+0000 7f4487fff700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4488106140 0x7f4488073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:47:34.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.633+0000 7f4486ffd700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4488101a10 0x7f4488073620 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4470000c00 tx=0x7f4470009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:34.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.633+0000 7f4484ff9700 1 -- 172.21.15.22:0/765490831 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447001d070 con 0x7f4488101a10 2023-09-20T21:47:34.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.633+0000 7f4484ff9700 1 -- 172.21.15.22:0/765490831 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4470022470 con 0x7f4488101a10 2023-09-20T21:47:34.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.633+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44881a59b0 con 0x7f4488101a10 2023-09-20T21:47:34.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.633+0000 7f4484ff9700 1 -- 172.21.15.22:0/765490831 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470017650 con 0x7f4488101a10 2023-09-20T21:47:34.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.633+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44881a5e20 con 0x7f4488101a10 2023-09-20T21:47:34.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.634+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4488110f70 con 0x7f4488101a10 2023-09-20T21:47:34.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.636+0000 7f4484ff9700 1 -- 172.21.15.22:0/765490831 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f4470022870 con 0x7f4488101a10 2023-09-20T21:47:34.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.639+0000 7f4484ff9700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44740a1380 0x7f44740a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:34.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.639+0000 7f4484ff9700 1 -- 172.21.15.22:0/765490831 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f44700ce170 con 0x7f4488101a10 2023-09-20T21:47:34.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.639+0000 7f44877fe700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44740a1380 0x7f44740a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:34.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.640+0000 7f4484ff9700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] conn(0x7f44740ac320 0x7f44740ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:34.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.640+0000 7f4484ff9700 1 -- 172.21.15.22:0/765490831 --> [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44740aee90 con 0x7f44740ac320 2023-09-20T21:47:34.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.640+0000 7f4484ff9700 1 -- 172.21.15.22:0/765490831 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44700d0750 con 0x7f4488101a10 2023-09-20T21:47:34.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.640+0000 7f44877fe700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44740a1380 0x7f44740a3840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4480009c60 tx=0x7f4480009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:34.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.640+0000 7f4487fff700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] conn(0x7f44740ac320 0x7f44740ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:35.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.641+0000 7f4487fff700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] conn(0x7f44740ac320 0x7f44740ae7c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:35.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.641+0000 7f4484ff9700 1 -- 172.21.15.22:0/765490831 <== osd.2 v2:172.21.15.123:6800/877674627 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f44740aee90 con 0x7f44740ac320 2023-09-20T21:47:35.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.666+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 --> [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f448804eae0 con 0x7f44740ac320 2023-09-20T21:47:35.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.666+0000 7f4484ff9700 1 -- 172.21.15.22:0/765490831 <== osd.2 v2:172.21.15.123:6800/877674627 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f448804eae0 con 0x7f44740ac320 2023-09-20T21:47:35.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 >> [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] conn(0x7f44740ac320 msgr2=0x7f44740ae7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:35.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f448deab700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] conn(0x7f44740ac320 0x7f44740ae7c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:35.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44740a1380 msgr2=0x7f44740a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:35.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f448deab700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f44740a1380 0x7f44740a3840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4480009c60 tx=0x7f4480009260 comp rx=0 tx=0).stop 2023-09-20T21:47:35.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4488101a10 msgr2=0x7f4488073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:35.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f448deab700 1 --2- 172.21.15.22:0/765490831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4488101a10 0x7f4488073620 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4470000c00 tx=0x7f4470009580 comp rx=0 tx=0).stop 2023-09-20T21:47:35.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f4487fff700 1 -- 172.21.15.22:0/765490831 reap_dead start 2023-09-20T21:47:35.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 shutdown_connections 2023-09-20T21:47:35.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 >> 172.21.15.22:0/765490831 conn(0x7f44880fc8d0 msgr2=0x7f44880fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:35.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 shutdown_connections 2023-09-20T21:47:35.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.667+0000 7f448deab700 1 -- 172.21.15.22:0/765490831 wait complete. 2023-09-20T21:47:35.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.206+0000 7fedfe023700 1 -- 172.21.15.22:0/2417596396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedf8106140 msgr2=0x7fedf810dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:35.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.206+0000 7fedfe023700 1 --2- 172.21.15.22:0/2417596396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedf8106140 0x7fedf810dd10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fedf000b0a0 tx=0x7fedf000b3b0 comp rx=0 tx=0).stop 2023-09-20T21:47:35.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.206+0000 7fedfe023700 1 -- 172.21.15.22:0/2417596396 shutdown_connections 2023-09-20T21:47:35.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.206+0000 7fedfe023700 1 --2- 172.21.15.22:0/2417596396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedf8106140 0x7fedf810dd10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:35.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.206+0000 7fedfe023700 1 --2- 172.21.15.22:0/2417596396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedf8101060 0x7fedf8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:35.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.206+0000 7fedfe023700 1 --2- 172.21.15.22:0/2417596396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedf8101a10 0x7fedf8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:35.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.206+0000 7fedfe023700 1 -- 172.21.15.22:0/2417596396 >> 172.21.15.22:0/2417596396 conn(0x7fedf80fc8d0 msgr2=0x7fedf80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:35.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.206+0000 7fedfe023700 1 -- 172.21.15.22:0/2417596396 shutdown_connections 2023-09-20T21:47:35.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.206+0000 7fedfe023700 1 -- 172.21.15.22:0/2417596396 wait complete. 2023-09-20T21:47:35.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.207+0000 7fedfe023700 1 Processor -- start 2023-09-20T21:47:35.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedfe023700 1 -- start start 2023-09-20T21:47:35.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedfe023700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedf8101060 0x7fedf8196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:35.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedfe023700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedf8101a10 0x7fedf8197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:35.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedfe023700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedf8106140 0x7fedf819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:35.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedfe023700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf819bed0 con 0x7fedf8101a10 2023-09-20T21:47:35.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedfe023700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf819c040 con 0x7fedf8101060 2023-09-20T21:47:35.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedfe023700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf819c200 con 0x7fedf8106140 2023-09-20T21:47:35.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedf6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedf8101a10 0x7fedf8197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:35.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedf6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedf8101a10 0x7fedf8197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60108/0 (socket says 172.21.15.22:60108) 2023-09-20T21:47:35.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedf6ffd700 1 -- 172.21.15.22:0/388861867 learned_addr learned my addr 172.21.15.22:0/388861867 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:35.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.208+0000 7fedf7fff700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedf8106140 0x7fedf819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:35.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedf6ffd700 1 -- 172.21.15.22:0/388861867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedf8106140 msgr2=0x7fedf819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:35.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedf6ffd700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedf8106140 0x7fedf819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:35.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedf6ffd700 1 -- 172.21.15.22:0/388861867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedf8101060 msgr2=0x7fedf8196f10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-20T21:47:35.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedf6ffd700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fedf8101060 0x7fedf8196f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:35.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedf6ffd700 1 -- 172.21.15.22:0/388861867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0009d00 con 0x7fedf8101a10 2023-09-20T21:47:35.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedf6ffd700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedf8101a10 0x7fedf8197450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b550 tx=0x7fedec00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:35.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedf4ff9700 1 -- 172.21.15.22:0/388861867 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec011830 con 0x7fedf8101a10 2023-09-20T21:47:35.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf819c4e0 con 0x7fedf8101a10 2023-09-20T21:47:35.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedf4ff9700 1 -- 172.21.15.22:0/388861867 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedec011e50 con 0x7fedf8101a10 2023-09-20T21:47:35.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedf4ff9700 1 -- 172.21.15.22:0/388861867 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec00f2e0 con 0x7fedf8101a10 2023-09-20T21:47:35.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.209+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf81a3d40 con 0x7fedf8101a10 2023-09-20T21:47:35.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.210+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fedf81110d0 con 0x7fedf8101a10 2023-09-20T21:47:35.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.213+0000 7fedf4ff9700 1 -- 172.21.15.22:0/388861867 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fedec010580 con 0x7fedf8101a10 2023-09-20T21:47:35.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.214+0000 7fedf4ff9700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fede00a1380 0x7fede00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:35.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.214+0000 7fedf4ff9700 1 -- 172.21.15.22:0/388861867 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fedec0ccf90 con 0x7fedf8101a10 2023-09-20T21:47:35.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.214+0000 7fedf77fe700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fede00a1380 0x7fede00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:35.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.215+0000 7fedf4ff9700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] conn(0x7fede00ac320 0x7fede00ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:35.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.215+0000 7fedf4ff9700 1 -- 172.21.15.22:0/388861867 --> [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fede00aee90 con 0x7fede00ac320 2023-09-20T21:47:35.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.215+0000 7fedf4ff9700 1 -- 172.21.15.22:0/388861867 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedec0cd300 con 0x7fedf8101a10 2023-09-20T21:47:35.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.215+0000 7fedf7fff700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] conn(0x7fede00ac320 0x7fede00ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:35.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.215+0000 7fedf77fe700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fede00a1380 0x7fede00a3840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fede8009af0 tx=0x7fede8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:35.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.215+0000 7fedf7fff700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] conn(0x7fede00ac320 0x7fede00ae7c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:35.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.216+0000 7fedf4ff9700 1 -- 172.21.15.22:0/388861867 <== osd.8 v2:172.21.15.123:6816/1023938376 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fede00aee90 con 0x7fede00ac320 2023-09-20T21:47:35.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.234+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 --> [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fedf804eae0 con 0x7fede00ac320 2023-09-20T21:47:35.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.234+0000 7fedf4ff9700 1 -- 172.21.15.22:0/388861867 <== osd.8 v2:172.21.15.123:6816/1023938376 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fedf804eae0 con 0x7fede00ac320 2023-09-20T21:47:35.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 >> [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] conn(0x7fede00ac320 msgr2=0x7fede00ae7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:35.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedfe023700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] conn(0x7fede00ac320 0x7fede00ae7c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:35.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fede00a1380 msgr2=0x7fede00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:35.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedfe023700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fede00a1380 0x7fede00a3840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fede8009af0 tx=0x7fede8009260 comp rx=0 tx=0).stop 2023-09-20T21:47:35.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedf8101a10 msgr2=0x7fedf8197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:35.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedfe023700 1 --2- 172.21.15.22:0/388861867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fedf8101a10 0x7fedf8197450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b550 tx=0x7fedec00b860 comp rx=0 tx=0).stop 2023-09-20T21:47:35.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedf7fff700 1 -- 172.21.15.22:0/388861867 reap_dead start 2023-09-20T21:47:35.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 shutdown_connections 2023-09-20T21:47:35.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 >> 172.21.15.22:0/388861867 conn(0x7fedf80fc8d0 msgr2=0x7fedf80fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:35.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 shutdown_connections 2023-09-20T21:47:35.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:34.235+0000 7fedfe023700 1 -- 172.21.15.22:0/388861867 wait complete. 2023-09-20T21:47:36.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:36 smithi022 ceph-mon[97651]: pgmap v305: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:47:36.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:36 smithi123 ceph-mon[134433]: pgmap v305: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:47:36.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:36 smithi089 ceph-mon[134090]: pgmap v305: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:47:36.861 INFO:teuthology.orchestra.run.smithi022.stdout:253403070500 2023-09-20T21:47:36.862 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.8 2023-09-20T21:47:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.619+0000 7fbb22e43700 1 -- 172.21.15.22:0/3949383441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c075c30 msgr2=0x7fbb1c111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.619+0000 7fbb22e43700 1 --2- 172.21.15.22:0/3949383441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c075c30 0x7fbb1c111780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009990 tx=0x7fbb08009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.619+0000 7fbb22e43700 1 -- 172.21.15.22:0/3949383441 shutdown_connections 2023-09-20T21:47:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.619+0000 7fbb22e43700 1 --2- 172.21.15.22:0/3949383441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb1c111cc0 0x7fbb1c1140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.619+0000 7fbb22e43700 1 --2- 172.21.15.22:0/3949383441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1c075310 0x7fbb1c0756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.619+0000 7fbb22e43700 1 --2- 172.21.15.22:0/3949383441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c075c30 0x7fbb1c111780 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.619+0000 7fbb22e43700 1 -- 172.21.15.22:0/3949383441 >> 172.21.15.22:0/3949383441 conn(0x7fbb1c0feaf0 msgr2=0x7fbb1c100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:37.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.620+0000 7fbb22e43700 1 -- 172.21.15.22:0/3949383441 shutdown_connections 2023-09-20T21:47:37.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.620+0000 7fbb22e43700 1 -- 172.21.15.22:0/3949383441 wait complete. 2023-09-20T21:47:37.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.621+0000 7fbb22e43700 1 Processor -- start 2023-09-20T21:47:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.622+0000 7fbb22e43700 1 -- start start 2023-09-20T21:47:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.622+0000 7fbb22e43700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c075310 0x7fbb1c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.622+0000 7fbb22e43700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1c075c30 0x7fbb1c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.622+0000 7fbb22e43700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb1c111cc0 0x7fbb1c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.622+0000 7fbb22e43700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a2520 con 0x7fbb1c111cc0 2023-09-20T21:47:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.622+0000 7fbb22e43700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a2690 con 0x7fbb1c075310 2023-09-20T21:47:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.622+0000 7fbb22e43700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a2850 con 0x7fbb1c075c30 2023-09-20T21:47:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.623+0000 7fbb20bdf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c075310 0x7fbb1c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.623+0000 7fbb1bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1c075c30 0x7fbb1c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.623+0000 7fbb20bdf700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c075310 0x7fbb1c19d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:59216/0 (socket says 172.21.15.22:59216) 2023-09-20T21:47:37.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.623+0000 7fbb20bdf700 1 -- 172.21.15.22:0/1782539182 learned_addr learned my addr 172.21.15.22:0/1782539182 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:37.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.623+0000 7fbb213e0700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb1c111cc0 0x7fbb1c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb20bdf700 1 -- 172.21.15.22:0/1782539182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1c075c30 msgr2=0x7fbb1c19daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb20bdf700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1c075c30 0x7fbb1c19daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb20bdf700 1 -- 172.21.15.22:0/1782539182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb1c111cc0 msgr2=0x7fbb1c1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb20bdf700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb1c111cc0 0x7fbb1c1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb20bdf700 1 -- 172.21.15.22:0/1782539182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb080095f0 con 0x7fbb1c075310 2023-09-20T21:47:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb213e0700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb1c111cc0 0x7fbb1c1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:47:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb20bdf700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c075310 0x7fbb1c19d560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000d950 tx=0x7fbb1000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb19ffb700 1 -- 172.21.15.22:0/1782539182 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10009940 con 0x7fbb1c075310 2023-09-20T21:47:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb19ffb700 1 -- 172.21.15.22:0/1782539182 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb10010460 con 0x7fbb1c075310 2023-09-20T21:47:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb19ffb700 1 -- 172.21.15.22:0/1782539182 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1000f620 con 0x7fbb1c075310 2023-09-20T21:47:37.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.624+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c1a2b30 con 0x7fbb1c075310 2023-09-20T21:47:37.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.625+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c1aa390 con 0x7fbb1c075310 2023-09-20T21:47:37.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.626+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb1c04eae0 con 0x7fbb1c075310 2023-09-20T21:47:37.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.629+0000 7fbb19ffb700 1 -- 172.21.15.22:0/1782539182 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fbb100108d0 con 0x7fbb1c075310 2023-09-20T21:47:37.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.632+0000 7fbb19ffb700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb0c0a13d0 0x7fbb0c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:37.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.632+0000 7fbb19ffb700 1 -- 172.21.15.22:0/1782539182 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fbb100ccd40 con 0x7fbb1c075310 2023-09-20T21:47:37.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.632+0000 7fbb1bfff700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb0c0a13d0 0x7fbb0c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:37.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.633+0000 7fbb19ffb700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] conn(0x7fbb0c0ac370 0x7fbb0c0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:37.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.633+0000 7fbb19ffb700 1 -- 172.21.15.22:0/1782539182 --> [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb0c0aeee0 con 0x7fbb0c0ac370 2023-09-20T21:47:37.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.633+0000 7fbb19ffb700 1 -- 172.21.15.22:0/1782539182 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb100cf320 con 0x7fbb1c075310 2023-09-20T21:47:37.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.633+0000 7fbb213e0700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] conn(0x7fbb0c0ac370 0x7fbb0c0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:37.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.633+0000 7fbb1bfff700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb0c0a13d0 0x7fbb0c0a3890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800b2e0 tx=0x7fbb080058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:37.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.634+0000 7fbb213e0700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] conn(0x7fbb0c0ac370 0x7fbb0c0ae810 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:37.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.635+0000 7fbb19ffb700 1 -- 172.21.15.22:0/1782539182 <== osd.7 v2:172.21.15.89:6816/4165516997 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fbb0c0aeee0 con 0x7fbb0c0ac370 2023-09-20T21:47:37.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.670+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 --> [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb1c117770 con 0x7fbb0c0ac370 2023-09-20T21:47:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.671+0000 7fbb19ffb700 1 -- 172.21.15.22:0/1782539182 <== osd.7 v2:172.21.15.89:6816/4165516997 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbb1400f040 con 0x7fbb0c0ac370 2023-09-20T21:47:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 >> [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] conn(0x7fbb0c0ac370 msgr2=0x7fbb0c0ae810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb22e43700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] conn(0x7fbb0c0ac370 0x7fbb0c0ae810 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb0c0a13d0 msgr2=0x7fbb0c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb22e43700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb0c0a13d0 0x7fbb0c0a3890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800b2e0 tx=0x7fbb080058e0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c075310 msgr2=0x7fbb1c19d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb22e43700 1 --2- 172.21.15.22:0/1782539182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb1c075310 0x7fbb1c19d560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000d950 tx=0x7fbb1000dc60 comp rx=0 tx=0).stop 2023-09-20T21:47:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb213e0700 1 -- 172.21.15.22:0/1782539182 reap_dead start 2023-09-20T21:47:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 shutdown_connections 2023-09-20T21:47:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 >> 172.21.15.22:0/1782539182 conn(0x7fbb1c0feaf0 msgr2=0x7fbb1c100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 shutdown_connections 2023-09-20T21:47:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.672+0000 7fbb22e43700 1 -- 172.21.15.22:0/1782539182 wait complete. 2023-09-20T21:47:37.733 INFO:teuthology.orchestra.run.smithi022.stdout:77309411393 2023-09-20T21:47:37.734 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.2 2023-09-20T21:47:37.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.796+0000 7f4977ce9700 1 -- 172.21.15.22:0/4257417321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49700692c0 msgr2=0x7f497010dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.796+0000 7f4977ce9700 1 --2- 172.21.15.22:0/4257417321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49700692c0 0x7f497010dc20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4968009f90 tx=0x7f496800b330 comp rx=0 tx=0).stop 2023-09-20T21:47:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.797+0000 7f4977ce9700 1 -- 172.21.15.22:0/4257417321 shutdown_connections 2023-09-20T21:47:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.797+0000 7f4977ce9700 1 --2- 172.21.15.22:0/4257417321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49700692c0 0x7f497010dc20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.797+0000 7f4977ce9700 1 --2- 172.21.15.22:0/4257417321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4970106f70 0x7f4970107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.797+0000 7f4977ce9700 1 --2- 172.21.15.22:0/4257417321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4970068900 0x7f4970068d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.797+0000 7f4977ce9700 1 -- 172.21.15.22:0/4257417321 >> 172.21.15.22:0/4257417321 conn(0x7f4970076de0 msgr2=0x7f49700771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:37.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.797+0000 7f4977ce9700 1 -- 172.21.15.22:0/4257417321 shutdown_connections 2023-09-20T21:47:37.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.797+0000 7f4977ce9700 1 -- 172.21.15.22:0/4257417321 wait complete. 2023-09-20T21:47:37.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.799+0000 7f4977ce9700 1 Processor -- start 2023-09-20T21:47:37.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.799+0000 7f4977ce9700 1 -- start start 2023-09-20T21:47:37.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.799+0000 7f4977ce9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4970068900 0x7f497019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:37.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4977ce9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49700692c0 0x7f497019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:37.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4977ce9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4970106f70 0x7f49701a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:37.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4977ce9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49701a2500 con 0x7f4970068900 2023-09-20T21:47:37.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4977ce9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49701a2670 con 0x7f49700692c0 2023-09-20T21:47:37.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4977ce9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49701a2830 con 0x7f4970106f70 2023-09-20T21:47:37.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4975284700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49700692c0 0x7f497019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:37.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4975284700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49700692c0 0x7f497019da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:59254/0 (socket says 172.21.15.22:59254) 2023-09-20T21:47:37.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4975284700 1 -- 172.21.15.22:0/2761887820 learned_addr learned my addr 172.21.15.22:0/2761887820 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:37.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4976286700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4970106f70 0x7f49701a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:37.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4975a85700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4970068900 0x7f497019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:37.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4975284700 1 -- 172.21.15.22:0/2761887820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4970106f70 msgr2=0x7f49701a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4975284700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4970106f70 0x7f49701a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4975284700 1 -- 172.21.15.22:0/2761887820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4970068900 msgr2=0x7f497019d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4975284700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4970068900 0x7f497019d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.800+0000 7f4975284700 1 -- 172.21.15.22:0/2761887820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4968009c40 con 0x7f49700692c0 2023-09-20T21:47:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.801+0000 7f4975a85700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4970068900 0x7f497019d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:47:37.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.801+0000 7f4975284700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49700692c0 0x7f497019da80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f496000d950 tx=0x7f496000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.801+0000 7f4966ffd700 1 -- 172.21.15.22:0/2761887820 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49600098c0 con 0x7f49700692c0 2023-09-20T21:47:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.801+0000 7f4966ffd700 1 -- 172.21.15.22:0/2761887820 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4960010460 con 0x7f49700692c0 2023-09-20T21:47:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.801+0000 7f4966ffd700 1 -- 172.21.15.22:0/2761887820 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000f620 con 0x7f49700692c0 2023-09-20T21:47:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.801+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49701a2b10 con 0x7f49700692c0 2023-09-20T21:47:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.801+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49701aa370 con 0x7f49700692c0 2023-09-20T21:47:37.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.802+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f497010ef10 con 0x7f49700692c0 2023-09-20T21:47:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.804+0000 7f4966ffd700 1 -- 172.21.15.22:0/2761887820 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f49600108d0 con 0x7f49700692c0 2023-09-20T21:47:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.806+0000 7f4966ffd700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f495c0a1330 0x7f495c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.806+0000 7f4966ffd700 1 -- 172.21.15.22:0/2761887820 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f49600cce50 con 0x7f49700692c0 2023-09-20T21:47:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.807+0000 7f4966ffd700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] conn(0x7f495c0ac2d0 0x7f495c0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:37.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.807+0000 7f4966ffd700 1 -- 172.21.15.22:0/2761887820 --> [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f495c0aee40 con 0x7f495c0ac2d0 2023-09-20T21:47:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.807+0000 7f4966ffd700 1 -- 172.21.15.22:0/2761887820 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49600cd1c0 con 0x7f49700692c0 2023-09-20T21:47:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.807+0000 7f4975a85700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f495c0a1330 0x7f495c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.807+0000 7f4976286700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] conn(0x7f495c0ac2d0 0x7f495c0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.808+0000 7f4975a85700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f495c0a1330 0x7f495c0a37f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f496c005950 tx=0x7f496c0058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.808+0000 7f4976286700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] conn(0x7f495c0ac2d0 0x7f495c0ae770 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:37.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.809+0000 7f4966ffd700 1 -- 172.21.15.22:0/2761887820 <== osd.11 v2:172.21.15.123:6824/2107937412 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f495c0aee40 con 0x7f495c0ac2d0 2023-09-20T21:47:37.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.837+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 --> [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f497004eae0 con 0x7f495c0ac2d0 2023-09-20T21:47:37.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.838+0000 7f4966ffd700 1 -- 172.21.15.22:0/2761887820 <== osd.11 v2:172.21.15.123:6824/2107937412 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f497004eae0 con 0x7f495c0ac2d0 2023-09-20T21:47:37.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.839+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 >> [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] conn(0x7f495c0ac2d0 msgr2=0x7f495c0ae770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.839+0000 7f4977ce9700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] conn(0x7f495c0ac2d0 0x7f495c0ae770 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.839+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f495c0a1330 msgr2=0x7f495c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.839+0000 7f4977ce9700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f495c0a1330 0x7f495c0a37f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f496c005950 tx=0x7f496c0058e0 comp rx=0 tx=0).stop 2023-09-20T21:47:37.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.839+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49700692c0 msgr2=0x7f497019da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:37.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.839+0000 7f4977ce9700 1 --2- 172.21.15.22:0/2761887820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49700692c0 0x7f497019da80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f496000d950 tx=0x7f496000dc60 comp rx=0 tx=0).stop 2023-09-20T21:47:37.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.839+0000 7f4976286700 1 -- 172.21.15.22:0/2761887820 reap_dead start 2023-09-20T21:47:37.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.839+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 shutdown_connections 2023-09-20T21:47:37.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.839+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 >> 172.21.15.22:0/2761887820 conn(0x7f4970076de0 msgr2=0x7f4970107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:37.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.840+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 shutdown_connections 2023-09-20T21:47:37.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.840+0000 7f4977ce9700 1 -- 172.21.15.22:0/2761887820 wait complete. 2023-09-20T21:47:38.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.720+0000 7fd11baea700 1 -- 172.21.15.22:0/191666885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd114106f70 msgr2=0x7fd114107350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.720+0000 7fd11baea700 1 --2- 172.21.15.22:0/191666885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd114106f70 0x7fd114107350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd110009990 tx=0x7fd110009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.720+0000 7fd11baea700 1 -- 172.21.15.22:0/191666885 shutdown_connections 2023-09-20T21:47:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.720+0000 7fd11baea700 1 --2- 172.21.15.22:0/191666885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1140692c0 0x7fd11410dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.720+0000 7fd11baea700 1 --2- 172.21.15.22:0/191666885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd114106f70 0x7fd114107350 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.720+0000 7fd11baea700 1 --2- 172.21.15.22:0/191666885 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd114068900 0x7fd114068d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:38.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.720+0000 7fd11baea700 1 -- 172.21.15.22:0/191666885 >> 172.21.15.22:0/191666885 conn(0x7fd114076de0 msgr2=0x7fd1140771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:38.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.720+0000 7fd11baea700 1 -- 172.21.15.22:0/191666885 shutdown_connections 2023-09-20T21:47:38.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.720+0000 7fd11baea700 1 -- 172.21.15.22:0/191666885 wait complete. 2023-09-20T21:47:38.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.722+0000 7fd11baea700 1 Processor -- start 2023-09-20T21:47:38.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.722+0000 7fd11baea700 1 -- start start 2023-09-20T21:47:38.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.722+0000 7fd11baea700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd114068900 0x7fd114196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:38.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.722+0000 7fd11baea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1140692c0 0x7fd1141974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:38.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.722+0000 7fd11baea700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd114106f70 0x7fd11419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:38.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd11baea700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11419bf30 con 0x7fd114106f70 2023-09-20T21:47:38.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd11baea700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11419c0a0 con 0x7fd114068900 2023-09-20T21:47:38.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd11baea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11419c260 con 0x7fd1140692c0 2023-09-20T21:47:38.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd119886700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd114068900 0x7fd114196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:38.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd119886700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd114068900 0x7fd114196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:59246/0 (socket says 172.21.15.22:59246) 2023-09-20T21:47:38.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd119886700 1 -- 172.21.15.22:0/1275810332 learned_addr learned my addr 172.21.15.22:0/1275810332 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:38.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd11a087700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd114106f70 0x7fd11419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:38.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd119085700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1140692c0 0x7fd1141974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd119886700 1 -- 172.21.15.22:0/1275810332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1140692c0 msgr2=0x7fd1141974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd119886700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1140692c0 0x7fd1141974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd119886700 1 -- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd114106f70 msgr2=0x7fd11419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd119886700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd114106f70 0x7fd11419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.723+0000 7fd119886700 1 -- 172.21.15.22:0/1275810332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd104009520 con 0x7fd114068900 2023-09-20T21:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.724+0000 7fd119886700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd114068900 0x7fd114196f70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd110005910 tx=0x7fd110005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.724+0000 7fd10affd700 1 -- 172.21.15.22:0/1275810332 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11001d070 con 0x7fd114068900 2023-09-20T21:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.724+0000 7fd10affd700 1 -- 172.21.15.22:0/1275810332 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd110022470 con 0x7fd114068900 2023-09-20T21:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.724+0000 7fd10affd700 1 -- 172.21.15.22:0/1275810332 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110017650 con 0x7fd114068900 2023-09-20T21:47:38.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.724+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1100095f0 con 0x7fd114068900 2023-09-20T21:47:38.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.724+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11419c710 con 0x7fd114068900 2023-09-20T21:47:38.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.725+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd114111010 con 0x7fd114068900 2023-09-20T21:47:38.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.729+0000 7fd10affd700 1 -- 172.21.15.22:0/1275810332 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fd1100225e0 con 0x7fd114068900 2023-09-20T21:47:38.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.730+0000 7fd10affd700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1000a1300 0x7fd1000a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:38.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.731+0000 7fd10affd700 1 -- 172.21.15.22:0/1275810332 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd1100ced40 con 0x7fd114068900 2023-09-20T21:47:38.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.731+0000 7fd119085700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1000a1300 0x7fd1000a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:38.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.731+0000 7fd10affd700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] conn(0x7fd1000ac2a0 0x7fd1000ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:38.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.731+0000 7fd10affd700 1 -- 172.21.15.22:0/1275810332 --> [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1000aee10 con 0x7fd1000ac2a0 2023-09-20T21:47:38.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.731+0000 7fd10affd700 1 -- 172.21.15.22:0/1275810332 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1100cf0b0 con 0x7fd114068900 2023-09-20T21:47:38.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.731+0000 7fd11a087700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] conn(0x7fd1000ac2a0 0x7fd1000ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:38.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.731+0000 7fd119085700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1000a1300 0x7fd1000a37c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd114198590 tx=0x7fd104009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:38.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.732+0000 7fd11a087700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] conn(0x7fd1000ac2a0 0x7fd1000ae740 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:38.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.733+0000 7fd10affd700 1 -- 172.21.15.22:0/1275810332 <== osd.0 v2:172.21.15.22:6802/3853681188 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd1000aee10 con 0x7fd1000ac2a0 2023-09-20T21:47:38.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.753+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 --> [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd11404eae0 con 0x7fd1000ac2a0 2023-09-20T21:47:38.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.753+0000 7fd10affd700 1 -- 172.21.15.22:0/1275810332 <== osd.0 v2:172.21.15.22:6802/3853681188 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd11404eae0 con 0x7fd1000ac2a0 2023-09-20T21:47:38.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] conn(0x7fd1000ac2a0 msgr2=0x7fd1000ae740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:38.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11baea700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] conn(0x7fd1000ac2a0 0x7fd1000ae740 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:38.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1000a1300 msgr2=0x7fd1000a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:38.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11baea700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1000a1300 0x7fd1000a37c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd114198590 tx=0x7fd104009260 comp rx=0 tx=0).stop 2023-09-20T21:47:38.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd114068900 msgr2=0x7fd114196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:38.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11baea700 1 --2- 172.21.15.22:0/1275810332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd114068900 0x7fd114196f70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd110005910 tx=0x7fd110005720 comp rx=0 tx=0).stop 2023-09-20T21:47:38.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11a087700 1 -- 172.21.15.22:0/1275810332 reap_dead start 2023-09-20T21:47:38.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 shutdown_connections 2023-09-20T21:47:38.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 >> 172.21.15.22:0/1275810332 conn(0x7fd114076de0 msgr2=0x7fd114104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:38.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 shutdown_connections 2023-09-20T21:47:38.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:37.754+0000 7fd11baea700 1 -- 172.21.15.22:0/1275810332 wait complete. 2023-09-20T21:47:38.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:38 smithi022 ceph-mon[97651]: pgmap v306: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:38.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:38 smithi123 ceph-mon[134433]: pgmap v306: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:38.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:38 smithi089 ceph-mon[134090]: pgmap v306: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:38.706 INFO:teuthology.orchestra.run.smithi022.stdout:343597383704 2023-09-20T21:47:38.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.11 2023-09-20T21:47:39.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.241+0000 7f40b7411700 1 -- 172.21.15.22:0/1205660212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0101050 msgr2=0x7f40b0101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.241+0000 7f40b7411700 1 --2- 172.21.15.22:0/1205660212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0101050 0x7f40b0101430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0098f0 tx=0x7f40ac009c00 comp rx=0 tx=0).stop 2023-09-20T21:47:39.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.241+0000 7f40b7411700 1 -- 172.21.15.22:0/1205660212 shutdown_connections 2023-09-20T21:47:39.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.241+0000 7f40b7411700 1 --2- 172.21.15.22:0/1205660212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b0106130 0x7f40b010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.241+0000 7f40b7411700 1 --2- 172.21.15.22:0/1205660212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0101050 0x7f40b0101430 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.241+0000 7f40b7411700 1 --2- 172.21.15.22:0/1205660212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b0101a00 0x7f40b0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.241+0000 7f40b7411700 1 -- 172.21.15.22:0/1205660212 >> 172.21.15.22:0/1205660212 conn(0x7f40b00fc8e0 msgr2=0x7f40b00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:39.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.241+0000 7f40b7411700 1 -- 172.21.15.22:0/1205660212 shutdown_connections 2023-09-20T21:47:39.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.241+0000 7f40b7411700 1 -- 172.21.15.22:0/1205660212 wait complete. 2023-09-20T21:47:39.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.242+0000 7f40b7411700 1 Processor -- start 2023-09-20T21:47:39.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.242+0000 7f40b7411700 1 -- start start 2023-09-20T21:47:39.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b7411700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b0101050 0x7f40b0196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b7411700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0101a00 0x7f40b0197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b7411700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b0106130 0x7f40b019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b7411700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b019bf10 con 0x7f40b0106130 2023-09-20T21:47:39.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b7411700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b019c080 con 0x7f40b0101a00 2023-09-20T21:47:39.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b7411700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b019c240 con 0x7f40b0101050 2023-09-20T21:47:39.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b59ae700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b0106130 0x7f40b019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b59ae700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b0106130 0x7f40b019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57540/0 (socket says 172.21.15.22:57540) 2023-09-20T21:47:39.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b49ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0101a00 0x7f40b0197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b59ae700 1 -- 172.21.15.22:0/3557853869 learned_addr learned my addr 172.21.15.22:0/3557853869 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:39.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b51ad700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b0101050 0x7f40b0196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b49ac700 1 -- 172.21.15.22:0/3557853869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b0101050 msgr2=0x7f40b0196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b49ac700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b0101050 0x7f40b0196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b49ac700 1 -- 172.21.15.22:0/3557853869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b0106130 msgr2=0x7f40b019b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b49ac700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b0106130 0x7f40b019b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.243+0000 7f40b49ac700 1 -- 172.21.15.22:0/3557853869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac009520 con 0x7f40b0101a00 2023-09-20T21:47:39.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.244+0000 7f40b49ac700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0101a00 0x7f40b0197490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f40a000e9c0 tx=0x7f40a000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.244+0000 7f40a67fc700 1 -- 172.21.15.22:0/3557853869 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a000c9c0 con 0x7f40b0101a00 2023-09-20T21:47:39.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.244+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b019c520 con 0x7f40b0101a00 2023-09-20T21:47:39.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.244+0000 7f40a67fc700 1 -- 172.21.15.22:0/3557853869 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40a0010430 con 0x7f40b0101a00 2023-09-20T21:47:39.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.244+0000 7f40a67fc700 1 -- 172.21.15.22:0/3557853869 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0018680 con 0x7f40b0101a00 2023-09-20T21:47:39.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.244+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b01a3d80 con 0x7f40b0101a00 2023-09-20T21:47:39.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.245+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40b0111090 con 0x7f40b0101a00 2023-09-20T21:47:39.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.247+0000 7f40a67fc700 1 -- 172.21.15.22:0/3557853869 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f40a000cb20 con 0x7f40b0101a00 2023-09-20T21:47:39.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.248+0000 7f40a67fc700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1300 0x7f409c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.249+0000 7f40a67fc700 1 -- 172.21.15.22:0/3557853869 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f40a0014070 con 0x7f40b0101a00 2023-09-20T21:47:39.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.249+0000 7f40b51ad700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1300 0x7f409c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.249+0000 7f40a67fc700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] conn(0x7f409c0ac2a0 0x7f409c0ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.249+0000 7f40a67fc700 1 -- 172.21.15.22:0/3557853869 --> [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f409c0aee10 con 0x7f409c0ac2a0 2023-09-20T21:47:39.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.249+0000 7f40a67fc700 1 -- 172.21.15.22:0/3557853869 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40a00cd6e0 con 0x7f40b0101a00 2023-09-20T21:47:39.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.249+0000 7f40b51ad700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1300 0x7f409c0a37c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f40ac005e90 tx=0x7f40ac019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.249+0000 7f40b59ae700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] conn(0x7f409c0ac2a0 0x7f409c0ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.250+0000 7f40b59ae700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] conn(0x7f409c0ac2a0 0x7f409c0ae740 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.251+0000 7f40a67fc700 1 -- 172.21.15.22:0/3557853869 <== osd.10 v2:172.21.15.89:6824/2445411927 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f409c0aee10 con 0x7f409c0ac2a0 2023-09-20T21:47:39.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.276+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 --> [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f40b004eae0 con 0x7f409c0ac2a0 2023-09-20T21:47:39.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.276+0000 7f40a67fc700 1 -- 172.21.15.22:0/3557853869 <== osd.10 v2:172.21.15.89:6824/2445411927 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f40b004eae0 con 0x7f409c0ac2a0 2023-09-20T21:47:39.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.277+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 >> [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] conn(0x7f409c0ac2a0 msgr2=0x7f409c0ae740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.277+0000 7f40b7411700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] conn(0x7f409c0ac2a0 0x7f409c0ae740 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.277+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1300 msgr2=0x7f409c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.277+0000 7f40b7411700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1300 0x7f409c0a37c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f40ac005e90 tx=0x7f40ac019040 comp rx=0 tx=0).stop 2023-09-20T21:47:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.277+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0101a00 msgr2=0x7f40b0197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.277+0000 7f40b7411700 1 --2- 172.21.15.22:0/3557853869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0101a00 0x7f40b0197490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f40a000e9c0 tx=0x7f40a000ecd0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.277+0000 7f40b59ae700 1 -- 172.21.15.22:0/3557853869 reap_dead start 2023-09-20T21:47:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.277+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 shutdown_connections 2023-09-20T21:47:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.277+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 >> 172.21.15.22:0/3557853869 conn(0x7f40b00fc8e0 msgr2=0x7f40b00fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.278+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 shutdown_connections 2023-09-20T21:47:39.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.278+0000 7f40b7411700 1 -- 172.21.15.22:0/3557853869 wait complete. 2023-09-20T21:47:39.311 INFO:teuthology.orchestra.run.smithi022.stdout:34359738441 2023-09-20T21:47:39.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.0 2023-09-20T21:47:39.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.638+0000 7f7595ea7700 1 -- 172.21.15.22:0/3845593089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7590101060 msgr2=0x7f7590101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.638+0000 7f7595ea7700 1 --2- 172.21.15.22:0/3845593089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7590101060 0x7f7590101440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f75800098f0 tx=0x7f7580009c00 comp rx=0 tx=0).stop 2023-09-20T21:47:39.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.638+0000 7f7595ea7700 1 -- 172.21.15.22:0/3845593089 shutdown_connections 2023-09-20T21:47:39.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.638+0000 7f7595ea7700 1 --2- 172.21.15.22:0/3845593089 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7590106140 0x7f759010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.638+0000 7f7595ea7700 1 --2- 172.21.15.22:0/3845593089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7590101060 0x7f7590101440 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.638+0000 7f7595ea7700 1 --2- 172.21.15.22:0/3845593089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7590101a10 0x7f7590105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.638+0000 7f7595ea7700 1 -- 172.21.15.22:0/3845593089 >> 172.21.15.22:0/3845593089 conn(0x7f75900fc8d0 msgr2=0x7f75900fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:39.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.638+0000 7f7595ea7700 1 -- 172.21.15.22:0/3845593089 shutdown_connections 2023-09-20T21:47:39.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.638+0000 7f7595ea7700 1 -- 172.21.15.22:0/3845593089 wait complete. 2023-09-20T21:47:39.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.639+0000 7f7595ea7700 1 Processor -- start 2023-09-20T21:47:39.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.639+0000 7f7595ea7700 1 -- start start 2023-09-20T21:47:39.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f7595ea7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7590101060 0x7f7590199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f7595ea7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7590101a10 0x7f75901996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f7595ea7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7590106140 0x7f759019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f7595ea7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019e120 con 0x7f7590101060 2023-09-20T21:47:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f7595ea7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019e290 con 0x7f7590106140 2023-09-20T21:47:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f7595ea7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019e450 con 0x7f7590101a10 2023-09-20T21:47:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f758f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7590101060 0x7f7590199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f758f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7590101060 0x7f7590199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57508/0 (socket says 172.21.15.22:57508) 2023-09-20T21:47:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f758f7fe700 1 -- 172.21.15.22:0/48139725 learned_addr learned my addr 172.21.15.22:0/48139725 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f758effd700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7590101a10 0x7f75901996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f758ffff700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7590106140 0x7f759019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f758f7fe700 1 -- 172.21.15.22:0/48139725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7590101a10 msgr2=0x7f75901996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f758f7fe700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7590101a10 0x7f75901996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f758f7fe700 1 -- 172.21.15.22:0/48139725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7590106140 msgr2=0x7f759019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f758f7fe700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7590106140 0x7f759019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.640+0000 7f758f7fe700 1 -- 172.21.15.22:0/48139725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7588009d00 con 0x7f7590101060 2023-09-20T21:47:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.641+0000 7f758f7fe700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7590101060 0x7f7590199160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7580009230 tx=0x7f758000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.641+0000 7f758cff9700 1 -- 172.21.15.22:0/48139725 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758001c070 con 0x7f7590101060 2023-09-20T21:47:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.641+0000 7f758cff9700 1 -- 172.21.15.22:0/48139725 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75800058e0 con 0x7f7590101060 2023-09-20T21:47:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.641+0000 7f758cff9700 1 -- 172.21.15.22:0/48139725 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800038e0 con 0x7f7590101060 2023-09-20T21:47:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.641+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7580009520 con 0x7f7590101060 2023-09-20T21:47:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.641+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75901a5dc0 con 0x7f7590101060 2023-09-20T21:47:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.642+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f759004eae0 con 0x7f7590101060 2023-09-20T21:47:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.644+0000 7f758cff9700 1 -- 172.21.15.22:0/48139725 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f7580020030 con 0x7f7590101060 2023-09-20T21:47:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.645+0000 7f758cff9700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75780a1380 0x7f75780a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.646+0000 7f758cff9700 1 -- 172.21.15.22:0/48139725 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f75800cdfd0 con 0x7f7590101060 2023-09-20T21:47:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.646+0000 7f758effd700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75780a1380 0x7f75780a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.646+0000 7f758cff9700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] conn(0x7f75780ac320 0x7f75780ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.646+0000 7f758cff9700 1 -- 172.21.15.22:0/48139725 --> [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75780aee90 con 0x7f75780ac320 2023-09-20T21:47:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.646+0000 7f758cff9700 1 -- 172.21.15.22:0/48139725 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75800ce340 con 0x7f7590101060 2023-09-20T21:47:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.646+0000 7f758ffff700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] conn(0x7f75780ac320 0x7f75780ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.646+0000 7f758effd700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75780a1380 0x7f75780a3840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f759019a780 tx=0x7f7584009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.647+0000 7f758ffff700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] conn(0x7f75780ac320 0x7f75780ae7c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.648+0000 7f758cff9700 1 -- 172.21.15.22:0/48139725 <== osd.3 v2:172.21.15.22:6810/143368577 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f75780aee90 con 0x7f75780ac320 2023-09-20T21:47:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.666+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 --> [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7590111060 con 0x7f75780ac320 2023-09-20T21:47:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.666+0000 7f758cff9700 1 -- 172.21.15.22:0/48139725 <== osd.3 v2:172.21.15.22:6810/143368577 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f758800f040 con 0x7f75780ac320 2023-09-20T21:47:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.667+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] conn(0x7f75780ac320 msgr2=0x7f75780ae7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.667+0000 7f7595ea7700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] conn(0x7f75780ac320 0x7f75780ae7c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.668+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75780a1380 msgr2=0x7f75780a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.668+0000 7f7595ea7700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f75780a1380 0x7f75780a3840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f759019a780 tx=0x7f7584009260 comp rx=0 tx=0).stop 2023-09-20T21:47:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.668+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7590101060 msgr2=0x7f7590199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.668+0000 7f7595ea7700 1 --2- 172.21.15.22:0/48139725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7590101060 0x7f7590199160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7580009230 tx=0x7f758000fa00 comp rx=0 tx=0).stop 2023-09-20T21:47:39.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.668+0000 7f758ffff700 1 -- 172.21.15.22:0/48139725 reap_dead start 2023-09-20T21:47:39.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.668+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 shutdown_connections 2023-09-20T21:47:39.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.668+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 >> 172.21.15.22:0/48139725 conn(0x7f75900fc8d0 msgr2=0x7f75900fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:39.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.669+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 shutdown_connections 2023-09-20T21:47:39.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:38.669+0000 7f7595ea7700 1 -- 172.21.15.22:0/48139725 wait complete. 2023-09-20T21:47:39.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.642+0000 7f60216ec700 1 -- 172.21.15.22:0/2975635276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c068900 msgr2=0x7f601c068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.642+0000 7f60216ec700 1 --2- 172.21.15.22:0/2975635276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c068900 0x7f601c068d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f601400b0a0 tx=0x7f601400b3b0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.642+0000 7f60216ec700 1 -- 172.21.15.22:0/2975635276 shutdown_connections 2023-09-20T21:47:39.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.642+0000 7f60216ec700 1 --2- 172.21.15.22:0/2975635276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c068900 0x7f601c068d80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.642+0000 7f60216ec700 1 --2- 172.21.15.22:0/2975635276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c105da0 0x7f601c103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.642+0000 7f60216ec700 1 --2- 172.21.15.22:0/2975635276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c103d70 0x7f601c1041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.642+0000 7f60216ec700 1 -- 172.21.15.22:0/2975635276 >> 172.21.15.22:0/2975635276 conn(0x7f601c076de0 msgr2=0x7f601c0771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.643+0000 7f60216ec700 1 -- 172.21.15.22:0/2975635276 shutdown_connections 2023-09-20T21:47:39.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.643+0000 7f60216ec700 1 -- 172.21.15.22:0/2975635276 wait complete. 2023-09-20T21:47:39.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.644+0000 7f60216ec700 1 Processor -- start 2023-09-20T21:47:39.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.644+0000 7f60216ec700 1 -- start start 2023-09-20T21:47:39.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.644+0000 7f60216ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c068900 0x7f601c10cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.644+0000 7f60216ec700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c103d70 0x7f601c10d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.644+0000 7f60216ec700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c105da0 0x7f601c1b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.645+0000 7f60216ec700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c1b1fd0 con 0x7f601c103d70 2023-09-20T21:47:39.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.645+0000 7f60216ec700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c1b2140 con 0x7f601c105da0 2023-09-20T21:47:39.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.645+0000 7f60216ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c1b2300 con 0x7f601c068900 2023-09-20T21:47:39.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.645+0000 7f601a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c103d70 0x7f601c10d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.645+0000 7f601a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c103d70 0x7f601c10d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57554/0 (socket says 172.21.15.22:57554) 2023-09-20T21:47:39.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.645+0000 7f601a7fc700 1 -- 172.21.15.22:0/4178307357 learned_addr learned my addr 172.21.15.22:0/4178307357 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:39.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.645+0000 7f601b7fe700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c105da0 0x7f601c1b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.646+0000 7f601a7fc700 1 -- 172.21.15.22:0/4178307357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c068900 msgr2=0x7f601c10cdf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-20T21:47:39.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.646+0000 7f601a7fc700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c068900 0x7f601c10cdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.646+0000 7f601a7fc700 1 -- 172.21.15.22:0/4178307357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c105da0 msgr2=0x7f601c1b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.646+0000 7f601a7fc700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c105da0 0x7f601c1b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.646+0000 7f601a7fc700 1 -- 172.21.15.22:0/4178307357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6014009d00 con 0x7f601c103d70 2023-09-20T21:47:39.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.646+0000 7f601a7fc700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c103d70 0x7f601c10d330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f601000e970 tx=0x7f601000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.647+0000 7f6003fff700 1 -- 172.21.15.22:0/4178307357 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601000cbd0 con 0x7f601c103d70 2023-09-20T21:47:39.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.647+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601c1b25e0 con 0x7f601c103d70 2023-09-20T21:47:39.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.647+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c1b2b30 con 0x7f601c103d70 2023-09-20T21:47:39.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.647+0000 7f6003fff700 1 -- 172.21.15.22:0/4178307357 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f601000eec0 con 0x7f601c103d70 2023-09-20T21:47:39.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.647+0000 7f6003fff700 1 -- 172.21.15.22:0/4178307357 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010003cc0 con 0x7f601c103d70 2023-09-20T21:47:39.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.648+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6008000ff0 con 0x7f601c103d70 2023-09-20T21:47:39.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.651+0000 7f6003fff700 1 -- 172.21.15.22:0/4178307357 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f601000cd30 con 0x7f601c103d70 2023-09-20T21:47:39.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.653+0000 7f6003fff700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60040a13b0 0x7f60040a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.653+0000 7f6003fff700 1 -- 172.21.15.22:0/4178307357 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6010014070 con 0x7f601c103d70 2023-09-20T21:47:39.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.653+0000 7f6003fff700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] conn(0x7f60040ac350 0x7f60040ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.653+0000 7f6003fff700 1 -- 172.21.15.22:0/4178307357 --> [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60040aeec0 con 0x7f60040ac350 2023-09-20T21:47:39.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.653+0000 7f6003fff700 1 -- 172.21.15.22:0/4178307357 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60100cd520 con 0x7f601c103d70 2023-09-20T21:47:39.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.654+0000 7f601b7fe700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] conn(0x7f60040ac350 0x7f60040ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.654+0000 7f601affd700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60040a13b0 0x7f60040a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.654+0000 7f601b7fe700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] conn(0x7f60040ac350 0x7f60040ae7f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.655+0000 7f601affd700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60040a13b0 0x7f60040a3870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f600c009710 tx=0x7f600c006c60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.655+0000 7f6003fff700 1 -- 172.21.15.22:0/4178307357 <== osd.5 v2:172.21.15.123:6808/402706185 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f60040aeec0 con 0x7f60040ac350 2023-09-20T21:47:39.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.678+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 --> [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6008002d70 con 0x7f60040ac350 2023-09-20T21:47:39.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.679+0000 7f6003fff700 1 -- 172.21.15.22:0/4178307357 <== osd.5 v2:172.21.15.123:6808/402706185 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6008002d70 con 0x7f60040ac350 2023-09-20T21:47:39.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.679+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 >> [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] conn(0x7f60040ac350 msgr2=0x7f60040ae7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.679+0000 7f60216ec700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] conn(0x7f60040ac350 0x7f60040ae7f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.680+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60040a13b0 msgr2=0x7f60040a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.680+0000 7f60216ec700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60040a13b0 0x7f60040a3870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f600c009710 tx=0x7f600c006c60 comp rx=0 tx=0).stop 2023-09-20T21:47:39.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.680+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c103d70 msgr2=0x7f601c10d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.680+0000 7f60216ec700 1 --2- 172.21.15.22:0/4178307357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c103d70 0x7f601c10d330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f601000e970 tx=0x7f601000ec80 comp rx=0 tx=0).stop 2023-09-20T21:47:39.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.680+0000 7f601b7fe700 1 -- 172.21.15.22:0/4178307357 reap_dead start 2023-09-20T21:47:39.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.680+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 shutdown_connections 2023-09-20T21:47:39.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.680+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 >> 172.21.15.22:0/4178307357 conn(0x7f601c076de0 msgr2=0x7f601c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:39.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.680+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 shutdown_connections 2023-09-20T21:47:39.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.680+0000 7f60216ec700 1 -- 172.21.15.22:0/4178307357 wait complete. 2023-09-20T21:47:39.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.894+0000 7fec94d5b700 1 -- 172.21.15.22:0/4062286386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec90100580 msgr2=0x7fec9010d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.894+0000 7fec94d5b700 1 --2- 172.21.15.22:0/4062286386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec90100580 0x7fec9010d240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fec84009990 tx=0x7fec84009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.894+0000 7fec94d5b700 1 -- 172.21.15.22:0/4062286386 shutdown_connections 2023-09-20T21:47:39.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.894+0000 7fec94d5b700 1 --2- 172.21.15.22:0/4062286386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec9010d780 0x7fec9010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.894+0000 7fec94d5b700 1 --2- 172.21.15.22:0/4062286386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec900ffbd0 0x7fec900fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.894+0000 7fec94d5b700 1 --2- 172.21.15.22:0/4062286386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec90100580 0x7fec9010d240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.894+0000 7fec94d5b700 1 -- 172.21.15.22:0/4062286386 >> 172.21.15.22:0/4062286386 conn(0x7fec900fb850 msgr2=0x7fec900fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.894+0000 7fec94d5b700 1 -- 172.21.15.22:0/4062286386 shutdown_connections 2023-09-20T21:47:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.894+0000 7fec94d5b700 1 -- 172.21.15.22:0/4062286386 wait complete. 2023-09-20T21:47:39.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.895+0000 7fec94d5b700 1 Processor -- start 2023-09-20T21:47:39.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.896+0000 7fec94d5b700 1 -- start start 2023-09-20T21:47:39.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.896+0000 7fec94d5b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec900ffbd0 0x7fec90199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.896+0000 7fec94d5b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec90100580 0x7fec90199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.896+0000 7fec94d5b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9010d780 0x7fec9019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.896+0000 7fec94d5b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9019e110 con 0x7fec90100580 2023-09-20T21:47:39.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.896+0000 7fec94d5b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9019e280 con 0x7fec900ffbd0 2023-09-20T21:47:39.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.896+0000 7fec94d5b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9019e440 con 0x7fec9010d780 2023-09-20T21:47:39.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec900ffbd0 0x7fec90199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec90100580 0x7fec90199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec900ffbd0 0x7fec90199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:59358/0 (socket says 172.21.15.22:59358) 2023-09-20T21:47:39.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8e59c700 1 -- 172.21.15.22:0/542862104 learned_addr learned my addr 172.21.15.22:0/542862104 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:39.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8ed9d700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9010d780 0x7fec9019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8e59c700 1 -- 172.21.15.22:0/542862104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9010d780 msgr2=0x7fec9019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8e59c700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9010d780 0x7fec9019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8e59c700 1 -- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec90100580 msgr2=0x7fec90199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8e59c700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec90100580 0x7fec90199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8e59c700 1 -- 172.21.15.22:0/542862104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840095f0 con 0x7fec900ffbd0 2023-09-20T21:47:39.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8ed9d700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec9010d780 0x7fec9019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.897+0000 7fec8e59c700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec900ffbd0 0x7fec90199150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec8000e9a0 tx=0x7fec8000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.898+0000 7fec7f7fe700 1 -- 172.21.15.22:0/542862104 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80009790 con 0x7fec900ffbd0 2023-09-20T21:47:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.898+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9019e720 con 0x7fec900ffbd0 2023-09-20T21:47:39.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.898+0000 7fec7f7fe700 1 -- 172.21.15.22:0/542862104 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec80010430 con 0x7fec900ffbd0 2023-09-20T21:47:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.898+0000 7fec7f7fe700 1 -- 172.21.15.22:0/542862104 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80018680 con 0x7fec900ffbd0 2023-09-20T21:47:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.898+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec901a5f80 con 0x7fec900ffbd0 2023-09-20T21:47:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.899+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec901132b0 con 0x7fec900ffbd0 2023-09-20T21:47:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.901+0000 7fec7f7fe700 1 -- 172.21.15.22:0/542862104 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fec800105a0 con 0x7fec900ffbd0 2023-09-20T21:47:39.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.902+0000 7fec7f7fe700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec780a1300 0x7fec780a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.902+0000 7fec7f7fe700 1 -- 172.21.15.22:0/542862104 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fec80014070 con 0x7fec900ffbd0 2023-09-20T21:47:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.902+0000 7fec8dd9b700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec780a1300 0x7fec780a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.903+0000 7fec7f7fe700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] conn(0x7fec780ac2a0 0x7fec780ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.903+0000 7fec7f7fe700 1 -- 172.21.15.22:0/542862104 --> [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec780aee10 con 0x7fec780ac2a0 2023-09-20T21:47:39.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.903+0000 7fec7f7fe700 1 -- 172.21.15.22:0/542862104 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec800cd750 con 0x7fec900ffbd0 2023-09-20T21:47:39.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.903+0000 7fec8ed9d700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] conn(0x7fec780ac2a0 0x7fec780ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:39.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.903+0000 7fec8dd9b700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec780a1300 0x7fec780a37c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec9019a770 tx=0x7fec84005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.903+0000 7fec8ed9d700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] conn(0x7fec780ac2a0 0x7fec780ae740 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.904+0000 7fec7f7fe700 1 -- 172.21.15.22:0/542862104 <== osd.6 v2:172.21.15.22:6818/1850518910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fec780aee10 con 0x7fec780ac2a0 2023-09-20T21:47:39.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.921+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 --> [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec90068a70 con 0x7fec780ac2a0 2023-09-20T21:47:39.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec7f7fe700 1 -- 172.21.15.22:0/542862104 <== osd.6 v2:172.21.15.22:6818/1850518910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fec90068a70 con 0x7fec780ac2a0 2023-09-20T21:47:39.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] conn(0x7fec780ac2a0 msgr2=0x7fec780ae740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec94d5b700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] conn(0x7fec780ac2a0 0x7fec780ae740 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec780a1300 msgr2=0x7fec780a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec94d5b700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec780a1300 0x7fec780a37c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec9019a770 tx=0x7fec84005fb0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec900ffbd0 msgr2=0x7fec90199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec94d5b700 1 --2- 172.21.15.22:0/542862104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec900ffbd0 0x7fec90199150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec8000e9a0 tx=0x7fec8000ecb0 comp rx=0 tx=0).stop 2023-09-20T21:47:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec8ed9d700 1 -- 172.21.15.22:0/542862104 reap_dead start 2023-09-20T21:47:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 shutdown_connections 2023-09-20T21:47:39.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 >> 172.21.15.22:0/542862104 conn(0x7fec900fb850 msgr2=0x7fec9010ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 shutdown_connections 2023-09-20T21:47:39.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.922+0000 7fec94d5b700 1 -- 172.21.15.22:0/542862104 wait complete. 2023-09-20T21:47:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:40.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.665+0000 7f0e1867d700 1 -- 172.21.15.22:0/3191699167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e10106150 msgr2=0x7f0e1010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.665+0000 7f0e1867d700 1 --2- 172.21.15.22:0/3191699167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e10106150 0x7f0e1010dd20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009f90 tx=0x7f0e0800b330 comp rx=0 tx=0).stop 2023-09-20T21:47:40.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.666+0000 7f0e1867d700 1 -- 172.21.15.22:0/3191699167 shutdown_connections 2023-09-20T21:47:40.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.666+0000 7f0e1867d700 1 --2- 172.21.15.22:0/3191699167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e10106150 0x7f0e1010dd20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.666+0000 7f0e1867d700 1 --2- 172.21.15.22:0/3191699167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e10101070 0x7f0e10101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.666+0000 7f0e1867d700 1 --2- 172.21.15.22:0/3191699167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e10101a20 0x7f0e10105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.666+0000 7f0e1867d700 1 -- 172.21.15.22:0/3191699167 >> 172.21.15.22:0/3191699167 conn(0x7f0e100fc900 msgr2=0x7f0e100fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:40.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.666+0000 7f0e1867d700 1 -- 172.21.15.22:0/3191699167 shutdown_connections 2023-09-20T21:47:40.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.666+0000 7f0e1867d700 1 -- 172.21.15.22:0/3191699167 wait complete. 2023-09-20T21:47:40.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.667+0000 7f0e1867d700 1 Processor -- start 2023-09-20T21:47:40.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.667+0000 7f0e1867d700 1 -- start start 2023-09-20T21:47:40.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.667+0000 7f0e1867d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e10101070 0x7f0e10199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:40.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.667+0000 7f0e1867d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e10101a20 0x7f0e10199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:40.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.667+0000 7f0e1867d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e10106150 0x7f0e1019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:40.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.667+0000 7f0e1867d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1019e1c0 con 0x7f0e10106150 2023-09-20T21:47:40.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.667+0000 7f0e1867d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1019e330 con 0x7f0e10101a20 2023-09-20T21:47:40.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.667+0000 7f0e1867d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1019e4f0 con 0x7f0e10101070 2023-09-20T21:47:40.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.668+0000 7f0e15c18700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e10101a20 0x7f0e10199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:40.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.668+0000 7f0e15c18700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e10101a20 0x7f0e10199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:59334/0 (socket says 172.21.15.22:59334) 2023-09-20T21:47:40.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.668+0000 7f0e15c18700 1 -- 172.21.15.22:0/2927858374 learned_addr learned my addr 172.21.15.22:0/2927858374 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:40.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.668+0000 7f0e16419700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e10101070 0x7f0e10199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:40.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.668+0000 7f0e16c1a700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e10106150 0x7f0e1019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:40.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.668+0000 7f0e15c18700 1 -- 172.21.15.22:0/2927858374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e10101070 msgr2=0x7f0e10199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.668+0000 7f0e15c18700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e10101070 0x7f0e10199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.668+0000 7f0e15c18700 1 -- 172.21.15.22:0/2927858374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e10106150 msgr2=0x7f0e1019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.668+0000 7f0e15c18700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e10106150 0x7f0e1019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.668+0000 7f0e15c18700 1 -- 172.21.15.22:0/2927858374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08009c40 con 0x7f0e10101a20 2023-09-20T21:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.669+0000 7f0e15c18700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e10101a20 0x7f0e10199740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000d980 tx=0x7f0e0000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.669+0000 7f0e077fe700 1 -- 172.21.15.22:0/2927858374 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00009b40 con 0x7f0e10101a20 2023-09-20T21:47:40.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.669+0000 7f0e077fe700 1 -- 172.21.15.22:0/2927858374 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e0000b640 con 0x7f0e10101a20 2023-09-20T21:47:40.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.669+0000 7f0e077fe700 1 -- 172.21.15.22:0/2927858374 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000f3c0 con 0x7f0e10101a20 2023-09-20T21:47:40.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.669+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1019e7d0 con 0x7f0e10101a20 2023-09-20T21:47:40.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.669+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e101a6030 con 0x7f0e10101a20 2023-09-20T21:47:40.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.670+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0e101110e0 con 0x7f0e10101a20 2023-09-20T21:47:40.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.674+0000 7f0e077fe700 1 -- 172.21.15.22:0/2927858374 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f0e00010460 con 0x7f0e10101a20 2023-09-20T21:47:40.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.676+0000 7f0e077fe700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0dfc0a1380 0x7f0dfc0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:40.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.677+0000 7f0e077fe700 1 -- 172.21.15.22:0/2927858374 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f0e000cc990 con 0x7f0e10101a20 2023-09-20T21:47:40.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.677+0000 7f0e16419700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0dfc0a1380 0x7f0dfc0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:40.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.677+0000 7f0e077fe700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] conn(0x7f0dfc0ac320 0x7f0dfc0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:40.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.677+0000 7f0e077fe700 1 -- 172.21.15.22:0/2927858374 --> [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0dfc0aee90 con 0x7f0dfc0ac320 2023-09-20T21:47:40.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.677+0000 7f0e077fe700 1 -- 172.21.15.22:0/2927858374 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e000ccd00 con 0x7f0e10101a20 2023-09-20T21:47:40.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.678+0000 7f0e16c1a700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] conn(0x7f0dfc0ac320 0x7f0dfc0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:40.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.678+0000 7f0e16419700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0dfc0a1380 0x7f0dfc0a3840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005fd0 tx=0x7f0e0c009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:40.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.678+0000 7f0e16c1a700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] conn(0x7f0dfc0ac320 0x7f0dfc0ae7c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.679+0000 7f0e077fe700 1 -- 172.21.15.22:0/2927858374 <== osd.1 v2:172.21.15.89:6800/2519597559 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0dfc0aee90 con 0x7f0dfc0ac320 2023-09-20T21:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.699+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 --> [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0e1004eae0 con 0x7f0dfc0ac320 2023-09-20T21:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.700+0000 7f0e077fe700 1 -- 172.21.15.22:0/2927858374 <== osd.1 v2:172.21.15.89:6800/2519597559 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0e1004eae0 con 0x7f0dfc0ac320 2023-09-20T21:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.700+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 >> [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] conn(0x7f0dfc0ac320 msgr2=0x7f0dfc0ae7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.700+0000 7f0e1867d700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] conn(0x7f0dfc0ac320 0x7f0dfc0ae7c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.700+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0dfc0a1380 msgr2=0x7f0dfc0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.700+0000 7f0e1867d700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0dfc0a1380 0x7f0dfc0a3840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005fd0 tx=0x7f0e0c009310 comp rx=0 tx=0).stop 2023-09-20T21:47:40.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.700+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e10101a20 msgr2=0x7f0e10199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.700+0000 7f0e1867d700 1 --2- 172.21.15.22:0/2927858374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e10101a20 0x7f0e10199740 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000d980 tx=0x7f0e0000dd40 comp rx=0 tx=0).stop 2023-09-20T21:47:40.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.701+0000 7f0e16c1a700 1 -- 172.21.15.22:0/2927858374 reap_dead start 2023-09-20T21:47:40.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.701+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 shutdown_connections 2023-09-20T21:47:40.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.701+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 >> 172.21.15.22:0/2927858374 conn(0x7f0e100fc900 msgr2=0x7f0e1010c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:40.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.701+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 shutdown_connections 2023-09-20T21:47:40.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:39.701+0000 7f0e1867d700 1 -- 172.21.15.22:0/2927858374 wait complete. 2023-09-20T21:47:40.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:40 smithi022 ceph-mon[97651]: pgmap v307: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:47:40.287 INFO:teuthology.orchestra.run.smithi022.stdout:107374182460 2023-09-20T21:47:40.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.3 2023-09-20T21:47:40.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:40 smithi123 ceph-mon[134433]: pgmap v307: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:47:40.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:40 smithi089 ceph-mon[134090]: pgmap v307: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:47:40.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.476+0000 7fadcfd8a700 1 -- 172.21.15.22:0/2314728711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadc8101a10 msgr2=0x7fadc8105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.476+0000 7fadcfd8a700 1 --2- 172.21.15.22:0/2314728711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadc8101a10 0x7fadc8105a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009990 tx=0x7fadb8009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.477+0000 7fadcfd8a700 1 -- 172.21.15.22:0/2314728711 shutdown_connections 2023-09-20T21:47:40.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.477+0000 7fadcfd8a700 1 --2- 172.21.15.22:0/2314728711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc8106140 0x7fadc810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.477+0000 7fadcfd8a700 1 --2- 172.21.15.22:0/2314728711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadc8101060 0x7fadc8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.477+0000 7fadcfd8a700 1 --2- 172.21.15.22:0/2314728711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadc8101a10 0x7fadc8105a60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.477+0000 7fadcfd8a700 1 -- 172.21.15.22:0/2314728711 >> 172.21.15.22:0/2314728711 conn(0x7fadc80fc8d0 msgr2=0x7fadc80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:40.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.477+0000 7fadcfd8a700 1 -- 172.21.15.22:0/2314728711 shutdown_connections 2023-09-20T21:47:40.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.477+0000 7fadcfd8a700 1 -- 172.21.15.22:0/2314728711 wait complete. 2023-09-20T21:47:40.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.478+0000 7fadcfd8a700 1 Processor -- start 2023-09-20T21:47:40.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.478+0000 7fadcfd8a700 1 -- start start 2023-09-20T21:47:40.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.478+0000 7fadcfd8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadc8101060 0x7fadc80730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:40.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.478+0000 7fadcfd8a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc8101a10 0x7fadc8073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:40.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.478+0000 7fadcfd8a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadc8106140 0x7fadc8073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:40.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.478+0000 7fadcfd8a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc81a5380 con 0x7fadc8106140 2023-09-20T21:47:40.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.478+0000 7fadcfd8a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc81a54f0 con 0x7fadc8101a10 2023-09-20T21:47:40.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.478+0000 7fadcfd8a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc81a56b0 con 0x7fadc8101060 2023-09-20T21:47:40.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadcdb26700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadc8101060 0x7fadc80730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:40.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadce327700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadc8106140 0x7fadc8073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:40.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadcdb26700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadc8101060 0x7fadc80730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:50794/0 (socket says 172.21.15.22:50794) 2023-09-20T21:47:40.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadcdb26700 1 -- 172.21.15.22:0/3707829975 learned_addr learned my addr 172.21.15.22:0/3707829975 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadcd325700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc8101a10 0x7fadc8073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadce327700 1 -- 172.21.15.22:0/3707829975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadc8101060 msgr2=0x7fadc80730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadce327700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadc8101060 0x7fadc80730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadce327700 1 -- 172.21.15.22:0/3707829975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc8101a10 msgr2=0x7fadc8073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadce327700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc8101a10 0x7fadc8073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadce327700 1 -- 172.21.15.22:0/3707829975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc4009520 con 0x7fadc8106140 2023-09-20T21:47:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadcd325700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc8101a10 0x7fadc8073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.479+0000 7fadce327700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadc8106140 0x7fadc8073ba0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fadc000efb0 tx=0x7fadc000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.480+0000 7fadbeffd700 1 -- 172.21.15.22:0/3707829975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0003a00 con 0x7fadc8106140 2023-09-20T21:47:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.480+0000 7fadbeffd700 1 -- 172.21.15.22:0/3707829975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc000f040 con 0x7fadc8106140 2023-09-20T21:47:40.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.480+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc0009c40 con 0x7fadc8106140 2023-09-20T21:47:40.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.480+0000 7fadbeffd700 1 -- 172.21.15.22:0/3707829975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0004540 con 0x7fadc8106140 2023-09-20T21:47:40.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.480+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb80095f0 con 0x7fadc8106140 2023-09-20T21:47:40.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.481+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fadc804eae0 con 0x7fadc8106140 2023-09-20T21:47:40.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.483+0000 7fadbeffd700 1 -- 172.21.15.22:0/3707829975 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fadc0003bf0 con 0x7fadc8106140 2023-09-20T21:47:40.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.485+0000 7fadbeffd700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fadb40a1300 0x7fadb40a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.486+0000 7fadbeffd700 1 -- 172.21.15.22:0/3707829975 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fadc00cd080 con 0x7fadc8106140 2023-09-20T21:47:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.486+0000 7fadcdb26700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fadb40a1300 0x7fadb40a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.486+0000 7fadbeffd700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] conn(0x7fadb40ac2a0 0x7fadb40ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.486+0000 7fadbeffd700 1 -- 172.21.15.22:0/3707829975 --> [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fadb40aee10 con 0x7fadb40ac2a0 2023-09-20T21:47:40.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.486+0000 7fadbeffd700 1 -- 172.21.15.22:0/3707829975 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadc00cd3f0 con 0x7fadc8106140 2023-09-20T21:47:40.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.486+0000 7fadcd325700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] conn(0x7fadb40ac2a0 0x7fadb40ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:40.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.486+0000 7fadcdb26700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fadb40a1300 0x7fadb40a37c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009ef0 tx=0x7fadc4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:40.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.487+0000 7fadcd325700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] conn(0x7fadb40ac2a0 0x7fadb40ae740 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:40.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.488+0000 7fadbeffd700 1 -- 172.21.15.22:0/3707829975 <== osd.9 v2:172.21.15.22:6826/4238100695 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fadb40aee10 con 0x7fadb40ac2a0 2023-09-20T21:47:40.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.505+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 --> [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fadc8111080 con 0x7fadb40ac2a0 2023-09-20T21:47:40.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.505+0000 7fadbeffd700 1 -- 172.21.15.22:0/3707829975 <== osd.9 v2:172.21.15.22:6826/4238100695 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fadb801e040 con 0x7fadb40ac2a0 2023-09-20T21:47:40.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] conn(0x7fadb40ac2a0 msgr2=0x7fadb40ae740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadcfd8a700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] conn(0x7fadb40ac2a0 0x7fadb40ae740 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fadb40a1300 msgr2=0x7fadb40a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadcfd8a700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fadb40a1300 0x7fadb40a37c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009ef0 tx=0x7fadc4009260 comp rx=0 tx=0).stop 2023-09-20T21:47:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadc8106140 msgr2=0x7fadc8073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadcfd8a700 1 --2- 172.21.15.22:0/3707829975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadc8106140 0x7fadc8073ba0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fadc000efb0 tx=0x7fadc000c3a0 comp rx=0 tx=0).stop 2023-09-20T21:47:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadce327700 1 -- 172.21.15.22:0/3707829975 reap_dead start 2023-09-20T21:47:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 shutdown_connections 2023-09-20T21:47:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 >> 172.21.15.22:0/3707829975 conn(0x7fadc80fc8d0 msgr2=0x7fadc80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 shutdown_connections 2023-09-20T21:47:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:40.506+0000 7fadcfd8a700 1 -- 172.21.15.22:0/3707829975 wait complete. 2023-09-20T21:47:41.005 INFO:teuthology.orchestra.run.smithi022.stdout:219043332138 2023-09-20T21:47:41.006 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.7 2023-09-20T21:47:41.089 INFO:teuthology.orchestra.run.smithi022.stdout:163208757300 2023-09-20T21:47:41.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.5 2023-09-20T21:47:41.915 INFO:teuthology.orchestra.run.smithi022.stdout:197568495662 2023-09-20T21:47:41.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.6 2023-09-20T21:47:41.988 INFO:teuthology.orchestra.run.smithi022.stdout:287762808864 2023-09-20T21:47:41.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.9 2023-09-20T21:47:42.119 INFO:teuthology.orchestra.run.smithi022.stdout:309237645339 2023-09-20T21:47:42.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.10 2023-09-20T21:47:42.290 INFO:teuthology.orchestra.run.smithi022.stdout:55834574918 2023-09-20T21:47:42.290 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph osd last-stat-seq osd.1 2023-09-20T21:47:42.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:42 smithi022 ceph-mon[97651]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:42.571 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:42.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:42 smithi123 ceph-mon[134433]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:42.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:42 smithi089 ceph-mon[134090]: pgmap v308: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:42.776 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:43.527 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:43.658 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:43.826 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:43.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:43 smithi123 ceph-mon[134433]: pgmap v309: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2023-09-20T21:47:43.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:43 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:47:43.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:43 smithi089 ceph-mon[134090]: pgmap v309: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2023-09-20T21:47:43.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:43 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:47:43.932 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:43 smithi022 ceph-mon[97651]: pgmap v309: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2023-09-20T21:47:43.933 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:47:44.091 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:44.788 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:45.172 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.283+0000 7f3093930700 1 -- 172.21.15.22:0/2521265823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c068900 msgr2=0x7f308c068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:45.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.283+0000 7f3093930700 1 --2- 172.21.15.22:0/2521265823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c068900 0x7f308c068d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f307c0098f0 tx=0x7f307c009c00 comp rx=0 tx=0).stop 2023-09-20T21:47:45.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.287+0000 7f3093930700 1 -- 172.21.15.22:0/2521265823 shutdown_connections 2023-09-20T21:47:45.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.287+0000 7f3093930700 1 --2- 172.21.15.22:0/2521265823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f308c0692c0 0x7f308c10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:45.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.287+0000 7f3093930700 1 --2- 172.21.15.22:0/2521265823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308c106f70 0x7f308c107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:45.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.287+0000 7f3093930700 1 --2- 172.21.15.22:0/2521265823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c068900 0x7f308c068d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:45.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.287+0000 7f3093930700 1 -- 172.21.15.22:0/2521265823 >> 172.21.15.22:0/2521265823 conn(0x7f308c076de0 msgr2=0x7f308c0771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:45.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.287+0000 7f3093930700 1 -- 172.21.15.22:0/2521265823 shutdown_connections 2023-09-20T21:47:45.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.288+0000 7f3093930700 1 -- 172.21.15.22:0/2521265823 wait complete. 2023-09-20T21:47:45.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.289+0000 7f3093930700 1 Processor -- start 2023-09-20T21:47:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.289+0000 7f3093930700 1 -- start start 2023-09-20T21:47:45.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.291+0000 7f3093930700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308c068900 0x7f308c0724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:45.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.291+0000 7f3093930700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f308c0692c0 0x7f308c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:45.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.291+0000 7f3093930700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c106f70 0x7f308c0ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:45.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.291+0000 7f3093930700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c100580 con 0x7f308c0692c0 2023-09-20T21:47:45.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.291+0000 7f3093930700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c1006f0 con 0x7f308c106f70 2023-09-20T21:47:45.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.291+0000 7f3093930700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c1008b0 con 0x7f308c068900 2023-09-20T21:47:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.291+0000 7f30916cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308c068900 0x7f308c0724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.291+0000 7f30916cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308c068900 0x7f308c0724f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:50818/0 (socket says 172.21.15.22:50818) 2023-09-20T21:47:45.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.291+0000 7f30916cc700 1 -- 172.21.15.22:0/771184748 learned_addr learned my addr 172.21.15.22:0/771184748 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.292+0000 7f30916cc700 1 -- 172.21.15.22:0/771184748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c106f70 msgr2=0x7f308c0ffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.292+0000 7f30916cc700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c106f70 0x7f308c0ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.292+0000 7f30916cc700 1 -- 172.21.15.22:0/771184748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f308c0692c0 msgr2=0x7f308c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:45.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.292+0000 7f30916cc700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f308c0692c0 0x7f308c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:45.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.292+0000 7f30916cc700 1 -- 172.21.15.22:0/771184748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c009520 con 0x7f308c068900 2023-09-20T21:47:45.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.293+0000 7f30916cc700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308c068900 0x7f308c0724f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f308800d980 tx=0x7f308800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:45.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.294+0000 7f30827fc700 1 -- 172.21.15.22:0/771184748 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088009ac0 con 0x7f308c068900 2023-09-20T21:47:45.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.294+0000 7f30827fc700 1 -- 172.21.15.22:0/771184748 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f308800de60 con 0x7f308c068900 2023-09-20T21:47:45.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.294+0000 7f30827fc700 1 -- 172.21.15.22:0/771184748 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088010ea0 con 0x7f308c068900 2023-09-20T21:47:45.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.296+0000 7f3093930700 1 -- 172.21.15.22:0/771184748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c100b90 con 0x7f308c068900 2023-09-20T21:47:45.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.296+0000 7f3093930700 1 -- 172.21.15.22:0/771184748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c1ae6c0 con 0x7f308c068900 2023-09-20T21:47:45.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.297+0000 7f3093930700 1 -- 172.21.15.22:0/771184748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308c04eae0 con 0x7f308c068900 2023-09-20T21:47:45.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.300+0000 7f30827fc700 1 -- 172.21.15.22:0/771184748 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f3088010460 con 0x7f308c068900 2023-09-20T21:47:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.301+0000 7f30827fc700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f30780a1450 0x7f30780a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:45.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.302+0000 7f30827fc700 1 -- 172.21.15.22:0/771184748 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f30880cce40 con 0x7f308c068900 2023-09-20T21:47:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.302+0000 7f3090ecb700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f30780a1450 0x7f30780a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:45.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.303+0000 7f30827fc700 1 -- 172.21.15.22:0/771184748 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f30880961b0 con 0x7f308c068900 2023-09-20T21:47:45.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.304+0000 7f3090ecb700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f30780a1450 0x7f30780a3910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f307c009810 tx=0x7f307c0058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:45.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.558+0000 7f3093930700 1 -- 172.21.15.22:0/771184748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f308c1012c0 con 0x7f308c068900 2023-09-20T21:47:45.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.559+0000 7f30827fc700 1 -- 172.21.15.22:0/771184748 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3088016090 con 0x7f308c068900 2023-09-20T21:47:45.560 INFO:teuthology.orchestra.run.smithi022.stdout:141733920825 2023-09-20T21:47:45.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.561+0000 7f3093930700 1 -- 172.21.15.22:0/771184748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f30780a1450 msgr2=0x7f30780a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:45.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.561+0000 7f3093930700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f30780a1450 0x7f30780a3910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f307c009810 tx=0x7f307c0058e0 comp rx=0 tx=0).stop 2023-09-20T21:47:45.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.561+0000 7f3093930700 1 -- 172.21.15.22:0/771184748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308c068900 msgr2=0x7f308c0724f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:45.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.561+0000 7f3093930700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308c068900 0x7f308c0724f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f308800d980 tx=0x7f308800dd40 comp rx=0 tx=0).stop 2023-09-20T21:47:45.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.564+0000 7f3093930700 1 -- 172.21.15.22:0/771184748 shutdown_connections 2023-09-20T21:47:45.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.564+0000 7f3093930700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f30780a1450 0x7f30780a3910 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:45.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.564+0000 7f3093930700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f308c106f70 0x7f308c0ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:45.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.564+0000 7f3093930700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f308c068900 0x7f308c0724f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:45.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.564+0000 7f3093930700 1 --2- 172.21.15.22:0/771184748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f308c0692c0 0x7f308c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:45.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.564+0000 7f3093930700 1 -- 172.21.15.22:0/771184748 >> 172.21.15.22:0/771184748 conn(0x7f308c076de0 msgr2=0x7f308c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:45.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.564+0000 7f3093930700 1 -- 172.21.15.22:0/771184748 shutdown_connections 2023-09-20T21:47:45.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:45.564+0000 7f3093930700 1 -- 172.21.15.22:0/771184748 wait complete. 2023-09-20T21:47:45.611 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:46.427 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:46 smithi022 ceph-mon[97651]: pgmap v310: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-09-20T21:47:46.427 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:46 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/771184748' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-20T21:47:46.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:46 smithi123 ceph-mon[134433]: pgmap v310: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-09-20T21:47:46.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:46 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/771184748' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-20T21:47:46.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:46 smithi089 ceph-mon[134090]: pgmap v310: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-09-20T21:47:46.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:46 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/771184748' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-09-20T21:47:47.435 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:47 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:47 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:47 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.436 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:47 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:47 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:47 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:47 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:47 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:47 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:47 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:47 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:47.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:47 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:48.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:48 smithi022 ceph-mon[97651]: pgmap v311: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-20T21:47:48.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:48 smithi123 ceph-mon[134433]: pgmap v311: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-20T21:47:48.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:48 smithi089 ceph-mon[134090]: pgmap v311: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-09-20T21:47:48.945 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920822 got 141733920825 for osd.4 2023-09-20T21:47:48.946 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:49.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.618+0000 7f66064fc700 1 -- 172.21.15.22:0/2129162549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6600106f70 msgr2=0x7f6600107350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:49.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.618+0000 7f66064fc700 1 --2- 172.21.15.22:0/2129162549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6600106f70 0x7f6600107350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009990 tx=0x7f65f0009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.618+0000 7f66064fc700 1 -- 172.21.15.22:0/2129162549 shutdown_connections 2023-09-20T21:47:49.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.618+0000 7f66064fc700 1 --2- 172.21.15.22:0/2129162549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66000692c0 0x7f660010dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.618+0000 7f66064fc700 1 --2- 172.21.15.22:0/2129162549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6600106f70 0x7f6600107350 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.618+0000 7f66064fc700 1 --2- 172.21.15.22:0/2129162549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6600068900 0x7f6600068d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.618+0000 7f66064fc700 1 -- 172.21.15.22:0/2129162549 >> 172.21.15.22:0/2129162549 conn(0x7f6600076de0 msgr2=0x7f66000771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:49.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.618+0000 7f66064fc700 1 -- 172.21.15.22:0/2129162549 shutdown_connections 2023-09-20T21:47:49.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.618+0000 7f66064fc700 1 -- 172.21.15.22:0/2129162549 wait complete. 2023-09-20T21:47:49.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.620+0000 7f66064fc700 1 Processor -- start 2023-09-20T21:47:49.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.620+0000 7f66064fc700 1 -- start start 2023-09-20T21:47:49.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.620+0000 7f66064fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6600068900 0x7f66000ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:49.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.620+0000 7f66064fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66000692c0 0x7f66001002d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:49.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.620+0000 7f66064fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6600106f70 0x7f6600100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:49.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.620+0000 7f66064fc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001a52b0 con 0x7f6600068900 2023-09-20T21:47:49.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.620+0000 7f66064fc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001a5420 con 0x7f66000692c0 2023-09-20T21:47:49.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.620+0000 7f66064fc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001a55e0 con 0x7f6600106f70 2023-09-20T21:47:49.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f65fffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6600068900 0x7f66000ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:49.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f6604a99700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6600106f70 0x7f6600100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:49.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f65fffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6600068900 0x7f66000ffd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37182/0 (socket says 172.21.15.22:37182) 2023-09-20T21:47:49.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f65fffff700 1 -- 172.21.15.22:0/2859444444 learned_addr learned my addr 172.21.15.22:0/2859444444 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:49.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f6604a99700 1 -- 172.21.15.22:0/2859444444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66000692c0 msgr2=0x7f66001002d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-20T21:47:49.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f6604a99700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66000692c0 0x7f66001002d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f6604a99700 1 -- 172.21.15.22:0/2859444444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6600068900 msgr2=0x7f66000ffd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:49.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f6604a99700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6600068900 0x7f66000ffd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f6604a99700 1 -- 172.21.15.22:0/2859444444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f4009520 con 0x7f6600106f70 2023-09-20T21:47:49.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f65fffff700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6600068900 0x7f66000ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:49.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.621+0000 7f6604a99700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6600106f70 0x7f6600100850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65f800efc0 tx=0x7f65f800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:49.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.622+0000 7f65fd7fa700 1 -- 172.21.15.22:0/2859444444 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8018070 con 0x7f6600106f70 2023-09-20T21:47:49.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.622+0000 7f66064fc700 1 -- 172.21.15.22:0/2859444444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f00095f0 con 0x7f6600106f70 2023-09-20T21:47:49.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.622+0000 7f66064fc700 1 -- 172.21.15.22:0/2859444444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66001a5bd0 con 0x7f6600106f70 2023-09-20T21:47:49.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.622+0000 7f65fd7fa700 1 -- 172.21.15.22:0/2859444444 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65f800f040 con 0x7f6600106f70 2023-09-20T21:47:49.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.622+0000 7f65fd7fa700 1 -- 172.21.15.22:0/2859444444 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800ccf0 con 0x7f6600106f70 2023-09-20T21:47:49.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.623+0000 7f66064fc700 1 -- 172.21.15.22:0/2859444444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec005320 con 0x7f6600106f70 2023-09-20T21:47:49.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.627+0000 7f65fd7fa700 1 -- 172.21.15.22:0/2859444444 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f65f8004ad0 con 0x7f6600106f70 2023-09-20T21:47:49.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.629+0000 7f65fd7fa700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65e80a1380 0x7f65e80a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:49.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.629+0000 7f65fd7fa700 1 -- 172.21.15.22:0/2859444444 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f65f80cc980 con 0x7f6600106f70 2023-09-20T21:47:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.630+0000 7f65fd7fa700 1 -- 172.21.15.22:0/2859444444 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f65f8098ea0 con 0x7f6600106f70 2023-09-20T21:47:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.630+0000 7f65fffff700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65e80a1380 0x7f65e80a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.631+0000 7f65fffff700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65e80a1380 0x7f65e80a3840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65f0006010 tx=0x7f65f000b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.869+0000 7f66064fc700 1 -- 172.21.15.22:0/2859444444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f65ec005190 con 0x7f6600106f70 2023-09-20T21:47:49.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.870+0000 7f65fd7fa700 1 -- 172.21.15.22:0/2859444444 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65f8099130 con 0x7f6600106f70 2023-09-20T21:47:49.056 INFO:teuthology.orchestra.run.smithi022.stdout:253403070502 2023-09-20T21:47:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 -- 172.21.15.22:0/2859444444 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65e80a1380 msgr2=0x7f65e80a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65e80a1380 0x7f65e80a3840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65f0006010 tx=0x7f65f000b260 comp rx=0 tx=0).stop 2023-09-20T21:47:49.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 -- 172.21.15.22:0/2859444444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6600106f70 msgr2=0x7f6600100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6600106f70 0x7f6600100850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65f800efc0 tx=0x7f65f800c3a0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 -- 172.21.15.22:0/2859444444 shutdown_connections 2023-09-20T21:47:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6600106f70 0x7f6600100850 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6600068900 0x7f66000ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65e80a1380 0x7f65e80a3840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 --2- 172.21.15.22:0/2859444444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f66000692c0 0x7f66001002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 -- 172.21.15.22:0/2859444444 >> 172.21.15.22:0/2859444444 conn(0x7f6600076de0 msgr2=0x7f66000fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:49.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.873+0000 7f66064fc700 1 -- 172.21.15.22:0/2859444444 shutdown_connections 2023-09-20T21:47:49.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.874+0000 7f66064fc700 1 -- 172.21.15.22:0/2859444444 wait complete. 2023-09-20T21:47:49.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.792+0000 7f3502bfc700 1 -- 172.21.15.22:0/2426435971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34fc101060 msgr2=0x7f34fc101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:49.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.792+0000 7f3502bfc700 1 --2- 172.21.15.22:0/2426435971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34fc101060 0x7f34fc101440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009990 tx=0x7f34f0009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.792+0000 7f3502bfc700 1 -- 172.21.15.22:0/2426435971 shutdown_connections 2023-09-20T21:47:49.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.792+0000 7f3502bfc700 1 --2- 172.21.15.22:0/2426435971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34fc106140 0x7f34fc10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.792+0000 7f3502bfc700 1 --2- 172.21.15.22:0/2426435971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34fc101060 0x7f34fc101440 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.792+0000 7f3502bfc700 1 --2- 172.21.15.22:0/2426435971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34fc101a10 0x7f34fc105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.792+0000 7f3502bfc700 1 -- 172.21.15.22:0/2426435971 >> 172.21.15.22:0/2426435971 conn(0x7f34fc0fc8d0 msgr2=0x7f34fc0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.792+0000 7f3502bfc700 1 -- 172.21.15.22:0/2426435971 shutdown_connections 2023-09-20T21:47:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.792+0000 7f3502bfc700 1 -- 172.21.15.22:0/2426435971 wait complete. 2023-09-20T21:47:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.793+0000 7f3502bfc700 1 Processor -- start 2023-09-20T21:47:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.794+0000 7f3502bfc700 1 -- start start 2023-09-20T21:47:49.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.794+0000 7f3502bfc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34fc101060 0x7f34fc0700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:49.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.794+0000 7f3502bfc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34fc101a10 0x7f34fc070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:49.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.794+0000 7f3502bfc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34fc106140 0x7f34fc06b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:49.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.794+0000 7f3502bfc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc06b950 con 0x7f34fc101060 2023-09-20T21:47:49.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.794+0000 7f3502bfc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc06bac0 con 0x7f34fc106140 2023-09-20T21:47:49.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.794+0000 7f3502bfc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc06bc80 con 0x7f34fc101a10 2023-09-20T21:47:49.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.794+0000 7f3501199700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34fc106140 0x7f34fc06b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:49.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.794+0000 7f3501199700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34fc106140 0x7f34fc06b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:34282/0 (socket says 172.21.15.22:34282) 2023-09-20T21:47:49.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.794+0000 7f3501199700 1 -- 172.21.15.22:0/304647743 learned_addr learned my addr 172.21.15.22:0/304647743 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:49.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.795+0000 7f3500998700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34fc101060 0x7f34fc0700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:49.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.795+0000 7f34fbfff700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34fc101a10 0x7f34fc070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:49.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.795+0000 7f3501199700 1 -- 172.21.15.22:0/304647743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34fc101a10 msgr2=0x7f34fc070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:49.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.795+0000 7f3501199700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34fc101a10 0x7f34fc070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.795+0000 7f3501199700 1 -- 172.21.15.22:0/304647743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34fc101060 msgr2=0x7f34fc0700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:49.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.795+0000 7f3501199700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34fc101060 0x7f34fc0700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.795+0000 7f3501199700 1 -- 172.21.15.22:0/304647743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4009c40 con 0x7f34fc106140 2023-09-20T21:47:49.328 INFO:teuthology.orchestra.run.smithi022.stdout:197568495664 2023-09-20T21:47:49.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.795+0000 7f3501199700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34fc106140 0x7f34fc06b180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34f400c370 tx=0x7f34f400c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:49.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.796+0000 7f34f9ffb700 1 -- 172.21.15.22:0/304647743 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400e050 con 0x7f34fc106140 2023-09-20T21:47:49.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.796+0000 7f3502bfc700 1 -- 172.21.15.22:0/304647743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f00095f0 con 0x7f34fc106140 2023-09-20T21:47:49.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.796+0000 7f3502bfc700 1 -- 172.21.15.22:0/304647743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc074020 con 0x7f34fc106140 2023-09-20T21:47:49.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.796+0000 7f34f9ffb700 1 -- 172.21.15.22:0/304647743 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34f400f040 con 0x7f34fc106140 2023-09-20T21:47:49.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.796+0000 7f34f9ffb700 1 -- 172.21.15.22:0/304647743 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4013660 con 0x7f34fc106140 2023-09-20T21:47:49.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.798+0000 7f3502bfc700 1 -- 172.21.15.22:0/304647743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e0005320 con 0x7f34fc106140 2023-09-20T21:47:49.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.802+0000 7f34f9ffb700 1 -- 172.21.15.22:0/304647743 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f34f4007500 con 0x7f34fc106140 2023-09-20T21:47:49.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.803+0000 7f34f9ffb700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f34ec0a1380 0x7f34ec0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:49.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.803+0000 7f34f9ffb700 1 -- 172.21.15.22:0/304647743 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f34f40cce90 con 0x7f34fc106140 2023-09-20T21:47:49.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.804+0000 7f3500998700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f34ec0a1380 0x7f34ec0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.805+0000 7f3500998700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f34ec0a1380 0x7f34ec0a3840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f34f0006010 tx=0x7f34f000b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:48.807+0000 7f34f9ffb700 1 -- 172.21.15.22:0/304647743 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f34f4096260 con 0x7f34fc106140 2023-09-20T21:47:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.085+0000 7f3502bfc700 1 -- 172.21.15.22:0/304647743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f34e0005190 con 0x7f34fc106140 2023-09-20T21:47:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.086+0000 7f34f9ffb700 1 -- 172.21.15.22:0/304647743 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34f40b9070 con 0x7f34fc106140 2023-09-20T21:47:49.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.089+0000 7f3502bfc700 1 -- 172.21.15.22:0/304647743 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f34ec0a1380 msgr2=0x7f34ec0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.089+0000 7f3502bfc700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f34ec0a1380 0x7f34ec0a3840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f34f0006010 tx=0x7f34f000b260 comp rx=0 tx=0).stop 2023-09-20T21:47:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.089+0000 7f3502bfc700 1 -- 172.21.15.22:0/304647743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34fc106140 msgr2=0x7f34fc06b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.089+0000 7f3502bfc700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34fc106140 0x7f34fc06b180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f34f400c370 tx=0x7f34f400c730 comp rx=0 tx=0).stop 2023-09-20T21:47:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.090+0000 7f3502bfc700 1 -- 172.21.15.22:0/304647743 shutdown_connections 2023-09-20T21:47:49.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.090+0000 7f3502bfc700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f34ec0a1380 0x7f34ec0a3840 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.090+0000 7f3502bfc700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f34fc106140 0x7f34fc06b180 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.090+0000 7f3502bfc700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34fc101060 0x7f34fc0700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.090+0000 7f3502bfc700 1 --2- 172.21.15.22:0/304647743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34fc101a10 0x7f34fc070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.090+0000 7f3502bfc700 1 -- 172.21.15.22:0/304647743 >> 172.21.15.22:0/304647743 conn(0x7f34fc0fc8d0 msgr2=0x7f34fc104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.091+0000 7f3502bfc700 1 -- 172.21.15.22:0/304647743 shutdown_connections 2023-09-20T21:47:49.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:49.091+0000 7f3502bfc700 1 -- 172.21.15.22:0/304647743 wait complete. 2023-09-20T21:47:49.340 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:49 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2859444444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-20T21:47:49.341 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:49 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/304647743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-20T21:47:49.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:49 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2859444444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-20T21:47:49.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:49 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/304647743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-20T21:47:49.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:49 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2859444444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-09-20T21:47:49.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:49 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/304647743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-09-20T21:47:50.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.131+0000 7f452ed99700 1 -- 172.21.15.22:0/1770504531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4528105da0 msgr2=0x7f4528103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:50.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.131+0000 7f452ed99700 1 --2- 172.21.15.22:0/1770504531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4528105da0 0x7f4528103830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f451c009990 tx=0x7f451c009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:50.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.132+0000 7f452ed99700 1 -- 172.21.15.22:0/1770504531 shutdown_connections 2023-09-20T21:47:50.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.132+0000 7f452ed99700 1 --2- 172.21.15.22:0/1770504531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528068900 0x7f4528068d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:50.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.132+0000 7f452ed99700 1 --2- 172.21.15.22:0/1770504531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4528105da0 0x7f4528103830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:50.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.132+0000 7f452ed99700 1 --2- 172.21.15.22:0/1770504531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4528103d70 0x7f45281041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:50.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.132+0000 7f452ed99700 1 -- 172.21.15.22:0/1770504531 >> 172.21.15.22:0/1770504531 conn(0x7f4528076de0 msgr2=0x7f45280771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.132+0000 7f452ed99700 1 -- 172.21.15.22:0/1770504531 shutdown_connections 2023-09-20T21:47:50.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.132+0000 7f452ed99700 1 -- 172.21.15.22:0/1770504531 wait complete. 2023-09-20T21:47:50.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.133+0000 7f452ed99700 1 Processor -- start 2023-09-20T21:47:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.133+0000 7f452ed99700 1 -- start start 2023-09-20T21:47:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.133+0000 7f452ed99700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4528068900 0x7f452819d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.133+0000 7f452ed99700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4528103d70 0x7f452819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.133+0000 7f452ed99700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528105da0 0x7f45281a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:50.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.133+0000 7f452ed99700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45281a24a0 con 0x7f4528068900 2023-09-20T21:47:50.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.133+0000 7f452ed99700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45281a2610 con 0x7f4528105da0 2023-09-20T21:47:50.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.133+0000 7f452ed99700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45281a27d0 con 0x7f4528103d70 2023-09-20T21:47:50.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f4527fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4528103d70 0x7f452819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:50.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f4527fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4528103d70 0x7f452819da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43814/0 (socket says 172.21.15.22:43814) 2023-09-20T21:47:50.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f4527fff700 1 -- 172.21.15.22:0/177834307 learned_addr learned my addr 172.21.15.22:0/177834307 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:50.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f452cb35700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4528068900 0x7f452819d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:50.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f452d336700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528105da0 0x7f45281a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:50.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f4527fff700 1 -- 172.21.15.22:0/177834307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528105da0 msgr2=0x7f45281a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:50.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f4527fff700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528105da0 0x7f45281a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:50.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f4527fff700 1 -- 172.21.15.22:0/177834307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4528068900 msgr2=0x7f452819d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:50.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f4527fff700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4528068900 0x7f452819d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:50.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f4527fff700 1 -- 172.21.15.22:0/177834307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c0095f0 con 0x7f4528103d70 2023-09-20T21:47:50.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.134+0000 7f4527fff700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4528103d70 0x7f452819da20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f451400e970 tx=0x7f451400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:50.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.135+0000 7f4525ffb700 1 -- 172.21.15.22:0/177834307 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514009790 con 0x7f4528103d70 2023-09-20T21:47:50.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.135+0000 7f4525ffb700 1 -- 172.21.15.22:0/177834307 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451400c7e0 con 0x7f4528103d70 2023-09-20T21:47:50.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.135+0000 7f4525ffb700 1 -- 172.21.15.22:0/177834307 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514003c70 con 0x7f4528103d70 2023-09-20T21:47:50.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.135+0000 7f452ed99700 1 -- 172.21.15.22:0/177834307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45281a2ab0 con 0x7f4528103d70 2023-09-20T21:47:50.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.135+0000 7f452ed99700 1 -- 172.21.15.22:0/177834307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45281aa310 con 0x7f4528103d70 2023-09-20T21:47:50.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.136+0000 7f452ed99700 1 -- 172.21.15.22:0/177834307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452804eae0 con 0x7f4528103d70 2023-09-20T21:47:50.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.138+0000 7f4525ffb700 1 -- 172.21.15.22:0/177834307 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f451400cc50 con 0x7f4528103d70 2023-09-20T21:47:50.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.140+0000 7f4525ffb700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f45180a1380 0x7f45180a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:50.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.140+0000 7f4525ffb700 1 -- 172.21.15.22:0/177834307 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4514014070 con 0x7f4528103d70 2023-09-20T21:47:50.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.140+0000 7f452cb35700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f45180a1380 0x7f45180a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:50.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.141+0000 7f452cb35700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f45180a1380 0x7f45180a3840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f451c000c00 tx=0x7f451c00b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:50.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.142+0000 7f4525ffb700 1 -- 172.21.15.22:0/177834307 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f4514096520 con 0x7f4528103d70 2023-09-20T21:47:50.164 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070500 got 253403070502 for osd.8 2023-09-20T21:47:50.164 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:50.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.370+0000 7f452ed99700 1 -- 172.21.15.22:0/177834307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f452819e840 con 0x7f4528103d70 2023-09-20T21:47:50.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.370+0000 7f4525ffb700 1 -- 172.21.15.22:0/177834307 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f4514099b40 con 0x7f4528103d70 2023-09-20T21:47:50.371 INFO:teuthology.orchestra.run.smithi022.stdout:343597383707 2023-09-20T21:47:50.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 -- 172.21.15.22:0/177834307 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f45180a1380 msgr2=0x7f45180a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:50.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f45180a1380 0x7f45180a3840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f451c000c00 tx=0x7f451c00b260 comp rx=0 tx=0).stop 2023-09-20T21:47:50.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 -- 172.21.15.22:0/177834307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4528103d70 msgr2=0x7f452819da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:50.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4528103d70 0x7f452819da20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f451400e970 tx=0x7f451400ec80 comp rx=0 tx=0).stop 2023-09-20T21:47:50.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 -- 172.21.15.22:0/177834307 shutdown_connections 2023-09-20T21:47:50.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f45180a1380 0x7f45180a3840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:50.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4528105da0 0x7f45281a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:50.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4528068900 0x7f452819d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:50.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 --2- 172.21.15.22:0/177834307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4528103d70 0x7f452819da20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:50.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 -- 172.21.15.22:0/177834307 >> 172.21.15.22:0/177834307 conn(0x7f4528076de0 msgr2=0x7f45280fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:50.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 -- 172.21.15.22:0/177834307 shutdown_connections 2023-09-20T21:47:50.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:50.373+0000 7f452ed99700 1 -- 172.21.15.22:0/177834307 wait complete. 2023-09-20T21:47:50.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:50 smithi022 ceph-mon[97651]: pgmap v312: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:47:50.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:50 smithi123 ceph-mon[134433]: pgmap v312: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:47:50.664 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:50 smithi089 ceph-mon[134090]: pgmap v312: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:47:51.010 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495664 for osd.6 2023-09-20T21:47:51.011 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:51.193 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:51 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/177834307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-20T21:47:51.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.305+0000 7f032d360700 1 -- 172.21.15.22:0/2210013171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0328101060 msgr2=0x7f0328101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.305+0000 7f032d360700 1 --2- 172.21.15.22:0/2210013171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0328101060 0x7f0328101440 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0318009990 tx=0x7f0318009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.305+0000 7f032d360700 1 -- 172.21.15.22:0/2210013171 shutdown_connections 2023-09-20T21:47:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.305+0000 7f032d360700 1 --2- 172.21.15.22:0/2210013171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0328106140 0x7f032810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.305+0000 7f032d360700 1 --2- 172.21.15.22:0/2210013171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0328101060 0x7f0328101440 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.305+0000 7f032d360700 1 --2- 172.21.15.22:0/2210013171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0328101a10 0x7f0328105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.305+0000 7f032d360700 1 -- 172.21.15.22:0/2210013171 >> 172.21.15.22:0/2210013171 conn(0x7f03280fc8d0 msgr2=0x7f03280fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:51.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.305+0000 7f032d360700 1 -- 172.21.15.22:0/2210013171 shutdown_connections 2023-09-20T21:47:51.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.305+0000 7f032d360700 1 -- 172.21.15.22:0/2210013171 wait complete. 2023-09-20T21:47:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.306+0000 7f032d360700 1 Processor -- start 2023-09-20T21:47:51.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.307+0000 7f032d360700 1 -- start start 2023-09-20T21:47:51.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.307+0000 7f032d360700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0328101060 0x7f0328196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:51.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.307+0000 7f032d360700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0328101a10 0x7f0328197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:51.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.307+0000 7f032d360700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0328106140 0x7f032819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:51.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.307+0000 7f032d360700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032819bed0 con 0x7f0328106140 2023-09-20T21:47:51.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.307+0000 7f032d360700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032819c040 con 0x7f0328101a10 2023-09-20T21:47:51.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.307+0000 7f032d360700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032819c200 con 0x7f0328101060 2023-09-20T21:47:51.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f0326ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0328101060 0x7f0328196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:51.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03267fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0328101a10 0x7f0328197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:51.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03277fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0328106140 0x7f032819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:51.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03267fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0328101a10 0x7f0328197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:34338/0 (socket says 172.21.15.22:34338) 2023-09-20T21:47:51.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03267fc700 1 -- 172.21.15.22:0/1277946327 learned_addr learned my addr 172.21.15.22:0/1277946327 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:51.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f0326ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0328101060 0x7f0328196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43834/0 (socket says 172.21.15.22:43834) 2023-09-20T21:47:51.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03267fc700 1 -- 172.21.15.22:0/1277946327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0328101060 msgr2=0x7f0328196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:51.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03267fc700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0328101060 0x7f0328196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03267fc700 1 -- 172.21.15.22:0/1277946327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0328106140 msgr2=0x7f032819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:51.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03267fc700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0328106140 0x7f032819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03267fc700 1 -- 172.21.15.22:0/1277946327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180095f0 con 0x7f0328101a10 2023-09-20T21:47:51.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f0326ffd700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0328101060 0x7f0328196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:51.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03277fe700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0328106140 0x7f032819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:51.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.308+0000 7f03267fc700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0328101a10 0x7f0328197450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f031c00e970 tx=0x7f031c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:51.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.309+0000 7f030ffff700 1 -- 172.21.15.22:0/1277946327 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c009790 con 0x7f0328101a10 2023-09-20T21:47:51.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.309+0000 7f030ffff700 1 -- 172.21.15.22:0/1277946327 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f031c00c7e0 con 0x7f0328101a10 2023-09-20T21:47:51.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.309+0000 7f032d360700 1 -- 172.21.15.22:0/1277946327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032819c4e0 con 0x7f0328101a10 2023-09-20T21:47:51.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.309+0000 7f030ffff700 1 -- 172.21.15.22:0/1277946327 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c0054e0 con 0x7f0328101a10 2023-09-20T21:47:51.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.309+0000 7f032d360700 1 -- 172.21.15.22:0/1277946327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03281a3d40 con 0x7f0328101a10 2023-09-20T21:47:51.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.310+0000 7f032d360700 1 -- 172.21.15.22:0/1277946327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032804eae0 con 0x7f0328101a10 2023-09-20T21:47:51.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.312+0000 7f030ffff700 1 -- 172.21.15.22:0/1277946327 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f031c00c950 con 0x7f0328101a10 2023-09-20T21:47:51.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.313+0000 7f030ffff700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f03100a1300 0x7f03100a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:51.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.313+0000 7f030ffff700 1 -- 172.21.15.22:0/1277946327 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f031c014070 con 0x7f0328101a10 2023-09-20T21:47:51.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.314+0000 7f0326ffd700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f03100a1300 0x7f03100a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:51.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.314+0000 7f0326ffd700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f03100a1300 0x7f03100a37c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f031800b2e0 tx=0x7f0318005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:51.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.317+0000 7f030ffff700 1 -- 172.21.15.22:0/1277946327 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f031c096520 con 0x7f0328101a10 2023-09-20T21:47:51.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.352+0000 7f6a2094e700 1 -- 172.21.15.22:0/103369542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a1c073180 msgr2=0x7f6a1c073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.352+0000 7f6a2094e700 1 --2- 172.21.15.22:0/103369542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a1c073180 0x7f6a1c073560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0098f0 tx=0x7f6a0c009c00 comp rx=0 tx=0).stop 2023-09-20T21:47:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.352+0000 7f6a2094e700 1 -- 172.21.15.22:0/103369542 shutdown_connections 2023-09-20T21:47:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.352+0000 7f6a2094e700 1 --2- 172.21.15.22:0/103369542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a1c111e10 0x7f6a1c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.352+0000 7f6a2094e700 1 --2- 172.21.15.22:0/103369542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a1c073180 0x7f6a1c073560 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.352+0000 7f6a2094e700 1 --2- 172.21.15.22:0/103369542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a1c073aa0 0x7f6a1c1118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.352+0000 7f6a2094e700 1 -- 172.21.15.22:0/103369542 >> 172.21.15.22:0/103369542 conn(0x7f6a1c0fc8d0 msgr2=0x7f6a1c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.352+0000 7f6a2094e700 1 -- 172.21.15.22:0/103369542 shutdown_connections 2023-09-20T21:47:51.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.352+0000 7f6a2094e700 1 -- 172.21.15.22:0/103369542 wait complete. 2023-09-20T21:47:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.354+0000 7f6a2094e700 1 Processor -- start 2023-09-20T21:47:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.354+0000 7f6a2094e700 1 -- start start 2023-09-20T21:47:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a2094e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a1c073180 0x7f6a1c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a2094e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a1c073aa0 0x7f6a1c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a2094e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a1c111e10 0x7f6a1c1a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a2094e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c1a2580 con 0x7f6a1c073180 2023-09-20T21:47:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a2094e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c1a26f0 con 0x7f6a1c073aa0 2023-09-20T21:47:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a2094e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c1a28b0 con 0x7f6a1c111e10 2023-09-20T21:47:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a1a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a1c073180 0x7f6a1c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a1a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a1c073180 0x7f6a1c19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37270/0 (socket says 172.21.15.22:37270) 2023-09-20T21:47:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a1a59c700 1 -- 172.21.15.22:0/3386264693 learned_addr learned my addr 172.21.15.22:0/3386264693 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a19d9b700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a1c073aa0 0x7f6a1c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.355+0000 7f6a1ad9d700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a1c111e10 0x7f6a1c1a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.356+0000 7f6a1a59c700 1 -- 172.21.15.22:0/3386264693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a1c111e10 msgr2=0x7f6a1c1a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:51.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.356+0000 7f6a1a59c700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a1c111e10 0x7f6a1c1a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.356+0000 7f6a1a59c700 1 -- 172.21.15.22:0/3386264693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a1c073aa0 msgr2=0x7f6a1c19db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:51.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.356+0000 7f6a1a59c700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a1c073aa0 0x7f6a1c19db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.356+0000 7f6a1a59c700 1 -- 172.21.15.22:0/3386264693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a100095f0 con 0x7f6a1c073180 2023-09-20T21:47:51.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.356+0000 7f6a19d9b700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a1c073aa0 0x7f6a1c19db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.356+0000 7f6a1a59c700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a1c073180 0x7f6a1c19d5c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009810 tx=0x7f6a0c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.356+0000 7f6a0b7fe700 1 -- 172.21.15.22:0/3386264693 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c01c070 con 0x7f6a1c073180 2023-09-20T21:47:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.357+0000 7f6a2094e700 1 -- 172.21.15.22:0/3386264693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a14009c40 con 0x7f6a1c073180 2023-09-20T21:47:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.357+0000 7f6a0b7fe700 1 -- 172.21.15.22:0/3386264693 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a0c00bbd0 con 0x7f6a1c073180 2023-09-20T21:47:51.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.357+0000 7f6a0b7fe700 1 -- 172.21.15.22:0/3386264693 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c004930 con 0x7f6a1c073180 2023-09-20T21:47:51.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.357+0000 7f6a2094e700 1 -- 172.21.15.22:0/3386264693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c009520 con 0x7f6a1c073180 2023-09-20T21:47:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.359+0000 7f6a2094e700 1 -- 172.21.15.22:0/3386264693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1c1042f0 con 0x7f6a1c073180 2023-09-20T21:47:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.360+0000 7f6a0b7fe700 1 -- 172.21.15.22:0/3386264693 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f6a0c020070 con 0x7f6a1c073180 2023-09-20T21:47:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.361+0000 7f6a0b7fe700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6a040a1300 0x7f6a040a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.361+0000 7f6a0b7fe700 1 -- 172.21.15.22:0/3386264693 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6a0c0cdb80 con 0x7f6a1c073180 2023-09-20T21:47:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.362+0000 7f6a19d9b700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6a040a1300 0x7f6a040a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.362+0000 7f6a19d9b700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6a040a1300 0x7f6a040a37c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c19ebe0 tx=0x7f6a10009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.365+0000 7f6a0b7fe700 1 -- 172.21.15.22:0/3386264693 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6a0c096f80 con 0x7f6a1c073180 2023-09-20T21:47:51.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.566+0000 7f032d360700 1 -- 172.21.15.22:0/1277946327 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0328110e70 con 0x7f0328101a10 2023-09-20T21:47:51.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.566+0000 7f030ffff700 1 -- 172.21.15.22:0/1277946327 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f031c099b40 con 0x7f0328101a10 2023-09-20T21:47:51.567 INFO:teuthology.orchestra.run.smithi022.stdout:77309411397 2023-09-20T21:47:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 -- 172.21.15.22:0/1277946327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f03100a1300 msgr2=0x7f03100a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:51.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f03100a1300 0x7f03100a37c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f031800b2e0 tx=0x7f0318005fd0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 -- 172.21.15.22:0/1277946327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0328101a10 msgr2=0x7f0328197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:51.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0328101a10 0x7f0328197450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f031c00e970 tx=0x7f031c00ec80 comp rx=0 tx=0).stop 2023-09-20T21:47:51.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 -- 172.21.15.22:0/1277946327 shutdown_connections 2023-09-20T21:47:51.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0328106140 0x7f032819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0328101060 0x7f0328196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f03100a1300 0x7f03100a37c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 --2- 172.21.15.22:0/1277946327 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0328101a10 0x7f0328197450 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 -- 172.21.15.22:0/1277946327 >> 172.21.15.22:0/1277946327 conn(0x7f03280fc8d0 msgr2=0x7f03280fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:51.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 -- 172.21.15.22:0/1277946327 shutdown_connections 2023-09-20T21:47:51.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.569+0000 7f032d360700 1 -- 172.21.15.22:0/1277946327 wait complete. 2023-09-20T21:47:51.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:51 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/177834307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-20T21:47:51.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.599+0000 7f6a2094e700 1 -- 172.21.15.22:0/3386264693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6a1c19ea20 con 0x7f6a1c073180 2023-09-20T21:47:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.600+0000 7f6a0b7fe700 1 -- 172.21.15.22:0/3386264693 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6a0c09a5a0 con 0x7f6a1c073180 2023-09-20T21:47:51.601 INFO:teuthology.orchestra.run.smithi022.stdout:34359738443 2023-09-20T21:47:51.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.602+0000 7f6a2094e700 1 -- 172.21.15.22:0/3386264693 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6a040a1300 msgr2=0x7f6a040a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:51.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.602+0000 7f6a2094e700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6a040a1300 0x7f6a040a37c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c19ebe0 tx=0x7f6a10009310 comp rx=0 tx=0).stop 2023-09-20T21:47:51.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.602+0000 7f6a2094e700 1 -- 172.21.15.22:0/3386264693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a1c073180 msgr2=0x7f6a1c19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:51.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.602+0000 7f6a2094e700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a1c073180 0x7f6a1c19d5c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009810 tx=0x7f6a0c00fa00 comp rx=0 tx=0).stop 2023-09-20T21:47:51.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.602+0000 7f6a2094e700 1 -- 172.21.15.22:0/3386264693 shutdown_connections 2023-09-20T21:47:51.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.602+0000 7f6a2094e700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a1c111e10 0x7f6a1c1a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.602+0000 7f6a2094e700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a1c073180 0x7f6a1c19d5c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.602+0000 7f6a2094e700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6a040a1300 0x7f6a040a37c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.602+0000 7f6a2094e700 1 --2- 172.21.15.22:0/3386264693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a1c073aa0 0x7f6a1c19db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:51.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.602+0000 7f6a2094e700 1 -- 172.21.15.22:0/3386264693 >> 172.21.15.22:0/3386264693 conn(0x7f6a1c0fc8d0 msgr2=0x7f6a1c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:51.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.603+0000 7f6a2094e700 1 -- 172.21.15.22:0/3386264693 shutdown_connections 2023-09-20T21:47:51.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:51.603+0000 7f6a2094e700 1 -- 172.21.15.22:0/3386264693 wait complete. 2023-09-20T21:47:51.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:51 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/177834307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-09-20T21:47:52.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383704 got 343597383707 for osd.11 2023-09-20T21:47:52.321 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:52.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:52 smithi022 ceph-mon[97651]: pgmap v313: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:47:52.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:52 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1277946327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-20T21:47:52.448 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:52 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3386264693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-20T21:47:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:52 smithi123 ceph-mon[134433]: pgmap v313: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:47:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:52 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1277946327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-20T21:47:52.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:52 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3386264693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-20T21:47:52.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:52 smithi089 ceph-mon[134090]: pgmap v313: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:47:52.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:52 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1277946327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-09-20T21:47:52.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:52 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3386264693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-09-20T21:47:52.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411397 for osd.2 2023-09-20T21:47:52.787 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:52.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.795+0000 7f6e26f91700 1 -- 172.21.15.22:0/1366382812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20101060 msgr2=0x7f6e20101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:52.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.795+0000 7f6e26f91700 1 --2- 172.21.15.22:0/1366382812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20101060 0x7f6e20101440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009990 tx=0x7f6e14009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.795+0000 7f6e26f91700 1 -- 172.21.15.22:0/1366382812 shutdown_connections 2023-09-20T21:47:52.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.795+0000 7f6e26f91700 1 --2- 172.21.15.22:0/1366382812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e20106140 0x7f6e2010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.795+0000 7f6e26f91700 1 --2- 172.21.15.22:0/1366382812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20101060 0x7f6e20101440 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.795+0000 7f6e26f91700 1 --2- 172.21.15.22:0/1366382812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e20101a10 0x7f6e20105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.795+0000 7f6e26f91700 1 -- 172.21.15.22:0/1366382812 >> 172.21.15.22:0/1366382812 conn(0x7f6e200fc8d0 msgr2=0x7f6e200fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:52.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.795+0000 7f6e26f91700 1 -- 172.21.15.22:0/1366382812 shutdown_connections 2023-09-20T21:47:52.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.795+0000 7f6e26f91700 1 -- 172.21.15.22:0/1366382812 wait complete. 2023-09-20T21:47:52.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.796+0000 7f6e26f91700 1 Processor -- start 2023-09-20T21:47:52.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.797+0000 7f6e26f91700 1 -- start start 2023-09-20T21:47:52.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.797+0000 7f6e26f91700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20101060 0x7f6e201a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.797+0000 7f6e26f91700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e20101a10 0x7f6e201a2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.797+0000 7f6e26f91700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e20106140 0x7f6e2010c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.797+0000 7f6e26f91700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2010d160 con 0x7f6e20106140 2023-09-20T21:47:52.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.797+0000 7f6e26f91700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2010d2d0 con 0x7f6e20101a10 2023-09-20T21:47:52.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.797+0000 7f6e26f91700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2010d490 con 0x7f6e20101060 2023-09-20T21:47:52.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.797+0000 7f6e2552e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e20106140 0x7f6e2010c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:52.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.797+0000 7f6e2552e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e20106140 0x7f6e2010c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37318/0 (socket says 172.21.15.22:37318) 2023-09-20T21:47:52.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.797+0000 7f6e2552e700 1 -- 172.21.15.22:0/2844430146 learned_addr learned my addr 172.21.15.22:0/2844430146 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:52.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.798+0000 7f6e2552e700 1 -- 172.21.15.22:0/2844430146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20101060 msgr2=0x7f6e201a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:52.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.798+0000 7f6e2552e700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20101060 0x7f6e201a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.798+0000 7f6e2552e700 1 -- 172.21.15.22:0/2844430146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e20101a10 msgr2=0x7f6e201a2750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:47:52.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.798+0000 7f6e2552e700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e20101a10 0x7f6e201a2750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.798+0000 7f6e2552e700 1 -- 172.21.15.22:0/2844430146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c009520 con 0x7f6e20106140 2023-09-20T21:47:52.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.799+0000 7f6e2552e700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e20106140 0x7f6e2010c9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800ef30 tx=0x7f6e1800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.799+0000 7f6e1dffb700 1 -- 172.21.15.22:0/2844430146 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18018070 con 0x7f6e20106140 2023-09-20T21:47:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.799+0000 7f6e26f91700 1 -- 172.21.15.22:0/2844430146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e140095f0 con 0x7f6e20106140 2023-09-20T21:47:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.799+0000 7f6e26f91700 1 -- 172.21.15.22:0/2844430146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e201aa310 con 0x7f6e20106140 2023-09-20T21:47:52.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.799+0000 7f6e1dffb700 1 -- 172.21.15.22:0/2844430146 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e1800f040 con 0x7f6e20106140 2023-09-20T21:47:52.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.799+0000 7f6e1dffb700 1 -- 172.21.15.22:0/2844430146 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18003de0 con 0x7f6e20106140 2023-09-20T21:47:52.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.800+0000 7f6e26f91700 1 -- 172.21.15.22:0/2844430146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e04005320 con 0x7f6e20106140 2023-09-20T21:47:52.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.807+0000 7f6e1dffb700 1 -- 172.21.15.22:0/2844430146 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f6e18004ad0 con 0x7f6e20106140 2023-09-20T21:47:52.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.809+0000 7f6e1dffb700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6e100a1400 0x7f6e100a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.809+0000 7f6e1dffb700 1 -- 172.21.15.22:0/2844430146 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6e180cd3c0 con 0x7f6e20106140 2023-09-20T21:47:52.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.810+0000 7f6e24d2d700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6e100a1400 0x7f6e100a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:52.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.811+0000 7f6e1dffb700 1 -- 172.21.15.22:0/2844430146 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6e180967e0 con 0x7f6e20106140 2023-09-20T21:47:52.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.811+0000 7f6e24d2d700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6e100a1400 0x7f6e100a38c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6e140098e0 tx=0x7f6e14005e80 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:52.880 INFO:teuthology.orchestra.run.smithi022.stdout:287762808866 2023-09-20T21:47:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.488+0000 7f25f341f700 1 -- 172.21.15.22:0/1175180403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec0ffd20 msgr2=0x7f25ec1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.488+0000 7f25f341f700 1 --2- 172.21.15.22:0/1175180403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec0ffd20 0x7f25ec1001a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009990 tx=0x7f25dc009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.488+0000 7f25f341f700 1 -- 172.21.15.22:0/1175180403 shutdown_connections 2023-09-20T21:47:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.488+0000 7f25f341f700 1 --2- 172.21.15.22:0/1175180403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec1006e0 0x7f25ec10cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.488+0000 7f25f341f700 1 --2- 172.21.15.22:0/1175180403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec105dd0 0x7f25ec1061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.488+0000 7f25f341f700 1 --2- 172.21.15.22:0/1175180403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec0ffd20 0x7f25ec1001a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.488+0000 7f25f341f700 1 -- 172.21.15.22:0/1175180403 >> 172.21.15.22:0/1175180403 conn(0x7f25ec076de0 msgr2=0x7f25ec0771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.488+0000 7f25f341f700 1 -- 172.21.15.22:0/1175180403 shutdown_connections 2023-09-20T21:47:52.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.488+0000 7f25f341f700 1 -- 172.21.15.22:0/1175180403 wait complete. 2023-09-20T21:47:52.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.489+0000 7f25f341f700 1 Processor -- start 2023-09-20T21:47:52.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.489+0000 7f25f341f700 1 -- start start 2023-09-20T21:47:52.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f341f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec0ffd20 0x7f25ec1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f341f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec1006e0 0x7f25ec199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f341f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec105dd0 0x7f25ec19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f341f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec19e080 con 0x7f25ec1006e0 2023-09-20T21:47:52.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f341f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec19e1f0 con 0x7f25ec0ffd20 2023-09-20T21:47:52.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f341f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec19e3b0 con 0x7f25ec105dd0 2023-09-20T21:47:52.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f09ba700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec1006e0 0x7f25ec199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:52.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f11bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec0ffd20 0x7f25ec1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:52.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f11bb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec0ffd20 0x7f25ec1990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:34380/0 (socket says 172.21.15.22:34380) 2023-09-20T21:47:52.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f11bb700 1 -- 172.21.15.22:0/823120859 learned_addr learned my addr 172.21.15.22:0/823120859 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:52.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f19bc700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec105dd0 0x7f25ec19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:52.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f11bb700 1 -- 172.21.15.22:0/823120859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec105dd0 msgr2=0x7f25ec19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:52.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f11bb700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec105dd0 0x7f25ec19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f11bb700 1 -- 172.21.15.22:0/823120859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec1006e0 msgr2=0x7f25ec199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:52.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f11bb700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec1006e0 0x7f25ec199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f11bb700 1 -- 172.21.15.22:0/823120859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc0095f0 con 0x7f25ec0ffd20 2023-09-20T21:47:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.490+0000 7f25f19bc700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec105dd0 0x7f25ec19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:47:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.491+0000 7f25f11bb700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec0ffd20 0x7f25ec1990c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f25e800e9a0 tx=0x7f25e800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.491+0000 7f25e27fc700 1 -- 172.21.15.22:0/823120859 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e8009790 con 0x7f25ec0ffd20 2023-09-20T21:47:52.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.491+0000 7f25f341f700 1 -- 172.21.15.22:0/823120859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ec19e690 con 0x7f25ec0ffd20 2023-09-20T21:47:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.491+0000 7f25e27fc700 1 -- 172.21.15.22:0/823120859 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e800cbd0 con 0x7f25ec0ffd20 2023-09-20T21:47:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.491+0000 7f25f341f700 1 -- 172.21.15.22:0/823120859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec1a5ef0 con 0x7f25ec0ffd20 2023-09-20T21:47:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.491+0000 7f25e27fc700 1 -- 172.21.15.22:0/823120859 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e8018680 con 0x7f25ec0ffd20 2023-09-20T21:47:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.492+0000 7f25f341f700 1 -- 172.21.15.22:0/823120859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ec04eae0 con 0x7f25ec0ffd20 2023-09-20T21:47:52.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.494+0000 7f25e27fc700 1 -- 172.21.15.22:0/823120859 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f25e8010430 con 0x7f25ec0ffd20 2023-09-20T21:47:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.496+0000 7f25e27fc700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1380 0x7f25d80a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.496+0000 7f25e27fc700 1 -- 172.21.15.22:0/823120859 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f25e8014070 con 0x7f25ec0ffd20 2023-09-20T21:47:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.496+0000 7f25f09ba700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1380 0x7f25d80a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.496+0000 7f25f09ba700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1380 0x7f25d80a3840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00b6f0 tx=0x7f25dc00b280 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.498+0000 7f25e27fc700 1 -- 172.21.15.22:0/823120859 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f25e8051e10 con 0x7f25ec0ffd20 2023-09-20T21:47:52.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.717+0000 7f25f341f700 1 -- 172.21.15.22:0/823120859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f25ec110e80 con 0x7f25ec0ffd20 2023-09-20T21:47:52.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.718+0000 7f25e27fc700 1 -- 172.21.15.22:0/823120859 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f25e80cae60 con 0x7f25ec0ffd20 2023-09-20T21:47:52.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 -- 172.21.15.22:0/823120859 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1380 msgr2=0x7f25d80a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:52.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1380 0x7f25d80a3840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00b6f0 tx=0x7f25dc00b280 comp rx=0 tx=0).stop 2023-09-20T21:47:52.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 -- 172.21.15.22:0/823120859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec0ffd20 msgr2=0x7f25ec1990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec0ffd20 0x7f25ec1990c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f25e800e9a0 tx=0x7f25e800ecb0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 -- 172.21.15.22:0/823120859 shutdown_connections 2023-09-20T21:47:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec105dd0 0x7f25ec19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1380 0x7f25d80a3840 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec0ffd20 0x7f25ec1990c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 --2- 172.21.15.22:0/823120859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec1006e0 0x7f25ec199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 -- 172.21.15.22:0/823120859 >> 172.21.15.22:0/823120859 conn(0x7f25ec076de0 msgr2=0x7f25ec10d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 -- 172.21.15.22:0/823120859 shutdown_connections 2023-09-20T21:47:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.721+0000 7f25f341f700 1 -- 172.21.15.22:0/823120859 wait complete. 2023-09-20T21:47:52.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.988+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/3724137909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c8105dd0 msgr2=0x7ff8c81061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:52.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.988+0000 7ff8cd9ab700 1 --2- 172.21.15.22:0/3724137909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c8105dd0 0x7ff8c81061b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009990 tx=0x7ff8b8009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.989+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/3724137909 shutdown_connections 2023-09-20T21:47:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.989+0000 7ff8cd9ab700 1 --2- 172.21.15.22:0/3724137909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c81006e0 0x7ff8c810cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.989+0000 7ff8cd9ab700 1 --2- 172.21.15.22:0/3724137909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c8105dd0 0x7ff8c81061b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.989+0000 7ff8cd9ab700 1 --2- 172.21.15.22:0/3724137909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c80ffd20 0x7ff8c81001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.989+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/3724137909 >> 172.21.15.22:0/3724137909 conn(0x7ff8c8076de0 msgr2=0x7ff8c80771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:52.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.989+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/3724137909 shutdown_connections 2023-09-20T21:47:52.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.989+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/3724137909 wait complete. 2023-09-20T21:47:52.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.990+0000 7ff8cd9ab700 1 Processor -- start 2023-09-20T21:47:52.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.991+0000 7ff8cd9ab700 1 -- start start 2023-09-20T21:47:52.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.991+0000 7ff8cd9ab700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c80ffd20 0x7ff8c8195460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.991+0000 7ff8cd9ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81006e0 0x7ff8c81959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.991+0000 7ff8cd9ab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c8105dd0 0x7ff8c81995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:52.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.991+0000 7ff8cd9ab700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8199b30 con 0x7ff8c8105dd0 2023-09-20T21:47:52.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.991+0000 7ff8cd9ab700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8199c70 con 0x7ff8c80ffd20 2023-09-20T21:47:52.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.991+0000 7ff8cd9ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8199e30 con 0x7ff8c81006e0 2023-09-20T21:47:52.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.992+0000 7ff8c67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81006e0 0x7ff8c81959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:52.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.992+0000 7ff8c67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81006e0 0x7ff8c81959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43916/0 (socket says 172.21.15.22:43916) 2023-09-20T21:47:52.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.992+0000 7ff8c67fc700 1 -- 172.21.15.22:0/728952762 learned_addr learned my addr 172.21.15.22:0/728952762 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:52.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.992+0000 7ff8c67fc700 1 -- 172.21.15.22:0/728952762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c80ffd20 msgr2=0x7ff8c8195460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:47:52.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.992+0000 7ff8c67fc700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c80ffd20 0x7ff8c8195460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.992+0000 7ff8c67fc700 1 -- 172.21.15.22:0/728952762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c8105dd0 msgr2=0x7ff8c81995f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:47:52.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.992+0000 7ff8c67fc700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c8105dd0 0x7ff8c81995f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:52.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.992+0000 7ff8c67fc700 1 -- 172.21.15.22:0/728952762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b80095f0 con 0x7ff8c81006e0 2023-09-20T21:47:53.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.996+0000 7ff8c67fc700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81006e0 0x7ff8c81959a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00e9c0 tx=0x7ff8bc00ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:53.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.997+0000 7ff8cc9a9700 1 -- 172.21.15.22:0/728952762 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc00cbd0 con 0x7ff8c81006e0 2023-09-20T21:47:53.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.997+0000 7ff8cc9a9700 1 -- 172.21.15.22:0/728952762 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8bc00cd30 con 0x7ff8c81006e0 2023-09-20T21:47:53.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.997+0000 7ff8cc9a9700 1 -- 172.21.15.22:0/728952762 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc005700 con 0x7ff8c81006e0 2023-09-20T21:47:53.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.997+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/728952762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c819a110 con 0x7ff8c81006e0 2023-09-20T21:47:53.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.997+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/728952762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c806a800 con 0x7ff8c81006e0 2023-09-20T21:47:53.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:52.999+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/728952762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c804eae0 con 0x7ff8c81006e0 2023-09-20T21:47:53.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.001+0000 7ff8cc9a9700 1 -- 172.21.15.22:0/728952762 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ff8bc004750 con 0x7ff8c81006e0 2023-09-20T21:47:53.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.002+0000 7ff8cc9a9700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff8b00a1380 0x7ff8b00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:53.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.002+0000 7ff8cc9a9700 1 -- 172.21.15.22:0/728952762 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff8bc014070 con 0x7ff8c81006e0 2023-09-20T21:47:53.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.003+0000 7ff8c6ffd700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff8b00a1380 0x7ff8b00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:53.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.005+0000 7ff8cc9a9700 1 -- 172.21.15.22:0/728952762 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff8bc096650 con 0x7ff8c81006e0 2023-09-20T21:47:53.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.005+0000 7ff8c6ffd700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff8b00a1380 0x7ff8b00a3840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80098e0 tx=0x7ff8b8000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:53.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.069+0000 7f6e26f91700 1 -- 172.21.15.22:0/2844430146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6e04005190 con 0x7f6e20106140 2023-09-20T21:47:53.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.069+0000 7f6e1dffb700 1 -- 172.21.15.22:0/2844430146 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e18025e60 con 0x7f6e20106140 2023-09-20T21:47:53.073 INFO:teuthology.orchestra.run.smithi022.stdout:163208757303 2023-09-20T21:47:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 -- 172.21.15.22:0/2844430146 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6e100a1400 msgr2=0x7f6e100a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6e100a1400 0x7f6e100a38c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6e140098e0 tx=0x7f6e14005e80 comp rx=0 tx=0).stop 2023-09-20T21:47:53.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 -- 172.21.15.22:0/2844430146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e20106140 msgr2=0x7f6e2010c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:53.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e20106140 0x7f6e2010c9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800ef30 tx=0x7f6e1800c450 comp rx=0 tx=0).stop 2023-09-20T21:47:53.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 -- 172.21.15.22:0/2844430146 shutdown_connections 2023-09-20T21:47:53.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e20106140 0x7f6e2010c9c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e20101060 0x7f6e201a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6e100a1400 0x7f6e100a38c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 --2- 172.21.15.22:0/2844430146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e20101a10 0x7f6e201a2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 -- 172.21.15.22:0/2844430146 >> 172.21.15.22:0/2844430146 conn(0x7f6e200fc8d0 msgr2=0x7f6e20102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.072+0000 7f6e26f91700 1 -- 172.21.15.22:0/2844430146 shutdown_connections 2023-09-20T21:47:53.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.073+0000 7f6e26f91700 1 -- 172.21.15.22:0/2844430146 wait complete. 2023-09-20T21:47:53.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.251+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/728952762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff8c810fce0 con 0x7ff8c81006e0 2023-09-20T21:47:53.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.252+0000 7ff8cc9a9700 1 -- 172.21.15.22:0/728952762 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8bc00cea0 con 0x7ff8c81006e0 2023-09-20T21:47:53.253 INFO:teuthology.orchestra.run.smithi022.stdout:107374182462 2023-09-20T21:47:53.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.254+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/728952762 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff8b00a1380 msgr2=0x7ff8b00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:53.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.254+0000 7ff8cd9ab700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff8b00a1380 0x7ff8b00a3840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80098e0 tx=0x7ff8b8000bc0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.255+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/728952762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81006e0 msgr2=0x7ff8c81959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:53.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.255+0000 7ff8cd9ab700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81006e0 0x7ff8c81959a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00e9c0 tx=0x7ff8bc00ed80 comp rx=0 tx=0).stop 2023-09-20T21:47:53.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.255+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/728952762 shutdown_connections 2023-09-20T21:47:53.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.255+0000 7ff8cd9ab700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8c8105dd0 0x7ff8c81995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.255+0000 7ff8cd9ab700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff8b00a1380 0x7ff8b00a3840 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.255+0000 7ff8cd9ab700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8c80ffd20 0x7ff8c8195460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.255+0000 7ff8cd9ab700 1 --2- 172.21.15.22:0/728952762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81006e0 0x7ff8c81959a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.255+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/728952762 >> 172.21.15.22:0/728952762 conn(0x7ff8c8076de0 msgr2=0x7ff8c80688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.255+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/728952762 shutdown_connections 2023-09-20T21:47:53.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.255+0000 7ff8cd9ab700 1 -- 172.21.15.22:0/728952762 wait complete. 2023-09-20T21:47:53.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738441 got 34359738443 for osd.0 2023-09-20T21:47:53.566 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:53.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:53 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/823120859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-20T21:47:53.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:53 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2844430146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-20T21:47:53.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:53 smithi123 ceph-mon[134433]: pgmap v314: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:47:53.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:53 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/728952762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-20T21:47:53.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:53 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/823120859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-20T21:47:53.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:53 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2844430146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-20T21:47:53.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:53 smithi089 ceph-mon[134090]: pgmap v314: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:47:53.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:53 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/728952762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-20T21:47:53.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.743+0000 7fc8463f3700 1 -- 172.21.15.22:0/4014318487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8400731b0 msgr2=0x7fc840073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:53.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.743+0000 7fc8463f3700 1 --2- 172.21.15.22:0/4014318487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8400731b0 0x7fc840073590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc830009990 tx=0x7fc830009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.743+0000 7fc8463f3700 1 -- 172.21.15.22:0/4014318487 shutdown_connections 2023-09-20T21:47:53.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.743+0000 7fc8463f3700 1 --2- 172.21.15.22:0/4014318487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84010d6b0 0x7fc84010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.743+0000 7fc8463f3700 1 --2- 172.21.15.22:0/4014318487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8400731b0 0x7fc840073590 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.743+0000 7fc8463f3700 1 --2- 172.21.15.22:0/4014318487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc840073ad0 0x7fc84010d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.743+0000 7fc8463f3700 1 -- 172.21.15.22:0/4014318487 >> 172.21.15.22:0/4014318487 conn(0x7fc8400fc900 msgr2=0x7fc8400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:53.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.743+0000 7fc8463f3700 1 -- 172.21.15.22:0/4014318487 shutdown_connections 2023-09-20T21:47:53.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.743+0000 7fc8463f3700 1 -- 172.21.15.22:0/4014318487 wait complete. 2023-09-20T21:47:53.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.744+0000 7fc8463f3700 1 Processor -- start 2023-09-20T21:47:53.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.744+0000 7fc8463f3700 1 -- start start 2023-09-20T21:47:53.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc8463f3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8400731b0 0x7fc840199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:53.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc8463f3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc840073ad0 0x7fc840199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:53.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc8463f3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84010d6b0 0x7fc84019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:53.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc8463f3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84019e0f0 con 0x7fc8400731b0 2023-09-20T21:47:53.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc8463f3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84019e260 con 0x7fc84010d6b0 2023-09-20T21:47:53.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc8463f3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84019e420 con 0x7fc840073ad0 2023-09-20T21:47:53.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc83ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8400731b0 0x7fc840199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:53.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc83ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8400731b0 0x7fc840199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37340/0 (socket says 172.21.15.22:37340) 2023-09-20T21:47:53.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc83ffff700 1 -- 172.21.15.22:0/1065790424 learned_addr learned my addr 172.21.15.22:0/1065790424 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:53.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc83f7fe700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc840073ad0 0x7fc840199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:53.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc844990700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84010d6b0 0x7fc84019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:53.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc83ffff700 1 -- 172.21.15.22:0/1065790424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc840073ad0 msgr2=0x7fc840199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:53.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc83ffff700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc840073ad0 0x7fc840199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc83ffff700 1 -- 172.21.15.22:0/1065790424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84010d6b0 msgr2=0x7fc84019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:53.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc83ffff700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84010d6b0 0x7fc84019dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc83ffff700 1 -- 172.21.15.22:0/1065790424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300095f0 con 0x7fc8400731b0 2023-09-20T21:47:53.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc83f7fe700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc840073ad0 0x7fc840199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:47:53.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.745+0000 7fc844990700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84010d6b0 0x7fc84019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:47:53.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.746+0000 7fc83ffff700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8400731b0 0x7fc840199130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc830000c00 tx=0x7fc8300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:53.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.746+0000 7fc83d7fa700 1 -- 172.21.15.22:0/1065790424 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83001d070 con 0x7fc8400731b0 2023-09-20T21:47:53.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.746+0000 7fc8463f3700 1 -- 172.21.15.22:0/1065790424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84019e6a0 con 0x7fc8400731b0 2023-09-20T21:47:53.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.746+0000 7fc83d7fa700 1 -- 172.21.15.22:0/1065790424 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc830022470 con 0x7fc8400731b0 2023-09-20T21:47:53.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.746+0000 7fc83d7fa700 1 -- 172.21.15.22:0/1065790424 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830017650 con 0x7fc8400731b0 2023-09-20T21:47:53.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.746+0000 7fc8463f3700 1 -- 172.21.15.22:0/1065790424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8401a5f60 con 0x7fc8400731b0 2023-09-20T21:47:53.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.747+0000 7fc8463f3700 1 -- 172.21.15.22:0/1065790424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84004eae0 con 0x7fc8400731b0 2023-09-20T21:47:53.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.749+0000 7fc83d7fa700 1 -- 172.21.15.22:0/1065790424 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fc8300225e0 con 0x7fc8400731b0 2023-09-20T21:47:53.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.751+0000 7fc83d7fa700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc8280a1300 0x7fc8280a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:53.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.751+0000 7fc83d7fa700 1 -- 172.21.15.22:0/1065790424 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc8300ce480 con 0x7fc8400731b0 2023-09-20T21:47:53.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.751+0000 7fc83f7fe700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc8280a1300 0x7fc8280a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:53.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.752+0000 7fc83f7fe700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc8280a1300 0x7fc8280a37c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc84019a750 tx=0x7fc834009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:53.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.755+0000 7fc83d7fa700 1 -- 172.21.15.22:0/1065790424 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fc830097880 con 0x7fc8400731b0 2023-09-20T21:47:53.928 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:53 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/823120859' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-09-20T21:47:53.928 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:53 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2844430146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-09-20T21:47:53.928 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:53 smithi022 ceph-mon[97651]: pgmap v314: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:47:53.929 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:53 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/728952762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-09-20T21:47:53.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.976+0000 7fc8463f3700 1 -- 172.21.15.22:0/1065790424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fc840113020 con 0x7fc8400731b0 2023-09-20T21:47:53.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.976+0000 7fc83d7fa700 1 -- 172.21.15.22:0/1065790424 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc830027090 con 0x7fc8400731b0 2023-09-20T21:47:53.977 INFO:teuthology.orchestra.run.smithi022.stdout:309237645342 2023-09-20T21:47:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 -- 172.21.15.22:0/1065790424 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc8280a1300 msgr2=0x7fc8280a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:53.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc8280a1300 0x7fc8280a37c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc84019a750 tx=0x7fc834009260 comp rx=0 tx=0).stop 2023-09-20T21:47:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 -- 172.21.15.22:0/1065790424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8400731b0 msgr2=0x7fc840199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8400731b0 0x7fc840199130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc830000c00 tx=0x7fc8300057f0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 -- 172.21.15.22:0/1065790424 shutdown_connections 2023-09-20T21:47:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc8280a1300 0x7fc8280a37c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc84010d6b0 0x7fc84019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8400731b0 0x7fc840199130 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 --2- 172.21.15.22:0/1065790424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc840073ad0 0x7fc840199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 -- 172.21.15.22:0/1065790424 >> 172.21.15.22:0/1065790424 conn(0x7fc8400fc900 msgr2=0x7fc84010f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 -- 172.21.15.22:0/1065790424 shutdown_connections 2023-09-20T21:47:53.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:53.979+0000 7fc8463f3700 1 -- 172.21.15.22:0/1065790424 wait complete. 2023-09-20T21:47:54.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.095+0000 7f2406c9f700 1 -- 172.21.15.22:0/2609268568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24000694f0 msgr2=0x7f24000698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.095+0000 7f2406c9f700 1 --2- 172.21.15.22:0/2609268568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24000694f0 0x7f24000698d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f23f40099c0 tx=0x7f23f4009cd0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.095+0000 7f2406c9f700 1 -- 172.21.15.22:0/2609268568 shutdown_connections 2023-09-20T21:47:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.095+0000 7f2406c9f700 1 --2- 172.21.15.22:0/2609268568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24001031b0 0x7f240010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.095+0000 7f2406c9f700 1 --2- 172.21.15.22:0/2609268568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24000694f0 0x7f24000698d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.095+0000 7f2406c9f700 1 --2- 172.21.15.22:0/2609268568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2400069e10 0x7f2400102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.095+0000 7f2406c9f700 1 -- 172.21.15.22:0/2609268568 >> 172.21.15.22:0/2609268568 conn(0x7f2400076dd0 msgr2=0x7f24000771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:54.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.095+0000 7f2406c9f700 1 -- 172.21.15.22:0/2609268568 shutdown_connections 2023-09-20T21:47:54.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.095+0000 7f2406c9f700 1 -- 172.21.15.22:0/2609268568 wait complete. 2023-09-20T21:47:54.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.096+0000 7f2406c9f700 1 Processor -- start 2023-09-20T21:47:54.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.096+0000 7f2406c9f700 1 -- start start 2023-09-20T21:47:54.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f2406c9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24000694f0 0x7f2400197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:54.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f2406c9f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2400069e10 0x7f2400197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:54.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f2406c9f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24001031b0 0x7f240019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:54.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f2406c9f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240019c170 con 0x7f24001031b0 2023-09-20T21:47:54.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f2406c9f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240019c2e0 con 0x7f24000694f0 2023-09-20T21:47:54.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f2406c9f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240019c4a0 con 0x7f2400069e10 2023-09-20T21:47:54.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f240523c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24001031b0 0x7f240019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:54.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f2404a3b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24000694f0 0x7f2400197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:54.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f2404a3b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24000694f0 0x7f2400197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:34436/0 (socket says 172.21.15.22:34436) 2023-09-20T21:47:54.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f240523c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24001031b0 0x7f240019bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37356/0 (socket says 172.21.15.22:37356) 2023-09-20T21:47:54.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f2404a3b700 1 -- 172.21.15.22:0/989903522 learned_addr learned my addr 172.21.15.22:0/989903522 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:54.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.097+0000 7f23fffff700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2400069e10 0x7f2400197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:54.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f240523c700 1 -- 172.21.15.22:0/989903522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2400069e10 msgr2=0x7f2400197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:54.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f240523c700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2400069e10 0x7f2400197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f240523c700 1 -- 172.21.15.22:0/989903522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24000694f0 msgr2=0x7f2400197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:54.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f240523c700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24000694f0 0x7f2400197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f240523c700 1 -- 172.21.15.22:0/989903522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f8009c40 con 0x7f24001031b0 2023-09-20T21:47:54.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f240523c700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24001031b0 0x7f240019bb60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f23f800efb0 tx=0x7f23f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:54.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f23fdffb700 1 -- 172.21.15.22:0/989903522 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8003a00 con 0x7f24001031b0 2023-09-20T21:47:54.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f23fdffb700 1 -- 172.21.15.22:0/989903522 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f800f040 con 0x7f24001031b0 2023-09-20T21:47:54.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f2406c9f700 1 -- 172.21.15.22:0/989903522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f40095f0 con 0x7f24001031b0 2023-09-20T21:47:54.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f23fdffb700 1 -- 172.21.15.22:0/989903522 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8004540 con 0x7f24001031b0 2023-09-20T21:47:54.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.098+0000 7f2406c9f700 1 -- 172.21.15.22:0/989903522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24001a3e10 con 0x7f24001031b0 2023-09-20T21:47:54.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.099+0000 7f2406c9f700 1 -- 172.21.15.22:0/989903522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240004eae0 con 0x7f24001031b0 2023-09-20T21:47:54.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.101+0000 7f23fdffb700 1 -- 172.21.15.22:0/989903522 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f23f8003bf0 con 0x7f24001031b0 2023-09-20T21:47:54.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.103+0000 7f23fdffb700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f23f00a1300 0x7f23f00a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:54.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.103+0000 7f23fdffb700 1 -- 172.21.15.22:0/989903522 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f23f80ccb10 con 0x7f24001031b0 2023-09-20T21:47:54.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.103+0000 7f2404a3b700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f23f00a1300 0x7f23f00a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:54.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.103+0000 7f2404a3b700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f23f00a1300 0x7f23f00a37c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23f40036c0 tx=0x7f23f4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:54.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.106+0000 7f23fdffb700 1 -- 172.21.15.22:0/989903522 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f23f8095f10 con 0x7f24001031b0 2023-09-20T21:47:54.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182462 for osd.3 2023-09-20T21:47:54.143 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.330+0000 7f2406c9f700 1 -- 172.21.15.22:0/989903522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2400110ef0 con 0x7f24001031b0 2023-09-20T21:47:54.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.331+0000 7f23fdffb700 1 -- 172.21.15.22:0/989903522 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f23f8099530 con 0x7f24001031b0 2023-09-20T21:47:54.332 INFO:teuthology.orchestra.run.smithi022.stdout:219043332141 2023-09-20T21:47:54.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 -- 172.21.15.22:0/989903522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f23f00a1300 msgr2=0x7f23f00a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:54.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f23f00a1300 0x7f23f00a37c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23f40036c0 tx=0x7f23f4005fb0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 -- 172.21.15.22:0/989903522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24001031b0 msgr2=0x7f240019bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24001031b0 0x7f240019bb60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f23f800efb0 tx=0x7f23f800c3a0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 -- 172.21.15.22:0/989903522 shutdown_connections 2023-09-20T21:47:54.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24001031b0 0x7f240019bb60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f23f00a1300 0x7f23f00a37c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24000694f0 0x7f2400197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 --2- 172.21.15.22:0/989903522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2400069e10 0x7f2400197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:54.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 -- 172.21.15.22:0/989903522 >> 172.21.15.22:0/989903522 conn(0x7f2400076dd0 msgr2=0x7f24000fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:54.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 -- 172.21.15.22:0/989903522 shutdown_connections 2023-09-20T21:47:54.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:54.333+0000 7f2406c9f700 1 -- 172.21.15.22:0/989903522 wait complete. 2023-09-20T21:47:54.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808864 got 287762808866 for osd.9 2023-09-20T21:47:54.538 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:54.619 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:54 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1065790424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-20T21:47:54.619 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:54 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/989903522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-20T21:47:54.738 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757303 for osd.5 2023-09-20T21:47:54.739 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:54.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:54 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1065790424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-20T21:47:54.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:54 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/989903522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-20T21:47:54.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:54 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1065790424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-09-20T21:47:54.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:54 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/989903522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-09-20T21:47:55.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.025+0000 7fef1e122700 1 -- 172.21.15.22:0/2741113812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef18101060 msgr2=0x7fef18101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.025+0000 7fef1e122700 1 --2- 172.21.15.22:0/2741113812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef18101060 0x7fef18101440 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef08009990 tx=0x7fef08009ca0 comp rx=0 tx=0).stop 2023-09-20T21:47:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.025+0000 7fef1e122700 1 -- 172.21.15.22:0/2741113812 shutdown_connections 2023-09-20T21:47:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.025+0000 7fef1e122700 1 --2- 172.21.15.22:0/2741113812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef18106140 0x7fef1810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:55.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.025+0000 7fef1e122700 1 --2- 172.21.15.22:0/2741113812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef18101060 0x7fef18101440 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.025+0000 7fef1e122700 1 --2- 172.21.15.22:0/2741113812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18101a10 0x7fef18105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.025+0000 7fef1e122700 1 -- 172.21.15.22:0/2741113812 >> 172.21.15.22:0/2741113812 conn(0x7fef180fc8d0 msgr2=0x7fef180fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.025+0000 7fef1e122700 1 -- 172.21.15.22:0/2741113812 shutdown_connections 2023-09-20T21:47:55.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.025+0000 7fef1e122700 1 -- 172.21.15.22:0/2741113812 wait complete. 2023-09-20T21:47:55.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.026+0000 7fef1e122700 1 Processor -- start 2023-09-20T21:47:55.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.026+0000 7fef1e122700 1 -- start start 2023-09-20T21:47:55.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.026+0000 7fef1e122700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18101060 0x7fef18196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:55.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.026+0000 7fef1e122700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef18101a10 0x7fef18197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:55.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.026+0000 7fef1e122700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef18106140 0x7fef1819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:55.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef1e122700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1819bed0 con 0x7fef18101a10 2023-09-20T21:47:55.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef1e122700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1819c040 con 0x7fef18106140 2023-09-20T21:47:55.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef1e122700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1819c200 con 0x7fef18101060 2023-09-20T21:47:55.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef16ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef18101a10 0x7fef18197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:55.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef177fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18101060 0x7fef18196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:55.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef16ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef18101a10 0x7fef18197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37368/0 (socket says 172.21.15.22:37368) 2023-09-20T21:47:55.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef16ffd700 1 -- 172.21.15.22:0/1241900742 learned_addr learned my addr 172.21.15.22:0/1241900742 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:55.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef17fff700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef18106140 0x7fef1819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:55.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef177fe700 1 -- 172.21.15.22:0/1241900742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef18106140 msgr2=0x7fef1819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:55.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef177fe700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef18106140 0x7fef1819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:55.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef177fe700 1 -- 172.21.15.22:0/1241900742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef18101a10 msgr2=0x7fef18197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:55.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef177fe700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef18101a10 0x7fef18197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:55.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef177fe700 1 -- 172.21.15.22:0/1241900742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef080095f0 con 0x7fef18101060 2023-09-20T21:47:55.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.027+0000 7fef17fff700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef18106140 0x7fef1819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:55.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.028+0000 7fef177fe700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18101060 0x7fef18196f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef08000c00 tx=0x7fef08005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:55.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.028+0000 7fef14ff9700 1 -- 172.21.15.22:0/1241900742 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0801d070 con 0x7fef18101060 2023-09-20T21:47:55.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.028+0000 7fef1e122700 1 -- 172.21.15.22:0/1241900742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1819c480 con 0x7fef18101060 2023-09-20T21:47:55.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.028+0000 7fef14ff9700 1 -- 172.21.15.22:0/1241900742 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef08021410 con 0x7fef18101060 2023-09-20T21:47:55.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.028+0000 7fef14ff9700 1 -- 172.21.15.22:0/1241900742 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08017650 con 0x7fef18101060 2023-09-20T21:47:55.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.028+0000 7fef1e122700 1 -- 172.21.15.22:0/1241900742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef181a3d40 con 0x7fef18101060 2023-09-20T21:47:55.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.029+0000 7fef1e122700 1 -- 172.21.15.22:0/1241900742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1804eae0 con 0x7fef18101060 2023-09-20T21:47:55.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.031+0000 7fef14ff9700 1 -- 172.21.15.22:0/1241900742 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fef080217f0 con 0x7fef18101060 2023-09-20T21:47:55.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.033+0000 7fef14ff9700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef000a1300 0x7fef000a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:55.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.033+0000 7fef14ff9700 1 -- 172.21.15.22:0/1241900742 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fef080ce390 con 0x7fef18101060 2023-09-20T21:47:55.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.033+0000 7fef16ffd700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef000a1300 0x7fef000a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:55.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.033+0000 7fef16ffd700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef000a1300 0x7fef000a37c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef18198530 tx=0x7fef0c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:55.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.035+0000 7fef14ff9700 1 -- 172.21.15.22:0/1241900742 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fef080977f0 con 0x7fef18101060 2023-09-20T21:47:55.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.259+0000 7fef1e122700 1 -- 172.21.15.22:0/1241900742 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fef18110da0 con 0x7fef18101060 2023-09-20T21:47:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.260+0000 7fef14ff9700 1 -- 172.21.15.22:0/1241900742 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef08026020 con 0x7fef18101060 2023-09-20T21:47:55.261 INFO:teuthology.orchestra.run.smithi022.stdout:55834574920 2023-09-20T21:47:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 -- 172.21.15.22:0/1241900742 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef000a1300 msgr2=0x7fef000a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef000a1300 0x7fef000a37c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef18198530 tx=0x7fef0c009260 comp rx=0 tx=0).stop 2023-09-20T21:47:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 -- 172.21.15.22:0/1241900742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18101060 msgr2=0x7fef18196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18101060 0x7fef18196f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef08000c00 tx=0x7fef08005740 comp rx=0 tx=0).stop 2023-09-20T21:47:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 -- 172.21.15.22:0/1241900742 shutdown_connections 2023-09-20T21:47:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef000a1300 0x7fef000a37c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef18106140 0x7fef1819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18101060 0x7fef18196f10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 --2- 172.21.15.22:0/1241900742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef18101a10 0x7fef18197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 -- 172.21.15.22:0/1241900742 >> 172.21.15.22:0/1241900742 conn(0x7fef180fc8d0 msgr2=0x7fef180fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 -- 172.21.15.22:0/1241900742 shutdown_connections 2023-09-20T21:47:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:55.262+0000 7fef1e122700 1 -- 172.21.15.22:0/1241900742 wait complete. 2023-09-20T21:47:55.791 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332138 got 219043332141 for osd.7 2023-09-20T21:47:55.792 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:55.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:55 smithi123 ceph-mon[134433]: pgmap v315: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-09-20T21:47:55.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:55 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1241900742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-20T21:47:55.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:55 smithi089 ceph-mon[134090]: pgmap v315: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-09-20T21:47:55.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:55 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1241900742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-20T21:47:55.888 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:55 smithi022 ceph-mon[97651]: pgmap v315: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-09-20T21:47:55.889 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:55 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1241900742' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-09-20T21:47:55.891 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645339 got 309237645342 for osd.10 2023-09-20T21:47:55.891 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:56.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574920 for osd.1 2023-09-20T21:47:56.143 DEBUG:teuthology.parallel:result is None 2023-09-20T21:47:56.144 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-09-20T21:47:56.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph pg dump --format=json 2023-09-20T21:47:56.901 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:47:58.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:58 smithi022 ceph-mon[97651]: pgmap v316: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:58.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:58.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:58 smithi123 ceph-mon[134433]: pgmap v316: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:58 smithi089 ceph-mon[134090]: pgmap v316: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:47:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:58.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.658+0000 7f20d3dce700 1 -- 172.21.15.22:0/1132283932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20cc106140 msgr2=0x7f20cc10dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:58.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.658+0000 7f20d3dce700 1 --2- 172.21.15.22:0/1132283932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20cc106140 0x7f20cc10dd10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b0a0 tx=0x7f20c400b3b0 comp rx=0 tx=0).stop 2023-09-20T21:47:58.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.658+0000 7f20d3dce700 1 -- 172.21.15.22:0/1132283932 shutdown_connections 2023-09-20T21:47:58.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.658+0000 7f20d3dce700 1 --2- 172.21.15.22:0/1132283932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20cc106140 0x7f20cc10dd10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:58.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.658+0000 7f20d3dce700 1 --2- 172.21.15.22:0/1132283932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc101060 0x7f20cc101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:58.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.658+0000 7f20d3dce700 1 --2- 172.21.15.22:0/1132283932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc101a10 0x7f20cc105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:58.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.658+0000 7f20d3dce700 1 -- 172.21.15.22:0/1132283932 >> 172.21.15.22:0/1132283932 conn(0x7f20cc0fc8d0 msgr2=0x7f20cc0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:58.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.658+0000 7f20d3dce700 1 -- 172.21.15.22:0/1132283932 shutdown_connections 2023-09-20T21:47:58.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.659+0000 7f20d3dce700 1 -- 172.21.15.22:0/1132283932 wait complete. 2023-09-20T21:47:58.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.660+0000 7f20d3dce700 1 Processor -- start 2023-09-20T21:47:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.660+0000 7f20d3dce700 1 -- start start 2023-09-20T21:47:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.660+0000 7f20d3dce700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc101060 0x7f20cc1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.660+0000 7f20d3dce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc101a10 0x7f20cc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d3dce700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20cc106140 0x7f20cc19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:58.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d3dce700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc19e180 con 0x7f20cc106140 2023-09-20T21:47:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d3dce700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc19e2f0 con 0x7f20cc101060 2023-09-20T21:47:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d3dce700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc19e4b0 con 0x7f20cc101a10 2023-09-20T21:47:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d236b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20cc106140 0x7f20cc19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:58.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d1b6a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc101060 0x7f20cc1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:58.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d236b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20cc106140 0x7f20cc19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39884/0 (socket says 172.21.15.22:39884) 2023-09-20T21:47:58.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d1369700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc101a10 0x7f20cc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:58.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d236b700 1 -- 172.21.15.22:0/3648716185 learned_addr learned my addr 172.21.15.22:0/3648716185 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:47:58.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d236b700 1 -- 172.21.15.22:0/3648716185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc101a10 msgr2=0x7f20cc199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:58.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d236b700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc101a10 0x7f20cc199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:58.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d236b700 1 -- 172.21.15.22:0/3648716185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc101060 msgr2=0x7f20cc1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:58.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d236b700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc101060 0x7f20cc1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:58.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d236b700 1 -- 172.21.15.22:0/3648716185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c8009520 con 0x7f20cc106140 2023-09-20T21:47:58.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.661+0000 7f20d1b6a700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc101060 0x7f20cc1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:47:58.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.662+0000 7f20d236b700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20cc106140 0x7f20cc19db70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b780 tx=0x7f20c4007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:58.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.662+0000 7f20c2ffd700 1 -- 172.21.15.22:0/3648716185 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c400e040 con 0x7f20cc106140 2023-09-20T21:47:58.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.662+0000 7f20d3dce700 1 -- 172.21.15.22:0/3648716185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c4009d00 con 0x7f20cc106140 2023-09-20T21:47:58.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.662+0000 7f20c2ffd700 1 -- 172.21.15.22:0/3648716185 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20c401d470 con 0x7f20cc106140 2023-09-20T21:47:58.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.662+0000 7f20c2ffd700 1 -- 172.21.15.22:0/3648716185 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40052a0 con 0x7f20cc106140 2023-09-20T21:47:58.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.662+0000 7f20d3dce700 1 -- 172.21.15.22:0/3648716185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20cc19e9c0 con 0x7f20cc106140 2023-09-20T21:47:58.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.663+0000 7f20d3dce700 1 -- 172.21.15.22:0/3648716185 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20cc04eae0 con 0x7f20cc106140 2023-09-20T21:47:58.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.665+0000 7f20c2ffd700 1 -- 172.21.15.22:0/3648716185 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f20c401d5e0 con 0x7f20cc106140 2023-09-20T21:47:58.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.668+0000 7f20c2ffd700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f20b80a1300 0x7f20b80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:47:58.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.668+0000 7f20c2ffd700 1 -- 172.21.15.22:0/3648716185 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f20c40ce4a0 con 0x7f20cc106140 2023-09-20T21:47:58.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.668+0000 7f20d1b6a700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f20b80a1300 0x7f20b80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:47:58.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.669+0000 7f20d1b6a700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f20b80a1300 0x7f20b80a37c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009fd0 tx=0x7f20c8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:47:58.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.671+0000 7f20c2ffd700 1 -- 172.21.15.22:0/3648716185 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f20c40978a0 con 0x7f20cc106140 2023-09-20T21:47:58.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.906+0000 7f20d3dce700 1 -- 172.21.15.22:0/3648716185 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20cc110fc0 con 0x7f20b80a1300 2023-09-20T21:47:58.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.920+0000 7f20c2ffd700 1 -- 172.21.15.22:0/3648716185 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426371 (secure 0 0 0) 0x7f20cc110fc0 con 0x7f20b80a1300 2023-09-20T21:47:58.929 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:47:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 -- 172.21.15.22:0/3648716185 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f20b80a1300 msgr2=0x7f20b80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:58.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f20b80a1300 0x7f20b80a37c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009fd0 tx=0x7f20c8009260 comp rx=0 tx=0).stop 2023-09-20T21:47:58.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 -- 172.21.15.22:0/3648716185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20cc106140 msgr2=0x7f20cc19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:47:58.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20cc106140 0x7f20cc19db70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b780 tx=0x7f20c4007990 comp rx=0 tx=0).stop 2023-09-20T21:47:58.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 -- 172.21.15.22:0/3648716185 shutdown_connections 2023-09-20T21:47:58.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20cc106140 0x7f20cc19db70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:58.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f20b80a1300 0x7f20b80a37c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:58.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f20cc101060 0x7f20cc1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:58.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 --2- 172.21.15.22:0/3648716185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20cc101a10 0x7f20cc199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:47:58.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 -- 172.21.15.22:0/3648716185 >> 172.21.15.22:0/3648716185 conn(0x7f20cc0fc8d0 msgr2=0x7f20cc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:47:58.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 -- 172.21.15.22:0/3648716185 shutdown_connections 2023-09-20T21:47:58.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:47:58.928+0000 7f20d3dce700 1 -- 172.21.15.22:0/3648716185 wait complete. 2023-09-20T21:47:58.935 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-09-20T21:47:59.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:59 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:47:59.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:59 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:47:59.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:47:59 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:59.415 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2023-09-20T21:47:57.117300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600632,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":628,"num_read_kb":380,"num_write":390,"num_write_kb":3005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338228,"kb_used_data":13976,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603596,"statfs":{"total":1151940427776,"available":1151594082304,"internally_reserved":0,"allocated":14311424,"data_stored":7748275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003747"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598937+0000","last_change":"2023-09-20T21:46:41.506092+0000","last_active":"2023-09-20T21:47:07.598937+0000","last_peered":"2023-09-20T21:47:07.598937+0000","last_clean":"2023-09-20T21:47:07.598937+0000","last_became_active":"2023-09-20T21:46:41.505779+0000","last_became_peered":"2023-09-20T21:46:41.505779+0000","last_unstale":"2023-09-20T21:47:07.598937+0000","last_undegraded":"2023-09-20T21:47:07.598937+0000","last_fullsized":"2023-09-20T21:47:07.598937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:25.284068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599485+0000","last_change":"2023-09-20T21:46:41.505000+0000","last_active":"2023-09-20T21:47:07.599485+0000","last_peered":"2023-09-20T21:47:07.599485+0000","last_clean":"2023-09-20T21:47:07.599485+0000","last_became_active":"2023-09-20T21:46:41.504464+0000","last_became_peered":"2023-09-20T21:46:41.504464+0000","last_unstale":"2023-09-20T21:47:07.599485+0000","last_undegraded":"2023-09-20T21:47:07.599485+0000","last_fullsized":"2023-09-20T21:47:07.599485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:42:44.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599342+0000","last_change":"2023-09-20T21:46:41.513134+0000","last_active":"2023-09-20T21:47:07.599342+0000","last_peered":"2023-09-20T21:47:07.599342+0000","last_clean":"2023-09-20T21:47:07.599342+0000","last_became_active":"2023-09-20T21:46:41.512793+0000","last_became_peered":"2023-09-20T21:46:41.512793+0000","last_unstale":"2023-09-20T21:47:07.599342+0000","last_undegraded":"2023-09-20T21:47:07.599342+0000","last_fullsized":"2023-09-20T21:47:07.599342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:59.978630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597341+0000","last_change":"2023-09-20T21:46:41.500200+0000","last_active":"2023-09-20T21:47:07.597341+0000","last_peered":"2023-09-20T21:47:07.597341+0000","last_clean":"2023-09-20T21:47:07.597341+0000","last_became_active":"2023-09-20T21:46:41.499866+0000","last_became_peered":"2023-09-20T21:46:41.499866+0000","last_unstale":"2023-09-20T21:47:07.597341+0000","last_undegraded":"2023-09-20T21:47:07.597341+0000","last_fullsized":"2023-09-20T21:47:07.597341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:20:52.989008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594880+0000","last_change":"2023-09-20T21:46:41.507584+0000","last_active":"2023-09-20T21:47:07.594880+0000","last_peered":"2023-09-20T21:47:07.594880+0000","last_clean":"2023-09-20T21:47:07.594880+0000","last_became_active":"2023-09-20T21:46:41.507298+0000","last_became_peered":"2023-09-20T21:46:41.507298+0000","last_unstale":"2023-09-20T21:47:07.594880+0000","last_undegraded":"2023-09-20T21:47:07.594880+0000","last_fullsized":"2023-09-20T21:47:07.594880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:32:14.042414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595455+0000","last_change":"2023-09-20T21:46:41.497002+0000","last_active":"2023-09-20T21:47:07.595455+0000","last_peered":"2023-09-20T21:47:07.595455+0000","last_clean":"2023-09-20T21:47:07.595455+0000","last_became_active":"2023-09-20T21:46:41.496485+0000","last_became_peered":"2023-09-20T21:46:41.496485+0000","last_unstale":"2023-09-20T21:47:07.595455+0000","last_undegraded":"2023-09-20T21:47:07.595455+0000","last_fullsized":"2023-09-20T21:47:07.595455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:48:24.481575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598725+0000","last_change":"2023-09-20T21:46:41.503303+0000","last_active":"2023-09-20T21:47:07.598725+0000","last_peered":"2023-09-20T21:47:07.598725+0000","last_clean":"2023-09-20T21:47:07.598725+0000","last_became_active":"2023-09-20T21:46:41.502982+0000","last_became_peered":"2023-09-20T21:46:41.502982+0000","last_unstale":"2023-09-20T21:47:07.598725+0000","last_undegraded":"2023-09-20T21:47:07.598725+0000","last_fullsized":"2023-09-20T21:47:07.598725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:23:27.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595388+0000","last_change":"2023-09-20T21:46:41.508243+0000","last_active":"2023-09-20T21:47:07.595388+0000","last_peered":"2023-09-20T21:47:07.595388+0000","last_clean":"2023-09-20T21:47:07.595388+0000","last_became_active":"2023-09-20T21:46:41.506407+0000","last_became_peered":"2023-09-20T21:46:41.506407+0000","last_unstale":"2023-09-20T21:47:07.595388+0000","last_undegraded":"2023-09-20T21:47:07.595388+0000","last_fullsized":"2023-09-20T21:47:07.595388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:39.388716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594461+0000","last_change":"2023-09-20T21:46:41.501513+0000","last_active":"2023-09-20T21:47:07.594461+0000","last_peered":"2023-09-20T21:47:07.594461+0000","last_clean":"2023-09-20T21:47:07.594461+0000","last_became_active":"2023-09-20T21:46:41.500051+0000","last_became_peered":"2023-09-20T21:46:41.500051+0000","last_unstale":"2023-09-20T21:47:07.594461+0000","last_undegraded":"2023-09-20T21:47:07.594461+0000","last_fullsized":"2023-09-20T21:47:07.594461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:40.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596488+0000","last_change":"2023-09-20T21:46:41.501045+0000","last_active":"2023-09-20T21:47:07.596488+0000","last_peered":"2023-09-20T21:47:07.596488+0000","last_clean":"2023-09-20T21:47:07.596488+0000","last_became_active":"2023-09-20T21:46:41.500744+0000","last_became_peered":"2023-09-20T21:46:41.500744+0000","last_unstale":"2023-09-20T21:47:07.596488+0000","last_undegraded":"2023-09-20T21:47:07.596488+0000","last_fullsized":"2023-09-20T21:47:07.596488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:57:29.120472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599791+0000","last_change":"2023-09-20T21:46:41.510235+0000","last_active":"2023-09-20T21:47:07.599791+0000","last_peered":"2023-09-20T21:47:07.599791+0000","last_clean":"2023-09-20T21:47:07.599791+0000","last_became_active":"2023-09-20T21:46:41.509943+0000","last_became_peered":"2023-09-20T21:46:41.509943+0000","last_unstale":"2023-09-20T21:47:07.599791+0000","last_undegraded":"2023-09-20T21:47:07.599791+0000","last_fullsized":"2023-09-20T21:47:07.599791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:09:19.383505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597247+0000","last_change":"2023-09-20T21:46:41.501194+0000","last_active":"2023-09-20T21:47:07.597247+0000","last_peered":"2023-09-20T21:47:07.597247+0000","last_clean":"2023-09-20T21:47:07.597247+0000","last_became_active":"2023-09-20T21:46:41.500095+0000","last_became_peered":"2023-09-20T21:46:41.500095+0000","last_unstale":"2023-09-20T21:47:07.597247+0000","last_undegraded":"2023-09-20T21:47:07.597247+0000","last_fullsized":"2023-09-20T21:47:07.597247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:46:51.343332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602350+0000","last_change":"2023-09-20T21:46:41.507562+0000","last_active":"2023-09-20T21:47:07.602350+0000","last_peered":"2023-09-20T21:47:07.602350+0000","last_clean":"2023-09-20T21:47:07.602350+0000","last_became_active":"2023-09-20T21:46:41.507094+0000","last_became_peered":"2023-09-20T21:46:41.507094+0000","last_unstale":"2023-09-20T21:47:07.602350+0000","last_undegraded":"2023-09-20T21:47:07.602350+0000","last_fullsized":"2023-09-20T21:47:07.602350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:34:32.401415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598795+0000","last_change":"2023-09-20T21:46:41.502864+0000","last_active":"2023-09-20T21:47:07.598795+0000","last_peered":"2023-09-20T21:47:07.598795+0000","last_clean":"2023-09-20T21:47:07.598795+0000","last_became_active":"2023-09-20T21:46:41.502543+0000","last_became_peered":"2023-09-20T21:46:41.502543+0000","last_unstale":"2023-09-20T21:47:07.598795+0000","last_undegraded":"2023-09-20T21:47:07.598795+0000","last_fullsized":"2023-09-20T21:47:07.598795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:02:04.631874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.600445+0000","last_change":"2023-09-20T21:47:08.600445+0000","last_active":"2023-09-20T21:47:08.600445+0000","last_peered":"2023-09-20T21:47:08.600445+0000","last_clean":"2023-09-20T21:47:08.600445+0000","last_became_active":"2023-09-20T21:47:08.600054+0000","last_became_peered":"2023-09-20T21:47:08.600054+0000","last_unstale":"2023-09-20T21:47:08.600445+0000","last_undegraded":"2023-09-20T21:47:08.600445+0000","last_fullsized":"2023-09-20T21:47:08.600445+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:43:18.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593608+0000","last_change":"2023-09-20T21:46:41.502036+0000","last_active":"2023-09-20T21:47:07.593608+0000","last_peered":"2023-09-20T21:47:07.593608+0000","last_clean":"2023-09-20T21:47:07.593608+0000","last_became_active":"2023-09-20T21:46:41.501745+0000","last_became_peered":"2023-09-20T21:46:41.501745+0000","last_unstale":"2023-09-20T21:47:07.593608+0000","last_undegraded":"2023-09-20T21:47:07.593608+0000","last_fullsized":"2023-09-20T21:47:07.593608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:08:06.699543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598739+0000","last_change":"2023-09-20T21:46:41.506666+0000","last_active":"2023-09-20T21:47:07.598739+0000","last_peered":"2023-09-20T21:47:07.598739+0000","last_clean":"2023-09-20T21:47:07.598739+0000","last_became_active":"2023-09-20T21:46:41.506266+0000","last_became_peered":"2023-09-20T21:46:41.506266+0000","last_unstale":"2023-09-20T21:47:07.598739+0000","last_undegraded":"2023-09-20T21:47:07.598739+0000","last_fullsized":"2023-09-20T21:47:07.598739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:06:16.809609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592904+0000","last_change":"2023-09-20T21:46:41.497391+0000","last_active":"2023-09-20T21:47:07.592904+0000","last_peered":"2023-09-20T21:47:07.592904+0000","last_clean":"2023-09-20T21:47:07.592904+0000","last_became_active":"2023-09-20T21:46:41.497064+0000","last_became_peered":"2023-09-20T21:46:41.497064+0000","last_unstale":"2023-09-20T21:47:07.592904+0000","last_undegraded":"2023-09-20T21:47:07.592904+0000","last_fullsized":"2023-09-20T21:47:07.592904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:56:11.489585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599538+0000","last_change":"2023-09-20T21:46:41.510602+0000","last_active":"2023-09-20T21:47:07.599538+0000","last_peered":"2023-09-20T21:47:07.599538+0000","last_clean":"2023-09-20T21:47:07.599538+0000","last_became_active":"2023-09-20T21:46:41.510308+0000","last_became_peered":"2023-09-20T21:46:41.510308+0000","last_unstale":"2023-09-20T21:47:07.599538+0000","last_undegraded":"2023-09-20T21:47:07.599538+0000","last_fullsized":"2023-09-20T21:47:07.599538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:43:41.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598788+0000","last_change":"2023-09-20T21:46:45.581583+0000","last_active":"2023-09-20T21:47:07.598788+0000","last_peered":"2023-09-20T21:47:07.598788+0000","last_clean":"2023-09-20T21:47:07.598788+0000","last_became_active":"2023-09-20T21:46:45.581165+0000","last_became_peered":"2023-09-20T21:46:45.581165+0000","last_unstale":"2023-09-20T21:47:07.598788+0000","last_undegraded":"2023-09-20T21:47:07.598788+0000","last_fullsized":"2023-09-20T21:47:07.598788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:39:36.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:52.686590+0000","last_change":"2023-09-20T21:46:40.500010+0000","last_active":"2023-09-20T21:47:52.686590+0000","last_peered":"2023-09-20T21:47:52.686590+0000","last_clean":"2023-09-20T21:47:52.686590+0000","last_became_active":"2023-09-20T21:46:40.499632+0000","last_became_peered":"2023-09-20T21:46:40.499632+0000","last_unstale":"2023-09-20T21:47:52.686590+0000","last_undegraded":"2023-09-20T21:47:52.686590+0000","last_fullsized":"2023-09-20T21:47:52.686590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:41:35.590982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595846+0000","last_change":"2023-09-20T21:46:41.516465+0000","last_active":"2023-09-20T21:47:07.595846+0000","last_peered":"2023-09-20T21:47:07.595846+0000","last_clean":"2023-09-20T21:47:07.595846+0000","last_became_active":"2023-09-20T21:46:41.516150+0000","last_became_peered":"2023-09-20T21:46:41.516150+0000","last_unstale":"2023-09-20T21:47:07.595846+0000","last_undegraded":"2023-09-20T21:47:07.595846+0000","last_fullsized":"2023-09-20T21:47:07.595846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:33:37.507049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596541+0000","last_change":"2023-09-20T21:46:45.582916+0000","last_active":"2023-09-20T21:47:07.596541+0000","last_peered":"2023-09-20T21:47:07.596541+0000","last_clean":"2023-09-20T21:47:07.596541+0000","last_became_active":"2023-09-20T21:46:45.582511+0000","last_became_peered":"2023-09-20T21:46:45.582511+0000","last_unstale":"2023-09-20T21:47:07.596541+0000","last_undegraded":"2023-09-20T21:47:07.596541+0000","last_fullsized":"2023-09-20T21:47:07.596541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:43:47.006768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596015+0000","last_change":"2023-09-20T21:46:40.516867+0000","last_active":"2023-09-20T21:47:07.596015+0000","last_peered":"2023-09-20T21:47:07.596015+0000","last_clean":"2023-09-20T21:47:07.596015+0000","last_became_active":"2023-09-20T21:46:40.516343+0000","last_became_peered":"2023-09-20T21:46:40.516343+0000","last_unstale":"2023-09-20T21:47:07.596015+0000","last_undegraded":"2023-09-20T21:47:07.596015+0000","last_fullsized":"2023-09-20T21:47:07.596015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:17:18.348921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599149+0000","last_change":"2023-09-20T21:46:41.507093+0000","last_active":"2023-09-20T21:47:07.599149+0000","last_peered":"2023-09-20T21:47:07.599149+0000","last_clean":"2023-09-20T21:47:07.599149+0000","last_became_active":"2023-09-20T21:46:41.506587+0000","last_became_peered":"2023-09-20T21:46:41.506587+0000","last_unstale":"2023-09-20T21:47:07.599149+0000","last_undegraded":"2023-09-20T21:47:07.599149+0000","last_fullsized":"2023-09-20T21:47:07.599149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:26:44.825499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597188+0000","last_change":"2023-09-20T21:46:45.583942+0000","last_active":"2023-09-20T21:47:07.597188+0000","last_peered":"2023-09-20T21:47:07.597188+0000","last_clean":"2023-09-20T21:47:07.597188+0000","last_became_active":"2023-09-20T21:46:45.583484+0000","last_became_peered":"2023-09-20T21:46:45.583484+0000","last_unstale":"2023-09-20T21:47:07.597188+0000","last_undegraded":"2023-09-20T21:47:07.597188+0000","last_fullsized":"2023-09-20T21:47:07.597188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:48.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:09.612757+0000","last_change":"2023-09-20T21:47:08.615780+0000","last_active":"2023-09-20T21:47:09.612757+0000","last_peered":"2023-09-20T21:47:09.612757+0000","last_clean":"2023-09-20T21:47:09.612757+0000","last_became_active":"2023-09-20T21:47:08.603974+0000","last_became_peered":"2023-09-20T21:47:08.603974+0000","last_unstale":"2023-09-20T21:47:09.612757+0000","last_undegraded":"2023-09-20T21:47:09.612757+0000","last_fullsized":"2023-09-20T21:47:09.612757+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:53:23.172401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594055+0000","last_change":"2023-09-20T21:46:41.504449+0000","last_active":"2023-09-20T21:47:07.594055+0000","last_peered":"2023-09-20T21:47:07.594055+0000","last_clean":"2023-09-20T21:47:07.594055+0000","last_became_active":"2023-09-20T21:46:41.504169+0000","last_became_peered":"2023-09-20T21:46:41.504169+0000","last_unstale":"2023-09-20T21:47:07.594055+0000","last_undegraded":"2023-09-20T21:47:07.594055+0000","last_fullsized":"2023-09-20T21:47:07.594055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:36:43.980902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596618+0000","last_change":"2023-09-20T21:46:45.582269+0000","last_active":"2023-09-20T21:47:07.596618+0000","last_peered":"2023-09-20T21:47:07.596618+0000","last_clean":"2023-09-20T21:47:07.596618+0000","last_became_active":"2023-09-20T21:46:45.581782+0000","last_became_peered":"2023-09-20T21:46:45.581782+0000","last_unstale":"2023-09-20T21:47:07.596618+0000","last_undegraded":"2023-09-20T21:47:07.596618+0000","last_fullsized":"2023-09-20T21:47:07.596618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:32:09.518715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.598645+0000","last_change":"2023-09-20T21:46:40.504099+0000","last_active":"2023-09-20T21:47:08.598645+0000","last_peered":"2023-09-20T21:47:08.598645+0000","last_clean":"2023-09-20T21:47:08.598645+0000","last_became_active":"2023-09-20T21:46:40.503774+0000","last_became_peered":"2023-09-20T21:46:40.503774+0000","last_unstale":"2023-09-20T21:47:08.598645+0000","last_undegraded":"2023-09-20T21:47:08.598645+0000","last_fullsized":"2023-09-20T21:47:08.598645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:17.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595338+0000","last_change":"2023-09-20T21:46:41.499802+0000","last_active":"2023-09-20T21:47:07.595338+0000","last_peered":"2023-09-20T21:47:07.595338+0000","last_clean":"2023-09-20T21:47:07.595338+0000","last_became_active":"2023-09-20T21:46:41.499491+0000","last_became_peered":"2023-09-20T21:46:41.499491+0000","last_unstale":"2023-09-20T21:47:07.595338+0000","last_undegraded":"2023-09-20T21:47:07.595338+0000","last_fullsized":"2023-09-20T21:47:07.595338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:44.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599395+0000","last_change":"2023-09-20T21:46:45.578753+0000","last_active":"2023-09-20T21:47:07.599395+0000","last_peered":"2023-09-20T21:47:07.599395+0000","last_clean":"2023-09-20T21:47:07.599395+0000","last_became_active":"2023-09-20T21:46:45.578297+0000","last_became_peered":"2023-09-20T21:46:45.578297+0000","last_unstale":"2023-09-20T21:47:07.599395+0000","last_undegraded":"2023-09-20T21:47:07.599395+0000","last_fullsized":"2023-09-20T21:47:07.599395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:18:22.596550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:53.638656+0000","last_change":"2023-09-20T21:46:40.505675+0000","last_active":"2023-09-20T21:47:53.638656+0000","last_peered":"2023-09-20T21:47:53.638656+0000","last_clean":"2023-09-20T21:47:53.638656+0000","last_became_active":"2023-09-20T21:46:40.505432+0000","last_became_peered":"2023-09-20T21:46:40.505432+0000","last_unstale":"2023-09-20T21:47:53.638656+0000","last_undegraded":"2023-09-20T21:47:53.638656+0000","last_fullsized":"2023-09-20T21:47:53.638656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:14:53.652904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595519+0000","last_change":"2023-09-20T21:46:41.505924+0000","last_active":"2023-09-20T21:47:07.595519+0000","last_peered":"2023-09-20T21:47:07.595519+0000","last_clean":"2023-09-20T21:47:07.595519+0000","last_became_active":"2023-09-20T21:46:41.505609+0000","last_became_peered":"2023-09-20T21:46:41.505609+0000","last_unstale":"2023-09-20T21:47:07.595519+0000","last_undegraded":"2023-09-20T21:47:07.595519+0000","last_fullsized":"2023-09-20T21:47:07.595519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:13:46.506786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600999+0000","last_change":"2023-09-20T21:46:45.587818+0000","last_active":"2023-09-20T21:47:07.600999+0000","last_peered":"2023-09-20T21:47:07.600999+0000","last_clean":"2023-09-20T21:47:07.600999+0000","last_became_active":"2023-09-20T21:46:45.587376+0000","last_became_peered":"2023-09-20T21:46:45.587376+0000","last_unstale":"2023-09-20T21:47:07.600999+0000","last_undegraded":"2023-09-20T21:47:07.600999+0000","last_fullsized":"2023-09-20T21:47:07.600999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:28:53.605158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595338+0000","last_change":"2023-09-20T21:46:40.505610+0000","last_active":"2023-09-20T21:47:07.595338+0000","last_peered":"2023-09-20T21:47:07.595338+0000","last_clean":"2023-09-20T21:47:07.595338+0000","last_became_active":"2023-09-20T21:46:40.505279+0000","last_became_peered":"2023-09-20T21:46:40.505279+0000","last_unstale":"2023-09-20T21:47:07.595338+0000","last_undegraded":"2023-09-20T21:47:07.595338+0000","last_fullsized":"2023-09-20T21:47:07.595338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:41:19.575154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.909510+0000","last_change":"2023-09-20T21:47:08.908513+0000","last_active":"2023-09-20T21:47:08.909510+0000","last_peered":"2023-09-20T21:47:08.909510+0000","last_clean":"2023-09-20T21:47:08.909510+0000","last_became_active":"2023-09-20T21:47:08.908142+0000","last_became_peered":"2023-09-20T21:47:08.908142+0000","last_unstale":"2023-09-20T21:47:08.909510+0000","last_undegraded":"2023-09-20T21:47:08.909510+0000","last_fullsized":"2023-09-20T21:47:08.909510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:31:33.992202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598682+0000","last_change":"2023-09-20T21:46:45.581738+0000","last_active":"2023-09-20T21:47:07.598682+0000","last_peered":"2023-09-20T21:47:07.598682+0000","last_clean":"2023-09-20T21:47:07.598682+0000","last_became_active":"2023-09-20T21:46:45.581279+0000","last_became_peered":"2023-09-20T21:46:45.581279+0000","last_unstale":"2023-09-20T21:47:07.598682+0000","last_undegraded":"2023-09-20T21:47:07.598682+0000","last_fullsized":"2023-09-20T21:47:07.598682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:41:41.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577335+0000","last_change":"2023-09-20T21:46:40.510539+0000","last_active":"2023-09-20T21:47:08.577335+0000","last_peered":"2023-09-20T21:47:08.577335+0000","last_clean":"2023-09-20T21:47:08.577335+0000","last_became_active":"2023-09-20T21:46:40.509880+0000","last_became_peered":"2023-09-20T21:46:40.509880+0000","last_unstale":"2023-09-20T21:47:08.577335+0000","last_undegraded":"2023-09-20T21:47:08.577335+0000","last_fullsized":"2023-09-20T21:47:08.577335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:52.877786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595248+0000","last_change":"2023-09-20T21:46:41.505105+0000","last_active":"2023-09-20T21:47:07.595248+0000","last_peered":"2023-09-20T21:47:07.595248+0000","last_clean":"2023-09-20T21:47:07.595248+0000","last_became_active":"2023-09-20T21:46:41.504785+0000","last_became_peered":"2023-09-20T21:46:41.504785+0000","last_unstale":"2023-09-20T21:47:07.595248+0000","last_undegraded":"2023-09-20T21:47:07.595248+0000","last_fullsized":"2023-09-20T21:47:07.595248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:07:57.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599298+0000","last_change":"2023-09-20T21:46:45.583160+0000","last_active":"2023-09-20T21:47:07.599298+0000","last_peered":"2023-09-20T21:47:07.599298+0000","last_clean":"2023-09-20T21:47:07.599298+0000","last_became_active":"2023-09-20T21:46:45.582709+0000","last_became_peered":"2023-09-20T21:46:45.582709+0000","last_unstale":"2023-09-20T21:47:07.599298+0000","last_undegraded":"2023-09-20T21:47:07.599298+0000","last_fullsized":"2023-09-20T21:47:07.599298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:24:57.563682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.906996+0000","last_change":"2023-09-20T21:46:40.502591+0000","last_active":"2023-09-20T21:47:08.906996+0000","last_peered":"2023-09-20T21:47:08.906996+0000","last_clean":"2023-09-20T21:47:08.906996+0000","last_became_active":"2023-09-20T21:46:40.502288+0000","last_became_peered":"2023-09-20T21:46:40.502288+0000","last_unstale":"2023-09-20T21:47:08.906996+0000","last_undegraded":"2023-09-20T21:47:08.906996+0000","last_fullsized":"2023-09-20T21:47:08.906996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:09:37.983866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.599912+0000","last_change":"2023-09-20T21:47:08.599912+0000","last_active":"2023-09-20T21:47:08.599912+0000","last_peered":"2023-09-20T21:47:08.599912+0000","last_clean":"2023-09-20T21:47:08.599912+0000","last_became_active":"2023-09-20T21:47:08.599434+0000","last_became_peered":"2023-09-20T21:47:08.599434+0000","last_unstale":"2023-09-20T21:47:08.599912+0000","last_undegraded":"2023-09-20T21:47:08.599912+0000","last_fullsized":"2023-09-20T21:47:08.599912+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:37:01.125731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595936+0000","last_change":"2023-09-20T21:46:45.580917+0000","last_active":"2023-09-20T21:47:07.595936+0000","last_peered":"2023-09-20T21:47:07.595936+0000","last_clean":"2023-09-20T21:47:07.595936+0000","last_became_active":"2023-09-20T21:46:45.580505+0000","last_became_peered":"2023-09-20T21:46:45.580505+0000","last_unstale":"2023-09-20T21:47:07.595936+0000","last_undegraded":"2023-09-20T21:47:07.595936+0000","last_fullsized":"2023-09-20T21:47:07.595936+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:52:19.733919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:55.641481+0000","last_change":"2023-09-20T21:46:40.507040+0000","last_active":"2023-09-20T21:47:55.641481+0000","last_peered":"2023-09-20T21:47:55.641481+0000","last_clean":"2023-09-20T21:47:55.641481+0000","last_became_active":"2023-09-20T21:46:40.506712+0000","last_became_peered":"2023-09-20T21:46:40.506712+0000","last_unstale":"2023-09-20T21:47:55.641481+0000","last_undegraded":"2023-09-20T21:47:55.641481+0000","last_fullsized":"2023-09-20T21:47:55.641481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:02:56.040911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594395+0000","last_change":"2023-09-20T21:46:41.504794+0000","last_active":"2023-09-20T21:47:07.594395+0000","last_peered":"2023-09-20T21:47:07.594395+0000","last_clean":"2023-09-20T21:47:07.594395+0000","last_became_active":"2023-09-20T21:46:41.504372+0000","last_became_peered":"2023-09-20T21:46:41.504372+0000","last_unstale":"2023-09-20T21:47:07.594395+0000","last_undegraded":"2023-09-20T21:47:07.594395+0000","last_fullsized":"2023-09-20T21:47:07.594395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:42:20.527306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592403+0000","last_change":"2023-09-20T21:46:45.586364+0000","last_active":"2023-09-20T21:47:07.592403+0000","last_peered":"2023-09-20T21:47:07.592403+0000","last_clean":"2023-09-20T21:47:07.592403+0000","last_became_active":"2023-09-20T21:46:45.584620+0000","last_became_peered":"2023-09-20T21:46:45.584620+0000","last_unstale":"2023-09-20T21:47:07.592403+0000","last_undegraded":"2023-09-20T21:47:07.592403+0000","last_fullsized":"2023-09-20T21:47:07.592403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:05:20.158118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577704+0000","last_change":"2023-09-20T21:46:40.500797+0000","last_active":"2023-09-20T21:47:08.577704+0000","last_peered":"2023-09-20T21:47:08.577704+0000","last_clean":"2023-09-20T21:47:08.577704+0000","last_became_active":"2023-09-20T21:46:40.500337+0000","last_became_peered":"2023-09-20T21:46:40.500337+0000","last_unstale":"2023-09-20T21:47:08.577704+0000","last_undegraded":"2023-09-20T21:47:08.577704+0000","last_fullsized":"2023-09-20T21:47:08.577704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:19.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.908615+0000","last_change":"2023-09-20T21:47:08.908615+0000","last_active":"2023-09-20T21:47:08.908615+0000","last_peered":"2023-09-20T21:47:08.908615+0000","last_clean":"2023-09-20T21:47:08.908615+0000","last_became_active":"2023-09-20T21:47:08.908228+0000","last_became_peered":"2023-09-20T21:47:08.908228+0000","last_unstale":"2023-09-20T21:47:08.908615+0000","last_undegraded":"2023-09-20T21:47:08.908615+0000","last_fullsized":"2023-09-20T21:47:08.908615+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:15:28.772129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597442+0000","last_change":"2023-09-20T21:46:45.583081+0000","last_active":"2023-09-20T21:47:07.597442+0000","last_peered":"2023-09-20T21:47:07.597442+0000","last_clean":"2023-09-20T21:47:07.597442+0000","last_became_active":"2023-09-20T21:46:45.580496+0000","last_became_peered":"2023-09-20T21:46:45.580496+0000","last_unstale":"2023-09-20T21:47:07.597442+0000","last_undegraded":"2023-09-20T21:47:07.597442+0000","last_fullsized":"2023-09-20T21:47:07.597442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:42.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595250+0000","last_change":"2023-09-20T21:46:40.512739+0000","last_active":"2023-09-20T21:47:07.595250+0000","last_peered":"2023-09-20T21:47:07.595250+0000","last_clean":"2023-09-20T21:47:07.595250+0000","last_became_active":"2023-09-20T21:46:40.512485+0000","last_became_peered":"2023-09-20T21:46:40.512485+0000","last_unstale":"2023-09-20T21:47:07.595250+0000","last_undegraded":"2023-09-20T21:47:07.595250+0000","last_fullsized":"2023-09-20T21:47:07.595250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:02:49.392560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.604682+0000","last_change":"2023-09-20T21:47:08.604682+0000","last_active":"2023-09-20T21:47:08.604682+0000","last_peered":"2023-09-20T21:47:08.604682+0000","last_clean":"2023-09-20T21:47:08.604682+0000","last_became_active":"2023-09-20T21:47:08.604271+0000","last_became_peered":"2023-09-20T21:47:08.604271+0000","last_unstale":"2023-09-20T21:47:08.604682+0000","last_undegraded":"2023-09-20T21:47:08.604682+0000","last_fullsized":"2023-09-20T21:47:08.604682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:40:51.542781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592702+0000","last_change":"2023-09-20T21:46:45.579478+0000","last_active":"2023-09-20T21:47:07.592702+0000","last_peered":"2023-09-20T21:47:07.592702+0000","last_clean":"2023-09-20T21:47:07.592702+0000","last_became_active":"2023-09-20T21:46:45.579055+0000","last_became_peered":"2023-09-20T21:46:45.579055+0000","last_unstale":"2023-09-20T21:47:07.592702+0000","last_undegraded":"2023-09-20T21:47:07.592702+0000","last_fullsized":"2023-09-20T21:47:07.592702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:17:32.013280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'14","reported_seq":132,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:55.642560+0000","last_change":"2023-09-20T21:46:40.507321+0000","last_active":"2023-09-20T21:47:55.642560+0000","last_peered":"2023-09-20T21:47:55.642560+0000","last_clean":"2023-09-20T21:47:55.642560+0000","last_became_active":"2023-09-20T21:46:40.506983+0000","last_became_peered":"2023-09-20T21:46:40.506983+0000","last_unstale":"2023-09-20T21:47:55.642560+0000","last_undegraded":"2023-09-20T21:47:55.642560+0000","last_fullsized":"2023-09-20T21:47:55.642560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:46:40.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":100,"num_read_kb":54,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596051+0000","last_change":"2023-09-20T21:46:41.509646+0000","last_active":"2023-09-20T21:47:07.596051+0000","last_peered":"2023-09-20T21:47:07.596051+0000","last_clean":"2023-09-20T21:47:07.596051+0000","last_became_active":"2023-09-20T21:46:41.509183+0000","last_became_peered":"2023-09-20T21:46:41.509183+0000","last_unstale":"2023-09-20T21:47:07.596051+0000","last_undegraded":"2023-09-20T21:47:07.596051+0000","last_fullsized":"2023-09-20T21:47:07.596051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:17:21.084830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595627+0000","last_change":"2023-09-20T21:46:45.583709+0000","last_active":"2023-09-20T21:47:07.595627+0000","last_peered":"2023-09-20T21:47:07.595627+0000","last_clean":"2023-09-20T21:47:07.595627+0000","last_became_active":"2023-09-20T21:46:45.583302+0000","last_became_peered":"2023-09-20T21:46:45.583302+0000","last_unstale":"2023-09-20T21:47:07.595627+0000","last_undegraded":"2023-09-20T21:47:07.595627+0000","last_fullsized":"2023-09-20T21:47:07.595627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:31:35.472750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.594685+0000","last_change":"2023-09-20T21:46:40.506411+0000","last_active":"2023-09-20T21:47:08.594685+0000","last_peered":"2023-09-20T21:47:08.594685+0000","last_clean":"2023-09-20T21:47:08.594685+0000","last_became_active":"2023-09-20T21:46:40.506116+0000","last_became_peered":"2023-09-20T21:46:40.506116+0000","last_unstale":"2023-09-20T21:47:08.594685+0000","last_undegraded":"2023-09-20T21:47:08.594685+0000","last_fullsized":"2023-09-20T21:47:08.594685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:23:19.175007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602083+0000","last_change":"2023-09-20T21:46:41.513793+0000","last_active":"2023-09-20T21:47:07.602083+0000","last_peered":"2023-09-20T21:47:07.602083+0000","last_clean":"2023-09-20T21:47:07.602083+0000","last_became_active":"2023-09-20T21:46:41.513434+0000","last_became_peered":"2023-09-20T21:46:41.513434+0000","last_unstale":"2023-09-20T21:47:07.602083+0000","last_undegraded":"2023-09-20T21:47:07.602083+0000","last_fullsized":"2023-09-20T21:47:07.602083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:00:57.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598091+0000","last_change":"2023-09-20T21:46:45.578188+0000","last_active":"2023-09-20T21:47:07.598091+0000","last_peered":"2023-09-20T21:47:07.598091+0000","last_clean":"2023-09-20T21:47:07.598091+0000","last_became_active":"2023-09-20T21:46:45.577548+0000","last_became_peered":"2023-09-20T21:46:45.577548+0000","last_unstale":"2023-09-20T21:47:07.598091+0000","last_undegraded":"2023-09-20T21:47:07.598091+0000","last_fullsized":"2023-09-20T21:47:07.598091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:54:58.894269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.907407+0000","last_change":"2023-09-20T21:46:40.505465+0000","last_active":"2023-09-20T21:47:08.907407+0000","last_peered":"2023-09-20T21:47:08.907407+0000","last_clean":"2023-09-20T21:47:08.907407+0000","last_became_active":"2023-09-20T21:46:40.505174+0000","last_became_peered":"2023-09-20T21:46:40.505174+0000","last_unstale":"2023-09-20T21:47:08.907407+0000","last_undegraded":"2023-09-20T21:47:08.907407+0000","last_fullsized":"2023-09-20T21:47:08.907407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:50:53.191963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598850+0000","last_change":"2023-09-20T21:46:41.507370+0000","last_active":"2023-09-20T21:47:07.598850+0000","last_peered":"2023-09-20T21:47:07.598850+0000","last_clean":"2023-09-20T21:47:07.598850+0000","last_became_active":"2023-09-20T21:46:41.507035+0000","last_became_peered":"2023-09-20T21:46:41.507035+0000","last_unstale":"2023-09-20T21:47:07.598850+0000","last_undegraded":"2023-09-20T21:47:07.598850+0000","last_fullsized":"2023-09-20T21:47:07.598850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:50.298899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592822+0000","last_change":"2023-09-20T21:46:45.584458+0000","last_active":"2023-09-20T21:47:07.592822+0000","last_peered":"2023-09-20T21:47:07.592822+0000","last_clean":"2023-09-20T21:47:07.592822+0000","last_became_active":"2023-09-20T21:46:45.584033+0000","last_became_peered":"2023-09-20T21:46:45.584033+0000","last_unstale":"2023-09-20T21:47:07.592822+0000","last_undegraded":"2023-09-20T21:47:07.592822+0000","last_fullsized":"2023-09-20T21:47:07.592822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:48:44.653880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:52.640190+0000","last_change":"2023-09-20T21:46:40.501329+0000","last_active":"2023-09-20T21:47:52.640190+0000","last_peered":"2023-09-20T21:47:52.640190+0000","last_clean":"2023-09-20T21:47:52.640190+0000","last_became_active":"2023-09-20T21:46:40.500999+0000","last_became_peered":"2023-09-20T21:46:40.500999+0000","last_unstale":"2023-09-20T21:47:52.640190+0000","last_undegraded":"2023-09-20T21:47:52.640190+0000","last_fullsized":"2023-09-20T21:47:52.640190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:38.028398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.602123+0000","last_change":"2023-09-20T21:47:08.602123+0000","last_active":"2023-09-20T21:47:08.602123+0000","last_peered":"2023-09-20T21:47:08.602123+0000","last_clean":"2023-09-20T21:47:08.602123+0000","last_became_active":"2023-09-20T21:47:08.601738+0000","last_became_peered":"2023-09-20T21:47:08.601738+0000","last_unstale":"2023-09-20T21:47:08.602123+0000","last_undegraded":"2023-09-20T21:47:08.602123+0000","last_fullsized":"2023-09-20T21:47:08.602123+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:26:36.710230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602591+0000","last_change":"2023-09-20T21:46:45.587116+0000","last_active":"2023-09-20T21:47:07.602591+0000","last_peered":"2023-09-20T21:47:07.602591+0000","last_clean":"2023-09-20T21:47:07.602591+0000","last_became_active":"2023-09-20T21:46:45.586629+0000","last_became_peered":"2023-09-20T21:46:45.586629+0000","last_unstale":"2023-09-20T21:47:07.602591+0000","last_undegraded":"2023-09-20T21:47:07.602591+0000","last_fullsized":"2023-09-20T21:47:07.602591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:10:33.024235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577220+0000","last_change":"2023-09-20T21:46:40.499985+0000","last_active":"2023-09-20T21:47:08.577220+0000","last_peered":"2023-09-20T21:47:08.577220+0000","last_clean":"2023-09-20T21:47:08.577220+0000","last_became_active":"2023-09-20T21:46:40.499646+0000","last_became_peered":"2023-09-20T21:46:40.499646+0000","last_unstale":"2023-09-20T21:47:08.577220+0000","last_undegraded":"2023-09-20T21:47:08.577220+0000","last_fullsized":"2023-09-20T21:47:08.577220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:52:02.758556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594909+0000","last_change":"2023-09-20T21:46:41.501241+0000","last_active":"2023-09-20T21:47:07.594909+0000","last_peered":"2023-09-20T21:47:07.594909+0000","last_clean":"2023-09-20T21:47:07.594909+0000","last_became_active":"2023-09-20T21:46:41.500902+0000","last_became_peered":"2023-09-20T21:46:41.500902+0000","last_unstale":"2023-09-20T21:47:07.594909+0000","last_undegraded":"2023-09-20T21:47:07.594909+0000","last_fullsized":"2023-09-20T21:47:07.594909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:21:55.210065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594812+0000","last_change":"2023-09-20T21:46:45.583405+0000","last_active":"2023-09-20T21:47:07.594812+0000","last_peered":"2023-09-20T21:47:07.594812+0000","last_clean":"2023-09-20T21:47:07.594812+0000","last_became_active":"2023-09-20T21:46:45.582945+0000","last_became_peered":"2023-09-20T21:46:45.582945+0000","last_unstale":"2023-09-20T21:47:07.594812+0000","last_undegraded":"2023-09-20T21:47:07.594812+0000","last_fullsized":"2023-09-20T21:47:07.594812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:27:14.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.598738+0000","last_change":"2023-09-20T21:46:40.507744+0000","last_active":"2023-09-20T21:47:08.598738+0000","last_peered":"2023-09-20T21:47:08.598738+0000","last_clean":"2023-09-20T21:47:08.598738+0000","last_became_active":"2023-09-20T21:46:40.507454+0000","last_became_peered":"2023-09-20T21:46:40.507454+0000","last_unstale":"2023-09-20T21:47:08.598738+0000","last_undegraded":"2023-09-20T21:47:08.598738+0000","last_fullsized":"2023-09-20T21:47:08.598738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:00:24.492789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598829+0000","last_change":"2023-09-20T21:46:41.504263+0000","last_active":"2023-09-20T21:47:07.598829+0000","last_peered":"2023-09-20T21:47:07.598829+0000","last_clean":"2023-09-20T21:47:07.598829+0000","last_became_active":"2023-09-20T21:46:41.503961+0000","last_became_peered":"2023-09-20T21:46:41.503961+0000","last_unstale":"2023-09-20T21:47:07.598829+0000","last_undegraded":"2023-09-20T21:47:07.598829+0000","last_fullsized":"2023-09-20T21:47:07.598829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:31:58.035517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595426+0000","last_change":"2023-09-20T21:46:45.581343+0000","last_active":"2023-09-20T21:47:07.595426+0000","last_peered":"2023-09-20T21:47:07.595426+0000","last_clean":"2023-09-20T21:47:07.595426+0000","last_became_active":"2023-09-20T21:46:45.580830+0000","last_became_peered":"2023-09-20T21:46:45.580830+0000","last_unstale":"2023-09-20T21:47:07.595426+0000","last_undegraded":"2023-09-20T21:47:07.595426+0000","last_fullsized":"2023-09-20T21:47:07.595426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:20:49.865825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592677+0000","last_change":"2023-09-20T21:46:40.501850+0000","last_active":"2023-09-20T21:47:07.592677+0000","last_peered":"2023-09-20T21:47:07.592677+0000","last_clean":"2023-09-20T21:47:07.592677+0000","last_became_active":"2023-09-20T21:46:40.501500+0000","last_became_peered":"2023-09-20T21:46:40.501500+0000","last_unstale":"2023-09-20T21:47:07.592677+0000","last_undegraded":"2023-09-20T21:47:07.592677+0000","last_fullsized":"2023-09-20T21:47:07.592677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:14:15.907335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595648+0000","last_change":"2023-09-20T21:46:41.507432+0000","last_active":"2023-09-20T21:47:07.595648+0000","last_peered":"2023-09-20T21:47:07.595648+0000","last_clean":"2023-09-20T21:47:07.595648+0000","last_became_active":"2023-09-20T21:46:41.507119+0000","last_became_peered":"2023-09-20T21:46:41.507119+0000","last_unstale":"2023-09-20T21:47:07.595648+0000","last_undegraded":"2023-09-20T21:47:07.595648+0000","last_fullsized":"2023-09-20T21:47:07.595648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:14:10.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602550+0000","last_change":"2023-09-20T21:46:45.588455+0000","last_active":"2023-09-20T21:47:07.602550+0000","last_peered":"2023-09-20T21:47:07.602550+0000","last_clean":"2023-09-20T21:47:07.602550+0000","last_became_active":"2023-09-20T21:46:45.587980+0000","last_became_peered":"2023-09-20T21:46:45.587980+0000","last_unstale":"2023-09-20T21:47:07.602550+0000","last_undegraded":"2023-09-20T21:47:07.602550+0000","last_fullsized":"2023-09-20T21:47:07.602550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:07:35.697330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:50.684479+0000","last_change":"2023-09-20T21:46:40.506260+0000","last_active":"2023-09-20T21:47:50.684479+0000","last_peered":"2023-09-20T21:47:50.684479+0000","last_clean":"2023-09-20T21:47:50.684479+0000","last_became_active":"2023-09-20T21:46:40.505999+0000","last_became_peered":"2023-09-20T21:46:40.505999+0000","last_unstale":"2023-09-20T21:47:50.684479+0000","last_undegraded":"2023-09-20T21:47:50.684479+0000","last_fullsized":"2023-09-20T21:47:50.684479+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:39:01.748038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.599481+0000","last_change":"2023-09-20T21:47:08.599481+0000","last_active":"2023-09-20T21:47:08.599481+0000","last_peered":"2023-09-20T21:47:08.599481+0000","last_clean":"2023-09-20T21:47:08.599481+0000","last_became_active":"2023-09-20T21:47:08.599104+0000","last_became_peered":"2023-09-20T21:47:08.599104+0000","last_unstale":"2023-09-20T21:47:08.599481+0000","last_undegraded":"2023-09-20T21:47:08.599481+0000","last_fullsized":"2023-09-20T21:47:08.599481+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:33:56.845446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602710+0000","last_change":"2023-09-20T21:46:45.587410+0000","last_active":"2023-09-20T21:47:07.602710+0000","last_peered":"2023-09-20T21:47:07.602710+0000","last_clean":"2023-09-20T21:47:07.602710+0000","last_became_active":"2023-09-20T21:46:45.586944+0000","last_became_peered":"2023-09-20T21:46:45.586944+0000","last_unstale":"2023-09-20T21:47:07.602710+0000","last_undegraded":"2023-09-20T21:47:07.602710+0000","last_fullsized":"2023-09-20T21:47:07.602710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:57.053765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594876+0000","last_change":"2023-09-20T21:46:41.511526+0000","last_active":"2023-09-20T21:47:07.594876+0000","last_peered":"2023-09-20T21:47:07.594876+0000","last_clean":"2023-09-20T21:47:07.594876+0000","last_became_active":"2023-09-20T21:46:41.511213+0000","last_became_peered":"2023-09-20T21:46:41.511213+0000","last_unstale":"2023-09-20T21:47:07.594876+0000","last_undegraded":"2023-09-20T21:47:07.594876+0000","last_fullsized":"2023-09-20T21:47:07.594876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:29:04.199190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:53.640949+0000","last_change":"2023-09-20T21:46:40.514916+0000","last_active":"2023-09-20T21:47:53.640949+0000","last_peered":"2023-09-20T21:47:53.640949+0000","last_clean":"2023-09-20T21:47:53.640949+0000","last_became_active":"2023-09-20T21:46:40.514553+0000","last_became_peered":"2023-09-20T21:46:40.514553+0000","last_unstale":"2023-09-20T21:47:53.640949+0000","last_undegraded":"2023-09-20T21:47:53.640949+0000","last_fullsized":"2023-09-20T21:47:53.640949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:12:32.481852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599672+0000","last_change":"2023-09-20T21:46:45.587489+0000","last_active":"2023-09-20T21:47:07.599672+0000","last_peered":"2023-09-20T21:47:07.599672+0000","last_clean":"2023-09-20T21:47:07.599672+0000","last_became_active":"2023-09-20T21:46:45.587022+0000","last_became_peered":"2023-09-20T21:46:45.587022+0000","last_unstale":"2023-09-20T21:47:07.599672+0000","last_undegraded":"2023-09-20T21:47:07.599672+0000","last_fullsized":"2023-09-20T21:47:07.599672+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:09:05.870763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595937+0000","last_change":"2023-09-20T21:46:41.508886+0000","last_active":"2023-09-20T21:47:07.595937+0000","last_peered":"2023-09-20T21:47:07.595937+0000","last_clean":"2023-09-20T21:47:07.595937+0000","last_became_active":"2023-09-20T21:46:41.508470+0000","last_became_peered":"2023-09-20T21:46:41.508470+0000","last_unstale":"2023-09-20T21:47:07.595937+0000","last_undegraded":"2023-09-20T21:47:07.595937+0000","last_fullsized":"2023-09-20T21:47:07.595937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:45:04.888300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.595089+0000","last_change":"2023-09-20T21:46:40.503350+0000","last_active":"2023-09-20T21:47:08.595089+0000","last_peered":"2023-09-20T21:47:08.595089+0000","last_clean":"2023-09-20T21:47:08.595089+0000","last_became_active":"2023-09-20T21:46:40.503045+0000","last_became_peered":"2023-09-20T21:46:40.503045+0000","last_unstale":"2023-09-20T21:47:08.595089+0000","last_undegraded":"2023-09-20T21:47:08.595089+0000","last_fullsized":"2023-09-20T21:47:08.595089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:31:09.004342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":315,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599438+0000","last_change":"2023-09-20T21:45:35.180242+0000","last_active":"2023-09-20T21:47:07.599438+0000","last_peered":"2023-09-20T21:47:07.599438+0000","last_clean":"2023-09-20T21:47:07.599438+0000","last_became_active":"2023-09-20T21:45:35.179705+0000","last_became_peered":"2023-09-20T21:45:35.179705+0000","last_unstale":"2023-09-20T21:47:07.599438+0000","last_undegraded":"2023-09-20T21:47:07.599438+0000","last_fullsized":"2023-09-20T21:47:07.599438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_clean_scrub_stamp":"2023-09-20T21:42:25.826683+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:02:03.448188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594840+0000","last_change":"2023-09-20T21:46:45.581396+0000","last_active":"2023-09-20T21:47:07.594840+0000","last_peered":"2023-09-20T21:47:07.594840+0000","last_clean":"2023-09-20T21:47:07.594840+0000","last_became_active":"2023-09-20T21:46:45.581018+0000","last_became_peered":"2023-09-20T21:46:45.581018+0000","last_unstale":"2023-09-20T21:47:07.594840+0000","last_undegraded":"2023-09-20T21:47:07.594840+0000","last_fullsized":"2023-09-20T21:47:07.594840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:16:35.934460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.601645+0000","last_change":"2023-09-20T21:46:41.514392+0000","last_active":"2023-09-20T21:47:07.601645+0000","last_peered":"2023-09-20T21:47:07.601645+0000","last_clean":"2023-09-20T21:47:07.601645+0000","last_became_active":"2023-09-20T21:46:41.514090+0000","last_became_peered":"2023-09-20T21:46:41.514090+0000","last_unstale":"2023-09-20T21:47:07.601645+0000","last_undegraded":"2023-09-20T21:47:07.601645+0000","last_fullsized":"2023-09-20T21:47:07.601645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:05:47.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577137+0000","last_change":"2023-09-20T21:46:40.507346+0000","last_active":"2023-09-20T21:47:08.577137+0000","last_peered":"2023-09-20T21:47:08.577137+0000","last_clean":"2023-09-20T21:47:08.577137+0000","last_became_active":"2023-09-20T21:46:40.506969+0000","last_became_peered":"2023-09-20T21:46:40.506969+0000","last_unstale":"2023-09-20T21:47:08.577137+0000","last_undegraded":"2023-09-20T21:47:08.577137+0000","last_fullsized":"2023-09-20T21:47:08.577137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:01:20.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600062+0000","last_change":"2023-09-20T21:46:45.578033+0000","last_active":"2023-09-20T21:47:07.600062+0000","last_peered":"2023-09-20T21:47:07.600062+0000","last_clean":"2023-09-20T21:47:07.600062+0000","last_became_active":"2023-09-20T21:46:45.577540+0000","last_became_peered":"2023-09-20T21:46:45.577540+0000","last_unstale":"2023-09-20T21:47:07.600062+0000","last_undegraded":"2023-09-20T21:47:07.600062+0000","last_fullsized":"2023-09-20T21:47:07.600062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:24:31.020660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577276+0000","last_change":"2023-09-20T21:46:40.510634+0000","last_active":"2023-09-20T21:47:08.577276+0000","last_peered":"2023-09-20T21:47:08.577276+0000","last_clean":"2023-09-20T21:47:08.577276+0000","last_became_active":"2023-09-20T21:46:40.510390+0000","last_became_peered":"2023-09-20T21:46:40.510390+0000","last_unstale":"2023-09-20T21:47:08.577276+0000","last_undegraded":"2023-09-20T21:47:08.577276+0000","last_fullsized":"2023-09-20T21:47:08.577276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:44:49.272987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602533+0000","last_change":"2023-09-20T21:46:41.502981+0000","last_active":"2023-09-20T21:47:07.602533+0000","last_peered":"2023-09-20T21:47:07.602533+0000","last_clean":"2023-09-20T21:47:07.602533+0000","last_became_active":"2023-09-20T21:46:41.502594+0000","last_became_peered":"2023-09-20T21:46:41.502594+0000","last_unstale":"2023-09-20T21:47:07.602533+0000","last_undegraded":"2023-09-20T21:47:07.602533+0000","last_fullsized":"2023-09-20T21:47:07.602533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:41:15.261478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595222+0000","last_change":"2023-09-20T21:46:45.582903+0000","last_active":"2023-09-20T21:47:07.595222+0000","last_peered":"2023-09-20T21:47:07.595222+0000","last_clean":"2023-09-20T21:47:07.595222+0000","last_became_active":"2023-09-20T21:46:45.582433+0000","last_became_peered":"2023-09-20T21:46:45.582433+0000","last_unstale":"2023-09-20T21:47:07.595222+0000","last_undegraded":"2023-09-20T21:47:07.595222+0000","last_fullsized":"2023-09-20T21:47:07.595222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:38:22.174903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592961+0000","last_change":"2023-09-20T21:46:40.507671+0000","last_active":"2023-09-20T21:47:07.592961+0000","last_peered":"2023-09-20T21:47:07.592961+0000","last_clean":"2023-09-20T21:47:07.592961+0000","last_became_active":"2023-09-20T21:46:40.507288+0000","last_became_peered":"2023-09-20T21:46:40.507288+0000","last_unstale":"2023-09-20T21:47:07.592961+0000","last_undegraded":"2023-09-20T21:47:07.592961+0000","last_fullsized":"2023-09-20T21:47:07.592961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:27:02.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596349+0000","last_change":"2023-09-20T21:46:41.507654+0000","last_active":"2023-09-20T21:47:07.596349+0000","last_peered":"2023-09-20T21:47:07.596349+0000","last_clean":"2023-09-20T21:47:07.596349+0000","last_became_active":"2023-09-20T21:46:41.507362+0000","last_became_peered":"2023-09-20T21:46:41.507362+0000","last_unstale":"2023-09-20T21:47:07.596349+0000","last_undegraded":"2023-09-20T21:47:07.596349+0000","last_fullsized":"2023-09-20T21:47:07.596349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:32:08.973379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598688+0000","last_change":"2023-09-20T21:46:45.578437+0000","last_active":"2023-09-20T21:47:07.598688+0000","last_peered":"2023-09-20T21:47:07.598688+0000","last_clean":"2023-09-20T21:47:07.598688+0000","last_became_active":"2023-09-20T21:46:45.578010+0000","last_became_peered":"2023-09-20T21:46:45.578010+0000","last_unstale":"2023-09-20T21:47:07.598688+0000","last_undegraded":"2023-09-20T21:47:07.598688+0000","last_fullsized":"2023-09-20T21:47:07.598688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:59:05.494861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.595756+0000","last_change":"2023-09-20T21:46:40.508937+0000","last_active":"2023-09-20T21:47:08.595756+0000","last_peered":"2023-09-20T21:47:08.595756+0000","last_clean":"2023-09-20T21:47:08.595756+0000","last_became_active":"2023-09-20T21:46:40.508639+0000","last_became_peered":"2023-09-20T21:46:40.508639+0000","last_unstale":"2023-09-20T21:47:08.595756+0000","last_undegraded":"2023-09-20T21:47:08.595756+0000","last_fullsized":"2023-09-20T21:47:08.595756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:50:48.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599421+0000","last_change":"2023-09-20T21:46:41.510702+0000","last_active":"2023-09-20T21:47:07.599421+0000","last_peered":"2023-09-20T21:47:07.599421+0000","last_clean":"2023-09-20T21:47:07.599421+0000","last_became_active":"2023-09-20T21:46:41.510357+0000","last_became_peered":"2023-09-20T21:46:41.510357+0000","last_unstale":"2023-09-20T21:47:07.599421+0000","last_undegraded":"2023-09-20T21:47:07.599421+0000","last_fullsized":"2023-09-20T21:47:07.599421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:35:44.455981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596780+0000","last_change":"2023-09-20T21:46:45.581110+0000","last_active":"2023-09-20T21:47:07.596780+0000","last_peered":"2023-09-20T21:47:07.596780+0000","last_clean":"2023-09-20T21:47:07.596780+0000","last_became_active":"2023-09-20T21:46:45.580498+0000","last_became_peered":"2023-09-20T21:46:45.580498+0000","last_unstale":"2023-09-20T21:47:07.596780+0000","last_undegraded":"2023-09-20T21:47:07.596780+0000","last_fullsized":"2023-09-20T21:47:07.596780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:19:22.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593487+0000","last_change":"2023-09-20T21:46:40.497017+0000","last_active":"2023-09-20T21:47:07.593487+0000","last_peered":"2023-09-20T21:47:07.593487+0000","last_clean":"2023-09-20T21:47:07.593487+0000","last_became_active":"2023-09-20T21:46:40.496722+0000","last_became_peered":"2023-09-20T21:46:40.496722+0000","last_unstale":"2023-09-20T21:47:07.593487+0000","last_undegraded":"2023-09-20T21:47:07.593487+0000","last_fullsized":"2023-09-20T21:47:07.593487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:12:07.017318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596148+0000","last_change":"2023-09-20T21:46:41.508793+0000","last_active":"2023-09-20T21:47:07.596148+0000","last_peered":"2023-09-20T21:47:07.596148+0000","last_clean":"2023-09-20T21:47:07.596148+0000","last_became_active":"2023-09-20T21:46:41.508453+0000","last_became_peered":"2023-09-20T21:46:41.508453+0000","last_unstale":"2023-09-20T21:47:07.596148+0000","last_undegraded":"2023-09-20T21:47:07.596148+0000","last_fullsized":"2023-09-20T21:47:07.596148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:35:12.711570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592993+0000","last_change":"2023-09-20T21:46:45.583519+0000","last_active":"2023-09-20T21:47:07.592993+0000","last_peered":"2023-09-20T21:47:07.592993+0000","last_clean":"2023-09-20T21:47:07.592993+0000","last_became_active":"2023-09-20T21:46:45.582933+0000","last_became_peered":"2023-09-20T21:46:45.582933+0000","last_unstale":"2023-09-20T21:47:07.592993+0000","last_undegraded":"2023-09-20T21:47:07.592993+0000","last_fullsized":"2023-09-20T21:47:07.592993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:13:20.237865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.578270+0000","last_change":"2023-09-20T21:46:40.507384+0000","last_active":"2023-09-20T21:47:08.578270+0000","last_peered":"2023-09-20T21:47:08.578270+0000","last_clean":"2023-09-20T21:47:08.578270+0000","last_became_active":"2023-09-20T21:46:40.507021+0000","last_became_peered":"2023-09-20T21:46:40.507021+0000","last_unstale":"2023-09-20T21:47:08.578270+0000","last_undegraded":"2023-09-20T21:47:08.578270+0000","last_fullsized":"2023-09-20T21:47:08.578270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:48:04.360664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598093+0000","last_change":"2023-09-20T21:46:41.511537+0000","last_active":"2023-09-20T21:47:07.598093+0000","last_peered":"2023-09-20T21:47:07.598093+0000","last_clean":"2023-09-20T21:47:07.598093+0000","last_became_active":"2023-09-20T21:46:41.511238+0000","last_became_peered":"2023-09-20T21:46:41.511238+0000","last_unstale":"2023-09-20T21:47:07.598093+0000","last_undegraded":"2023-09-20T21:47:07.598093+0000","last_fullsized":"2023-09-20T21:47:07.598093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:01:41.492824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596769+0000","last_change":"2023-09-20T21:46:45.585750+0000","last_active":"2023-09-20T21:47:07.596769+0000","last_peered":"2023-09-20T21:47:07.596769+0000","last_clean":"2023-09-20T21:47:07.596769+0000","last_became_active":"2023-09-20T21:46:45.585213+0000","last_became_peered":"2023-09-20T21:46:45.585213+0000","last_unstale":"2023-09-20T21:47:07.596769+0000","last_undegraded":"2023-09-20T21:47:07.596769+0000","last_fullsized":"2023-09-20T21:47:07.596769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:31:15.650766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.594299+0000","last_change":"2023-09-20T21:46:40.506394+0000","last_active":"2023-09-20T21:47:08.594299+0000","last_peered":"2023-09-20T21:47:08.594299+0000","last_clean":"2023-09-20T21:47:08.594299+0000","last_became_active":"2023-09-20T21:46:40.506085+0000","last_became_peered":"2023-09-20T21:46:40.506085+0000","last_unstale":"2023-09-20T21:47:08.594299+0000","last_undegraded":"2023-09-20T21:47:08.594299+0000","last_fullsized":"2023-09-20T21:47:08.594299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:02.738600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596347+0000","last_change":"2023-09-20T21:46:41.509491+0000","last_active":"2023-09-20T21:47:07.596347+0000","last_peered":"2023-09-20T21:47:07.596347+0000","last_clean":"2023-09-20T21:47:07.596347+0000","last_became_active":"2023-09-20T21:46:41.509039+0000","last_became_peered":"2023-09-20T21:46:41.509039+0000","last_unstale":"2023-09-20T21:47:07.596347+0000","last_undegraded":"2023-09-20T21:47:07.596347+0000","last_fullsized":"2023-09-20T21:47:07.596347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:57:39.003154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597192+0000","last_change":"2023-09-20T21:46:45.581651+0000","last_active":"2023-09-20T21:47:07.597192+0000","last_peered":"2023-09-20T21:47:07.597192+0000","last_clean":"2023-09-20T21:47:07.597192+0000","last_became_active":"2023-09-20T21:46:45.581224+0000","last_became_peered":"2023-09-20T21:46:45.581224+0000","last_unstale":"2023-09-20T21:47:07.597192+0000","last_undegraded":"2023-09-20T21:47:07.597192+0000","last_fullsized":"2023-09-20T21:47:07.597192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:32:17.778301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.610835+0000","last_change":"2023-09-20T21:47:08.610413+0000","last_active":"2023-09-20T21:47:08.610835+0000","last_peered":"2023-09-20T21:47:08.610835+0000","last_clean":"2023-09-20T21:47:08.610835+0000","last_became_active":"2023-09-20T21:47:08.603734+0000","last_became_peered":"2023-09-20T21:47:08.603734+0000","last_unstale":"2023-09-20T21:47:08.610835+0000","last_undegraded":"2023-09-20T21:47:08.610835+0000","last_fullsized":"2023-09-20T21:47:08.610835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:43:42.569371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598787+0000","last_change":"2023-09-20T21:46:41.507218+0000","last_active":"2023-09-20T21:47:07.598787+0000","last_peered":"2023-09-20T21:47:07.598787+0000","last_clean":"2023-09-20T21:47:07.598787+0000","last_became_active":"2023-09-20T21:46:41.506889+0000","last_became_peered":"2023-09-20T21:46:41.506889+0000","last_unstale":"2023-09-20T21:47:07.598787+0000","last_undegraded":"2023-09-20T21:47:07.598787+0000","last_fullsized":"2023-09-20T21:47:07.598787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:08:40.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593769+0000","last_change":"2023-09-20T21:46:45.579269+0000","last_active":"2023-09-20T21:47:07.593769+0000","last_peered":"2023-09-20T21:47:07.593769+0000","last_clean":"2023-09-20T21:47:07.593769+0000","last_became_active":"2023-09-20T21:46:45.578541+0000","last_became_peered":"2023-09-20T21:46:45.578541+0000","last_unstale":"2023-09-20T21:47:07.593769+0000","last_undegraded":"2023-09-20T21:47:07.593769+0000","last_fullsized":"2023-09-20T21:47:07.593769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:07:50.241947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:54.731314+0000","last_change":"2023-09-20T21:46:40.508774+0000","last_active":"2023-09-20T21:47:54.731314+0000","last_peered":"2023-09-20T21:47:54.731314+0000","last_clean":"2023-09-20T21:47:54.731314+0000","last_became_active":"2023-09-20T21:46:40.508480+0000","last_became_peered":"2023-09-20T21:46:40.508480+0000","last_unstale":"2023-09-20T21:47:54.731314+0000","last_undegraded":"2023-09-20T21:47:54.731314+0000","last_fullsized":"2023-09-20T21:47:54.731314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:22.643724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599875+0000","last_change":"2023-09-20T21:46:41.503763+0000","last_active":"2023-09-20T21:47:07.599875+0000","last_peered":"2023-09-20T21:47:07.599875+0000","last_clean":"2023-09-20T21:47:07.599875+0000","last_became_active":"2023-09-20T21:46:41.503360+0000","last_became_peered":"2023-09-20T21:46:41.503360+0000","last_unstale":"2023-09-20T21:47:07.599875+0000","last_undegraded":"2023-09-20T21:47:07.599875+0000","last_fullsized":"2023-09-20T21:47:07.599875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:03:00.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598933+0000","last_change":"2023-09-20T21:46:45.586702+0000","last_active":"2023-09-20T21:47:07.598933+0000","last_peered":"2023-09-20T21:47:07.598933+0000","last_clean":"2023-09-20T21:47:07.598933+0000","last_became_active":"2023-09-20T21:46:45.586177+0000","last_became_peered":"2023-09-20T21:46:45.586177+0000","last_unstale":"2023-09-20T21:47:07.598933+0000","last_undegraded":"2023-09-20T21:47:07.598933+0000","last_fullsized":"2023-09-20T21:47:07.598933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:19:03.883440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.576396+0000","last_change":"2023-09-20T21:46:40.507270+0000","last_active":"2023-09-20T21:47:08.576396+0000","last_peered":"2023-09-20T21:47:08.576396+0000","last_clean":"2023-09-20T21:47:08.576396+0000","last_became_active":"2023-09-20T21:46:40.506933+0000","last_became_peered":"2023-09-20T21:46:40.506933+0000","last_unstale":"2023-09-20T21:47:08.576396+0000","last_undegraded":"2023-09-20T21:47:08.576396+0000","last_fullsized":"2023-09-20T21:47:08.576396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:59.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599660+0000","last_change":"2023-09-20T21:46:41.507868+0000","last_active":"2023-09-20T21:47:07.599660+0000","last_peered":"2023-09-20T21:47:07.599660+0000","last_clean":"2023-09-20T21:47:07.599660+0000","last_became_active":"2023-09-20T21:46:41.507568+0000","last_became_peered":"2023-09-20T21:46:41.507568+0000","last_unstale":"2023-09-20T21:47:07.599660+0000","last_undegraded":"2023-09-20T21:47:07.599660+0000","last_fullsized":"2023-09-20T21:47:07.599660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:44.615620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594494+0000","last_change":"2023-09-20T21:46:45.578726+0000","last_active":"2023-09-20T21:47:07.594494+0000","last_peered":"2023-09-20T21:47:07.594494+0000","last_clean":"2023-09-20T21:47:07.594494+0000","last_became_active":"2023-09-20T21:46:45.578326+0000","last_became_peered":"2023-09-20T21:46:45.578326+0000","last_unstale":"2023-09-20T21:47:07.594494+0000","last_undegraded":"2023-09-20T21:47:07.594494+0000","last_fullsized":"2023-09-20T21:47:07.594494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:21:34.564840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577629+0000","last_change":"2023-09-20T21:46:40.509024+0000","last_active":"2023-09-20T21:47:08.577629+0000","last_peered":"2023-09-20T21:47:08.577629+0000","last_clean":"2023-09-20T21:47:08.577629+0000","last_became_active":"2023-09-20T21:46:40.508659+0000","last_became_peered":"2023-09-20T21:46:40.508659+0000","last_unstale":"2023-09-20T21:47:08.577629+0000","last_undegraded":"2023-09-20T21:47:08.577629+0000","last_fullsized":"2023-09-20T21:47:08.577629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:12:08.302068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599976+0000","last_change":"2023-09-20T21:46:41.510307+0000","last_active":"2023-09-20T21:47:07.599976+0000","last_peered":"2023-09-20T21:47:07.599976+0000","last_clean":"2023-09-20T21:47:07.599976+0000","last_became_active":"2023-09-20T21:46:41.510014+0000","last_became_peered":"2023-09-20T21:46:41.510014+0000","last_unstale":"2023-09-20T21:47:07.599976+0000","last_undegraded":"2023-09-20T21:47:07.599976+0000","last_fullsized":"2023-09-20T21:47:07.599976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:05:55.062515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598156+0000","last_change":"2023-09-20T21:46:41.511525+0000","last_active":"2023-09-20T21:47:07.598156+0000","last_peered":"2023-09-20T21:47:07.598156+0000","last_clean":"2023-09-20T21:47:07.598156+0000","last_became_active":"2023-09-20T21:46:41.511203+0000","last_became_peered":"2023-09-20T21:46:41.511203+0000","last_unstale":"2023-09-20T21:47:07.598156+0000","last_undegraded":"2023-09-20T21:47:07.598156+0000","last_fullsized":"2023-09-20T21:47:07.598156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:42:15.757422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599580+0000","last_change":"2023-09-20T21:46:41.502936+0000","last_active":"2023-09-20T21:47:07.599580+0000","last_peered":"2023-09-20T21:47:07.599580+0000","last_clean":"2023-09-20T21:47:07.599580+0000","last_became_active":"2023-09-20T21:46:41.502617+0000","last_became_peered":"2023-09-20T21:46:41.502617+0000","last_unstale":"2023-09-20T21:47:07.599580+0000","last_undegraded":"2023-09-20T21:47:07.599580+0000","last_fullsized":"2023-09-20T21:47:07.599580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:45:21.197652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599689+0000","last_change":"2023-09-20T21:46:41.510766+0000","last_active":"2023-09-20T21:47:07.599689+0000","last_peered":"2023-09-20T21:47:07.599689+0000","last_clean":"2023-09-20T21:47:07.599689+0000","last_became_active":"2023-09-20T21:46:41.510433+0000","last_became_peered":"2023-09-20T21:46:41.510433+0000","last_unstale":"2023-09-20T21:47:07.599689+0000","last_undegraded":"2023-09-20T21:47:07.599689+0000","last_fullsized":"2023-09-20T21:47:07.599689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:56:12.085926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599196+0000","last_change":"2023-09-20T21:46:41.508846+0000","last_active":"2023-09-20T21:47:07.599196+0000","last_peered":"2023-09-20T21:47:07.599196+0000","last_clean":"2023-09-20T21:47:07.599196+0000","last_became_active":"2023-09-20T21:46:41.508518+0000","last_became_peered":"2023-09-20T21:46:41.508518+0000","last_unstale":"2023-09-20T21:47:07.599196+0000","last_undegraded":"2023-09-20T21:47:07.599196+0000","last_fullsized":"2023-09-20T21:47:07.599196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:01:58.195327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602625+0000","last_change":"2023-09-20T21:46:41.505929+0000","last_active":"2023-09-20T21:47:07.602625+0000","last_peered":"2023-09-20T21:47:07.602625+0000","last_clean":"2023-09-20T21:47:07.602625+0000","last_became_active":"2023-09-20T21:46:41.505542+0000","last_became_peered":"2023-09-20T21:46:41.505542+0000","last_unstale":"2023-09-20T21:47:07.602625+0000","last_undegraded":"2023-09-20T21:47:07.602625+0000","last_fullsized":"2023-09-20T21:47:07.602625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:24:47.520344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598727+0000","last_change":"2023-09-20T21:46:41.507603+0000","last_active":"2023-09-20T21:47:07.598727+0000","last_peered":"2023-09-20T21:47:07.598727+0000","last_clean":"2023-09-20T21:47:07.598727+0000","last_became_active":"2023-09-20T21:46:41.507340+0000","last_became_peered":"2023-09-20T21:46:41.507340+0000","last_unstale":"2023-09-20T21:47:07.598727+0000","last_undegraded":"2023-09-20T21:47:07.598727+0000","last_fullsized":"2023-09-20T21:47:07.598727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:50:19.756491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600180+0000","last_change":"2023-09-20T21:46:41.509625+0000","last_active":"2023-09-20T21:47:07.600180+0000","last_peered":"2023-09-20T21:47:07.600180+0000","last_clean":"2023-09-20T21:47:07.600180+0000","last_became_active":"2023-09-20T21:46:41.509301+0000","last_became_peered":"2023-09-20T21:46:41.509301+0000","last_unstale":"2023-09-20T21:47:07.600180+0000","last_undegraded":"2023-09-20T21:47:07.600180+0000","last_fullsized":"2023-09-20T21:47:07.600180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:15:03.108060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596444+0000","last_change":"2023-09-20T21:46:41.516781+0000","last_active":"2023-09-20T21:47:07.596444+0000","last_peered":"2023-09-20T21:47:07.596444+0000","last_clean":"2023-09-20T21:47:07.596444+0000","last_became_active":"2023-09-20T21:46:41.516496+0000","last_became_peered":"2023-09-20T21:46:41.516496+0000","last_unstale":"2023-09-20T21:47:07.596444+0000","last_undegraded":"2023-09-20T21:47:07.596444+0000","last_fullsized":"2023-09-20T21:47:07.596444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:17:26.492851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594775+0000","last_change":"2023-09-20T21:46:41.499927+0000","last_active":"2023-09-20T21:47:07.594775+0000","last_peered":"2023-09-20T21:47:07.594775+0000","last_clean":"2023-09-20T21:47:07.594775+0000","last_became_active":"2023-09-20T21:46:41.499605+0000","last_became_peered":"2023-09-20T21:46:41.499605+0000","last_unstale":"2023-09-20T21:47:07.594775+0000","last_undegraded":"2023-09-20T21:47:07.594775+0000","last_fullsized":"2023-09-20T21:47:07.594775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:18:16.306349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595650+0000","last_change":"2023-09-20T21:46:41.515842+0000","last_active":"2023-09-20T21:47:07.595650+0000","last_peered":"2023-09-20T21:47:07.595650+0000","last_clean":"2023-09-20T21:47:07.595650+0000","last_became_active":"2023-09-20T21:46:41.515600+0000","last_became_peered":"2023-09-20T21:46:41.515600+0000","last_unstale":"2023-09-20T21:47:07.595650+0000","last_undegraded":"2023-09-20T21:47:07.595650+0000","last_fullsized":"2023-09-20T21:47:07.595650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:50:58.088103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593931+0000","last_change":"2023-09-20T21:46:41.510695+0000","last_active":"2023-09-20T21:47:07.593931+0000","last_peered":"2023-09-20T21:47:07.593931+0000","last_clean":"2023-09-20T21:47:07.593931+0000","last_became_active":"2023-09-20T21:46:41.510345+0000","last_became_peered":"2023-09-20T21:46:41.510345+0000","last_unstale":"2023-09-20T21:47:07.593931+0000","last_undegraded":"2023-09-20T21:47:07.593931+0000","last_fullsized":"2023-09-20T21:47:07.593931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:38:24.201382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602278+0000","last_change":"2023-09-20T21:46:41.507809+0000","last_active":"2023-09-20T21:47:07.602278+0000","last_peered":"2023-09-20T21:47:07.602278+0000","last_clean":"2023-09-20T21:47:07.602278+0000","last_became_active":"2023-09-20T21:46:41.507513+0000","last_became_peered":"2023-09-20T21:46:41.507513+0000","last_unstale":"2023-09-20T21:47:07.602278+0000","last_undegraded":"2023-09-20T21:47:07.602278+0000","last_fullsized":"2023-09-20T21:47:07.602278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:30:43.132728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598414+0000","last_change":"2023-09-20T21:46:41.503590+0000","last_active":"2023-09-20T21:47:07.598414+0000","last_peered":"2023-09-20T21:47:07.598414+0000","last_clean":"2023-09-20T21:47:07.598414+0000","last_became_active":"2023-09-20T21:46:41.503295+0000","last_became_peered":"2023-09-20T21:46:41.503295+0000","last_unstale":"2023-09-20T21:47:07.598414+0000","last_undegraded":"2023-09-20T21:47:07.598414+0000","last_fullsized":"2023-09-20T21:47:07.598414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:28:33.088873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10264,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":502,"num_read_kb":271,"num_write":209,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":30792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":496747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.47999999999999998}]},{"osd":1,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.50700000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.379},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.45600000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.438},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.64100000000000001}]},{"osd":9,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.80300000000000005}]},{"osd":10,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.68700000000000006}]}]},{"osd":10,"up_from":72,"seq":309237645343,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1662976,"data_stored":1091501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":7.9690000000000003,"5min":7.9690000000000003,"15min":7.9690000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.6280000000000001,"5min":5.6280000000000001,"15min":5.6280000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47399999999999998,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.59599999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48099999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.48999999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79200000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67900000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68799999999999994,"15min":0.68700000000000006},"min":{"1min":0.42199999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54500000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.71299999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82899999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.437,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65500000000000003}]},{"osd":5,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67800000000000005,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91800000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.47199999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84399999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72899999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.51700000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84299999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.345,"5min":0.90200000000000002,"15min":0.82799999999999996},"min":{"1min":0.69899999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":8.0440000000000005,"5min":8.0440000000000005,"15min":8.0440000000000005},"last":0.77100000000000002}]},{"osd":7,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.752,"15min":0.755},"min":{"1min":0.58699999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.90300000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.77100000000000002,"15min":0.78100000000000003},"min":{"1min":0.58099999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94499999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80700000000000005}]},{"osd":9,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":0.90400000000000003,"15min":0.86199999999999999},"min":{"1min":0.65300000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":5.6449999999999996,"5min":5.6449999999999996,"15min":5.6449999999999996},"last":0.88},{"interface":"front","average":{"1min":1.123,"5min":0.90800000000000003,"15min":0.872},"min":{"1min":0.63200000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.6669999999999998,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":0.86799999999999999}]},{"osd":11,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.83899999999999997}]}]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":496677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.3600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6799999999999997,"5min":4.6799999999999997,"15min":4.6799999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82499999999999996,"15min":0.83299999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.94599999999999995}]},{"osd":3,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.84199999999999997,"15min":0.85099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.96899999999999997,"15min":0.94099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":6.3449999999999998,"5min":6.3449999999999998,"15min":6.3449999999999998},"last":1},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.875,"15min":0.871},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":4.7279999999999998,"5min":4.7279999999999998,"15min":4.7279999999999998},"last":0.92200000000000004}]},{"osd":5,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.97699999999999998,"15min":0.999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.6469999999999998,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":4.7679999999999998,"5min":4.7679999999999998,"15min":4.7679999999999998},"last":1.048}]},{"osd":6,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97399999999999998,"15min":0.97099999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.806,"5min":4.806,"15min":4.806},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.86299999999999999,"5min":1.0089999999999999,"15min":1.0329999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.67400000000000004}]},{"osd":8,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.103,"15min":1.1319999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.069},{"interface":"front","average":{"1min":1.048,"5min":1.0660000000000001,"15min":1.069},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":4.843,"5min":4.843,"15min":4.843},"last":1.0129999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":1.0660000000000001,"15min":1.095},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.835,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.109},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.113,"15min":1.1399999999999999},"min":{"1min":0.78600000000000003,"5min":0.623,"15min":0.623},"max":{"1min":1.867,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.1399999999999999}]},{"osd":11,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.1759999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070504,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":497055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":0.57999999999999996}]},{"osd":1,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.497,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.61699999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.48099999999999998,"15min":0.46300000000000002},"min":{"1min":0.38300000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.70399999999999996}]},{"osd":3,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.53900000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.90900000000000003}]},{"osd":4,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.67500000000000004,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.99399999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.93400000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.93400000000000005}]},{"osd":6,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.76600000000000001,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.042},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.96699999999999997}]},{"osd":7,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.57699999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0109999999999999,"5min":1.548,"15min":1.548},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.57499999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.0249999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:47:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76100000000000001,"15min":0.75700000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.03,"5min":1.073,"15min":1.073},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78000000000000003,"15min":0.77300000000000002},"min":{"1min":0.621,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.0980000000000001}]},{"osd":10,"last update":"Wed Sep 20 21:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.748,"15min":0.73399999999999999},"min":{"1min":0.55600000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77300000000000002,"15min":0.76000000000000001},"min":{"1min":0.624,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.077}]},{"osd":11,"last update":"Wed Sep 20 21:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.95199999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1638400,"data_stored":1089065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.29599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56499999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.437},"min":{"1min":0.33200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.67900000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.45200000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.747,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58899999999999997,"15min":0.57399999999999995},"min":{"1min":0.49299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61399999999999999,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61799999999999999,"15min":0.59699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80700000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.97499999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.45300000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72299999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77600000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.63300000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.52300000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83099999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.65400000000000003,"15min":0.63100000000000001},"min":{"1min":0.56200000000000006,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91300000000000003,"5min":1.006,"15min":1.006},"last":0.96099999999999997}]},{"osd":6,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.437,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.495,"5min":0.435,"15min":0.435},"max":{"1min":0.85799999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.85399999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.60699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91900000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.68400000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91400000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.88600000000000001}]},{"osd":9,"last update":"Wed Sep 20 21:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.76000000000000001},"min":{"1min":0.59099999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.89900000000000002}]},{"osd":10,"last update":"Wed Sep 20 21:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.628,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.67400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93300000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.82899999999999996}]},{"osd":11,"last update":"Wed Sep 20 21:47:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.873}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":496277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":1.018,"15min":1.018},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64500000000000002,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":1.2,"15min":1.2},"last":0.74299999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.503},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.57199999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.466,"15min":1.466},"last":0.63100000000000001}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":1.012,"15min":1.012},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.175,"5min":1.179,"15min":1.179},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0640000000000001,"5min":1.093,"15min":1.093},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78700000000000003,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.022,"5min":1.407,"15min":1.407},"last":0.86199999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0549999999999999,"5min":1.266,"15min":1.266},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.046,"5min":1.073,"15min":1.073},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:47:24 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.85699999999999998,"15min":0.83999999999999997},"min":{"1min":0.64700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86799999999999999,"15min":0.84899999999999998},"min":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.038}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.82599999999999996,"15min":0.85299999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.55900000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.206,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.79700000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.94099999999999995,"15min":0.95199999999999996},"min":{"1min":0.64800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.97599999999999998,"15min":0.98799999999999999},"min":{"1min":0.66600000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3260000000000001,"5min":1.333,"15min":1.333},"last":1.1619999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:47:36 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.74299999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":1.2370000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.98099999999999998,"15min":0.98299999999999998},"min":{"1min":0.70699999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.079}]},{"osd":11,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.119}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":501149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90200000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.33400000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.77900000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56699999999999995,"15min":0.53300000000000003},"min":{"1min":0.41599999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80000000000000004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.50800000000000001},"min":{"1min":0.48499999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77700000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73499999999999999}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.41899999999999998,"15min":0.39300000000000002},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.50600000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83499999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63200000000000001,"15min":0.59999999999999998},"min":{"1min":0.56299999999999994,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88700000000000001,"5min":1.01,"15min":1.01},"last":0.78100000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67900000000000005,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70799999999999996,"15min":0.68400000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.70199999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:47:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90200000000000002,"5min":1.218,"15min":1.218},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60299999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.321,"15min":1.321},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.82499999999999996}]},{"osd":9,"last update":"Wed Sep 20 21:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.93000000000000005,"5min":1.054,"15min":1.054},"last":0.79200000000000004}]},{"osd":10,"last update":"Wed Sep 20 21:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.53500000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.72999999999999998,"15min":0.72599999999999998},"min":{"1min":0.437,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.75}]},{"osd":11,"last update":"Wed Sep 20 21:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80500000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":497733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 20 21:47:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77800000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.64600000000000002},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Wed Sep 20 21:47:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":0.71599999999999997,"15min":0.65000000000000002},"min":{"1min":0.52100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":8.1639999999999997,"5min":8.1639999999999997,"15min":8.1639999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.72799999999999998,"15min":0.67200000000000004},"min":{"1min":0.55700000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":6.5270000000000001,"5min":6.5270000000000001,"15min":6.5270000000000001},"last":0.52100000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.9499999999999993,"5min":8.9499999999999993,"15min":8.9499999999999993},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":8.6620000000000008,"5min":8.6620000000000008,"15min":8.6620000000000008},"last":0.55400000000000005}]},{"osd":4,"last update":"Wed Sep 20 21:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66600000000000004,"15min":0.64200000000000002},"min":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76700000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.624},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70399999999999996,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.76900000000000002}]},{"osd":5,"last update":"Wed Sep 20 21:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":0.82999999999999996,"15min":0.72899999999999998},"min":{"1min":0.52200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":8.2919999999999998,"5min":8.2919999999999998,"15min":8.2919999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.091,"5min":0.78300000000000003,"15min":0.70099999999999996},"min":{"1min":0.55800000000000005,"5min":0.434,"15min":0.434},"max":{"1min":6.601,"5min":6.601,"15min":6.601},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:47:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.72199999999999998,"15min":0.67900000000000005},"min":{"1min":0.39700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94499999999999995,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":6.6440000000000001,"5min":6.6440000000000001,"15min":6.6440000000000001},"last":0.747},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":8.2159999999999993,"5min":8.2159999999999993,"15min":8.2159999999999993},"last":0.73499999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":0.93999999999999995,"15min":0.86799999999999999},"min":{"1min":0.628,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":8.3290000000000006,"5min":8.3290000000000006,"15min":8.3290000000000006},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.1990000000000001,"5min":0.96099999999999997,"15min":0.93500000000000005},"min":{"1min":0.79600000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":6.6799999999999997,"5min":6.6799999999999997,"15min":6.6799999999999997},"last":0.80200000000000005}]},{"osd":9,"last update":"Wed Sep 20 21:47:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.92800000000000005,"15min":0.93400000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.874},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.78700000000000003,"15min":0.80000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0089999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.755}]},{"osd":10,"last update":"Wed Sep 20 21:47:32 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.01,"15min":0.96399999999999997},"min":{"1min":0.81599999999999995,"5min":0.629,"15min":0.629},"max":{"1min":6.7149999999999999,"5min":6.7149999999999999,"15min":6.7149999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.159,"5min":0.876,"15min":0.82899999999999996},"min":{"1min":0.627,"5min":0.498,"15min":0.498},"max":{"1min":6.8520000000000003,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":0.86099999999999999}]},{"osd":11,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.91400000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":501369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.58799999999999997,"15min":0.55400000000000005},"min":{"1min":0.35399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.0890000000000004,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.441},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.63300000000000001,"15min":0.59299999999999997},"min":{"1min":0.40500000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.628,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":7.0330000000000004,"5min":7.0330000000000004,"15min":7.0330000000000004},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55300000000000005,"15min":0.51800000000000002},"min":{"1min":0.45700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71399999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.645,"5min":3.645,"15min":3.645},"last":0.59299999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:47:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.621},{"interface":"front","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":9.8659999999999997,"5min":9.8659999999999997,"15min":9.8659999999999997},"last":0.64600000000000002}]},{"osd":5,"last update":"Wed Sep 20 21:47:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.41199999999999998},"min":{"1min":0.36199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.92100000000000004,"5min":3.798,"15min":3.798},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.502},"min":{"1min":0.40600000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88,"5min":7.1500000000000004,"15min":7.1500000000000004},"last":0.49099999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.69499999999999995,"15min":0.63600000000000001},"min":{"1min":0.497,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":7.367,"5min":7.367,"15min":7.367},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.72699999999999998,"15min":0.66500000000000004},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":6.4649999999999999,"5min":6.4649999999999999,"15min":6.4649999999999999},"last":0.79900000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.54900000000000004}]},{"osd":8,"last update":"Wed Sep 20 21:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.59799999999999998,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.626},"min":{"1min":0.498,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.85199999999999998,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.60799999999999998}]},{"osd":9,"last update":"Wed Sep 20 21:47:11 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.77000000000000002,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":7.1200000000000001,"5min":7.1200000000000001,"15min":7.1200000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.80500000000000005,"15min":0.77700000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.82399999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.78000000000000003,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":7.1799999999999997,"5min":7.1799999999999997,"15min":7.1799999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.72699999999999998,"15min":0.70699999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.67800000000000005}]},{"osd":11,"last update":"Wed Sep 20 21:47:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":7.2110000000000003,"5min":7.2110000000000003,"15min":7.2110000000000003},"last":0.75}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":495985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.432,"15min":0.39000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81699999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.38}]},{"osd":1,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":1,"15min":1},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.84899999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63300000000000001,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:47:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66400000000000003,"15min":0.63},"min":{"1min":0.47399999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.752},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72599999999999998,"15min":0.65800000000000003},"min":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.89200000000000002}]},{"osd":5,"last update":"Wed Sep 20 21:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70899999999999996,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.625},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76700000000000002,"15min":0.749},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98599999999999999,"5min":1.167,"15min":1.167},"last":0.52700000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.51200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.018,"5min":1.147,"15min":1.147},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.018,"5min":1.296,"15min":1.296},"last":0.81899999999999995}]},{"osd":7,"last update":"Wed Sep 20 21:47:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78300000000000003,"15min":0.77700000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.143,"5min":1.248,"15min":1.248},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.82899999999999996,"15min":0.81899999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.78500000000000003}]},{"osd":8,"last update":"Wed Sep 20 21:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.92900000000000005,"15min":0.94899999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.093,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.92000000000000004,"15min":0.93799999999999994},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.204,"5min":1.361,"15min":1.361},"last":0.54900000000000004}]},{"osd":10,"last update":"Wed Sep 20 21:47:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.90600000000000003,"15min":0.91500000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.141,"5min":1.228,"15min":1.228},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93200000000000005,"15min":0.92700000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":0.85599999999999998}]},{"osd":11,"last update":"Wed Sep 20 21:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.73799999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920827,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":496255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.39600000000000002,"15min":0.36399999999999999},"min":{"1min":0.34200000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.39900000000000002,"15min":0.375},"min":{"1min":0.29499999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42999999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.35799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.41999999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.55000000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57099999999999995,"15min":0.55800000000000005},"min":{"1min":0.41199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.624},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.57399999999999995},"min":{"1min":0.443,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90900000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53000000000000003}]},{"osd":5,"last update":"Wed Sep 20 21:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83199999999999996,"5min":0.871,"15min":0.871},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.497,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:47:07 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":1.554},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67000000000000004,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":1.3919999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59599999999999997,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48099999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.54200000000000004,"5min":0.371,"15min":0.371},"max":{"1min":0.89500000000000002,"5min":1,"15min":1},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73699999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:47:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0820000000000001,"5min":1.087,"15min":1.087},"last":1.429},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.70799999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:47:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.75,"15min":0.751},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.54700000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.59299999999999997}]},{"osd":11,"last update":"Wed Sep 20 21:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.752},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.76700000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1646592,"data_stored":1088462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.42699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80800000000000005,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.54400000000000004}]},{"osd":1,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55800000000000005,"15min":0.53800000000000003},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.77400000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.52800000000000002},"min":{"1min":0.45400000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73299999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.629}]},{"osd":2,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.375},"min":{"1min":0.34899999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.39500000000000002},"min":{"1min":0.36099999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.66400000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38600000000000001}]},{"osd":3,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.45000000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88900000000000001,"5min":1.615,"15min":1.615},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.44700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79300000000000004,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.496}]},{"osd":4,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60699999999999998,"15min":0.56899999999999995},"min":{"1min":0.57299999999999995,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73999999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77500000000000002,"5min":1.008,"15min":1.008},"last":0.70499999999999996}]},{"osd":6,"last update":"Wed Sep 20 21:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65800000000000003,"15min":0.65200000000000002},"min":{"1min":0.53900000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80400000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.76100000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59699999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:47:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.57599999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84899999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.46999999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88600000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72699999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":0.879,"15min":0.879},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.51800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64600000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79100000000000004,"15min":0.79600000000000004},"min":{"1min":0.61099999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.93300000000000005,"5min":1.02,"15min":1.02},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.751,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.84099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66000000000000003}]},{"osd":10,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.745,"15min":0.73499999999999999},"min":{"1min":0.68899999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99399999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.70599999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92100000000000004,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.69199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-20T21:47:59.425 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph pg dump --format=json 2023-09-20T21:47:59.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:59 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:47:59.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:59 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:47:59.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:47:59 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:47:59.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:59 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:47:59.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:59 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:47:59.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:47:59 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:48:00.159 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:48:00.450 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:00 smithi022 ceph-mon[97651]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:00.450 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:00 smithi022 ceph-mon[97651]: pgmap v317: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:48:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:00 smithi123 ceph-mon[134433]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:00 smithi123 ceph-mon[134433]: pgmap v317: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:48:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:00 smithi089 ceph-mon[134090]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:00.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:00 smithi089 ceph-mon[134090]: pgmap v317: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:48:01.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.804+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/650731806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e1410d640 msgr2=0x7f6e1410fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:01.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.804+0000 7f6e1a3c8700 1 --2- 172.21.15.22:0/650731806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e1410d640 0x7f6e1410fab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00b0a0 tx=0x7f6e0c00b3b0 comp rx=0 tx=0).stop 2023-09-20T21:48:01.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.804+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/650731806 shutdown_connections 2023-09-20T21:48:01.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.804+0000 7f6e1a3c8700 1 --2- 172.21.15.22:0/650731806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e1410d640 0x7f6e1410fab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:01.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.804+0000 7f6e1a3c8700 1 --2- 172.21.15.22:0/650731806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14073210 0x7f6e140735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:01.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.804+0000 7f6e1a3c8700 1 --2- 172.21.15.22:0/650731806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14073b30 0x7f6e1410d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:01.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.804+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/650731806 >> 172.21.15.22:0/650731806 conn(0x7f6e140fc8d0 msgr2=0x7f6e140fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:01.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.804+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/650731806 shutdown_connections 2023-09-20T21:48:01.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.804+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/650731806 wait complete. 2023-09-20T21:48:01.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.805+0000 7f6e1a3c8700 1 Processor -- start 2023-09-20T21:48:01.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.805+0000 7f6e1a3c8700 1 -- start start 2023-09-20T21:48:01.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e1a3c8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14073210 0x7f6e14199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:01.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e1a3c8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14073b30 0x7f6e141997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:01.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e1a3c8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e1410d640 0x7f6e1419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:01.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e1a3c8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1419e240 con 0x7f6e1410d640 2023-09-20T21:48:01.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e1a3c8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1419e3b0 con 0x7f6e14073210 2023-09-20T21:48:01.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e1a3c8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1419e570 con 0x7f6e14073b30 2023-09-20T21:48:01.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14073210 0x7f6e14199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:01.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e13fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14073210 0x7f6e14199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:56556/0 (socket says 172.21.15.22:56556) 2023-09-20T21:48:01.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e13fff700 1 -- 172.21.15.22:0/2961664786 learned_addr learned my addr 172.21.15.22:0/2961664786 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:01.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e18965700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e1410d640 0x7f6e1419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:01.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e137fe700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14073b30 0x7f6e141997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:01.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e13fff700 1 -- 172.21.15.22:0/2961664786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14073b30 msgr2=0x7f6e141997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:01.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.806+0000 7f6e13fff700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14073b30 0x7f6e141997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:01.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.807+0000 7f6e13fff700 1 -- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e1410d640 msgr2=0x7f6e1419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:01.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.807+0000 7f6e13fff700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e1410d640 0x7f6e1419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:01.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.807+0000 7f6e13fff700 1 -- 172.21.15.22:0/2961664786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c009d00 con 0x7f6e14073210 2023-09-20T21:48:01.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.807+0000 7f6e18965700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e1410d640 0x7f6e1419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:01.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.807+0000 7f6e13fff700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14073210 0x7f6e14199280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400d980 tx=0x7f6e0400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:01.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.807+0000 7f6e117fa700 1 -- 172.21.15.22:0/2961664786 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04004220 con 0x7f6e14073210 2023-09-20T21:48:01.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.807+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/2961664786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1419e850 con 0x7f6e14073210 2023-09-20T21:48:01.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.807+0000 7f6e117fa700 1 -- 172.21.15.22:0/2961664786 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e04007640 con 0x7f6e14073210 2023-09-20T21:48:01.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.807+0000 7f6e117fa700 1 -- 172.21.15.22:0/2961664786 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04005610 con 0x7f6e14073210 2023-09-20T21:48:01.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.807+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/2961664786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e141a60b0 con 0x7f6e14073210 2023-09-20T21:48:01.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.808+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/2961664786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e14107f60 con 0x7f6e14073210 2023-09-20T21:48:01.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.810+0000 7f6e117fa700 1 -- 172.21.15.22:0/2961664786 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f6e04004750 con 0x7f6e14073210 2023-09-20T21:48:01.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.812+0000 7f6e117fa700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6dfc0a1300 0x7f6dfc0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:01.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.812+0000 7f6e117fa700 1 -- 172.21.15.22:0/2961664786 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6e040cd270 con 0x7f6e14073210 2023-09-20T21:48:01.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.812+0000 7f6e137fe700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6dfc0a1300 0x7f6dfc0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:01.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.813+0000 7f6e137fe700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6dfc0a1300 0x7f6dfc0a37c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e1419a8a0 tx=0x7f6e08009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:01.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:01.814+0000 7f6e117fa700 1 -- 172.21.15.22:0/2961664786 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6e040965e0 con 0x7f6e14073210 2023-09-20T21:48:02.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.018+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/2961664786 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e14112f50 con 0x7f6dfc0a1300 2023-09-20T21:48:02.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.031+0000 7f6e117fa700 1 -- 172.21.15.22:0/2961664786 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426401 (secure 0 0 0) 0x7f6e14112f50 con 0x7f6dfc0a1300 2023-09-20T21:48:02.036 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:02.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.036+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6dfc0a1300 msgr2=0x7f6dfc0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:02.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.036+0000 7f6e1a3c8700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6dfc0a1300 0x7f6dfc0a37c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e1419a8a0 tx=0x7f6e08009260 comp rx=0 tx=0).stop 2023-09-20T21:48:02.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.036+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/2961664786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14073210 msgr2=0x7f6e14199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:02.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.036+0000 7f6e1a3c8700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14073210 0x7f6e14199280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400d980 tx=0x7f6e0400dc90 comp rx=0 tx=0).stop 2023-09-20T21:48:02.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.037+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/2961664786 shutdown_connections 2023-09-20T21:48:02.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.037+0000 7f6e1a3c8700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e1410d640 0x7f6e1419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:02.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.037+0000 7f6e1a3c8700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6dfc0a1300 0x7f6dfc0a37c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:02.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.037+0000 7f6e1a3c8700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14073210 0x7f6e14199280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:02.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.037+0000 7f6e1a3c8700 1 --2- 172.21.15.22:0/2961664786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e14073b30 0x7f6e141997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:02.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.037+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/2961664786 >> 172.21.15.22:0/2961664786 conn(0x7f6e140fc8d0 msgr2=0x7f6e141050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:02.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.037+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/2961664786 shutdown_connections 2023-09-20T21:48:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:02.037+0000 7f6e1a3c8700 1 -- 172.21.15.22:0/2961664786 wait complete. 2023-09-20T21:48:02.042 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-09-20T21:48:02.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:02 smithi022 ceph-mon[97651]: pgmap v318: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:02.553 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2023-09-20T21:48:01.118369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600632,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":658,"num_read_kb":390,"num_write":390,"num_write_kb":3005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":320,"ondisk_log_size":320,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338228,"kb_used_data":13976,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603596,"statfs":{"total":1151940427776,"available":1151594082304,"internally_reserved":0,"allocated":14311424,"data_stored":7748275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003603"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598937+0000","last_change":"2023-09-20T21:46:41.506092+0000","last_active":"2023-09-20T21:47:07.598937+0000","last_peered":"2023-09-20T21:47:07.598937+0000","last_clean":"2023-09-20T21:47:07.598937+0000","last_became_active":"2023-09-20T21:46:41.505779+0000","last_became_peered":"2023-09-20T21:46:41.505779+0000","last_unstale":"2023-09-20T21:47:07.598937+0000","last_undegraded":"2023-09-20T21:47:07.598937+0000","last_fullsized":"2023-09-20T21:47:07.598937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:25.284068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599485+0000","last_change":"2023-09-20T21:46:41.505000+0000","last_active":"2023-09-20T21:47:07.599485+0000","last_peered":"2023-09-20T21:47:07.599485+0000","last_clean":"2023-09-20T21:47:07.599485+0000","last_became_active":"2023-09-20T21:46:41.504464+0000","last_became_peered":"2023-09-20T21:46:41.504464+0000","last_unstale":"2023-09-20T21:47:07.599485+0000","last_undegraded":"2023-09-20T21:47:07.599485+0000","last_fullsized":"2023-09-20T21:47:07.599485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:42:44.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599342+0000","last_change":"2023-09-20T21:46:41.513134+0000","last_active":"2023-09-20T21:47:07.599342+0000","last_peered":"2023-09-20T21:47:07.599342+0000","last_clean":"2023-09-20T21:47:07.599342+0000","last_became_active":"2023-09-20T21:46:41.512793+0000","last_became_peered":"2023-09-20T21:46:41.512793+0000","last_unstale":"2023-09-20T21:47:07.599342+0000","last_undegraded":"2023-09-20T21:47:07.599342+0000","last_fullsized":"2023-09-20T21:47:07.599342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:59.978630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597341+0000","last_change":"2023-09-20T21:46:41.500200+0000","last_active":"2023-09-20T21:47:07.597341+0000","last_peered":"2023-09-20T21:47:07.597341+0000","last_clean":"2023-09-20T21:47:07.597341+0000","last_became_active":"2023-09-20T21:46:41.499866+0000","last_became_peered":"2023-09-20T21:46:41.499866+0000","last_unstale":"2023-09-20T21:47:07.597341+0000","last_undegraded":"2023-09-20T21:47:07.597341+0000","last_fullsized":"2023-09-20T21:47:07.597341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:20:52.989008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594880+0000","last_change":"2023-09-20T21:46:41.507584+0000","last_active":"2023-09-20T21:47:07.594880+0000","last_peered":"2023-09-20T21:47:07.594880+0000","last_clean":"2023-09-20T21:47:07.594880+0000","last_became_active":"2023-09-20T21:46:41.507298+0000","last_became_peered":"2023-09-20T21:46:41.507298+0000","last_unstale":"2023-09-20T21:47:07.594880+0000","last_undegraded":"2023-09-20T21:47:07.594880+0000","last_fullsized":"2023-09-20T21:47:07.594880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:32:14.042414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595455+0000","last_change":"2023-09-20T21:46:41.497002+0000","last_active":"2023-09-20T21:47:07.595455+0000","last_peered":"2023-09-20T21:47:07.595455+0000","last_clean":"2023-09-20T21:47:07.595455+0000","last_became_active":"2023-09-20T21:46:41.496485+0000","last_became_peered":"2023-09-20T21:46:41.496485+0000","last_unstale":"2023-09-20T21:47:07.595455+0000","last_undegraded":"2023-09-20T21:47:07.595455+0000","last_fullsized":"2023-09-20T21:47:07.595455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:48:24.481575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598725+0000","last_change":"2023-09-20T21:46:41.503303+0000","last_active":"2023-09-20T21:47:07.598725+0000","last_peered":"2023-09-20T21:47:07.598725+0000","last_clean":"2023-09-20T21:47:07.598725+0000","last_became_active":"2023-09-20T21:46:41.502982+0000","last_became_peered":"2023-09-20T21:46:41.502982+0000","last_unstale":"2023-09-20T21:47:07.598725+0000","last_undegraded":"2023-09-20T21:47:07.598725+0000","last_fullsized":"2023-09-20T21:47:07.598725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:23:27.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595388+0000","last_change":"2023-09-20T21:46:41.508243+0000","last_active":"2023-09-20T21:47:07.595388+0000","last_peered":"2023-09-20T21:47:07.595388+0000","last_clean":"2023-09-20T21:47:07.595388+0000","last_became_active":"2023-09-20T21:46:41.506407+0000","last_became_peered":"2023-09-20T21:46:41.506407+0000","last_unstale":"2023-09-20T21:47:07.595388+0000","last_undegraded":"2023-09-20T21:47:07.595388+0000","last_fullsized":"2023-09-20T21:47:07.595388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:39.388716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594461+0000","last_change":"2023-09-20T21:46:41.501513+0000","last_active":"2023-09-20T21:47:07.594461+0000","last_peered":"2023-09-20T21:47:07.594461+0000","last_clean":"2023-09-20T21:47:07.594461+0000","last_became_active":"2023-09-20T21:46:41.500051+0000","last_became_peered":"2023-09-20T21:46:41.500051+0000","last_unstale":"2023-09-20T21:47:07.594461+0000","last_undegraded":"2023-09-20T21:47:07.594461+0000","last_fullsized":"2023-09-20T21:47:07.594461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:40.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596488+0000","last_change":"2023-09-20T21:46:41.501045+0000","last_active":"2023-09-20T21:47:07.596488+0000","last_peered":"2023-09-20T21:47:07.596488+0000","last_clean":"2023-09-20T21:47:07.596488+0000","last_became_active":"2023-09-20T21:46:41.500744+0000","last_became_peered":"2023-09-20T21:46:41.500744+0000","last_unstale":"2023-09-20T21:47:07.596488+0000","last_undegraded":"2023-09-20T21:47:07.596488+0000","last_fullsized":"2023-09-20T21:47:07.596488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:57:29.120472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599791+0000","last_change":"2023-09-20T21:46:41.510235+0000","last_active":"2023-09-20T21:47:07.599791+0000","last_peered":"2023-09-20T21:47:07.599791+0000","last_clean":"2023-09-20T21:47:07.599791+0000","last_became_active":"2023-09-20T21:46:41.509943+0000","last_became_peered":"2023-09-20T21:46:41.509943+0000","last_unstale":"2023-09-20T21:47:07.599791+0000","last_undegraded":"2023-09-20T21:47:07.599791+0000","last_fullsized":"2023-09-20T21:47:07.599791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:09:19.383505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597247+0000","last_change":"2023-09-20T21:46:41.501194+0000","last_active":"2023-09-20T21:47:07.597247+0000","last_peered":"2023-09-20T21:47:07.597247+0000","last_clean":"2023-09-20T21:47:07.597247+0000","last_became_active":"2023-09-20T21:46:41.500095+0000","last_became_peered":"2023-09-20T21:46:41.500095+0000","last_unstale":"2023-09-20T21:47:07.597247+0000","last_undegraded":"2023-09-20T21:47:07.597247+0000","last_fullsized":"2023-09-20T21:47:07.597247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:46:51.343332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602350+0000","last_change":"2023-09-20T21:46:41.507562+0000","last_active":"2023-09-20T21:47:07.602350+0000","last_peered":"2023-09-20T21:47:07.602350+0000","last_clean":"2023-09-20T21:47:07.602350+0000","last_became_active":"2023-09-20T21:46:41.507094+0000","last_became_peered":"2023-09-20T21:46:41.507094+0000","last_unstale":"2023-09-20T21:47:07.602350+0000","last_undegraded":"2023-09-20T21:47:07.602350+0000","last_fullsized":"2023-09-20T21:47:07.602350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:34:32.401415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598795+0000","last_change":"2023-09-20T21:46:41.502864+0000","last_active":"2023-09-20T21:47:07.598795+0000","last_peered":"2023-09-20T21:47:07.598795+0000","last_clean":"2023-09-20T21:47:07.598795+0000","last_became_active":"2023-09-20T21:46:41.502543+0000","last_became_peered":"2023-09-20T21:46:41.502543+0000","last_unstale":"2023-09-20T21:47:07.598795+0000","last_undegraded":"2023-09-20T21:47:07.598795+0000","last_fullsized":"2023-09-20T21:47:07.598795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:02:04.631874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.600445+0000","last_change":"2023-09-20T21:47:08.600445+0000","last_active":"2023-09-20T21:47:08.600445+0000","last_peered":"2023-09-20T21:47:08.600445+0000","last_clean":"2023-09-20T21:47:08.600445+0000","last_became_active":"2023-09-20T21:47:08.600054+0000","last_became_peered":"2023-09-20T21:47:08.600054+0000","last_unstale":"2023-09-20T21:47:08.600445+0000","last_undegraded":"2023-09-20T21:47:08.600445+0000","last_fullsized":"2023-09-20T21:47:08.600445+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:43:18.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593608+0000","last_change":"2023-09-20T21:46:41.502036+0000","last_active":"2023-09-20T21:47:07.593608+0000","last_peered":"2023-09-20T21:47:07.593608+0000","last_clean":"2023-09-20T21:47:07.593608+0000","last_became_active":"2023-09-20T21:46:41.501745+0000","last_became_peered":"2023-09-20T21:46:41.501745+0000","last_unstale":"2023-09-20T21:47:07.593608+0000","last_undegraded":"2023-09-20T21:47:07.593608+0000","last_fullsized":"2023-09-20T21:47:07.593608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:08:06.699543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598739+0000","last_change":"2023-09-20T21:46:41.506666+0000","last_active":"2023-09-20T21:47:07.598739+0000","last_peered":"2023-09-20T21:47:07.598739+0000","last_clean":"2023-09-20T21:47:07.598739+0000","last_became_active":"2023-09-20T21:46:41.506266+0000","last_became_peered":"2023-09-20T21:46:41.506266+0000","last_unstale":"2023-09-20T21:47:07.598739+0000","last_undegraded":"2023-09-20T21:47:07.598739+0000","last_fullsized":"2023-09-20T21:47:07.598739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:06:16.809609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592904+0000","last_change":"2023-09-20T21:46:41.497391+0000","last_active":"2023-09-20T21:47:07.592904+0000","last_peered":"2023-09-20T21:47:07.592904+0000","last_clean":"2023-09-20T21:47:07.592904+0000","last_became_active":"2023-09-20T21:46:41.497064+0000","last_became_peered":"2023-09-20T21:46:41.497064+0000","last_unstale":"2023-09-20T21:47:07.592904+0000","last_undegraded":"2023-09-20T21:47:07.592904+0000","last_fullsized":"2023-09-20T21:47:07.592904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:56:11.489585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599538+0000","last_change":"2023-09-20T21:46:41.510602+0000","last_active":"2023-09-20T21:47:07.599538+0000","last_peered":"2023-09-20T21:47:07.599538+0000","last_clean":"2023-09-20T21:47:07.599538+0000","last_became_active":"2023-09-20T21:46:41.510308+0000","last_became_peered":"2023-09-20T21:46:41.510308+0000","last_unstale":"2023-09-20T21:47:07.599538+0000","last_undegraded":"2023-09-20T21:47:07.599538+0000","last_fullsized":"2023-09-20T21:47:07.599538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:43:41.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598788+0000","last_change":"2023-09-20T21:46:45.581583+0000","last_active":"2023-09-20T21:47:07.598788+0000","last_peered":"2023-09-20T21:47:07.598788+0000","last_clean":"2023-09-20T21:47:07.598788+0000","last_became_active":"2023-09-20T21:46:45.581165+0000","last_became_peered":"2023-09-20T21:46:45.581165+0000","last_unstale":"2023-09-20T21:47:07.598788+0000","last_undegraded":"2023-09-20T21:47:07.598788+0000","last_fullsized":"2023-09-20T21:47:07.598788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:39:36.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":83,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:57.693872+0000","last_change":"2023-09-20T21:46:40.500010+0000","last_active":"2023-09-20T21:47:57.693872+0000","last_peered":"2023-09-20T21:47:57.693872+0000","last_clean":"2023-09-20T21:47:57.693872+0000","last_became_active":"2023-09-20T21:46:40.499632+0000","last_became_peered":"2023-09-20T21:46:40.499632+0000","last_unstale":"2023-09-20T21:47:57.693872+0000","last_undegraded":"2023-09-20T21:47:57.693872+0000","last_fullsized":"2023-09-20T21:47:57.693872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:41:35.590982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595846+0000","last_change":"2023-09-20T21:46:41.516465+0000","last_active":"2023-09-20T21:47:07.595846+0000","last_peered":"2023-09-20T21:47:07.595846+0000","last_clean":"2023-09-20T21:47:07.595846+0000","last_became_active":"2023-09-20T21:46:41.516150+0000","last_became_peered":"2023-09-20T21:46:41.516150+0000","last_unstale":"2023-09-20T21:47:07.595846+0000","last_undegraded":"2023-09-20T21:47:07.595846+0000","last_fullsized":"2023-09-20T21:47:07.595846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:33:37.507049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596541+0000","last_change":"2023-09-20T21:46:45.582916+0000","last_active":"2023-09-20T21:47:07.596541+0000","last_peered":"2023-09-20T21:47:07.596541+0000","last_clean":"2023-09-20T21:47:07.596541+0000","last_became_active":"2023-09-20T21:46:45.582511+0000","last_became_peered":"2023-09-20T21:46:45.582511+0000","last_unstale":"2023-09-20T21:47:07.596541+0000","last_undegraded":"2023-09-20T21:47:07.596541+0000","last_fullsized":"2023-09-20T21:47:07.596541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:43:47.006768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596015+0000","last_change":"2023-09-20T21:46:40.516867+0000","last_active":"2023-09-20T21:47:07.596015+0000","last_peered":"2023-09-20T21:47:07.596015+0000","last_clean":"2023-09-20T21:47:07.596015+0000","last_became_active":"2023-09-20T21:46:40.516343+0000","last_became_peered":"2023-09-20T21:46:40.516343+0000","last_unstale":"2023-09-20T21:47:07.596015+0000","last_undegraded":"2023-09-20T21:47:07.596015+0000","last_fullsized":"2023-09-20T21:47:07.596015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:17:18.348921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599149+0000","last_change":"2023-09-20T21:46:41.507093+0000","last_active":"2023-09-20T21:47:07.599149+0000","last_peered":"2023-09-20T21:47:07.599149+0000","last_clean":"2023-09-20T21:47:07.599149+0000","last_became_active":"2023-09-20T21:46:41.506587+0000","last_became_peered":"2023-09-20T21:46:41.506587+0000","last_unstale":"2023-09-20T21:47:07.599149+0000","last_undegraded":"2023-09-20T21:47:07.599149+0000","last_fullsized":"2023-09-20T21:47:07.599149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:26:44.825499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597188+0000","last_change":"2023-09-20T21:46:45.583942+0000","last_active":"2023-09-20T21:47:07.597188+0000","last_peered":"2023-09-20T21:47:07.597188+0000","last_clean":"2023-09-20T21:47:07.597188+0000","last_became_active":"2023-09-20T21:46:45.583484+0000","last_became_peered":"2023-09-20T21:46:45.583484+0000","last_unstale":"2023-09-20T21:47:07.597188+0000","last_undegraded":"2023-09-20T21:47:07.597188+0000","last_fullsized":"2023-09-20T21:47:07.597188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:48.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:09.612757+0000","last_change":"2023-09-20T21:47:08.615780+0000","last_active":"2023-09-20T21:47:09.612757+0000","last_peered":"2023-09-20T21:47:09.612757+0000","last_clean":"2023-09-20T21:47:09.612757+0000","last_became_active":"2023-09-20T21:47:08.603974+0000","last_became_peered":"2023-09-20T21:47:08.603974+0000","last_unstale":"2023-09-20T21:47:09.612757+0000","last_undegraded":"2023-09-20T21:47:09.612757+0000","last_fullsized":"2023-09-20T21:47:09.612757+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:53:23.172401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594055+0000","last_change":"2023-09-20T21:46:41.504449+0000","last_active":"2023-09-20T21:47:07.594055+0000","last_peered":"2023-09-20T21:47:07.594055+0000","last_clean":"2023-09-20T21:47:07.594055+0000","last_became_active":"2023-09-20T21:46:41.504169+0000","last_became_peered":"2023-09-20T21:46:41.504169+0000","last_unstale":"2023-09-20T21:47:07.594055+0000","last_undegraded":"2023-09-20T21:47:07.594055+0000","last_fullsized":"2023-09-20T21:47:07.594055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:36:43.980902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596618+0000","last_change":"2023-09-20T21:46:45.582269+0000","last_active":"2023-09-20T21:47:07.596618+0000","last_peered":"2023-09-20T21:47:07.596618+0000","last_clean":"2023-09-20T21:47:07.596618+0000","last_became_active":"2023-09-20T21:46:45.581782+0000","last_became_peered":"2023-09-20T21:46:45.581782+0000","last_unstale":"2023-09-20T21:47:07.596618+0000","last_undegraded":"2023-09-20T21:47:07.596618+0000","last_fullsized":"2023-09-20T21:47:07.596618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:32:09.518715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.598645+0000","last_change":"2023-09-20T21:46:40.504099+0000","last_active":"2023-09-20T21:47:08.598645+0000","last_peered":"2023-09-20T21:47:08.598645+0000","last_clean":"2023-09-20T21:47:08.598645+0000","last_became_active":"2023-09-20T21:46:40.503774+0000","last_became_peered":"2023-09-20T21:46:40.503774+0000","last_unstale":"2023-09-20T21:47:08.598645+0000","last_undegraded":"2023-09-20T21:47:08.598645+0000","last_fullsized":"2023-09-20T21:47:08.598645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:17.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595338+0000","last_change":"2023-09-20T21:46:41.499802+0000","last_active":"2023-09-20T21:47:07.595338+0000","last_peered":"2023-09-20T21:47:07.595338+0000","last_clean":"2023-09-20T21:47:07.595338+0000","last_became_active":"2023-09-20T21:46:41.499491+0000","last_became_peered":"2023-09-20T21:46:41.499491+0000","last_unstale":"2023-09-20T21:47:07.595338+0000","last_undegraded":"2023-09-20T21:47:07.595338+0000","last_fullsized":"2023-09-20T21:47:07.595338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:44.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599395+0000","last_change":"2023-09-20T21:46:45.578753+0000","last_active":"2023-09-20T21:47:07.599395+0000","last_peered":"2023-09-20T21:47:07.599395+0000","last_clean":"2023-09-20T21:47:07.599395+0000","last_became_active":"2023-09-20T21:46:45.578297+0000","last_became_peered":"2023-09-20T21:46:45.578297+0000","last_unstale":"2023-09-20T21:47:07.599395+0000","last_undegraded":"2023-09-20T21:47:07.599395+0000","last_fullsized":"2023-09-20T21:47:07.599395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:18:22.596550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":79,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:58.647225+0000","last_change":"2023-09-20T21:46:40.505675+0000","last_active":"2023-09-20T21:47:58.647225+0000","last_peered":"2023-09-20T21:47:58.647225+0000","last_clean":"2023-09-20T21:47:58.647225+0000","last_became_active":"2023-09-20T21:46:40.505432+0000","last_became_peered":"2023-09-20T21:46:40.505432+0000","last_unstale":"2023-09-20T21:47:58.647225+0000","last_undegraded":"2023-09-20T21:47:58.647225+0000","last_fullsized":"2023-09-20T21:47:58.647225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:14:53.652904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595519+0000","last_change":"2023-09-20T21:46:41.505924+0000","last_active":"2023-09-20T21:47:07.595519+0000","last_peered":"2023-09-20T21:47:07.595519+0000","last_clean":"2023-09-20T21:47:07.595519+0000","last_became_active":"2023-09-20T21:46:41.505609+0000","last_became_peered":"2023-09-20T21:46:41.505609+0000","last_unstale":"2023-09-20T21:47:07.595519+0000","last_undegraded":"2023-09-20T21:47:07.595519+0000","last_fullsized":"2023-09-20T21:47:07.595519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:13:46.506786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600999+0000","last_change":"2023-09-20T21:46:45.587818+0000","last_active":"2023-09-20T21:47:07.600999+0000","last_peered":"2023-09-20T21:47:07.600999+0000","last_clean":"2023-09-20T21:47:07.600999+0000","last_became_active":"2023-09-20T21:46:45.587376+0000","last_became_peered":"2023-09-20T21:46:45.587376+0000","last_unstale":"2023-09-20T21:47:07.600999+0000","last_undegraded":"2023-09-20T21:47:07.600999+0000","last_fullsized":"2023-09-20T21:47:07.600999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:28:53.605158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595338+0000","last_change":"2023-09-20T21:46:40.505610+0000","last_active":"2023-09-20T21:47:07.595338+0000","last_peered":"2023-09-20T21:47:07.595338+0000","last_clean":"2023-09-20T21:47:07.595338+0000","last_became_active":"2023-09-20T21:46:40.505279+0000","last_became_peered":"2023-09-20T21:46:40.505279+0000","last_unstale":"2023-09-20T21:47:07.595338+0000","last_undegraded":"2023-09-20T21:47:07.595338+0000","last_fullsized":"2023-09-20T21:47:07.595338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:41:19.575154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.909510+0000","last_change":"2023-09-20T21:47:08.908513+0000","last_active":"2023-09-20T21:47:08.909510+0000","last_peered":"2023-09-20T21:47:08.909510+0000","last_clean":"2023-09-20T21:47:08.909510+0000","last_became_active":"2023-09-20T21:47:08.908142+0000","last_became_peered":"2023-09-20T21:47:08.908142+0000","last_unstale":"2023-09-20T21:47:08.909510+0000","last_undegraded":"2023-09-20T21:47:08.909510+0000","last_fullsized":"2023-09-20T21:47:08.909510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:31:33.992202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598682+0000","last_change":"2023-09-20T21:46:45.581738+0000","last_active":"2023-09-20T21:47:07.598682+0000","last_peered":"2023-09-20T21:47:07.598682+0000","last_clean":"2023-09-20T21:47:07.598682+0000","last_became_active":"2023-09-20T21:46:45.581279+0000","last_became_peered":"2023-09-20T21:46:45.581279+0000","last_unstale":"2023-09-20T21:47:07.598682+0000","last_undegraded":"2023-09-20T21:47:07.598682+0000","last_fullsized":"2023-09-20T21:47:07.598682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:41:41.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577335+0000","last_change":"2023-09-20T21:46:40.510539+0000","last_active":"2023-09-20T21:47:08.577335+0000","last_peered":"2023-09-20T21:47:08.577335+0000","last_clean":"2023-09-20T21:47:08.577335+0000","last_became_active":"2023-09-20T21:46:40.509880+0000","last_became_peered":"2023-09-20T21:46:40.509880+0000","last_unstale":"2023-09-20T21:47:08.577335+0000","last_undegraded":"2023-09-20T21:47:08.577335+0000","last_fullsized":"2023-09-20T21:47:08.577335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:52.877786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595248+0000","last_change":"2023-09-20T21:46:41.505105+0000","last_active":"2023-09-20T21:47:07.595248+0000","last_peered":"2023-09-20T21:47:07.595248+0000","last_clean":"2023-09-20T21:47:07.595248+0000","last_became_active":"2023-09-20T21:46:41.504785+0000","last_became_peered":"2023-09-20T21:46:41.504785+0000","last_unstale":"2023-09-20T21:47:07.595248+0000","last_undegraded":"2023-09-20T21:47:07.595248+0000","last_fullsized":"2023-09-20T21:47:07.595248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:07:57.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599298+0000","last_change":"2023-09-20T21:46:45.583160+0000","last_active":"2023-09-20T21:47:07.599298+0000","last_peered":"2023-09-20T21:47:07.599298+0000","last_clean":"2023-09-20T21:47:07.599298+0000","last_became_active":"2023-09-20T21:46:45.582709+0000","last_became_peered":"2023-09-20T21:46:45.582709+0000","last_unstale":"2023-09-20T21:47:07.599298+0000","last_undegraded":"2023-09-20T21:47:07.599298+0000","last_fullsized":"2023-09-20T21:47:07.599298+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:24:57.563682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.906996+0000","last_change":"2023-09-20T21:46:40.502591+0000","last_active":"2023-09-20T21:47:08.906996+0000","last_peered":"2023-09-20T21:47:08.906996+0000","last_clean":"2023-09-20T21:47:08.906996+0000","last_became_active":"2023-09-20T21:46:40.502288+0000","last_became_peered":"2023-09-20T21:46:40.502288+0000","last_unstale":"2023-09-20T21:47:08.906996+0000","last_undegraded":"2023-09-20T21:47:08.906996+0000","last_fullsized":"2023-09-20T21:47:08.906996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:09:37.983866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.599912+0000","last_change":"2023-09-20T21:47:08.599912+0000","last_active":"2023-09-20T21:47:08.599912+0000","last_peered":"2023-09-20T21:47:08.599912+0000","last_clean":"2023-09-20T21:47:08.599912+0000","last_became_active":"2023-09-20T21:47:08.599434+0000","last_became_peered":"2023-09-20T21:47:08.599434+0000","last_unstale":"2023-09-20T21:47:08.599912+0000","last_undegraded":"2023-09-20T21:47:08.599912+0000","last_fullsized":"2023-09-20T21:47:08.599912+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:37:01.125731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595936+0000","last_change":"2023-09-20T21:46:45.580917+0000","last_active":"2023-09-20T21:47:07.595936+0000","last_peered":"2023-09-20T21:47:07.595936+0000","last_clean":"2023-09-20T21:47:07.595936+0000","last_became_active":"2023-09-20T21:46:45.580505+0000","last_became_peered":"2023-09-20T21:46:45.580505+0000","last_unstale":"2023-09-20T21:47:07.595936+0000","last_undegraded":"2023-09-20T21:47:07.595936+0000","last_fullsized":"2023-09-20T21:47:07.595936+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:52:19.733919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:55.641481+0000","last_change":"2023-09-20T21:46:40.507040+0000","last_active":"2023-09-20T21:47:55.641481+0000","last_peered":"2023-09-20T21:47:55.641481+0000","last_clean":"2023-09-20T21:47:55.641481+0000","last_became_active":"2023-09-20T21:46:40.506712+0000","last_became_peered":"2023-09-20T21:46:40.506712+0000","last_unstale":"2023-09-20T21:47:55.641481+0000","last_undegraded":"2023-09-20T21:47:55.641481+0000","last_fullsized":"2023-09-20T21:47:55.641481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:02:56.040911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594395+0000","last_change":"2023-09-20T21:46:41.504794+0000","last_active":"2023-09-20T21:47:07.594395+0000","last_peered":"2023-09-20T21:47:07.594395+0000","last_clean":"2023-09-20T21:47:07.594395+0000","last_became_active":"2023-09-20T21:46:41.504372+0000","last_became_peered":"2023-09-20T21:46:41.504372+0000","last_unstale":"2023-09-20T21:47:07.594395+0000","last_undegraded":"2023-09-20T21:47:07.594395+0000","last_fullsized":"2023-09-20T21:47:07.594395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:42:20.527306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592403+0000","last_change":"2023-09-20T21:46:45.586364+0000","last_active":"2023-09-20T21:47:07.592403+0000","last_peered":"2023-09-20T21:47:07.592403+0000","last_clean":"2023-09-20T21:47:07.592403+0000","last_became_active":"2023-09-20T21:46:45.584620+0000","last_became_peered":"2023-09-20T21:46:45.584620+0000","last_unstale":"2023-09-20T21:47:07.592403+0000","last_undegraded":"2023-09-20T21:47:07.592403+0000","last_fullsized":"2023-09-20T21:47:07.592403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:05:20.158118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577704+0000","last_change":"2023-09-20T21:46:40.500797+0000","last_active":"2023-09-20T21:47:08.577704+0000","last_peered":"2023-09-20T21:47:08.577704+0000","last_clean":"2023-09-20T21:47:08.577704+0000","last_became_active":"2023-09-20T21:46:40.500337+0000","last_became_peered":"2023-09-20T21:46:40.500337+0000","last_unstale":"2023-09-20T21:47:08.577704+0000","last_undegraded":"2023-09-20T21:47:08.577704+0000","last_fullsized":"2023-09-20T21:47:08.577704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:19.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.908615+0000","last_change":"2023-09-20T21:47:08.908615+0000","last_active":"2023-09-20T21:47:08.908615+0000","last_peered":"2023-09-20T21:47:08.908615+0000","last_clean":"2023-09-20T21:47:08.908615+0000","last_became_active":"2023-09-20T21:47:08.908228+0000","last_became_peered":"2023-09-20T21:47:08.908228+0000","last_unstale":"2023-09-20T21:47:08.908615+0000","last_undegraded":"2023-09-20T21:47:08.908615+0000","last_fullsized":"2023-09-20T21:47:08.908615+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:15:28.772129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597442+0000","last_change":"2023-09-20T21:46:45.583081+0000","last_active":"2023-09-20T21:47:07.597442+0000","last_peered":"2023-09-20T21:47:07.597442+0000","last_clean":"2023-09-20T21:47:07.597442+0000","last_became_active":"2023-09-20T21:46:45.580496+0000","last_became_peered":"2023-09-20T21:46:45.580496+0000","last_unstale":"2023-09-20T21:47:07.597442+0000","last_undegraded":"2023-09-20T21:47:07.597442+0000","last_fullsized":"2023-09-20T21:47:07.597442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:42.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595250+0000","last_change":"2023-09-20T21:46:40.512739+0000","last_active":"2023-09-20T21:47:07.595250+0000","last_peered":"2023-09-20T21:47:07.595250+0000","last_clean":"2023-09-20T21:47:07.595250+0000","last_became_active":"2023-09-20T21:46:40.512485+0000","last_became_peered":"2023-09-20T21:46:40.512485+0000","last_unstale":"2023-09-20T21:47:07.595250+0000","last_undegraded":"2023-09-20T21:47:07.595250+0000","last_fullsized":"2023-09-20T21:47:07.595250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:02:49.392560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.604682+0000","last_change":"2023-09-20T21:47:08.604682+0000","last_active":"2023-09-20T21:47:08.604682+0000","last_peered":"2023-09-20T21:47:08.604682+0000","last_clean":"2023-09-20T21:47:08.604682+0000","last_became_active":"2023-09-20T21:47:08.604271+0000","last_became_peered":"2023-09-20T21:47:08.604271+0000","last_unstale":"2023-09-20T21:47:08.604682+0000","last_undegraded":"2023-09-20T21:47:08.604682+0000","last_fullsized":"2023-09-20T21:47:08.604682+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:40:51.542781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592702+0000","last_change":"2023-09-20T21:46:45.579478+0000","last_active":"2023-09-20T21:47:07.592702+0000","last_peered":"2023-09-20T21:47:07.592702+0000","last_clean":"2023-09-20T21:47:07.592702+0000","last_became_active":"2023-09-20T21:46:45.579055+0000","last_became_peered":"2023-09-20T21:46:45.579055+0000","last_unstale":"2023-09-20T21:47:07.592702+0000","last_undegraded":"2023-09-20T21:47:07.592702+0000","last_fullsized":"2023-09-20T21:47:07.592702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:17:32.013280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'14","reported_seq":132,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:55.642560+0000","last_change":"2023-09-20T21:46:40.507321+0000","last_active":"2023-09-20T21:47:55.642560+0000","last_peered":"2023-09-20T21:47:55.642560+0000","last_clean":"2023-09-20T21:47:55.642560+0000","last_became_active":"2023-09-20T21:46:40.506983+0000","last_became_peered":"2023-09-20T21:46:40.506983+0000","last_unstale":"2023-09-20T21:47:55.642560+0000","last_undegraded":"2023-09-20T21:47:55.642560+0000","last_fullsized":"2023-09-20T21:47:55.642560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:46:40.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":100,"num_read_kb":54,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596051+0000","last_change":"2023-09-20T21:46:41.509646+0000","last_active":"2023-09-20T21:47:07.596051+0000","last_peered":"2023-09-20T21:47:07.596051+0000","last_clean":"2023-09-20T21:47:07.596051+0000","last_became_active":"2023-09-20T21:46:41.509183+0000","last_became_peered":"2023-09-20T21:46:41.509183+0000","last_unstale":"2023-09-20T21:47:07.596051+0000","last_undegraded":"2023-09-20T21:47:07.596051+0000","last_fullsized":"2023-09-20T21:47:07.596051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:17:21.084830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595627+0000","last_change":"2023-09-20T21:46:45.583709+0000","last_active":"2023-09-20T21:47:07.595627+0000","last_peered":"2023-09-20T21:47:07.595627+0000","last_clean":"2023-09-20T21:47:07.595627+0000","last_became_active":"2023-09-20T21:46:45.583302+0000","last_became_peered":"2023-09-20T21:46:45.583302+0000","last_unstale":"2023-09-20T21:47:07.595627+0000","last_undegraded":"2023-09-20T21:47:07.595627+0000","last_fullsized":"2023-09-20T21:47:07.595627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:31:35.472750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.594685+0000","last_change":"2023-09-20T21:46:40.506411+0000","last_active":"2023-09-20T21:47:08.594685+0000","last_peered":"2023-09-20T21:47:08.594685+0000","last_clean":"2023-09-20T21:47:08.594685+0000","last_became_active":"2023-09-20T21:46:40.506116+0000","last_became_peered":"2023-09-20T21:46:40.506116+0000","last_unstale":"2023-09-20T21:47:08.594685+0000","last_undegraded":"2023-09-20T21:47:08.594685+0000","last_fullsized":"2023-09-20T21:47:08.594685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:23:19.175007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602083+0000","last_change":"2023-09-20T21:46:41.513793+0000","last_active":"2023-09-20T21:47:07.602083+0000","last_peered":"2023-09-20T21:47:07.602083+0000","last_clean":"2023-09-20T21:47:07.602083+0000","last_became_active":"2023-09-20T21:46:41.513434+0000","last_became_peered":"2023-09-20T21:46:41.513434+0000","last_unstale":"2023-09-20T21:47:07.602083+0000","last_undegraded":"2023-09-20T21:47:07.602083+0000","last_fullsized":"2023-09-20T21:47:07.602083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:00:57.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598091+0000","last_change":"2023-09-20T21:46:45.578188+0000","last_active":"2023-09-20T21:47:07.598091+0000","last_peered":"2023-09-20T21:47:07.598091+0000","last_clean":"2023-09-20T21:47:07.598091+0000","last_became_active":"2023-09-20T21:46:45.577548+0000","last_became_peered":"2023-09-20T21:46:45.577548+0000","last_unstale":"2023-09-20T21:47:07.598091+0000","last_undegraded":"2023-09-20T21:47:07.598091+0000","last_fullsized":"2023-09-20T21:47:07.598091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:54:58.894269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.907407+0000","last_change":"2023-09-20T21:46:40.505465+0000","last_active":"2023-09-20T21:47:08.907407+0000","last_peered":"2023-09-20T21:47:08.907407+0000","last_clean":"2023-09-20T21:47:08.907407+0000","last_became_active":"2023-09-20T21:46:40.505174+0000","last_became_peered":"2023-09-20T21:46:40.505174+0000","last_unstale":"2023-09-20T21:47:08.907407+0000","last_undegraded":"2023-09-20T21:47:08.907407+0000","last_fullsized":"2023-09-20T21:47:08.907407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:50:53.191963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598850+0000","last_change":"2023-09-20T21:46:41.507370+0000","last_active":"2023-09-20T21:47:07.598850+0000","last_peered":"2023-09-20T21:47:07.598850+0000","last_clean":"2023-09-20T21:47:07.598850+0000","last_became_active":"2023-09-20T21:46:41.507035+0000","last_became_peered":"2023-09-20T21:46:41.507035+0000","last_unstale":"2023-09-20T21:47:07.598850+0000","last_undegraded":"2023-09-20T21:47:07.598850+0000","last_fullsized":"2023-09-20T21:47:07.598850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:50.298899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592822+0000","last_change":"2023-09-20T21:46:45.584458+0000","last_active":"2023-09-20T21:47:07.592822+0000","last_peered":"2023-09-20T21:47:07.592822+0000","last_clean":"2023-09-20T21:47:07.592822+0000","last_became_active":"2023-09-20T21:46:45.584033+0000","last_became_peered":"2023-09-20T21:46:45.584033+0000","last_unstale":"2023-09-20T21:47:07.592822+0000","last_undegraded":"2023-09-20T21:47:07.592822+0000","last_fullsized":"2023-09-20T21:47:07.592822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:48:44.653880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":77,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:57.646907+0000","last_change":"2023-09-20T21:46:40.501329+0000","last_active":"2023-09-20T21:47:57.646907+0000","last_peered":"2023-09-20T21:47:57.646907+0000","last_clean":"2023-09-20T21:47:57.646907+0000","last_became_active":"2023-09-20T21:46:40.500999+0000","last_became_peered":"2023-09-20T21:46:40.500999+0000","last_unstale":"2023-09-20T21:47:57.646907+0000","last_undegraded":"2023-09-20T21:47:57.646907+0000","last_fullsized":"2023-09-20T21:47:57.646907+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:38.028398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.602123+0000","last_change":"2023-09-20T21:47:08.602123+0000","last_active":"2023-09-20T21:47:08.602123+0000","last_peered":"2023-09-20T21:47:08.602123+0000","last_clean":"2023-09-20T21:47:08.602123+0000","last_became_active":"2023-09-20T21:47:08.601738+0000","last_became_peered":"2023-09-20T21:47:08.601738+0000","last_unstale":"2023-09-20T21:47:08.602123+0000","last_undegraded":"2023-09-20T21:47:08.602123+0000","last_fullsized":"2023-09-20T21:47:08.602123+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:26:36.710230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602591+0000","last_change":"2023-09-20T21:46:45.587116+0000","last_active":"2023-09-20T21:47:07.602591+0000","last_peered":"2023-09-20T21:47:07.602591+0000","last_clean":"2023-09-20T21:47:07.602591+0000","last_became_active":"2023-09-20T21:46:45.586629+0000","last_became_peered":"2023-09-20T21:46:45.586629+0000","last_unstale":"2023-09-20T21:47:07.602591+0000","last_undegraded":"2023-09-20T21:47:07.602591+0000","last_fullsized":"2023-09-20T21:47:07.602591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:10:33.024235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577220+0000","last_change":"2023-09-20T21:46:40.499985+0000","last_active":"2023-09-20T21:47:08.577220+0000","last_peered":"2023-09-20T21:47:08.577220+0000","last_clean":"2023-09-20T21:47:08.577220+0000","last_became_active":"2023-09-20T21:46:40.499646+0000","last_became_peered":"2023-09-20T21:46:40.499646+0000","last_unstale":"2023-09-20T21:47:08.577220+0000","last_undegraded":"2023-09-20T21:47:08.577220+0000","last_fullsized":"2023-09-20T21:47:08.577220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:52:02.758556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594909+0000","last_change":"2023-09-20T21:46:41.501241+0000","last_active":"2023-09-20T21:47:07.594909+0000","last_peered":"2023-09-20T21:47:07.594909+0000","last_clean":"2023-09-20T21:47:07.594909+0000","last_became_active":"2023-09-20T21:46:41.500902+0000","last_became_peered":"2023-09-20T21:46:41.500902+0000","last_unstale":"2023-09-20T21:47:07.594909+0000","last_undegraded":"2023-09-20T21:47:07.594909+0000","last_fullsized":"2023-09-20T21:47:07.594909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:21:55.210065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594812+0000","last_change":"2023-09-20T21:46:45.583405+0000","last_active":"2023-09-20T21:47:07.594812+0000","last_peered":"2023-09-20T21:47:07.594812+0000","last_clean":"2023-09-20T21:47:07.594812+0000","last_became_active":"2023-09-20T21:46:45.582945+0000","last_became_peered":"2023-09-20T21:46:45.582945+0000","last_unstale":"2023-09-20T21:47:07.594812+0000","last_undegraded":"2023-09-20T21:47:07.594812+0000","last_fullsized":"2023-09-20T21:47:07.594812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:27:14.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.598738+0000","last_change":"2023-09-20T21:46:40.507744+0000","last_active":"2023-09-20T21:47:08.598738+0000","last_peered":"2023-09-20T21:47:08.598738+0000","last_clean":"2023-09-20T21:47:08.598738+0000","last_became_active":"2023-09-20T21:46:40.507454+0000","last_became_peered":"2023-09-20T21:46:40.507454+0000","last_unstale":"2023-09-20T21:47:08.598738+0000","last_undegraded":"2023-09-20T21:47:08.598738+0000","last_fullsized":"2023-09-20T21:47:08.598738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:00:24.492789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598829+0000","last_change":"2023-09-20T21:46:41.504263+0000","last_active":"2023-09-20T21:47:07.598829+0000","last_peered":"2023-09-20T21:47:07.598829+0000","last_clean":"2023-09-20T21:47:07.598829+0000","last_became_active":"2023-09-20T21:46:41.503961+0000","last_became_peered":"2023-09-20T21:46:41.503961+0000","last_unstale":"2023-09-20T21:47:07.598829+0000","last_undegraded":"2023-09-20T21:47:07.598829+0000","last_fullsized":"2023-09-20T21:47:07.598829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:31:58.035517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595426+0000","last_change":"2023-09-20T21:46:45.581343+0000","last_active":"2023-09-20T21:47:07.595426+0000","last_peered":"2023-09-20T21:47:07.595426+0000","last_clean":"2023-09-20T21:47:07.595426+0000","last_became_active":"2023-09-20T21:46:45.580830+0000","last_became_peered":"2023-09-20T21:46:45.580830+0000","last_unstale":"2023-09-20T21:47:07.595426+0000","last_undegraded":"2023-09-20T21:47:07.595426+0000","last_fullsized":"2023-09-20T21:47:07.595426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:20:49.865825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592677+0000","last_change":"2023-09-20T21:46:40.501850+0000","last_active":"2023-09-20T21:47:07.592677+0000","last_peered":"2023-09-20T21:47:07.592677+0000","last_clean":"2023-09-20T21:47:07.592677+0000","last_became_active":"2023-09-20T21:46:40.501500+0000","last_became_peered":"2023-09-20T21:46:40.501500+0000","last_unstale":"2023-09-20T21:47:07.592677+0000","last_undegraded":"2023-09-20T21:47:07.592677+0000","last_fullsized":"2023-09-20T21:47:07.592677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:14:15.907335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595648+0000","last_change":"2023-09-20T21:46:41.507432+0000","last_active":"2023-09-20T21:47:07.595648+0000","last_peered":"2023-09-20T21:47:07.595648+0000","last_clean":"2023-09-20T21:47:07.595648+0000","last_became_active":"2023-09-20T21:46:41.507119+0000","last_became_peered":"2023-09-20T21:46:41.507119+0000","last_unstale":"2023-09-20T21:47:07.595648+0000","last_undegraded":"2023-09-20T21:47:07.595648+0000","last_fullsized":"2023-09-20T21:47:07.595648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:14:10.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602550+0000","last_change":"2023-09-20T21:46:45.588455+0000","last_active":"2023-09-20T21:47:07.602550+0000","last_peered":"2023-09-20T21:47:07.602550+0000","last_clean":"2023-09-20T21:47:07.602550+0000","last_became_active":"2023-09-20T21:46:45.587980+0000","last_became_peered":"2023-09-20T21:46:45.587980+0000","last_unstale":"2023-09-20T21:47:07.602550+0000","last_undegraded":"2023-09-20T21:47:07.602550+0000","last_fullsized":"2023-09-20T21:47:07.602550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:07:35.697330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:55.691657+0000","last_change":"2023-09-20T21:46:40.506260+0000","last_active":"2023-09-20T21:47:55.691657+0000","last_peered":"2023-09-20T21:47:55.691657+0000","last_clean":"2023-09-20T21:47:55.691657+0000","last_became_active":"2023-09-20T21:46:40.505999+0000","last_became_peered":"2023-09-20T21:46:40.505999+0000","last_unstale":"2023-09-20T21:47:55.691657+0000","last_undegraded":"2023-09-20T21:47:55.691657+0000","last_fullsized":"2023-09-20T21:47:55.691657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:39:01.748038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.599481+0000","last_change":"2023-09-20T21:47:08.599481+0000","last_active":"2023-09-20T21:47:08.599481+0000","last_peered":"2023-09-20T21:47:08.599481+0000","last_clean":"2023-09-20T21:47:08.599481+0000","last_became_active":"2023-09-20T21:47:08.599104+0000","last_became_peered":"2023-09-20T21:47:08.599104+0000","last_unstale":"2023-09-20T21:47:08.599481+0000","last_undegraded":"2023-09-20T21:47:08.599481+0000","last_fullsized":"2023-09-20T21:47:08.599481+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:33:56.845446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602710+0000","last_change":"2023-09-20T21:46:45.587410+0000","last_active":"2023-09-20T21:47:07.602710+0000","last_peered":"2023-09-20T21:47:07.602710+0000","last_clean":"2023-09-20T21:47:07.602710+0000","last_became_active":"2023-09-20T21:46:45.586944+0000","last_became_peered":"2023-09-20T21:46:45.586944+0000","last_unstale":"2023-09-20T21:47:07.602710+0000","last_undegraded":"2023-09-20T21:47:07.602710+0000","last_fullsized":"2023-09-20T21:47:07.602710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:57.053765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594876+0000","last_change":"2023-09-20T21:46:41.511526+0000","last_active":"2023-09-20T21:47:07.594876+0000","last_peered":"2023-09-20T21:47:07.594876+0000","last_clean":"2023-09-20T21:47:07.594876+0000","last_became_active":"2023-09-20T21:46:41.511213+0000","last_became_peered":"2023-09-20T21:46:41.511213+0000","last_unstale":"2023-09-20T21:47:07.594876+0000","last_undegraded":"2023-09-20T21:47:07.594876+0000","last_fullsized":"2023-09-20T21:47:07.594876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:29:04.199190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":77,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:58.647594+0000","last_change":"2023-09-20T21:46:40.514916+0000","last_active":"2023-09-20T21:47:58.647594+0000","last_peered":"2023-09-20T21:47:58.647594+0000","last_clean":"2023-09-20T21:47:58.647594+0000","last_became_active":"2023-09-20T21:46:40.514553+0000","last_became_peered":"2023-09-20T21:46:40.514553+0000","last_unstale":"2023-09-20T21:47:58.647594+0000","last_undegraded":"2023-09-20T21:47:58.647594+0000","last_fullsized":"2023-09-20T21:47:58.647594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:12:32.481852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599672+0000","last_change":"2023-09-20T21:46:45.587489+0000","last_active":"2023-09-20T21:47:07.599672+0000","last_peered":"2023-09-20T21:47:07.599672+0000","last_clean":"2023-09-20T21:47:07.599672+0000","last_became_active":"2023-09-20T21:46:45.587022+0000","last_became_peered":"2023-09-20T21:46:45.587022+0000","last_unstale":"2023-09-20T21:47:07.599672+0000","last_undegraded":"2023-09-20T21:47:07.599672+0000","last_fullsized":"2023-09-20T21:47:07.599672+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:09:05.870763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595937+0000","last_change":"2023-09-20T21:46:41.508886+0000","last_active":"2023-09-20T21:47:07.595937+0000","last_peered":"2023-09-20T21:47:07.595937+0000","last_clean":"2023-09-20T21:47:07.595937+0000","last_became_active":"2023-09-20T21:46:41.508470+0000","last_became_peered":"2023-09-20T21:46:41.508470+0000","last_unstale":"2023-09-20T21:47:07.595937+0000","last_undegraded":"2023-09-20T21:47:07.595937+0000","last_fullsized":"2023-09-20T21:47:07.595937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:45:04.888300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.595089+0000","last_change":"2023-09-20T21:46:40.503350+0000","last_active":"2023-09-20T21:47:08.595089+0000","last_peered":"2023-09-20T21:47:08.595089+0000","last_clean":"2023-09-20T21:47:08.595089+0000","last_became_active":"2023-09-20T21:46:40.503045+0000","last_became_peered":"2023-09-20T21:46:40.503045+0000","last_unstale":"2023-09-20T21:47:08.595089+0000","last_undegraded":"2023-09-20T21:47:08.595089+0000","last_fullsized":"2023-09-20T21:47:08.595089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:31:09.004342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":315,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599438+0000","last_change":"2023-09-20T21:45:35.180242+0000","last_active":"2023-09-20T21:47:07.599438+0000","last_peered":"2023-09-20T21:47:07.599438+0000","last_clean":"2023-09-20T21:47:07.599438+0000","last_became_active":"2023-09-20T21:45:35.179705+0000","last_became_peered":"2023-09-20T21:45:35.179705+0000","last_unstale":"2023-09-20T21:47:07.599438+0000","last_undegraded":"2023-09-20T21:47:07.599438+0000","last_fullsized":"2023-09-20T21:47:07.599438+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_clean_scrub_stamp":"2023-09-20T21:42:25.826683+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:02:03.448188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594840+0000","last_change":"2023-09-20T21:46:45.581396+0000","last_active":"2023-09-20T21:47:07.594840+0000","last_peered":"2023-09-20T21:47:07.594840+0000","last_clean":"2023-09-20T21:47:07.594840+0000","last_became_active":"2023-09-20T21:46:45.581018+0000","last_became_peered":"2023-09-20T21:46:45.581018+0000","last_unstale":"2023-09-20T21:47:07.594840+0000","last_undegraded":"2023-09-20T21:47:07.594840+0000","last_fullsized":"2023-09-20T21:47:07.594840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:16:35.934460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.601645+0000","last_change":"2023-09-20T21:46:41.514392+0000","last_active":"2023-09-20T21:47:07.601645+0000","last_peered":"2023-09-20T21:47:07.601645+0000","last_clean":"2023-09-20T21:47:07.601645+0000","last_became_active":"2023-09-20T21:46:41.514090+0000","last_became_peered":"2023-09-20T21:46:41.514090+0000","last_unstale":"2023-09-20T21:47:07.601645+0000","last_undegraded":"2023-09-20T21:47:07.601645+0000","last_fullsized":"2023-09-20T21:47:07.601645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:05:47.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577137+0000","last_change":"2023-09-20T21:46:40.507346+0000","last_active":"2023-09-20T21:47:08.577137+0000","last_peered":"2023-09-20T21:47:08.577137+0000","last_clean":"2023-09-20T21:47:08.577137+0000","last_became_active":"2023-09-20T21:46:40.506969+0000","last_became_peered":"2023-09-20T21:46:40.506969+0000","last_unstale":"2023-09-20T21:47:08.577137+0000","last_undegraded":"2023-09-20T21:47:08.577137+0000","last_fullsized":"2023-09-20T21:47:08.577137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:01:20.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600062+0000","last_change":"2023-09-20T21:46:45.578033+0000","last_active":"2023-09-20T21:47:07.600062+0000","last_peered":"2023-09-20T21:47:07.600062+0000","last_clean":"2023-09-20T21:47:07.600062+0000","last_became_active":"2023-09-20T21:46:45.577540+0000","last_became_peered":"2023-09-20T21:46:45.577540+0000","last_unstale":"2023-09-20T21:47:07.600062+0000","last_undegraded":"2023-09-20T21:47:07.600062+0000","last_fullsized":"2023-09-20T21:47:07.600062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:24:31.020660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577276+0000","last_change":"2023-09-20T21:46:40.510634+0000","last_active":"2023-09-20T21:47:08.577276+0000","last_peered":"2023-09-20T21:47:08.577276+0000","last_clean":"2023-09-20T21:47:08.577276+0000","last_became_active":"2023-09-20T21:46:40.510390+0000","last_became_peered":"2023-09-20T21:46:40.510390+0000","last_unstale":"2023-09-20T21:47:08.577276+0000","last_undegraded":"2023-09-20T21:47:08.577276+0000","last_fullsized":"2023-09-20T21:47:08.577276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:44:49.272987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602533+0000","last_change":"2023-09-20T21:46:41.502981+0000","last_active":"2023-09-20T21:47:07.602533+0000","last_peered":"2023-09-20T21:47:07.602533+0000","last_clean":"2023-09-20T21:47:07.602533+0000","last_became_active":"2023-09-20T21:46:41.502594+0000","last_became_peered":"2023-09-20T21:46:41.502594+0000","last_unstale":"2023-09-20T21:47:07.602533+0000","last_undegraded":"2023-09-20T21:47:07.602533+0000","last_fullsized":"2023-09-20T21:47:07.602533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:41:15.261478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595222+0000","last_change":"2023-09-20T21:46:45.582903+0000","last_active":"2023-09-20T21:47:07.595222+0000","last_peered":"2023-09-20T21:47:07.595222+0000","last_clean":"2023-09-20T21:47:07.595222+0000","last_became_active":"2023-09-20T21:46:45.582433+0000","last_became_peered":"2023-09-20T21:46:45.582433+0000","last_unstale":"2023-09-20T21:47:07.595222+0000","last_undegraded":"2023-09-20T21:47:07.595222+0000","last_fullsized":"2023-09-20T21:47:07.595222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:38:22.174903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592961+0000","last_change":"2023-09-20T21:46:40.507671+0000","last_active":"2023-09-20T21:47:07.592961+0000","last_peered":"2023-09-20T21:47:07.592961+0000","last_clean":"2023-09-20T21:47:07.592961+0000","last_became_active":"2023-09-20T21:46:40.507288+0000","last_became_peered":"2023-09-20T21:46:40.507288+0000","last_unstale":"2023-09-20T21:47:07.592961+0000","last_undegraded":"2023-09-20T21:47:07.592961+0000","last_fullsized":"2023-09-20T21:47:07.592961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:27:02.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596349+0000","last_change":"2023-09-20T21:46:41.507654+0000","last_active":"2023-09-20T21:47:07.596349+0000","last_peered":"2023-09-20T21:47:07.596349+0000","last_clean":"2023-09-20T21:47:07.596349+0000","last_became_active":"2023-09-20T21:46:41.507362+0000","last_became_peered":"2023-09-20T21:46:41.507362+0000","last_unstale":"2023-09-20T21:47:07.596349+0000","last_undegraded":"2023-09-20T21:47:07.596349+0000","last_fullsized":"2023-09-20T21:47:07.596349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:32:08.973379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598688+0000","last_change":"2023-09-20T21:46:45.578437+0000","last_active":"2023-09-20T21:47:07.598688+0000","last_peered":"2023-09-20T21:47:07.598688+0000","last_clean":"2023-09-20T21:47:07.598688+0000","last_became_active":"2023-09-20T21:46:45.578010+0000","last_became_peered":"2023-09-20T21:46:45.578010+0000","last_unstale":"2023-09-20T21:47:07.598688+0000","last_undegraded":"2023-09-20T21:47:07.598688+0000","last_fullsized":"2023-09-20T21:47:07.598688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:59:05.494861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.595756+0000","last_change":"2023-09-20T21:46:40.508937+0000","last_active":"2023-09-20T21:47:08.595756+0000","last_peered":"2023-09-20T21:47:08.595756+0000","last_clean":"2023-09-20T21:47:08.595756+0000","last_became_active":"2023-09-20T21:46:40.508639+0000","last_became_peered":"2023-09-20T21:46:40.508639+0000","last_unstale":"2023-09-20T21:47:08.595756+0000","last_undegraded":"2023-09-20T21:47:08.595756+0000","last_fullsized":"2023-09-20T21:47:08.595756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:50:48.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599421+0000","last_change":"2023-09-20T21:46:41.510702+0000","last_active":"2023-09-20T21:47:07.599421+0000","last_peered":"2023-09-20T21:47:07.599421+0000","last_clean":"2023-09-20T21:47:07.599421+0000","last_became_active":"2023-09-20T21:46:41.510357+0000","last_became_peered":"2023-09-20T21:46:41.510357+0000","last_unstale":"2023-09-20T21:47:07.599421+0000","last_undegraded":"2023-09-20T21:47:07.599421+0000","last_fullsized":"2023-09-20T21:47:07.599421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:35:44.455981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596780+0000","last_change":"2023-09-20T21:46:45.581110+0000","last_active":"2023-09-20T21:47:07.596780+0000","last_peered":"2023-09-20T21:47:07.596780+0000","last_clean":"2023-09-20T21:47:07.596780+0000","last_became_active":"2023-09-20T21:46:45.580498+0000","last_became_peered":"2023-09-20T21:46:45.580498+0000","last_unstale":"2023-09-20T21:47:07.596780+0000","last_undegraded":"2023-09-20T21:47:07.596780+0000","last_fullsized":"2023-09-20T21:47:07.596780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:19:22.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593487+0000","last_change":"2023-09-20T21:46:40.497017+0000","last_active":"2023-09-20T21:47:07.593487+0000","last_peered":"2023-09-20T21:47:07.593487+0000","last_clean":"2023-09-20T21:47:07.593487+0000","last_became_active":"2023-09-20T21:46:40.496722+0000","last_became_peered":"2023-09-20T21:46:40.496722+0000","last_unstale":"2023-09-20T21:47:07.593487+0000","last_undegraded":"2023-09-20T21:47:07.593487+0000","last_fullsized":"2023-09-20T21:47:07.593487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:12:07.017318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596148+0000","last_change":"2023-09-20T21:46:41.508793+0000","last_active":"2023-09-20T21:47:07.596148+0000","last_peered":"2023-09-20T21:47:07.596148+0000","last_clean":"2023-09-20T21:47:07.596148+0000","last_became_active":"2023-09-20T21:46:41.508453+0000","last_became_peered":"2023-09-20T21:46:41.508453+0000","last_unstale":"2023-09-20T21:47:07.596148+0000","last_undegraded":"2023-09-20T21:47:07.596148+0000","last_fullsized":"2023-09-20T21:47:07.596148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:35:12.711570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.592993+0000","last_change":"2023-09-20T21:46:45.583519+0000","last_active":"2023-09-20T21:47:07.592993+0000","last_peered":"2023-09-20T21:47:07.592993+0000","last_clean":"2023-09-20T21:47:07.592993+0000","last_became_active":"2023-09-20T21:46:45.582933+0000","last_became_peered":"2023-09-20T21:46:45.582933+0000","last_unstale":"2023-09-20T21:47:07.592993+0000","last_undegraded":"2023-09-20T21:47:07.592993+0000","last_fullsized":"2023-09-20T21:47:07.592993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:13:20.237865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.578270+0000","last_change":"2023-09-20T21:46:40.507384+0000","last_active":"2023-09-20T21:47:08.578270+0000","last_peered":"2023-09-20T21:47:08.578270+0000","last_clean":"2023-09-20T21:47:08.578270+0000","last_became_active":"2023-09-20T21:46:40.507021+0000","last_became_peered":"2023-09-20T21:46:40.507021+0000","last_unstale":"2023-09-20T21:47:08.578270+0000","last_undegraded":"2023-09-20T21:47:08.578270+0000","last_fullsized":"2023-09-20T21:47:08.578270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:48:04.360664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598093+0000","last_change":"2023-09-20T21:46:41.511537+0000","last_active":"2023-09-20T21:47:07.598093+0000","last_peered":"2023-09-20T21:47:07.598093+0000","last_clean":"2023-09-20T21:47:07.598093+0000","last_became_active":"2023-09-20T21:46:41.511238+0000","last_became_peered":"2023-09-20T21:46:41.511238+0000","last_unstale":"2023-09-20T21:47:07.598093+0000","last_undegraded":"2023-09-20T21:47:07.598093+0000","last_fullsized":"2023-09-20T21:47:07.598093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:01:41.492824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596769+0000","last_change":"2023-09-20T21:46:45.585750+0000","last_active":"2023-09-20T21:47:07.596769+0000","last_peered":"2023-09-20T21:47:07.596769+0000","last_clean":"2023-09-20T21:47:07.596769+0000","last_became_active":"2023-09-20T21:46:45.585213+0000","last_became_peered":"2023-09-20T21:46:45.585213+0000","last_unstale":"2023-09-20T21:47:07.596769+0000","last_undegraded":"2023-09-20T21:47:07.596769+0000","last_fullsized":"2023-09-20T21:47:07.596769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:31:15.650766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.594299+0000","last_change":"2023-09-20T21:46:40.506394+0000","last_active":"2023-09-20T21:47:08.594299+0000","last_peered":"2023-09-20T21:47:08.594299+0000","last_clean":"2023-09-20T21:47:08.594299+0000","last_became_active":"2023-09-20T21:46:40.506085+0000","last_became_peered":"2023-09-20T21:46:40.506085+0000","last_unstale":"2023-09-20T21:47:08.594299+0000","last_undegraded":"2023-09-20T21:47:08.594299+0000","last_fullsized":"2023-09-20T21:47:08.594299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:02.738600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596347+0000","last_change":"2023-09-20T21:46:41.509491+0000","last_active":"2023-09-20T21:47:07.596347+0000","last_peered":"2023-09-20T21:47:07.596347+0000","last_clean":"2023-09-20T21:47:07.596347+0000","last_became_active":"2023-09-20T21:46:41.509039+0000","last_became_peered":"2023-09-20T21:46:41.509039+0000","last_unstale":"2023-09-20T21:47:07.596347+0000","last_undegraded":"2023-09-20T21:47:07.596347+0000","last_fullsized":"2023-09-20T21:47:07.596347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:57:39.003154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.597192+0000","last_change":"2023-09-20T21:46:45.581651+0000","last_active":"2023-09-20T21:47:07.597192+0000","last_peered":"2023-09-20T21:47:07.597192+0000","last_clean":"2023-09-20T21:47:07.597192+0000","last_became_active":"2023-09-20T21:46:45.581224+0000","last_became_peered":"2023-09-20T21:46:45.581224+0000","last_unstale":"2023-09-20T21:47:07.597192+0000","last_undegraded":"2023-09-20T21:47:07.597192+0000","last_fullsized":"2023-09-20T21:47:07.597192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:32:17.778301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.610835+0000","last_change":"2023-09-20T21:47:08.610413+0000","last_active":"2023-09-20T21:47:08.610835+0000","last_peered":"2023-09-20T21:47:08.610835+0000","last_clean":"2023-09-20T21:47:08.610835+0000","last_became_active":"2023-09-20T21:47:08.603734+0000","last_became_peered":"2023-09-20T21:47:08.603734+0000","last_unstale":"2023-09-20T21:47:08.610835+0000","last_undegraded":"2023-09-20T21:47:08.610835+0000","last_fullsized":"2023-09-20T21:47:08.610835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:43:42.569371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598787+0000","last_change":"2023-09-20T21:46:41.507218+0000","last_active":"2023-09-20T21:47:07.598787+0000","last_peered":"2023-09-20T21:47:07.598787+0000","last_clean":"2023-09-20T21:47:07.598787+0000","last_became_active":"2023-09-20T21:46:41.506889+0000","last_became_peered":"2023-09-20T21:46:41.506889+0000","last_unstale":"2023-09-20T21:47:07.598787+0000","last_undegraded":"2023-09-20T21:47:07.598787+0000","last_fullsized":"2023-09-20T21:47:07.598787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:08:40.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593769+0000","last_change":"2023-09-20T21:46:45.579269+0000","last_active":"2023-09-20T21:47:07.593769+0000","last_peered":"2023-09-20T21:47:07.593769+0000","last_clean":"2023-09-20T21:47:07.593769+0000","last_became_active":"2023-09-20T21:46:45.578541+0000","last_became_peered":"2023-09-20T21:46:45.578541+0000","last_unstale":"2023-09-20T21:47:07.593769+0000","last_undegraded":"2023-09-20T21:47:07.593769+0000","last_fullsized":"2023-09-20T21:47:07.593769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:07:50.241947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-09-20T21:47:59.739738+0000","last_change":"2023-09-20T21:46:40.508774+0000","last_active":"2023-09-20T21:47:59.739738+0000","last_peered":"2023-09-20T21:47:59.739738+0000","last_clean":"2023-09-20T21:47:59.739738+0000","last_became_active":"2023-09-20T21:46:40.508480+0000","last_became_peered":"2023-09-20T21:46:40.508480+0000","last_unstale":"2023-09-20T21:47:59.739738+0000","last_undegraded":"2023-09-20T21:47:59.739738+0000","last_fullsized":"2023-09-20T21:47:59.739738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:22.643724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599875+0000","last_change":"2023-09-20T21:46:41.503763+0000","last_active":"2023-09-20T21:47:07.599875+0000","last_peered":"2023-09-20T21:47:07.599875+0000","last_clean":"2023-09-20T21:47:07.599875+0000","last_became_active":"2023-09-20T21:46:41.503360+0000","last_became_peered":"2023-09-20T21:46:41.503360+0000","last_unstale":"2023-09-20T21:47:07.599875+0000","last_undegraded":"2023-09-20T21:47:07.599875+0000","last_fullsized":"2023-09-20T21:47:07.599875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:03:00.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598933+0000","last_change":"2023-09-20T21:46:45.586702+0000","last_active":"2023-09-20T21:47:07.598933+0000","last_peered":"2023-09-20T21:47:07.598933+0000","last_clean":"2023-09-20T21:47:07.598933+0000","last_became_active":"2023-09-20T21:46:45.586177+0000","last_became_peered":"2023-09-20T21:46:45.586177+0000","last_unstale":"2023-09-20T21:47:07.598933+0000","last_undegraded":"2023-09-20T21:47:07.598933+0000","last_fullsized":"2023-09-20T21:47:07.598933+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:19:03.883440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.576396+0000","last_change":"2023-09-20T21:46:40.507270+0000","last_active":"2023-09-20T21:47:08.576396+0000","last_peered":"2023-09-20T21:47:08.576396+0000","last_clean":"2023-09-20T21:47:08.576396+0000","last_became_active":"2023-09-20T21:46:40.506933+0000","last_became_peered":"2023-09-20T21:46:40.506933+0000","last_unstale":"2023-09-20T21:47:08.576396+0000","last_undegraded":"2023-09-20T21:47:08.576396+0000","last_fullsized":"2023-09-20T21:47:08.576396+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:59.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599660+0000","last_change":"2023-09-20T21:46:41.507868+0000","last_active":"2023-09-20T21:47:07.599660+0000","last_peered":"2023-09-20T21:47:07.599660+0000","last_clean":"2023-09-20T21:47:07.599660+0000","last_became_active":"2023-09-20T21:46:41.507568+0000","last_became_peered":"2023-09-20T21:46:41.507568+0000","last_unstale":"2023-09-20T21:47:07.599660+0000","last_undegraded":"2023-09-20T21:47:07.599660+0000","last_fullsized":"2023-09-20T21:47:07.599660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:44.615620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594494+0000","last_change":"2023-09-20T21:46:45.578726+0000","last_active":"2023-09-20T21:47:07.594494+0000","last_peered":"2023-09-20T21:47:07.594494+0000","last_clean":"2023-09-20T21:47:07.594494+0000","last_became_active":"2023-09-20T21:46:45.578326+0000","last_became_peered":"2023-09-20T21:46:45.578326+0000","last_unstale":"2023-09-20T21:47:07.594494+0000","last_undegraded":"2023-09-20T21:47:07.594494+0000","last_fullsized":"2023-09-20T21:47:07.594494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:21:34.564840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:08.577629+0000","last_change":"2023-09-20T21:46:40.509024+0000","last_active":"2023-09-20T21:47:08.577629+0000","last_peered":"2023-09-20T21:47:08.577629+0000","last_clean":"2023-09-20T21:47:08.577629+0000","last_became_active":"2023-09-20T21:46:40.508659+0000","last_became_peered":"2023-09-20T21:46:40.508659+0000","last_unstale":"2023-09-20T21:47:08.577629+0000","last_undegraded":"2023-09-20T21:47:08.577629+0000","last_fullsized":"2023-09-20T21:47:08.577629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:12:08.302068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599976+0000","last_change":"2023-09-20T21:46:41.510307+0000","last_active":"2023-09-20T21:47:07.599976+0000","last_peered":"2023-09-20T21:47:07.599976+0000","last_clean":"2023-09-20T21:47:07.599976+0000","last_became_active":"2023-09-20T21:46:41.510014+0000","last_became_peered":"2023-09-20T21:46:41.510014+0000","last_unstale":"2023-09-20T21:47:07.599976+0000","last_undegraded":"2023-09-20T21:47:07.599976+0000","last_fullsized":"2023-09-20T21:47:07.599976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:05:55.062515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598156+0000","last_change":"2023-09-20T21:46:41.511525+0000","last_active":"2023-09-20T21:47:07.598156+0000","last_peered":"2023-09-20T21:47:07.598156+0000","last_clean":"2023-09-20T21:47:07.598156+0000","last_became_active":"2023-09-20T21:46:41.511203+0000","last_became_peered":"2023-09-20T21:46:41.511203+0000","last_unstale":"2023-09-20T21:47:07.598156+0000","last_undegraded":"2023-09-20T21:47:07.598156+0000","last_fullsized":"2023-09-20T21:47:07.598156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:42:15.757422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599580+0000","last_change":"2023-09-20T21:46:41.502936+0000","last_active":"2023-09-20T21:47:07.599580+0000","last_peered":"2023-09-20T21:47:07.599580+0000","last_clean":"2023-09-20T21:47:07.599580+0000","last_became_active":"2023-09-20T21:46:41.502617+0000","last_became_peered":"2023-09-20T21:46:41.502617+0000","last_unstale":"2023-09-20T21:47:07.599580+0000","last_undegraded":"2023-09-20T21:47:07.599580+0000","last_fullsized":"2023-09-20T21:47:07.599580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:45:21.197652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599689+0000","last_change":"2023-09-20T21:46:41.510766+0000","last_active":"2023-09-20T21:47:07.599689+0000","last_peered":"2023-09-20T21:47:07.599689+0000","last_clean":"2023-09-20T21:47:07.599689+0000","last_became_active":"2023-09-20T21:46:41.510433+0000","last_became_peered":"2023-09-20T21:46:41.510433+0000","last_unstale":"2023-09-20T21:47:07.599689+0000","last_undegraded":"2023-09-20T21:47:07.599689+0000","last_fullsized":"2023-09-20T21:47:07.599689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:56:12.085926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.599196+0000","last_change":"2023-09-20T21:46:41.508846+0000","last_active":"2023-09-20T21:47:07.599196+0000","last_peered":"2023-09-20T21:47:07.599196+0000","last_clean":"2023-09-20T21:47:07.599196+0000","last_became_active":"2023-09-20T21:46:41.508518+0000","last_became_peered":"2023-09-20T21:46:41.508518+0000","last_unstale":"2023-09-20T21:47:07.599196+0000","last_undegraded":"2023-09-20T21:47:07.599196+0000","last_fullsized":"2023-09-20T21:47:07.599196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:01:58.195327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602625+0000","last_change":"2023-09-20T21:46:41.505929+0000","last_active":"2023-09-20T21:47:07.602625+0000","last_peered":"2023-09-20T21:47:07.602625+0000","last_clean":"2023-09-20T21:47:07.602625+0000","last_became_active":"2023-09-20T21:46:41.505542+0000","last_became_peered":"2023-09-20T21:46:41.505542+0000","last_unstale":"2023-09-20T21:47:07.602625+0000","last_undegraded":"2023-09-20T21:47:07.602625+0000","last_fullsized":"2023-09-20T21:47:07.602625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:24:47.520344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598727+0000","last_change":"2023-09-20T21:46:41.507603+0000","last_active":"2023-09-20T21:47:07.598727+0000","last_peered":"2023-09-20T21:47:07.598727+0000","last_clean":"2023-09-20T21:47:07.598727+0000","last_became_active":"2023-09-20T21:46:41.507340+0000","last_became_peered":"2023-09-20T21:46:41.507340+0000","last_unstale":"2023-09-20T21:47:07.598727+0000","last_undegraded":"2023-09-20T21:47:07.598727+0000","last_fullsized":"2023-09-20T21:47:07.598727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:50:19.756491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.600180+0000","last_change":"2023-09-20T21:46:41.509625+0000","last_active":"2023-09-20T21:47:07.600180+0000","last_peered":"2023-09-20T21:47:07.600180+0000","last_clean":"2023-09-20T21:47:07.600180+0000","last_became_active":"2023-09-20T21:46:41.509301+0000","last_became_peered":"2023-09-20T21:46:41.509301+0000","last_unstale":"2023-09-20T21:47:07.600180+0000","last_undegraded":"2023-09-20T21:47:07.600180+0000","last_fullsized":"2023-09-20T21:47:07.600180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:15:03.108060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.596444+0000","last_change":"2023-09-20T21:46:41.516781+0000","last_active":"2023-09-20T21:47:07.596444+0000","last_peered":"2023-09-20T21:47:07.596444+0000","last_clean":"2023-09-20T21:47:07.596444+0000","last_became_active":"2023-09-20T21:46:41.516496+0000","last_became_peered":"2023-09-20T21:46:41.516496+0000","last_unstale":"2023-09-20T21:47:07.596444+0000","last_undegraded":"2023-09-20T21:47:07.596444+0000","last_fullsized":"2023-09-20T21:47:07.596444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:17:26.492851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.594775+0000","last_change":"2023-09-20T21:46:41.499927+0000","last_active":"2023-09-20T21:47:07.594775+0000","last_peered":"2023-09-20T21:47:07.594775+0000","last_clean":"2023-09-20T21:47:07.594775+0000","last_became_active":"2023-09-20T21:46:41.499605+0000","last_became_peered":"2023-09-20T21:46:41.499605+0000","last_unstale":"2023-09-20T21:47:07.594775+0000","last_undegraded":"2023-09-20T21:47:07.594775+0000","last_fullsized":"2023-09-20T21:47:07.594775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:18:16.306349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.595650+0000","last_change":"2023-09-20T21:46:41.515842+0000","last_active":"2023-09-20T21:47:07.595650+0000","last_peered":"2023-09-20T21:47:07.595650+0000","last_clean":"2023-09-20T21:47:07.595650+0000","last_became_active":"2023-09-20T21:46:41.515600+0000","last_became_peered":"2023-09-20T21:46:41.515600+0000","last_unstale":"2023-09-20T21:47:07.595650+0000","last_undegraded":"2023-09-20T21:47:07.595650+0000","last_fullsized":"2023-09-20T21:47:07.595650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:50:58.088103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.593931+0000","last_change":"2023-09-20T21:46:41.510695+0000","last_active":"2023-09-20T21:47:07.593931+0000","last_peered":"2023-09-20T21:47:07.593931+0000","last_clean":"2023-09-20T21:47:07.593931+0000","last_became_active":"2023-09-20T21:46:41.510345+0000","last_became_peered":"2023-09-20T21:46:41.510345+0000","last_unstale":"2023-09-20T21:47:07.593931+0000","last_undegraded":"2023-09-20T21:47:07.593931+0000","last_fullsized":"2023-09-20T21:47:07.593931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:38:24.201382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.602278+0000","last_change":"2023-09-20T21:46:41.507809+0000","last_active":"2023-09-20T21:47:07.602278+0000","last_peered":"2023-09-20T21:47:07.602278+0000","last_clean":"2023-09-20T21:47:07.602278+0000","last_became_active":"2023-09-20T21:46:41.507513+0000","last_became_peered":"2023-09-20T21:46:41.507513+0000","last_unstale":"2023-09-20T21:47:07.602278+0000","last_undegraded":"2023-09-20T21:47:07.602278+0000","last_fullsized":"2023-09-20T21:47:07.602278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:30:43.132728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-09-20T21:47:07.598414+0000","last_change":"2023-09-20T21:46:41.503590+0000","last_active":"2023-09-20T21:47:07.598414+0000","last_peered":"2023-09-20T21:47:07.598414+0000","last_clean":"2023-09-20T21:47:07.598414+0000","last_became_active":"2023-09-20T21:46:41.503295+0000","last_became_peered":"2023-09-20T21:46:41.503295+0000","last_unstale":"2023-09-20T21:47:07.598414+0000","last_undegraded":"2023-09-20T21:47:07.598414+0000","last_fullsized":"2023-09-20T21:47:07.598414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:28:33.088873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10264,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":532,"num_read_kb":281,"num_write":209,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":30792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":496747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.503}]},{"osd":1,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56599999999999995}]},{"osd":2,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.373},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.42799999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.60499999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.621}]},{"osd":7,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.75},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.70399999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65100000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.80400000000000005}]},{"osd":10,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.73199999999999998}]}]},{"osd":10,"up_from":72,"seq":309237645343,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1662976,"data_stored":1091501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:40 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":7.9690000000000003,"5min":7.9690000000000003,"15min":7.9690000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":5.6280000000000001,"5min":5.6280000000000001,"15min":5.6280000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47399999999999998,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.59599999999999997,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.46999999999999997},"min":{"1min":0.35299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.48099999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.48999999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79200000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67900000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68799999999999994,"15min":0.68700000000000006},"min":{"1min":0.42199999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54500000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.71299999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.45200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82899999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.437,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65500000000000003}]},{"osd":5,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67800000000000005,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91800000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.47199999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84399999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72899999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.51700000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84299999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.345,"5min":0.90200000000000002,"15min":0.82799999999999996},"min":{"1min":0.69899999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":8.0440000000000005,"5min":8.0440000000000005,"15min":8.0440000000000005},"last":0.77100000000000002}]},{"osd":7,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.752,"15min":0.755},"min":{"1min":0.58699999999999997,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.90300000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.77100000000000002,"15min":0.78100000000000003},"min":{"1min":0.58099999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94499999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.80700000000000005}]},{"osd":9,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":0.90400000000000003,"15min":0.86199999999999999},"min":{"1min":0.65300000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":5.6449999999999996,"5min":5.6449999999999996,"15min":5.6449999999999996},"last":0.88},{"interface":"front","average":{"1min":1.123,"5min":0.90800000000000003,"15min":0.872},"min":{"1min":0.63200000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":5.6669999999999998,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":0.86799999999999999}]},{"osd":11,"last update":"Wed Sep 20 21:46:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.83899999999999997}]}]},{"osd":9,"up_from":67,"seq":287762808868,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":496677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":0.51900000000000002}]},{"osd":1,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6799999999999997,"5min":4.6799999999999997,"15min":4.6799999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.64300000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82499999999999996,"15min":0.83299999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.86399999999999999}]},{"osd":3,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.84199999999999997,"15min":0.85099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.67600000000000005}]},{"osd":4,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.96899999999999997,"15min":0.94099999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":6.3449999999999998,"5min":6.3449999999999998,"15min":6.3449999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.875,"15min":0.871},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":4.7279999999999998,"5min":4.7279999999999998,"15min":4.7279999999999998},"last":0.77600000000000002}]},{"osd":5,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.97699999999999998,"15min":0.999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.6469999999999998,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":4.7679999999999998,"5min":4.7679999999999998,"15min":4.7679999999999998},"last":0.88600000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.97399999999999998,"15min":0.97099999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.806,"5min":4.806,"15min":4.806},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":1.0089999999999999,"15min":1.0329999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.79600000000000004}]},{"osd":8,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.103,"15min":1.1319999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.048,"5min":1.0660000000000001,"15min":1.069},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":4.843,"5min":4.843,"15min":4.843},"last":0.93500000000000005}]},{"osd":10,"last update":"Wed Sep 20 21:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":1.0660000000000001,"15min":1.095},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.835,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.113,"15min":1.1399999999999999},"min":{"1min":0.78600000000000003,"5min":0.623,"15min":0.623},"max":{"1min":1.867,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.748}]},{"osd":11,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.95799999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070505,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":497055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":0.755}]},{"osd":1,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.497,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.48099999999999998,"15min":0.46300000000000002},"min":{"1min":0.38300000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.58199999999999996}]},{"osd":3,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.53900000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.73099999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.67500000000000004,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.871}]},{"osd":5,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.93400000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.85599999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.76600000000000001,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.78400000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.57699999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0109999999999999,"5min":1.548,"15min":1.548},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.57499999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.95399999999999996}]},{"osd":9,"last update":"Wed Sep 20 21:47:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76100000000000001,"15min":0.75700000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.03,"5min":1.073,"15min":1.073},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78000000000000003,"15min":0.77300000000000002},"min":{"1min":0.621,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.012}]},{"osd":10,"last update":"Wed Sep 20 21:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.748,"15min":0.73399999999999999},"min":{"1min":0.55600000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77300000000000002,"15min":0.76000000000000001},"min":{"1min":0.624,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.044}]},{"osd":11,"last update":"Wed Sep 20 21:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.93100000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1638400,"data_stored":1089065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.42599999999999999,"15min":0.41999999999999998},"min":{"1min":0.29599999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56499999999999995,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.437},"min":{"1min":0.33200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.67900000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.37}]},{"osd":2,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.45200000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.747,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58899999999999997,"15min":0.57399999999999995},"min":{"1min":0.49299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65200000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61399999999999999,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.874,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61799999999999999,"15min":0.59699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80700000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.45300000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72299999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77600000000000002,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.442}]},{"osd":5,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.629,"15min":0.61899999999999999},"min":{"1min":0.52300000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83099999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.65400000000000003,"15min":0.63100000000000001},"min":{"1min":0.56200000000000006,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91300000000000003,"5min":1.006,"15min":1.006},"last":0.80300000000000005}]},{"osd":6,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.437,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66800000000000004,"15min":0.66100000000000003},"min":{"1min":0.495,"5min":0.435,"15min":0.435},"max":{"1min":0.85799999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.54700000000000004}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.60699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91900000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.68400000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91400000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.84099999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:47:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.76000000000000001},"min":{"1min":0.59099999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.746}]},{"osd":10,"last update":"Wed Sep 20 21:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.628,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.67400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93300000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.70099999999999996}]},{"osd":11,"last update":"Wed Sep 20 21:47:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.82099999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":496277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":1.018,"15min":1.018},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.64500000000000002,"15min":0.622},"min":{"1min":0.40500000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89700000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":1.2,"15min":1.2},"last":0.57399999999999995}]},{"osd":2,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.49099999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.57199999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0900000000000001,"5min":1.466,"15min":1.466},"last":0.59799999999999998}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.34699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.999,"5min":1.012,"15min":1.012},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69999999999999996,"15min":0.68700000000000006},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.175,"5min":1.179,"15min":1.179},"last":0.77200000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.72799999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0640000000000001,"5min":1.093,"15min":1.093},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78700000000000003,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.022,"5min":1.407,"15min":1.407},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0549999999999999,"5min":1.266,"15min":1.266},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.51500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.046,"5min":1.073,"15min":1.073},"last":0.75900000000000001}]},{"osd":7,"last update":"Wed Sep 20 21:47:24 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.85699999999999998,"15min":0.83999999999999997},"min":{"1min":0.64700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.873},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86799999999999999,"15min":0.84899999999999998},"min":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.91200000000000003}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.82599999999999996,"15min":0.85299999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.55900000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.206,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.99099999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.94099999999999995,"15min":0.95199999999999996},"min":{"1min":0.64800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.97599999999999998,"15min":0.98799999999999999},"min":{"1min":0.66600000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3260000000000001,"5min":1.333,"15min":1.333},"last":0.96799999999999997}]},{"osd":10,"last update":"Wed Sep 20 21:47:36 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.74299999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.98099999999999998,"15min":0.98299999999999998},"min":{"1min":0.70699999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1}]},{"osd":11,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.028}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":501149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90200000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.33400000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.77900000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.432}]},{"osd":2,"last update":"Wed Sep 20 21:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56699999999999995,"15min":0.53300000000000003},"min":{"1min":0.41599999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80000000000000004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.374},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.50800000000000001},"min":{"1min":0.48499999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77700000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:47:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.56399999999999995,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.56699999999999995,"15min":0.53400000000000003},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.41899999999999998,"15min":0.39300000000000002},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.60299999999999998,"15min":0.57899999999999996},"min":{"1min":0.50600000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83499999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63200000000000001,"15min":0.59999999999999998},"min":{"1min":0.56299999999999994,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88700000000000001,"5min":1.01,"15min":1.01},"last":0.58999999999999997}]},{"osd":6,"last update":"Wed Sep 20 21:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67900000000000005,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70799999999999996,"15min":0.68400000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed Sep 20 21:47:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.57699999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90200000000000002,"5min":1.218,"15min":1.218},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67300000000000004}]},{"osd":8,"last update":"Wed Sep 20 21:47:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.321,"15min":1.321},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.622}]},{"osd":9,"last update":"Wed Sep 20 21:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77200000000000002,"15min":0.77000000000000002},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.93000000000000005,"5min":1.054,"15min":1.054},"last":0.78300000000000003}]},{"osd":10,"last update":"Wed Sep 20 21:47:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.53500000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.72999999999999998,"15min":0.72599999999999998},"min":{"1min":0.437,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.751}]},{"osd":11,"last update":"Wed Sep 20 21:47:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68799999999999994}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":497733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 20 21:47:10 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61599999999999999,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77800000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.64600000000000002},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Wed Sep 20 21:47:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":0.71599999999999997,"15min":0.65000000000000002},"min":{"1min":0.52100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":8.1639999999999997,"5min":8.1639999999999997,"15min":8.1639999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.72799999999999998,"15min":0.67200000000000004},"min":{"1min":0.55700000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":6.5270000000000001,"5min":6.5270000000000001,"15min":6.5270000000000001},"last":0.52100000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":8.9499999999999993,"5min":8.9499999999999993,"15min":8.9499999999999993},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":8.6620000000000008,"5min":8.6620000000000008,"15min":8.6620000000000008},"last":0.55400000000000005}]},{"osd":4,"last update":"Wed Sep 20 21:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66600000000000004,"15min":0.64200000000000002},"min":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76700000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.624},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70399999999999996,"15min":0.63700000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.76900000000000002}]},{"osd":5,"last update":"Wed Sep 20 21:47:41 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":0.82999999999999996,"15min":0.72899999999999998},"min":{"1min":0.52200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":8.2919999999999998,"5min":8.2919999999999998,"15min":8.2919999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.091,"5min":0.78300000000000003,"15min":0.70099999999999996},"min":{"1min":0.55800000000000005,"5min":0.434,"15min":0.434},"max":{"1min":6.601,"5min":6.601,"15min":6.601},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:47:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.72199999999999998,"15min":0.67900000000000005},"min":{"1min":0.39700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94499999999999995,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":6.6440000000000001,"5min":6.6440000000000001,"15min":6.6440000000000001},"last":0.747},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":8.2159999999999993,"5min":8.2159999999999993,"15min":8.2159999999999993},"last":0.73499999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":0.93999999999999995,"15min":0.86799999999999999},"min":{"1min":0.628,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":8.3290000000000006,"5min":8.3290000000000006,"15min":8.3290000000000006},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.1990000000000001,"5min":0.96099999999999997,"15min":0.93500000000000005},"min":{"1min":0.79600000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":6.6799999999999997,"5min":6.6799999999999997,"15min":6.6799999999999997},"last":0.80200000000000005}]},{"osd":9,"last update":"Wed Sep 20 21:47:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.92800000000000005,"15min":0.93400000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.874},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.78700000000000003,"15min":0.80000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0089999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.755}]},{"osd":10,"last update":"Wed Sep 20 21:47:32 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.01,"15min":0.96399999999999997},"min":{"1min":0.81599999999999995,"5min":0.629,"15min":0.629},"max":{"1min":6.7149999999999999,"5min":6.7149999999999999,"15min":6.7149999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.159,"5min":0.876,"15min":0.82899999999999996},"min":{"1min":0.627,"5min":0.498,"15min":0.498},"max":{"1min":6.8520000000000003,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":0.86099999999999999}]},{"osd":11,"last update":"Wed Sep 20 21:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.91400000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":501369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.58799999999999997,"15min":0.55400000000000005},"min":{"1min":0.35399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":4.0890000000000004,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.373},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.63300000000000001,"15min":0.59299999999999997},"min":{"1min":0.40500000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.72999999999999998}]},{"osd":1,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.628,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":7.0330000000000004,"5min":7.0330000000000004,"15min":7.0330000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.55300000000000005,"15min":0.51800000000000002},"min":{"1min":0.45700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed Sep 20 21:47:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84599999999999997,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66800000000000004,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90900000000000003,"5min":3.645,"15min":3.645},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Sep 20 21:47:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.29,"15min":1.29},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":9.8659999999999997,"5min":9.8659999999999997,"15min":9.8659999999999997},"last":0.63100000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:47:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.41199999999999998},"min":{"1min":0.36199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.92100000000000004,"5min":3.798,"15min":3.798},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.502},"min":{"1min":0.40600000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88,"5min":7.1500000000000004,"15min":7.1500000000000004},"last":0.40300000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.69499999999999995,"15min":0.63600000000000001},"min":{"1min":0.497,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":7.367,"5min":7.367,"15min":7.367},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.72699999999999998,"15min":0.66500000000000004},"min":{"1min":0.48099999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":6.4649999999999999,"5min":6.4649999999999999,"15min":6.4649999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Sep 20 21:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66700000000000004}]},{"osd":8,"last update":"Wed Sep 20 21:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.59799999999999998,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.503},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.626},"min":{"1min":0.498,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.85199999999999998,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.69399999999999995}]},{"osd":9,"last update":"Wed Sep 20 21:47:11 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.77000000000000002,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":7.1200000000000001,"5min":7.1200000000000001,"15min":7.1200000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.80500000000000005,"15min":0.77700000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.7690000000000001,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.754}]},{"osd":10,"last update":"Wed Sep 20 21:47:34 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.78000000000000003,"15min":0.71499999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":7.1799999999999997,"5min":7.1799999999999997,"15min":7.1799999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.72699999999999998,"15min":0.70699999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.71899999999999997}]},{"osd":11,"last update":"Wed Sep 20 21:47:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":7.2110000000000003,"5min":7.2110000000000003,"15min":7.2110000000000003},"last":0.81299999999999994}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":495985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.432,"15min":0.39000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81699999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.45900000000000002,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.38}]},{"osd":1,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60399999999999998,"15min":0.56699999999999995},"min":{"1min":0.41699999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.98599999999999999,"5min":1,"15min":1},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58499999999999996,"15min":0.55700000000000005},"min":{"1min":0.435,"5min":0.37,"15min":0.37},"max":{"1min":0.84899999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Sep 20 21:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63900000000000001,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63300000000000001,"15min":0.57699999999999996},"min":{"1min":0.45400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:47:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66400000000000003,"15min":0.63},"min":{"1min":0.47399999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.752},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72599999999999998,"15min":0.65800000000000003},"min":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.9009999999999998,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.89200000000000002}]},{"osd":5,"last update":"Wed Sep 20 21:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70899999999999996,"15min":0.69499999999999995},"min":{"1min":0.50900000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.625},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76700000000000002,"15min":0.749},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98599999999999999,"5min":1.167,"15min":1.167},"last":0.52700000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.51200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.018,"5min":1.147,"15min":1.147},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.018,"5min":1.296,"15min":1.296},"last":0.81899999999999995}]},{"osd":7,"last update":"Wed Sep 20 21:47:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78300000000000003,"15min":0.77700000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.143,"5min":1.248,"15min":1.248},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.82899999999999996,"15min":0.81899999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.78500000000000003}]},{"osd":8,"last update":"Wed Sep 20 21:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:47:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.92900000000000005,"15min":0.94899999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.093,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.92000000000000004,"15min":0.93799999999999994},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.204,"5min":1.361,"15min":1.361},"last":0.54900000000000004}]},{"osd":10,"last update":"Wed Sep 20 21:47:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.90600000000000003,"15min":0.91500000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.141,"5min":1.228,"15min":1.228},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.93200000000000005,"15min":0.92700000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.0600000000000001,"5min":3.0600000000000001,"15min":3.0600000000000001},"last":0.85599999999999998}]},{"osd":11,"last update":"Wed Sep 20 21:47:44 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.73799999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":496255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.39600000000000002,"15min":0.36399999999999999},"min":{"1min":0.34200000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.39900000000000002,"15min":0.375},"min":{"1min":0.29499999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45100000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56499999999999995,"15min":0.52700000000000002},"min":{"1min":0.35799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.41999999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57699999999999996}]},{"osd":3,"last update":"Wed Sep 20 21:47:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57099999999999995,"15min":0.55800000000000005},"min":{"1min":0.41199999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59899999999999998,"15min":0.57399999999999995},"min":{"1min":0.443,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90900000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88}]},{"osd":5,"last update":"Wed Sep 20 21:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83199999999999996,"5min":0.871,"15min":0.871},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60499999999999998,"15min":0.59299999999999997},"min":{"1min":0.497,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:47:07 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.57199999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67000000000000004,"15min":0.64700000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.66600000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:47:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.40300000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.628},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59599999999999997,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60399999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.54200000000000004,"5min":0.371,"15min":0.371},"max":{"1min":0.89500000000000002,"5min":1,"15min":1},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84399999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:47:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.79200000000000004,"15min":0.79100000000000004},"min":{"1min":0.63300000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0820000000000001,"5min":1.087,"15min":1.087},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.68999999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:47:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.75,"15min":0.751},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.54700000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.82999999999999996}]},{"osd":11,"last update":"Wed Sep 20 21:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.81399999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1646592,"data_stored":1088462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.42699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78000000000000003,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80800000000000005,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.43099999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55800000000000005,"15min":0.53800000000000003},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.77400000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.52800000000000002},"min":{"1min":0.45400000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73299999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.72199999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.375},"min":{"1min":0.34899999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.39500000000000002},"min":{"1min":0.36099999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.66400000000000003,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.45000000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88900000000000001,"5min":1.615,"15min":1.615},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.44700000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79300000000000004,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.68500000000000005}]},{"osd":4,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60699999999999998,"15min":0.56899999999999995},"min":{"1min":0.57299999999999995,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73999999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77500000000000002,"5min":1.008,"15min":1.008},"last":0.59999999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65800000000000003,"15min":0.65200000000000002},"min":{"1min":0.53900000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80400000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63500000000000001,"15min":0.621},"min":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.76100000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:47:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.57599999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84899999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.46999999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88600000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61799999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":0.879,"15min":0.879},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.51800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.78000000000000003}]},{"osd":9,"last update":"Wed Sep 20 21:47:14 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.79100000000000004,"15min":0.79600000000000004},"min":{"1min":0.61099999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.93300000000000005,"5min":1.02,"15min":1.02},"last":0.75},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.751,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.84099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:47:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.745,"15min":0.73499999999999999},"min":{"1min":0.68899999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99399999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.70599999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92100000000000004,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-20T21:48:02.562 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-09-20T21:48:02.562 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-20T21:48:02.562 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-09-20T21:48:02.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph health --format=json 2023-09-20T21:48:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:02 smithi123 ceph-mon[134433]: pgmap v318: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:02.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:02 smithi089 ceph-mon[134090]: pgmap v318: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:03.301 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:48:03.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:03 smithi022 ceph-mon[97651]: from='client.25013 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:03.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:03 smithi123 ceph-mon[134433]: from='client.25013 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:03.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:03 smithi089 ceph-mon[134090]: from='client.25013 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:04.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:04 smithi022 ceph-mon[97651]: pgmap v319: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:48:04.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:04 smithi123 ceph-mon[134433]: pgmap v319: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:48:04.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:04 smithi089 ceph-mon[134090]: pgmap v319: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:48:04.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.953+0000 7f25f2950700 1 -- 172.21.15.22:0/2140805307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec10d6c0 msgr2=0x7f25ec10fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.953+0000 7f25f2950700 1 --2- 172.21.15.22:0/2140805307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec10d6c0 0x7f25ec10fab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f25e400b0a0 tx=0x7f25e400b3b0 comp rx=0 tx=0).stop 2023-09-20T21:48:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.954+0000 7f25f2950700 1 -- 172.21.15.22:0/2140805307 shutdown_connections 2023-09-20T21:48:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.954+0000 7f25f2950700 1 --2- 172.21.15.22:0/2140805307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec10d6c0 0x7f25ec10fab0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:04.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.954+0000 7f25f2950700 1 --2- 172.21.15.22:0/2140805307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec073100 0x7f25ec0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.954+0000 7f25f2950700 1 --2- 172.21.15.22:0/2140805307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec073a20 0x7f25ec10d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.954+0000 7f25f2950700 1 -- 172.21.15.22:0/2140805307 >> 172.21.15.22:0/2140805307 conn(0x7f25ec0fc8d0 msgr2=0x7f25ec0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.954+0000 7f25f2950700 1 -- 172.21.15.22:0/2140805307 shutdown_connections 2023-09-20T21:48:04.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.954+0000 7f25f2950700 1 -- 172.21.15.22:0/2140805307 wait complete. 2023-09-20T21:48:04.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.955+0000 7f25f2950700 1 Processor -- start 2023-09-20T21:48:04.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.955+0000 7f25f2950700 1 -- start start 2023-09-20T21:48:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.955+0000 7f25f2950700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec073100 0x7f25ec1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25f2950700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec073a20 0x7f25ec199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25f2950700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec10d6c0 0x7f25ec19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25f2950700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec19e1a0 con 0x7f25ec073100 2023-09-20T21:48:04.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25f2950700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec19e310 con 0x7f25ec10d6c0 2023-09-20T21:48:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25f2950700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec19e4d0 con 0x7f25ec073a20 2023-09-20T21:48:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25ebfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec073100 0x7f25ec1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25ebfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec073100 0x7f25ec1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39932/0 (socket says 172.21.15.22:39932) 2023-09-20T21:48:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25ebfff700 1 -- 172.21.15.22:0/946854496 learned_addr learned my addr 172.21.15.22:0/946854496 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:04.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25f0eed700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec10d6c0 0x7f25ec19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25e3fff700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec073a20 0x7f25ec199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25ebfff700 1 -- 172.21.15.22:0/946854496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec073a20 msgr2=0x7f25ec199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25ebfff700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec073a20 0x7f25ec199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25ebfff700 1 -- 172.21.15.22:0/946854496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec10d6c0 msgr2=0x7f25ec19db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:04.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25ebfff700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec10d6c0 0x7f25ec19db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:04.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25ebfff700 1 -- 172.21.15.22:0/946854496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e4009d00 con 0x7f25ec073100 2023-09-20T21:48:04.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.956+0000 7f25f0eed700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec10d6c0 0x7f25ec19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:04.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.957+0000 7f25ebfff700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec073100 0x7f25ec1991e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00e9a0 tx=0x7f25dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:04.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.957+0000 7f25e9ffb700 1 -- 172.21.15.22:0/946854496 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc009790 con 0x7f25ec073100 2023-09-20T21:48:04.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.957+0000 7f25e9ffb700 1 -- 172.21.15.22:0/946854496 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25dc010430 con 0x7f25ec073100 2023-09-20T21:48:04.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.957+0000 7f25e9ffb700 1 -- 172.21.15.22:0/946854496 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc018680 con 0x7f25ec073100 2023-09-20T21:48:04.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.957+0000 7f25f2950700 1 -- 172.21.15.22:0/946854496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ec19e7b0 con 0x7f25ec073100 2023-09-20T21:48:04.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.957+0000 7f25f2950700 1 -- 172.21.15.22:0/946854496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec1a6010 con 0x7f25ec073100 2023-09-20T21:48:04.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.958+0000 7f25f2950700 1 -- 172.21.15.22:0/946854496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ec19e9d0 con 0x7f25ec073100 2023-09-20T21:48:04.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.960+0000 7f25e9ffb700 1 -- 172.21.15.22:0/946854496 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f25dc0105a0 con 0x7f25ec073100 2023-09-20T21:48:04.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.962+0000 7f25e9ffb700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1300 0x7f25d80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:04.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.962+0000 7f25e9ffb700 1 -- 172.21.15.22:0/946854496 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f25dc014070 con 0x7f25ec073100 2023-09-20T21:48:04.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.962+0000 7f25e3fff700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1300 0x7f25d80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:04.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.963+0000 7f25e3fff700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1300 0x7f25d80a37c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25ec19a800 tx=0x7f25d4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:04.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:04.966+0000 7f25e9ffb700 1 -- 172.21.15.22:0/946854496 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f25dc051e10 con 0x7f25ec073100 2023-09-20T21:48:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.220+0000 7f25f2950700 1 -- 172.21.15.22:0/946854496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f25ec19a540 con 0x7f25ec073100 2023-09-20T21:48:05.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.220+0000 7f25e9ffb700 1 -- 172.21.15.22:0/946854496 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f25dc096470 con 0x7f25ec073100 2023-09-20T21:48:05.221 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:05.222 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-20T21:48:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 -- 172.21.15.22:0/946854496 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1300 msgr2=0x7f25d80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:05.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1300 0x7f25d80a37c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25ec19a800 tx=0x7f25d4009260 comp rx=0 tx=0).stop 2023-09-20T21:48:05.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 -- 172.21.15.22:0/946854496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec073100 msgr2=0x7f25ec1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:05.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec073100 0x7f25ec1991e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00e9a0 tx=0x7f25dc00ecb0 comp rx=0 tx=0).stop 2023-09-20T21:48:05.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 -- 172.21.15.22:0/946854496 shutdown_connections 2023-09-20T21:48:05.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f25d80a1300 0x7f25d80a37c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:05.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25ec10d6c0 0x7f25ec19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:05.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25ec073100 0x7f25ec1991e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:05.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 --2- 172.21.15.22:0/946854496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ec073a20 0x7f25ec199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:05.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 -- 172.21.15.22:0/946854496 >> 172.21.15.22:0/946854496 conn(0x7f25ec0fc8d0 msgr2=0x7f25ec10f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 -- 172.21.15.22:0/946854496 shutdown_connections 2023-09-20T21:48:05.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.223+0000 7f25f2950700 1 -- 172.21.15.22:0/946854496 wait complete. 2023-09-20T21:48:05.696 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-09-20T21:48:05.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:48:05.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.906+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2965828525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1e0102970 msgr2=0x7ff1e0109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:05.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.906+0000 7ff1e63ff700 1 --2- 172.21.15.22:0/2965828525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1e0102970 0x7ff1e0109220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009fc0 tx=0x7ff1d800b330 comp rx=0 tx=0).stop 2023-09-20T21:48:05.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.906+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2965828525 shutdown_connections 2023-09-20T21:48:05.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.906+0000 7ff1e63ff700 1 --2- 172.21.15.22:0/2965828525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1e0102970 0x7ff1e0109220 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:05.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.906+0000 7ff1e63ff700 1 --2- 172.21.15.22:0/2965828525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e0100de0 0x7ff1e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:05.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.906+0000 7ff1e63ff700 1 --2- 172.21.15.22:0/2965828525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e0101fd0 0x7ff1e0102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:05.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.906+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2965828525 >> 172.21.15.22:0/2965828525 conn(0x7ff1e00fc310 msgr2=0x7ff1e00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:05.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.906+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2965828525 shutdown_connections 2023-09-20T21:48:05.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.906+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2965828525 wait complete. 2023-09-20T21:48:05.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.907+0000 7ff1e63ff700 1 Processor -- start 2023-09-20T21:48:05.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.907+0000 7ff1e63ff700 1 -- start start 2023-09-20T21:48:05.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1e63ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e0100de0 0x7ff1e0074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:05.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1e63ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1e0101fd0 0x7ff1e00731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:05.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1e63ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e0102970 0x7ff1e0073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:05.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1e63ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e0073eb0 con 0x7ff1e0101fd0 2023-09-20T21:48:05.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1e63ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e0074020 con 0x7ff1e0102970 2023-09-20T21:48:05.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1e63ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e01a0c00 con 0x7ff1e0100de0 2023-09-20T21:48:05.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1e0101fd0 0x7ff1e00731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:05.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1e0101fd0 0x7ff1e00731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47798/0 (socket says 172.21.15.22:47798) 2023-09-20T21:48:05.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1df7fe700 1 -- 172.21.15.22:0/2545678379 learned_addr learned my addr 172.21.15.22:0/2545678379 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:05.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1dffff700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e0100de0 0x7ff1e0074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:05.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.908+0000 7ff1e499c700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e0102970 0x7ff1e0073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:05.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1df7fe700 1 -- 172.21.15.22:0/2545678379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e0100de0 msgr2=0x7ff1e0074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:05.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1df7fe700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e0100de0 0x7ff1e0074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:05.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1df7fe700 1 -- 172.21.15.22:0/2545678379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e0102970 msgr2=0x7ff1e0073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:05.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1df7fe700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e0102970 0x7ff1e0073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:05.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1df7fe700 1 -- 172.21.15.22:0/2545678379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d40095f0 con 0x7ff1e0101fd0 2023-09-20T21:48:05.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1df7fe700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1e0101fd0 0x7ff1e00731d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400ec10 tx=0x7ff1d400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:05.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1dd7fa700 1 -- 172.21.15.22:0/2545678379 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400cc90 con 0x7ff1e0101fd0 2023-09-20T21:48:05.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2545678379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d8009c40 con 0x7ff1e0101fd0 2023-09-20T21:48:05.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1dd7fa700 1 -- 172.21.15.22:0/2545678379 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1d40044e0 con 0x7ff1e0101fd0 2023-09-20T21:48:05.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1dd7fa700 1 -- 172.21.15.22:0/2545678379 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4010430 con 0x7ff1e0101fd0 2023-09-20T21:48:05.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.909+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2545678379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e01a1250 con 0x7ff1e0101fd0 2023-09-20T21:48:05.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.910+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2545678379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e010c650 con 0x7ff1e0101fd0 2023-09-20T21:48:05.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.912+0000 7ff1dd7fa700 1 -- 172.21.15.22:0/2545678379 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ff1d4003fb0 con 0x7ff1e0101fd0 2023-09-20T21:48:05.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.914+0000 7ff1dd7fa700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff1c80a1350 0x7ff1c80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:05.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.914+0000 7ff1dd7fa700 1 -- 172.21.15.22:0/2545678379 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff1d4014070 con 0x7ff1e0101fd0 2023-09-20T21:48:05.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.914+0000 7ff1dffff700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff1c80a1350 0x7ff1c80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:05.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.915+0000 7ff1dffff700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff1c80a1350 0x7ff1c80a3810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0101e30 tx=0x7ff1d0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:05.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:05.918+0000 7ff1dd7fa700 1 -- 172.21.15.22:0/2545678379 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff1d4096820 con 0x7ff1e0101fd0 2023-09-20T21:48:06.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.132+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2545678379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1e0107c60 con 0x7ff1e0101fd0 2023-09-20T21:48:06.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.134+0000 7ff1dd7fa700 1 -- 172.21.15.22:0/2545678379 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24411 (secure 0 0 0) 0x7ff1d4099e40 con 0x7ff1e0101fd0 2023-09-20T21:48:06.134 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:06.135 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":91,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:47:08.590323+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:48:06.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.136+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2545678379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff1c80a1350 msgr2=0x7ff1c80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:06.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.136+0000 7ff1e63ff700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff1c80a1350 0x7ff1c80a3810 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0101e30 tx=0x7ff1d0009260 comp rx=0 tx=0).stop 2023-09-20T21:48:06.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.136+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2545678379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1e0101fd0 msgr2=0x7ff1e00731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:06.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.136+0000 7ff1e63ff700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1e0101fd0 0x7ff1e00731d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400ec10 tx=0x7ff1d400ef20 comp rx=0 tx=0).stop 2023-09-20T21:48:06.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.136+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2545678379 shutdown_connections 2023-09-20T21:48:06.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.136+0000 7ff1e63ff700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff1c80a1350 0x7ff1c80a3810 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.136+0000 7ff1e63ff700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1e0102970 0x7ff1e0073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.136+0000 7ff1e63ff700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e0100de0 0x7ff1e0074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.136+0000 7ff1e63ff700 1 --2- 172.21.15.22:0/2545678379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1e0101fd0 0x7ff1e00731d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.136+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2545678379 >> 172.21.15.22:0/2545678379 conn(0x7ff1e00fc310 msgr2=0x7ff1e00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:06.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.137+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2545678379 shutdown_connections 2023-09-20T21:48:06.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.137+0000 7ff1e63ff700 1 -- 172.21.15.22:0/2545678379 wait complete. 2023-09-20T21:48:06.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T21:48:06.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:48:06.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.365+0000 7fea189ff700 1 -- 172.21.15.22:0/2571037743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea14102960 msgr2=0x7fea14109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:06.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.365+0000 7fea189ff700 1 --2- 172.21.15.22:0/2571037743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea14102960 0x7fea14109210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00b0d0 tx=0x7fea0c00b3e0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.365+0000 7fea189ff700 1 -- 172.21.15.22:0/2571037743 shutdown_connections 2023-09-20T21:48:06.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.365+0000 7fea189ff700 1 --2- 172.21.15.22:0/2571037743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea14102960 0x7fea14109210 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.365+0000 7fea189ff700 1 --2- 172.21.15.22:0/2571037743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea14100dd0 0x7fea141011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.365+0000 7fea189ff700 1 --2- 172.21.15.22:0/2571037743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea14101fc0 0x7fea14102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.365+0000 7fea189ff700 1 -- 172.21.15.22:0/2571037743 >> 172.21.15.22:0/2571037743 conn(0x7fea140fc320 msgr2=0x7fea140fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:06.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.365+0000 7fea189ff700 1 -- 172.21.15.22:0/2571037743 shutdown_connections 2023-09-20T21:48:06.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.365+0000 7fea189ff700 1 -- 172.21.15.22:0/2571037743 wait complete. 2023-09-20T21:48:06.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.366+0000 7fea189ff700 1 Processor -- start 2023-09-20T21:48:06.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.366+0000 7fea189ff700 1 -- start start 2023-09-20T21:48:06.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea189ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea14100dd0 0x7fea14198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:06.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea189ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea14101fc0 0x7fea14199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea189ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea14102960 0x7fea141a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea189ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea141a0950 con 0x7fea14100dd0 2023-09-20T21:48:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea189ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea141a0a90 con 0x7fea14101fc0 2023-09-20T21:48:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea189ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea141a0c20 con 0x7fea14102960 2023-09-20T21:48:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea1259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea14100dd0 0x7fea14198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea1259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea14100dd0 0x7fea14198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47820/0 (socket says 172.21.15.22:47820) 2023-09-20T21:48:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea1259c700 1 -- 172.21.15.22:0/2917122075 learned_addr learned my addr 172.21.15.22:0/2917122075 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea11d9b700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea14101fc0 0x7fea14199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea12d9d700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea14102960 0x7fea141a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea1259c700 1 -- 172.21.15.22:0/2917122075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea14102960 msgr2=0x7fea141a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea1259c700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea14102960 0x7fea141a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea1259c700 1 -- 172.21.15.22:0/2917122075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea14101fc0 msgr2=0x7fea14199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea1259c700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea14101fc0 0x7fea14199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea1259c700 1 -- 172.21.15.22:0/2917122075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea08009520 con 0x7fea14100dd0 2023-09-20T21:48:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.367+0000 7fea12d9d700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea14102960 0x7fea141a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.368+0000 7fea1259c700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea14100dd0 0x7fea14198dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fea0400e970 tx=0x7fea0400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.368+0000 7fea037fe700 1 -- 172.21.15.22:0/2917122075 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04009790 con 0x7fea14100dd0 2023-09-20T21:48:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.368+0000 7fea037fe700 1 -- 172.21.15.22:0/2917122075 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea0400eec0 con 0x7fea14100dd0 2023-09-20T21:48:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.368+0000 7fea189ff700 1 -- 172.21.15.22:0/2917122075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c009d00 con 0x7fea14100dd0 2023-09-20T21:48:06.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.368+0000 7fea037fe700 1 -- 172.21.15.22:0/2917122075 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04003cf0 con 0x7fea14100dd0 2023-09-20T21:48:06.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.368+0000 7fea189ff700 1 -- 172.21.15.22:0/2917122075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea141a1210 con 0x7fea14100dd0 2023-09-20T21:48:06.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.369+0000 7fea189ff700 1 -- 172.21.15.22:0/2917122075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1410c640 con 0x7fea14100dd0 2023-09-20T21:48:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.371+0000 7fea037fe700 1 -- 172.21.15.22:0/2917122075 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fea0400ccd0 con 0x7fea14100dd0 2023-09-20T21:48:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.373+0000 7fea037fe700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe9fc0a1380 0x7fe9fc0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.373+0000 7fea037fe700 1 -- 172.21.15.22:0/2917122075 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fea04014070 con 0x7fea14100dd0 2023-09-20T21:48:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.373+0000 7fea11d9b700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe9fc0a1380 0x7fe9fc0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:06.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.374+0000 7fea11d9b700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe9fc0a1380 0x7fe9fc0a3840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea08009d70 tx=0x7fea08009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:06.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.375+0000 7fea037fe700 1 -- 172.21.15.22:0/2917122075 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fea04096660 con 0x7fea14100dd0 2023-09-20T21:48:06.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:06 smithi022 ceph-mon[97651]: pgmap v320: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:48:06.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:06 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/946854496' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-20T21:48:06.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:06 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2545678379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.586+0000 7fea189ff700 1 -- 172.21.15.22:0/2917122075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea141a1460 con 0x7fea14100dd0 2023-09-20T21:48:06.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.587+0000 7fea037fe700 1 -- 172.21.15.22:0/2917122075 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea04099c80 con 0x7fea14100dd0 2023-09-20T21:48:06.588 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:48:06.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.589+0000 7fea189ff700 1 -- 172.21.15.22:0/2917122075 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe9fc0a1380 msgr2=0x7fe9fc0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:06.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.589+0000 7fea189ff700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe9fc0a1380 0x7fe9fc0a3840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea08009d70 tx=0x7fea08009260 comp rx=0 tx=0).stop 2023-09-20T21:48:06.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.589+0000 7fea189ff700 1 -- 172.21.15.22:0/2917122075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea14100dd0 msgr2=0x7fea14198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:06.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.589+0000 7fea189ff700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea14100dd0 0x7fea14198dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fea0400e970 tx=0x7fea0400ec80 comp rx=0 tx=0).stop 2023-09-20T21:48:06.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.590+0000 7fea189ff700 1 -- 172.21.15.22:0/2917122075 shutdown_connections 2023-09-20T21:48:06.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.590+0000 7fea189ff700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea14102960 0x7fea141a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.590+0000 7fea189ff700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea14100dd0 0x7fea14198dd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.590+0000 7fea189ff700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe9fc0a1380 0x7fe9fc0a3840 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.590+0000 7fea189ff700 1 --2- 172.21.15.22:0/2917122075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea14101fc0 0x7fea14199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.590+0000 7fea189ff700 1 -- 172.21.15.22:0/2917122075 >> 172.21.15.22:0/2917122075 conn(0x7fea140fc320 msgr2=0x7fea14107860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:06.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.590+0000 7fea189ff700 1 -- 172.21.15.22:0/2917122075 shutdown_connections 2023-09-20T21:48:06.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.590+0000 7fea189ff700 1 -- 172.21.15.22:0/2917122075 wait complete. 2023-09-20T21:48:06.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:06 smithi123 ceph-mon[134433]: pgmap v320: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:48:06.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:06 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/946854496' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-20T21:48:06.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:06 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2545678379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:06.609 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T21:48:06.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:06 smithi089 ceph-mon[134090]: pgmap v320: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:48:06.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:06 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/946854496' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-09-20T21:48:06.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:06 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2545678379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:06.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.818+0000 7f07e37ff700 1 -- 172.21.15.22:0/2931932158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07dc102960 msgr2=0x7f07dc109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.818+0000 7f07e37ff700 1 --2- 172.21.15.22:0/2931932158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07dc102960 0x7f07dc109210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f07d400b0d0 tx=0x7f07d400b3e0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.818+0000 7f07e37ff700 1 -- 172.21.15.22:0/2931932158 shutdown_connections 2023-09-20T21:48:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.818+0000 7f07e37ff700 1 --2- 172.21.15.22:0/2931932158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07dc102960 0x7f07dc109210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.818+0000 7f07e37ff700 1 --2- 172.21.15.22:0/2931932158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07dc100dd0 0x7f07dc1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.818+0000 7f07e37ff700 1 --2- 172.21.15.22:0/2931932158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07dc101fc0 0x7f07dc102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.818+0000 7f07e37ff700 1 -- 172.21.15.22:0/2931932158 >> 172.21.15.22:0/2931932158 conn(0x7f07dc0fc320 msgr2=0x7f07dc0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.818+0000 7f07e37ff700 1 -- 172.21.15.22:0/2931932158 shutdown_connections 2023-09-20T21:48:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.818+0000 7f07e37ff700 1 -- 172.21.15.22:0/2931932158 wait complete. 2023-09-20T21:48:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.819+0000 7f07e37ff700 1 Processor -- start 2023-09-20T21:48:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.819+0000 7f07e37ff700 1 -- start start 2023-09-20T21:48:06.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.819+0000 7f07e37ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07dc100dd0 0x7f07dc074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:06.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.819+0000 7f07e37ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07dc101fc0 0x7f07dc0731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:06.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e37ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07dc102960 0x7f07dc073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:06.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e37ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc19aab0 con 0x7f07dc102960 2023-09-20T21:48:06.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e37ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc073c50 con 0x7f07dc101fc0 2023-09-20T21:48:06.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e37ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc073e10 con 0x7f07dc100dd0 2023-09-20T21:48:06.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e0d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07dc101fc0 0x7f07dc0731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:06.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e0d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07dc101fc0 0x7f07dc0731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46434/0 (socket says 172.21.15.22:46434) 2023-09-20T21:48:06.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e0d9a700 1 -- 172.21.15.22:0/1310677598 learned_addr learned my addr 172.21.15.22:0/1310677598 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:06.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e159b700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07dc100dd0 0x7f07dc074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:06.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e1d9c700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07dc102960 0x7f07dc073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e0d9a700 1 -- 172.21.15.22:0/1310677598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07dc100dd0 msgr2=0x7f07dc074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e0d9a700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07dc100dd0 0x7f07dc074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e0d9a700 1 -- 172.21.15.22:0/1310677598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07dc102960 msgr2=0x7f07dc073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e0d9a700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07dc102960 0x7f07dc073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.820+0000 7f07e0d9a700 1 -- 172.21.15.22:0/1310677598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d4009d00 con 0x7f07dc101fc0 2023-09-20T21:48:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.821+0000 7f07e0d9a700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07dc101fc0 0x7f07dc0731d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e9a0 tx=0x7f07cc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.821+0000 7f07d27fc700 1 -- 172.21.15.22:0/1310677598 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc009790 con 0x7f07dc101fc0 2023-09-20T21:48:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.821+0000 7f07e37ff700 1 -- 172.21.15.22:0/1310677598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc1a5460 con 0x7f07dc101fc0 2023-09-20T21:48:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.821+0000 7f07d27fc700 1 -- 172.21.15.22:0/1310677598 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07cc010430 con 0x7f07dc101fc0 2023-09-20T21:48:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.821+0000 7f07d27fc700 1 -- 172.21.15.22:0/1310677598 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc018680 con 0x7f07dc101fc0 2023-09-20T21:48:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.821+0000 7f07e37ff700 1 -- 172.21.15.22:0/1310677598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc1a58e0 con 0x7f07dc101fc0 2023-09-20T21:48:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.822+0000 7f07e37ff700 1 -- 172.21.15.22:0/1310677598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc10c640 con 0x7f07dc101fc0 2023-09-20T21:48:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.824+0000 7f07d27fc700 1 -- 172.21.15.22:0/1310677598 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f07cc0105a0 con 0x7f07dc101fc0 2023-09-20T21:48:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.825+0000 7f07d27fc700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07c80a1300 0x7f07c80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.826+0000 7f07d27fc700 1 -- 172.21.15.22:0/1310677598 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f07cc014070 con 0x7f07dc101fc0 2023-09-20T21:48:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.826+0000 7f07e159b700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07c80a1300 0x7f07c80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.826+0000 7f07e159b700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07c80a1300 0x7f07c80a37c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f07dc101e20 tx=0x7f07d8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:06.828+0000 7f07d27fc700 1 -- 172.21.15.22:0/1310677598 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f07cc0966a0 con 0x7f07dc101fc0 2023-09-20T21:48:07.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.039+0000 7f07e37ff700 1 -- 172.21.15.22:0/1310677598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07dc10c800 con 0x7f07dc101fc0 2023-09-20T21:48:07.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.040+0000 7f07d27fc700 1 -- 172.21.15.22:0/1310677598 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07cc099cc0 con 0x7f07dc101fc0 2023-09-20T21:48:07.040 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:07.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.042+0000 7f07e37ff700 1 -- 172.21.15.22:0/1310677598 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07c80a1300 msgr2=0x7f07c80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.042+0000 7f07e37ff700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07c80a1300 0x7f07c80a37c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f07dc101e20 tx=0x7f07d8009260 comp rx=0 tx=0).stop 2023-09-20T21:48:07.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.042+0000 7f07e37ff700 1 -- 172.21.15.22:0/1310677598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07dc101fc0 msgr2=0x7f07dc0731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.042+0000 7f07e37ff700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07dc101fc0 0x7f07dc0731d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e9a0 tx=0x7f07cc00ecb0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.042+0000 7f07e37ff700 1 -- 172.21.15.22:0/1310677598 shutdown_connections 2023-09-20T21:48:07.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.042+0000 7f07e37ff700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f07dc102960 0x7f07dc073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.042+0000 7f07e37ff700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07dc100dd0 0x7f07dc074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.042+0000 7f07e37ff700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f07c80a1300 0x7f07c80a37c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.042+0000 7f07e37ff700 1 --2- 172.21.15.22:0/1310677598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f07dc101fc0 0x7f07dc0731d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.042+0000 7f07e37ff700 1 -- 172.21.15.22:0/1310677598 >> 172.21.15.22:0/1310677598 conn(0x7f07dc0fc320 msgr2=0x7f07dc0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:07.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.043+0000 7f07e37ff700 1 -- 172.21.15.22:0/1310677598 shutdown_connections 2023-09-20T21:48:07.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.043+0000 7f07e37ff700 1 -- 172.21.15.22:0/1310677598 wait complete. 2023-09-20T21:48:07.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T21:48:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.271+0000 7f60069ff700 1 -- 172.21.15.22:0/2343125563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6000100dd0 msgr2=0x7f60001011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.271+0000 7f60069ff700 1 --2- 172.21.15.22:0/2343125563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6000100dd0 0x7f60001011f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00099c0 tx=0x7f5ff0009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.271+0000 7f60069ff700 1 -- 172.21.15.22:0/2343125563 shutdown_connections 2023-09-20T21:48:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.271+0000 7f60069ff700 1 --2- 172.21.15.22:0/2343125563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6000102960 0x7f6000109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.271+0000 7f60069ff700 1 --2- 172.21.15.22:0/2343125563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6000100dd0 0x7f60001011f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.271+0000 7f60069ff700 1 --2- 172.21.15.22:0/2343125563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000101fc0 0x7f6000102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.271+0000 7f60069ff700 1 -- 172.21.15.22:0/2343125563 >> 172.21.15.22:0/2343125563 conn(0x7f60000fc320 msgr2=0x7f60000fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.271+0000 7f60069ff700 1 -- 172.21.15.22:0/2343125563 shutdown_connections 2023-09-20T21:48:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.271+0000 7f60069ff700 1 -- 172.21.15.22:0/2343125563 wait complete. 2023-09-20T21:48:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.272+0000 7f60069ff700 1 Processor -- start 2023-09-20T21:48:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.272+0000 7f60069ff700 1 -- start start 2023-09-20T21:48:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.272+0000 7f60069ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6000100dd0 0x7f6000198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:07.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.272+0000 7f60069ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000101fc0 0x7f6000199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:07.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.272+0000 7f60069ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6000102960 0x7f60001a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:07.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.272+0000 7f60069ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60001a0860 con 0x7f6000102960 2023-09-20T21:48:07.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.272+0000 7f60069ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60001a09d0 con 0x7f6000100dd0 2023-09-20T21:48:07.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.272+0000 7f60069ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60001a0b90 con 0x7f6000101fc0 2023-09-20T21:48:07.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5fff7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000101fc0 0x7f6000199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:07.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f6004f9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6000102960 0x7f60001a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:07.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5fff7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000101fc0 0x7f6000199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:40574/0 (socket says 172.21.15.22:40574) 2023-09-20T21:48:07.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5fff7fe700 1 -- 172.21.15.22:0/1518738372 learned_addr learned my addr 172.21.15.22:0/1518738372 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:07.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5ffffff700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6000100dd0 0x7f6000198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:07.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5fff7fe700 1 -- 172.21.15.22:0/1518738372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6000100dd0 msgr2=0x7f6000198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5fff7fe700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6000100dd0 0x7f6000198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5fff7fe700 1 -- 172.21.15.22:0/1518738372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6000102960 msgr2=0x7f60001a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5fff7fe700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6000102960 0x7f60001a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5fff7fe700 1 -- 172.21.15.22:0/1518738372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00095f0 con 0x7f6000101fc0 2023-09-20T21:48:07.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f6004f9c700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6000102960 0x7f60001a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:07.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5fff7fe700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000101fc0 0x7f6000199260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e9a0 tx=0x7f5ff400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:07.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.273+0000 7f5ffd7fa700 1 -- 172.21.15.22:0/1518738372 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4009790 con 0x7f6000101fc0 2023-09-20T21:48:07.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.274+0000 7f60069ff700 1 -- 172.21.15.22:0/1518738372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60001a0e70 con 0x7f6000101fc0 2023-09-20T21:48:07.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.274+0000 7f5ffd7fa700 1 -- 172.21.15.22:0/1518738372 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ff4010430 con 0x7f6000101fc0 2023-09-20T21:48:07.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.274+0000 7f5ffd7fa700 1 -- 172.21.15.22:0/1518738372 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4018680 con 0x7f6000101fc0 2023-09-20T21:48:07.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.274+0000 7f60069ff700 1 -- 172.21.15.22:0/1518738372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60001a12a0 con 0x7f6000101fc0 2023-09-20T21:48:07.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.275+0000 7f60069ff700 1 -- 172.21.15.22:0/1518738372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60001933d0 con 0x7f6000101fc0 2023-09-20T21:48:07.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.277+0000 7f5ffd7fa700 1 -- 172.21.15.22:0/1518738372 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5ff40105a0 con 0x7f6000101fc0 2023-09-20T21:48:07.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.278+0000 7f5ffd7fa700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5fe80a1300 0x7f5fe80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:07.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.278+0000 7f5ffd7fa700 1 -- 172.21.15.22:0/1518738372 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5ff4014070 con 0x7f6000101fc0 2023-09-20T21:48:07.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.278+0000 7f5ffffff700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5fe80a1300 0x7f5fe80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:07.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.279+0000 7f5ffffff700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5fe80a1300 0x7f5fe80a37c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f600019a1f0 tx=0x7f5ff001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:07.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.280+0000 7f5ffd7fa700 1 -- 172.21.15.22:0/1518738372 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5ff4096470 con 0x7f6000101fc0 2023-09-20T21:48:07.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.492+0000 7f60069ff700 1 -- 172.21.15.22:0/1518738372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f600010c830 con 0x7f6000101fc0 2023-09-20T21:48:07.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.493+0000 7f5ffd7fa700 1 -- 172.21.15.22:0/1518738372 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ff4099a90 con 0x7f6000101fc0 2023-09-20T21:48:07.493 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T21:48:07.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.495+0000 7f60069ff700 1 -- 172.21.15.22:0/1518738372 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5fe80a1300 msgr2=0x7f5fe80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.495+0000 7f60069ff700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5fe80a1300 0x7f5fe80a37c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f600019a1f0 tx=0x7f5ff001a040 comp rx=0 tx=0).stop 2023-09-20T21:48:07.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.495+0000 7f60069ff700 1 -- 172.21.15.22:0/1518738372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000101fc0 msgr2=0x7f6000199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.495+0000 7f60069ff700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000101fc0 0x7f6000199260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e9a0 tx=0x7f5ff400ecb0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.495+0000 7f60069ff700 1 -- 172.21.15.22:0/1518738372 shutdown_connections 2023-09-20T21:48:07.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.495+0000 7f60069ff700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6000102960 0x7f60001a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.495+0000 7f60069ff700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5fe80a1300 0x7f5fe80a37c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.495+0000 7f60069ff700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6000100dd0 0x7f6000198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.495+0000 7f60069ff700 1 --2- 172.21.15.22:0/1518738372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6000101fc0 0x7f6000199260 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.495+0000 7f60069ff700 1 -- 172.21.15.22:0/1518738372 >> 172.21.15.22:0/1518738372 conn(0x7f60000fc320 msgr2=0x7f6000107940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:07.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.496+0000 7f60069ff700 1 -- 172.21.15.22:0/1518738372 shutdown_connections 2023-09-20T21:48:07.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.496+0000 7f60069ff700 1 -- 172.21.15.22:0/1518738372 wait complete. 2023-09-20T21:48:07.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:07 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2917122075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:07.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:07 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1310677598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:07.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T21:48:07.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:07 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2917122075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:07.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:07 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1310677598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:07.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:07 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2917122075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:07.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:07 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1310677598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:07.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.723+0000 7f18527ff700 1 -- 172.21.15.22:0/2533863832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c101fc0 msgr2=0x7f184c102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.723+0000 7f18527ff700 1 --2- 172.21.15.22:0/2533863832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c101fc0 0x7f184c102420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1840009920 tx=0x7f1840009c30 comp rx=0 tx=0).stop 2023-09-20T21:48:07.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.723+0000 7f18527ff700 1 -- 172.21.15.22:0/2533863832 shutdown_connections 2023-09-20T21:48:07.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.723+0000 7f18527ff700 1 --2- 172.21.15.22:0/2533863832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c102960 0x7f184c109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.723+0000 7f18527ff700 1 --2- 172.21.15.22:0/2533863832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c100dd0 0x7f184c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.723+0000 7f18527ff700 1 --2- 172.21.15.22:0/2533863832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c101fc0 0x7f184c102420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.723+0000 7f18527ff700 1 -- 172.21.15.22:0/2533863832 >> 172.21.15.22:0/2533863832 conn(0x7f184c0fc320 msgr2=0x7f184c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:07.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.723+0000 7f18527ff700 1 -- 172.21.15.22:0/2533863832 shutdown_connections 2023-09-20T21:48:07.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.723+0000 7f18527ff700 1 -- 172.21.15.22:0/2533863832 wait complete. 2023-09-20T21:48:07.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.724+0000 7f18527ff700 1 Processor -- start 2023-09-20T21:48:07.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.725+0000 7f18527ff700 1 -- start start 2023-09-20T21:48:07.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.725+0000 7f18527ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c100dd0 0x7f184c196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:07.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.725+0000 7f18527ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c101fc0 0x7f184c1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:07.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.725+0000 7f18527ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c102960 0x7f184c19e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:07.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.725+0000 7f18527ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c19e7a0 con 0x7f184c102960 2023-09-20T21:48:07.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.725+0000 7f18527ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c19e910 con 0x7f184c100dd0 2023-09-20T21:48:07.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.725+0000 7f18527ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c19ead0 con 0x7f184c101fc0 2023-09-20T21:48:07.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c101fc0 0x7f184c1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:07.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c100dd0 0x7f184c196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:07.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c101fc0 0x7f184c1971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:40588/0 (socket says 172.21.15.22:40588) 2023-09-20T21:48:07.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184b7fe700 1 -- 172.21.15.22:0/1857665700 learned_addr learned my addr 172.21.15.22:0/1857665700 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:07.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c100dd0 0x7f184c196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46474/0 (socket says 172.21.15.22:46474) 2023-09-20T21:48:07.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f1850d9c700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c102960 0x7f184c19e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:07.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184b7fe700 1 -- 172.21.15.22:0/1857665700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c100dd0 msgr2=0x7f184c196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184b7fe700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c100dd0 0x7f184c196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184b7fe700 1 -- 172.21.15.22:0/1857665700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c102960 msgr2=0x7f184c19e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184b7fe700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c102960 0x7f184c19e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184b7fe700 1 -- 172.21.15.22:0/1857665700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1840009520 con 0x7f184c101fc0 2023-09-20T21:48:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184bfff700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c100dd0 0x7f184c196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f184b7fe700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c101fc0 0x7f184c1971a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1840009230 tx=0x7f184000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.726+0000 7f18497fa700 1 -- 172.21.15.22:0/1857665700 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184001c070 con 0x7f184c101fc0 2023-09-20T21:48:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.727+0000 7f18497fa700 1 -- 172.21.15.22:0/1857665700 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f184000f7d0 con 0x7f184c101fc0 2023-09-20T21:48:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.727+0000 7f18497fa700 1 -- 172.21.15.22:0/1857665700 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840003ae0 con 0x7f184c101fc0 2023-09-20T21:48:07.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.727+0000 7f18527ff700 1 -- 172.21.15.22:0/1857665700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184c19ed50 con 0x7f184c101fc0 2023-09-20T21:48:07.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.727+0000 7f18527ff700 1 -- 172.21.15.22:0/1857665700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c19f120 con 0x7f184c101fc0 2023-09-20T21:48:07.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.728+0000 7f18527ff700 1 -- 172.21.15.22:0/1857665700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184c108550 con 0x7f184c101fc0 2023-09-20T21:48:07.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.730+0000 7f18497fa700 1 -- 172.21.15.22:0/1857665700 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1840020030 con 0x7f184c101fc0 2023-09-20T21:48:07.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.731+0000 7f18497fa700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f18340a1380 0x7f18340a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:07.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.731+0000 7f18497fa700 1 -- 172.21.15.22:0/1857665700 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f18400ce100 con 0x7f184c101fc0 2023-09-20T21:48:07.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.731+0000 7f184bfff700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f18340a1380 0x7f18340a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:07.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.732+0000 7f184bfff700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f18340a1380 0x7f18340a3840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f184c198130 tx=0x7f183c00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:07.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.733+0000 7f18497fa700 1 -- 172.21.15.22:0/1857665700 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1840097500 con 0x7f184c101fc0 2023-09-20T21:48:07.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.945+0000 7f18527ff700 1 -- 172.21.15.22:0/1857665700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f184c10c840 con 0x7f184c101fc0 2023-09-20T21:48:07.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.946+0000 7f18497fa700 1 -- 172.21.15.22:0/1857665700 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f184009ab20 con 0x7f184c101fc0 2023-09-20T21:48:07.946 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:07.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 -- 172.21.15.22:0/1857665700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f18340a1380 msgr2=0x7f18340a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f18340a1380 0x7f18340a3840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f184c198130 tx=0x7f183c00b410 comp rx=0 tx=0).stop 2023-09-20T21:48:07.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 -- 172.21.15.22:0/1857665700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c101fc0 msgr2=0x7f184c1971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:07.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c101fc0 0x7f184c1971a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1840009230 tx=0x7f184000b790 comp rx=0 tx=0).stop 2023-09-20T21:48:07.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 -- 172.21.15.22:0/1857665700 shutdown_connections 2023-09-20T21:48:07.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c102960 0x7f184c19e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f18340a1380 0x7f18340a3840 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f184c100dd0 0x7f184c196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 --2- 172.21.15.22:0/1857665700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f184c101fc0 0x7f184c1971a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:07.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 -- 172.21.15.22:0/1857665700 >> 172.21.15.22:0/1857665700 conn(0x7f184c0fc320 msgr2=0x7f184c105690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:07.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 -- 172.21.15.22:0/1857665700 shutdown_connections 2023-09-20T21:48:07.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:07.948+0000 7f18527ff700 1 -- 172.21.15.22:0/1857665700 wait complete. 2023-09-20T21:48:07.967 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:48:08.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.171+0000 7f673ddff700 1 -- 172.21.15.22:0/3477814529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6738100de0 msgr2=0x7f6738101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:08.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.171+0000 7f673ddff700 1 --2- 172.21.15.22:0/3477814529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6738100de0 0x7f6738101200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f67280099c0 tx=0x7f6728009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.171+0000 7f673ddff700 1 -- 172.21.15.22:0/3477814529 shutdown_connections 2023-09-20T21:48:08.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.171+0000 7f673ddff700 1 --2- 172.21.15.22:0/3477814529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738102970 0x7f6738109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.171+0000 7f673ddff700 1 --2- 172.21.15.22:0/3477814529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6738100de0 0x7f6738101200 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.171+0000 7f673ddff700 1 --2- 172.21.15.22:0/3477814529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6738101fd0 0x7f6738102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.171+0000 7f673ddff700 1 -- 172.21.15.22:0/3477814529 >> 172.21.15.22:0/3477814529 conn(0x7f67380fc330 msgr2=0x7f67380fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:08.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.171+0000 7f673ddff700 1 -- 172.21.15.22:0/3477814529 shutdown_connections 2023-09-20T21:48:08.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.171+0000 7f673ddff700 1 -- 172.21.15.22:0/3477814529 wait complete. 2023-09-20T21:48:08.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.172+0000 7f673ddff700 1 Processor -- start 2023-09-20T21:48:08.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.172+0000 7f673ddff700 1 -- start start 2023-09-20T21:48:08.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f673ddff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6738100de0 0x7f6738198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:08.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f673ddff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101fd0 0x7f67381991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:08.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f673ddff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6738102970 0x7f67381a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:08.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f673ddff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67381a0800 con 0x7f6738100de0 2023-09-20T21:48:08.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f673ddff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67381a0970 con 0x7f6738102970 2023-09-20T21:48:08.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f673ddff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67381a0b30 con 0x7f6738101fd0 2023-09-20T21:48:08.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f67377fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6738100de0 0x7f6738198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:08.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f6737fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6738102970 0x7f67381a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:08.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f67377fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6738100de0 0x7f6738198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47894/0 (socket says 172.21.15.22:47894) 2023-09-20T21:48:08.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f67377fe700 1 -- 172.21.15.22:0/1449057867 learned_addr learned my addr 172.21.15.22:0/1449057867 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:08.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f6736ffd700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101fd0 0x7f67381991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:08.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f67377fe700 1 -- 172.21.15.22:0/1449057867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101fd0 msgr2=0x7f67381991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:08.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f67377fe700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101fd0 0x7f67381991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f67377fe700 1 -- 172.21.15.22:0/1449057867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6738102970 msgr2=0x7f67381a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:08.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f67377fe700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6738102970 0x7f67381a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f67377fe700 1 -- 172.21.15.22:0/1449057867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c009520 con 0x7f6738100de0 2023-09-20T21:48:08.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.173+0000 7f6737fff700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6738102970 0x7f67381a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:08.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.174+0000 7f67377fe700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6738100de0 0x7f6738198c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f67280058e0 tx=0x7f672800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:08.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.174+0000 7f6734ff9700 1 -- 172.21.15.22:0/1449057867 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672801d070 con 0x7f6738100de0 2023-09-20T21:48:08.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.174+0000 7f6734ff9700 1 -- 172.21.15.22:0/1449057867 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6728005050 con 0x7f6738100de0 2023-09-20T21:48:08.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.174+0000 7f6734ff9700 1 -- 172.21.15.22:0/1449057867 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728004b30 con 0x7f6738100de0 2023-09-20T21:48:08.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.174+0000 7f673ddff700 1 -- 172.21.15.22:0/1449057867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67280095f0 con 0x7f6738100de0 2023-09-20T21:48:08.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.174+0000 7f673ddff700 1 -- 172.21.15.22:0/1449057867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67381a1040 con 0x7f6738100de0 2023-09-20T21:48:08.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.175+0000 7f673ddff700 1 -- 172.21.15.22:0/1449057867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67381933e0 con 0x7f6738100de0 2023-09-20T21:48:08.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.177+0000 7f6734ff9700 1 -- 172.21.15.22:0/1449057867 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f67280051c0 con 0x7f6738100de0 2023-09-20T21:48:08.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.179+0000 7f6734ff9700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67200a13d0 0x7f67200a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:08.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.179+0000 7f6734ff9700 1 -- 172.21.15.22:0/1449057867 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f67280cdca0 con 0x7f6738100de0 2023-09-20T21:48:08.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.179+0000 7f6736ffd700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67200a13d0 0x7f67200a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:08.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.179+0000 7f6736ffd700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67200a13d0 0x7f67200a3890 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f673819a190 tx=0x7f672c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:08.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.183+0000 7f6734ff9700 1 -- 172.21.15.22:0/1449057867 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f67280971c0 con 0x7f6738100de0 2023-09-20T21:48:08.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:08 smithi022 ceph-mon[97651]: pgmap v321: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:08.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:08 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1518738372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:08.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-09-20T21:48:08.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-09-20T21:48:08.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]: dispatch 2023-09-20T21:48:08.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-09-20T21:48:08.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2023-09-20T21:48:08.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:08 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1857665700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:08.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.442+0000 7f673ddff700 1 -- 172.21.15.22:0/1449057867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f673810c5f0 con 0x7f6738100de0 2023-09-20T21:48:08.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.443+0000 7f6734ff9700 1 -- 172.21.15.22:0/1449057867 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f6728026090 con 0x7f6738100de0 2023-09-20T21:48:08.443 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:08.443 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:47:09.607119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T21:48:08.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 -- 172.21.15.22:0/1449057867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67200a13d0 msgr2=0x7f67200a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:08.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67200a13d0 0x7f67200a3890 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f673819a190 tx=0x7f672c009260 comp rx=0 tx=0).stop 2023-09-20T21:48:08.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 -- 172.21.15.22:0/1449057867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6738100de0 msgr2=0x7f6738198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:08.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6738100de0 0x7f6738198c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f67280058e0 tx=0x7f672800f690 comp rx=0 tx=0).stop 2023-09-20T21:48:08.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 -- 172.21.15.22:0/1449057867 shutdown_connections 2023-09-20T21:48:08.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67200a13d0 0x7f67200a3890 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6738102970 0x7f67381a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6738100de0 0x7f6738198c90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 --2- 172.21.15.22:0/1449057867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6738101fd0 0x7f67381991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 -- 172.21.15.22:0/1449057867 >> 172.21.15.22:0/1449057867 conn(0x7f67380fc330 msgr2=0x7f67381078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:08.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 -- 172.21.15.22:0/1449057867 shutdown_connections 2023-09-20T21:48:08.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.445+0000 7f673ddff700 1 -- 172.21.15.22:0/1449057867 wait complete. 2023-09-20T21:48:08.448 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2023-09-20T21:48:08.464 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:48:08.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:08 smithi123 ceph-mon[134433]: pgmap v321: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:08.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:08 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1518738372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:08.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-09-20T21:48:08.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-09-20T21:48:08.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]: dispatch 2023-09-20T21:48:08.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-09-20T21:48:08.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2023-09-20T21:48:08.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:08 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1857665700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:08.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:08 smithi089 ceph-mon[134090]: pgmap v321: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:08.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:08 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1518738372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:08.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-09-20T21:48:08.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-09-20T21:48:08.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]: dispatch 2023-09-20T21:48:08.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-09-20T21:48:08.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2023-09-20T21:48:08.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:08 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1857665700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:08.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.675+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3506752796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8101f70 msgr2=0x7fe6d81023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:08.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.675+0000 7fe6ddfff700 1 --2- 172.21.15.22:0/3506752796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8101f70 0x7fe6d81023f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009990 tx=0x7fe6cc009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.675+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3506752796 shutdown_connections 2023-09-20T21:48:08.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.675+0000 7fe6ddfff700 1 --2- 172.21.15.22:0/3506752796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6d8102930 0x7fe6d8109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.675+0000 7fe6ddfff700 1 --2- 172.21.15.22:0/3506752796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d8100e20 0x7fe6d8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.675+0000 7fe6ddfff700 1 --2- 172.21.15.22:0/3506752796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8101f70 0x7fe6d81023f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.675+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3506752796 >> 172.21.15.22:0/3506752796 conn(0x7fe6d80fc390 msgr2=0x7fe6d80fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:08.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.675+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3506752796 shutdown_connections 2023-09-20T21:48:08.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.675+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3506752796 wait complete. 2023-09-20T21:48:08.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.676+0000 7fe6ddfff700 1 Processor -- start 2023-09-20T21:48:08.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.676+0000 7fe6ddfff700 1 -- start start 2023-09-20T21:48:08.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6ddfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6d8100e20 0x7fe6d8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:08.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6ddfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d8101f70 0x7fe6d81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:08.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6ddfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8102930 0x7fe6d81a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:08.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6ddfff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81a07f0 con 0x7fe6d8102930 2023-09-20T21:48:08.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6ddfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81a0960 con 0x7fe6d8101f70 2023-09-20T21:48:08.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6ddfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81a0b20 con 0x7fe6d8100e20 2023-09-20T21:48:08.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6d7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8102930 0x7fe6d81a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:08.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6d6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d8101f70 0x7fe6d81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:08.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6d7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8102930 0x7fe6d81a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47922/0 (socket says 172.21.15.22:47922) 2023-09-20T21:48:08.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6d7fff700 1 -- 172.21.15.22:0/3614106655 learned_addr learned my addr 172.21.15.22:0/3614106655 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:08.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6d77fe700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6d8100e20 0x7fe6d8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:08.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6d6ffd700 1 -- 172.21.15.22:0/3614106655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6d8100e20 msgr2=0x7fe6d8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:08.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6d6ffd700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6d8100e20 0x7fe6d8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6d6ffd700 1 -- 172.21.15.22:0/3614106655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8102930 msgr2=0x7fe6d81a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.677+0000 7fe6d6ffd700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8102930 0x7fe6d81a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.678+0000 7fe6d6ffd700 1 -- 172.21.15.22:0/3614106655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8009520 con 0x7fe6d8101f70 2023-09-20T21:48:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.678+0000 7fe6d7fff700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8102930 0x7fe6d81a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:08.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.678+0000 7fe6d6ffd700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d8101f70 0x7fe6d81991c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0059d0 tx=0x7fe6cc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:08.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.678+0000 7fe6d4ff9700 1 -- 172.21.15.22:0/3614106655 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc01d070 con 0x7fe6d8101f70 2023-09-20T21:48:08.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.678+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3614106655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc0095f0 con 0x7fe6d8101f70 2023-09-20T21:48:08.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.678+0000 7fe6d4ff9700 1 -- 172.21.15.22:0/3614106655 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6cc022470 con 0x7fe6d8101f70 2023-09-20T21:48:08.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.678+0000 7fe6d4ff9700 1 -- 172.21.15.22:0/3614106655 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc017650 con 0x7fe6d8101f70 2023-09-20T21:48:08.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.678+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3614106655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d81a1040 con 0x7fe6d8101f70 2023-09-20T21:48:08.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.679+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3614106655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d810c650 con 0x7fe6d8101f70 2023-09-20T21:48:08.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.681+0000 7fe6d4ff9700 1 -- 172.21.15.22:0/3614106655 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe6cc0225e0 con 0x7fe6d8101f70 2023-09-20T21:48:08.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.683+0000 7fe6d4ff9700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe6c00a1350 0x7fe6c00a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:08.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.683+0000 7fe6d4ff9700 1 -- 172.21.15.22:0/3614106655 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe6cc0ce6f0 con 0x7fe6d8101f70 2023-09-20T21:48:08.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.683+0000 7fe6d77fe700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe6c00a1350 0x7fe6c00a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:08.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.684+0000 7fe6d77fe700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe6c00a1350 0x7fe6c00a3810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8199b50 tx=0x7fe6c8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:08.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.685+0000 7fe6d4ff9700 1 -- 172.21.15.22:0/3614106655 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe6cc097a60 con 0x7fe6d8101f70 2023-09-20T21:48:08.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.897+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3614106655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6d810c840 con 0x7fe6d8101f70 2023-09-20T21:48:08.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.900+0000 7fe6d4ff9700 1 -- 172.21.15.22:0/3614106655 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24645 (secure 0 0 0) 0x7fe6cc09b080 con 0x7fe6d8101f70 2023-09-20T21:48:08.900 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:08.901 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:08.580328+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:48:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3614106655 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe6c00a1350 msgr2=0x7fe6c00a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe6c00a1350 0x7fe6c00a3810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8199b50 tx=0x7fe6c8009260 comp rx=0 tx=0).stop 2023-09-20T21:48:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3614106655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d8101f70 msgr2=0x7fe6d81991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:08.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d8101f70 0x7fe6d81991c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0059d0 tx=0x7fe6cc004730 comp rx=0 tx=0).stop 2023-09-20T21:48:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3614106655 shutdown_connections 2023-09-20T21:48:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe6d8102930 0x7fe6d81a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6d8100e20 0x7fe6d8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe6c00a1350 0x7fe6c00a3810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 --2- 172.21.15.22:0/3614106655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6d8101f70 0x7fe6d81991c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:08.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3614106655 >> 172.21.15.22:0/3614106655 conn(0x7fe6d80fc390 msgr2=0x7fe6d81078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:08.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3614106655 shutdown_connections 2023-09-20T21:48:08.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:08.902+0000 7fe6ddfff700 1 -- 172.21.15.22:0/3614106655 wait complete. 2023-09-20T21:48:08.922 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T21:48:08.923 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:48:09.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.129+0000 7f6c131ff700 1 -- 172.21.15.22:0/3569676179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c0c0fffa0 msgr2=0x7f6c0c100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:09.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.129+0000 7f6c131ff700 1 --2- 172.21.15.22:0/3569676179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c0c0fffa0 0x7f6c0c100420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009990 tx=0x7f6bfc009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.129+0000 7f6c131ff700 1 -- 172.21.15.22:0/3569676179 shutdown_connections 2023-09-20T21:48:09.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.129+0000 7f6c131ff700 1 --2- 172.21.15.22:0/3569676179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0c100960 0x7f6c0c100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.129+0000 7f6c131ff700 1 --2- 172.21.15.22:0/3569676179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c0c0ff640 0x7f6c0c0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.129+0000 7f6c131ff700 1 --2- 172.21.15.22:0/3569676179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c0c0fffa0 0x7f6c0c100420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.129+0000 7f6c131ff700 1 -- 172.21.15.22:0/3569676179 >> 172.21.15.22:0/3569676179 conn(0x7f6c0c0fb240 msgr2=0x7f6c0c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:09.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.129+0000 7f6c131ff700 1 -- 172.21.15.22:0/3569676179 shutdown_connections 2023-09-20T21:48:09.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.129+0000 7f6c131ff700 1 -- 172.21.15.22:0/3569676179 wait complete. 2023-09-20T21:48:09.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.131+0000 7f6c131ff700 1 Processor -- start 2023-09-20T21:48:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.131+0000 7f6c131ff700 1 -- start start 2023-09-20T21:48:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.131+0000 7f6c131ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c0c0ff640 0x7f6c0c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.131+0000 7f6c131ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0c0fffa0 0x7f6c0c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.131+0000 7f6c131ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c0c100960 0x7f6c0c1a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.131+0000 7f6c131ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1a0890 con 0x7f6c0c100960 2023-09-20T21:48:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.131+0000 7f6c131ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1a0a00 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.131+0000 7f6c131ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1a0bc0 con 0x7f6c0c0ff640 2023-09-20T21:48:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c03fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0c0fffa0 0x7f6c0c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c03fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0c0fffa0 0x7f6c0c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46532/0 (socket says 172.21.15.22:46532) 2023-09-20T21:48:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c03fff700 1 -- 172.21.15.22:0/2214378353 learned_addr learned my addr 172.21.15.22:0/2214378353 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c10f9b700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c0c0ff640 0x7f6c0c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c1179c700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c0c100960 0x7f6c0c1a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c03fff700 1 -- 172.21.15.22:0/2214378353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c0c0ff640 msgr2=0x7f6c0c198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c03fff700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c0c0ff640 0x7f6c0c198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c03fff700 1 -- 172.21.15.22:0/2214378353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c0c100960 msgr2=0x7f6c0c1a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c03fff700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c0c100960 0x7f6c0c1a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c03fff700 1 -- 172.21.15.22:0/2214378353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0095f0 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.132+0000 7f6c03fff700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0c0fffa0 0x7f6c0c199290 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc004730 tx=0x7f6bfc004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.133+0000 7f6c01ffb700 1 -- 172.21.15.22:0/2214378353 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc01d070 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.133+0000 7f6c131ff700 1 -- 172.21.15.22:0/2214378353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c1a0e40 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.133+0000 7f6c01ffb700 1 -- 172.21.15.22:0/2214378353 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bfc004c90 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.133+0000 7f6c01ffb700 1 -- 172.21.15.22:0/2214378353 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0054b0 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.133+0000 7f6c131ff700 1 -- 172.21.15.22:0/2214378353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c1a11b0 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.134+0000 7f6c131ff700 1 -- 172.21.15.22:0/2214378353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c10a420 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.136+0000 7f6c01ffb700 1 -- 172.21.15.22:0/2214378353 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f6bfc00fbc0 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.137+0000 7f6c01ffb700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6bf40a1300 0x7f6bf40a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:09.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.138+0000 7f6c01ffb700 1 -- 172.21.15.22:0/2214378353 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6bfc0ce690 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.138+0000 7f6c10f9b700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6bf40a1300 0x7f6bf40a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:09.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.138+0000 7f6c10f9b700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6bf40a1300 0x7f6bf40a37c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c19a220 tx=0x7f6c08009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:09.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.139+0000 7f6c01ffb700 1 -- 172.21.15.22:0/2214378353 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6bfc097a00 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.352+0000 7f6c131ff700 1 -- 172.21.15.22:0/2214378353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c0c10e810 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.353+0000 7f6c01ffb700 1 -- 172.21.15.22:0/2214378353 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6bfc09b020 con 0x7f6c0c0fffa0 2023-09-20T21:48:09.353 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:48:09.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 -- 172.21.15.22:0/2214378353 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6bf40a1300 msgr2=0x7f6bf40a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:09.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6bf40a1300 0x7f6bf40a37c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c19a220 tx=0x7f6c08009260 comp rx=0 tx=0).stop 2023-09-20T21:48:09.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 -- 172.21.15.22:0/2214378353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0c0fffa0 msgr2=0x7f6c0c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:09.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0c0fffa0 0x7f6c0c199290 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc004730 tx=0x7f6bfc004760 comp rx=0 tx=0).stop 2023-09-20T21:48:09.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 -- 172.21.15.22:0/2214378353 shutdown_connections 2023-09-20T21:48:09.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c0c100960 0x7f6c0c1a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c0c0ff640 0x7f6c0c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6bf40a1300 0x7f6bf40a37c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 --2- 172.21.15.22:0/2214378353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c0c0fffa0 0x7f6c0c199290 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 -- 172.21.15.22:0/2214378353 >> 172.21.15.22:0/2214378353 conn(0x7f6c0c0fb240 msgr2=0x7f6c0c0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:09.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 -- 172.21.15.22:0/2214378353 shutdown_connections 2023-09-20T21:48:09.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.355+0000 7f6c131ff700 1 -- 172.21.15.22:0/2214378353 wait complete. 2023-09-20T21:48:09.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T21:48:09.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:09 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1449057867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:09.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-09-20T21:48:09.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-09-20T21:48:09.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]': finished 2023-09-20T21:48:09.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-09-20T21:48:09.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2023-09-20T21:48:09.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:09 smithi022 ceph-mon[97651]: osdmap e92: 12 total, 12 up, 12 in 2023-09-20T21:48:09.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:09 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3614106655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:09.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.586+0000 7fe2e5fff700 1 -- 172.21.15.22:0/2336940532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0103000 msgr2=0x7fe2e0103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:09.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.586+0000 7fe2e5fff700 1 --2- 172.21.15.22:0/2336940532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0103000 0x7fe2e0103420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00099c0 tx=0x7fe2d0009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.586+0000 7fe2e5fff700 1 -- 172.21.15.22:0/2336940532 shutdown_connections 2023-09-20T21:48:09.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.586+0000 7fe2e5fff700 1 --2- 172.21.15.22:0/2336940532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e0104b90 0x7fe2e010b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.586+0000 7fe2e5fff700 1 --2- 172.21.15.22:0/2336940532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0103000 0x7fe2e0103420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.586+0000 7fe2e5fff700 1 --2- 172.21.15.22:0/2336940532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2e01041f0 0x7fe2e0104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.586+0000 7fe2e5fff700 1 -- 172.21.15.22:0/2336940532 >> 172.21.15.22:0/2336940532 conn(0x7fe2e00fe590 msgr2=0x7fe2e01009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:09.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.586+0000 7fe2e5fff700 1 -- 172.21.15.22:0/2336940532 shutdown_connections 2023-09-20T21:48:09.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.586+0000 7fe2e5fff700 1 -- 172.21.15.22:0/2336940532 wait complete. 2023-09-20T21:48:09.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.587+0000 7fe2e5fff700 1 Processor -- start 2023-09-20T21:48:09.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.587+0000 7fe2e5fff700 1 -- start start 2023-09-20T21:48:09.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2e5fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0103000 0x7fe2e0198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:09.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2e5fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e01041f0 0x7fe2e0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:09.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2e5fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2e0104b90 0x7fe2e01a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:09.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2e5fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e01a0880 con 0x7fe2e0103000 2023-09-20T21:48:09.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2e5fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e01a09f0 con 0x7fe2e01041f0 2023-09-20T21:48:09.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2e5fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e01a0bb0 con 0x7fe2e0104b90 2023-09-20T21:48:09.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2df7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0103000 0x7fe2e0198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:09.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2deffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e01041f0 0x7fe2e0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2dffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2e0104b90 0x7fe2e01a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2deffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e01041f0 0x7fe2e0199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46550/0 (socket says 172.21.15.22:46550) 2023-09-20T21:48:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.588+0000 7fe2deffd700 1 -- 172.21.15.22:0/688985377 learned_addr learned my addr 172.21.15.22:0/688985377 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.589+0000 7fe2df7fe700 1 -- 172.21.15.22:0/688985377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2e0104b90 msgr2=0x7fe2e01a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:09.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.589+0000 7fe2df7fe700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2e0104b90 0x7fe2e01a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.589+0000 7fe2df7fe700 1 -- 172.21.15.22:0/688985377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e01041f0 msgr2=0x7fe2e0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:09.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.589+0000 7fe2df7fe700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e01041f0 0x7fe2e0199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.589+0000 7fe2df7fe700 1 -- 172.21.15.22:0/688985377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d00095f0 con 0x7fe2e0103000 2023-09-20T21:48:09.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.589+0000 7fe2deffd700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e01041f0 0x7fe2e0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:09.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.589+0000 7fe2df7fe700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0103000 0x7fe2e0198d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00098e0 tx=0x7fe2d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:09.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:09 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1449057867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:09.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-09-20T21:48:09.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-09-20T21:48:09.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]': finished 2023-09-20T21:48:09.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-09-20T21:48:09.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2023-09-20T21:48:09.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:09 smithi123 ceph-mon[134433]: osdmap e92: 12 total, 12 up, 12 in 2023-09-20T21:48:09.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:09 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3614106655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:09.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.594+0000 7fe2dcff9700 1 -- 172.21.15.22:0/688985377 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d001d070 con 0x7fe2e0103000 2023-09-20T21:48:09.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.594+0000 7fe2dcff9700 1 -- 172.21.15.22:0/688985377 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2d0005050 con 0x7fe2e0103000 2023-09-20T21:48:09.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.594+0000 7fe2e5fff700 1 -- 172.21.15.22:0/688985377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e01a0e30 con 0x7fe2e0103000 2023-09-20T21:48:09.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.594+0000 7fe2dcff9700 1 -- 172.21.15.22:0/688985377 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0004a70 con 0x7fe2e0103000 2023-09-20T21:48:09.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.594+0000 7fe2e5fff700 1 -- 172.21.15.22:0/688985377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e01a11a0 con 0x7fe2e0103000 2023-09-20T21:48:09.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.596+0000 7fe2e5fff700 1 -- 172.21.15.22:0/688985377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e010e860 con 0x7fe2e0103000 2023-09-20T21:48:09.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.599+0000 7fe2dcff9700 1 -- 172.21.15.22:0/688985377 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe2d0005450 con 0x7fe2e0103000 2023-09-20T21:48:09.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.601+0000 7fe2dcff9700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe2c80a1450 0x7fe2c80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:09.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.601+0000 7fe2dcff9700 1 -- 172.21.15.22:0/688985377 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe2d00cdd50 con 0x7fe2e0103000 2023-09-20T21:48:09.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.601+0000 7fe2deffd700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe2c80a1450 0x7fe2c80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:09.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.602+0000 7fe2deffd700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe2c80a1450 0x7fe2c80a3910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4011550 tx=0x7fe2d4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:09.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.603+0000 7fe2dcff9700 1 -- 172.21.15.22:0/688985377 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe2d00971a0 con 0x7fe2e0103000 2023-09-20T21:48:09.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:09 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1449057867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:09.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-09-20T21:48:09.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-09-20T21:48:09.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]': finished 2023-09-20T21:48:09.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-09-20T21:48:09.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2023-09-20T21:48:09.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:09 smithi089 ceph-mon[134090]: osdmap e92: 12 total, 12 up, 12 in 2023-09-20T21:48:09.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:09 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3614106655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:09.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.817+0000 7fe2e5fff700 1 -- 172.21.15.22:0/688985377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2e0109f50 con 0x7fe2e0103000 2023-09-20T21:48:09.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.817+0000 7fe2dcff9700 1 -- 172.21.15.22:0/688985377 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2d0026020 con 0x7fe2e0103000 2023-09-20T21:48:09.818 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 -- 172.21.15.22:0/688985377 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe2c80a1450 msgr2=0x7fe2c80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:09.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe2c80a1450 0x7fe2c80a3910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4011550 tx=0x7fe2d4009260 comp rx=0 tx=0).stop 2023-09-20T21:48:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 -- 172.21.15.22:0/688985377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0103000 msgr2=0x7fe2e0198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0103000 0x7fe2e0198d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00098e0 tx=0x7fe2d000f690 comp rx=0 tx=0).stop 2023-09-20T21:48:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 -- 172.21.15.22:0/688985377 shutdown_connections 2023-09-20T21:48:09.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2e0104b90 0x7fe2e01a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2e0103000 0x7fe2e0198d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe2c80a1450 0x7fe2c80a3910 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 --2- 172.21.15.22:0/688985377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe2e01041f0 0x7fe2e0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 -- 172.21.15.22:0/688985377 >> 172.21.15.22:0/688985377 conn(0x7fe2e00fe590 msgr2=0x7fe2e0100820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:09.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 -- 172.21.15.22:0/688985377 shutdown_connections 2023-09-20T21:48:09.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:09.820+0000 7fe2e5fff700 1 -- 172.21.15.22:0/688985377 wait complete. 2023-09-20T21:48:09.838 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T21:48:10.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.047+0000 7f3680fff700 1 -- 172.21.15.22:0/190888286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f367c101fe0 msgr2=0x7f367c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.047+0000 7f3680fff700 1 --2- 172.21.15.22:0/190888286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f367c101fe0 0x7f367c102440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f36700099c0 tx=0x7f3670009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.048+0000 7f3680fff700 1 -- 172.21.15.22:0/190888286 shutdown_connections 2023-09-20T21:48:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.048+0000 7f3680fff700 1 --2- 172.21.15.22:0/190888286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367c102980 0x7f367c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.048+0000 7f3680fff700 1 --2- 172.21.15.22:0/190888286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f367c100df0 0x7f367c101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.048+0000 7f3680fff700 1 --2- 172.21.15.22:0/190888286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f367c101fe0 0x7f367c102440 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.048+0000 7f3680fff700 1 -- 172.21.15.22:0/190888286 >> 172.21.15.22:0/190888286 conn(0x7f367c0fc360 msgr2=0x7f367c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:10.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.048+0000 7f3680fff700 1 -- 172.21.15.22:0/190888286 shutdown_connections 2023-09-20T21:48:10.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.048+0000 7f3680fff700 1 -- 172.21.15.22:0/190888286 wait complete. 2023-09-20T21:48:10.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.049+0000 7f3680fff700 1 Processor -- start 2023-09-20T21:48:10.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.049+0000 7f3680fff700 1 -- start start 2023-09-20T21:48:10.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f3680fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367c100df0 0x7f367c196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f3680fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f367c101fe0 0x7f367c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f3680fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f367c102980 0x7f367c19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f3680fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c19e660 con 0x7f367c101fe0 2023-09-20T21:48:10.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f3680fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c19e7a0 con 0x7f367c102980 2023-09-20T21:48:10.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f3680fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c19e930 con 0x7f367c100df0 2023-09-20T21:48:10.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f3679d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f367c101fe0 0x7f367c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f3679d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f367c101fe0 0x7f367c197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47972/0 (socket says 172.21.15.22:47972) 2023-09-20T21:48:10.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f3679d9b700 1 -- 172.21.15.22:0/2818906702 learned_addr learned my addr 172.21.15.22:0/2818906702 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:10.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f367ad9d700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f367c102980 0x7f367c19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.050+0000 7f367a59c700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367c100df0 0x7f367c196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f3679d9b700 1 -- 172.21.15.22:0/2818906702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367c100df0 msgr2=0x7f367c196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f3679d9b700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367c100df0 0x7f367c196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f3679d9b700 1 -- 172.21.15.22:0/2818906702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f367c102980 msgr2=0x7f367c19e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f3679d9b700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f367c102980 0x7f367c19e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f3679d9b700 1 -- 172.21.15.22:0/2818906702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36700095f0 con 0x7f367c101fe0 2023-09-20T21:48:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f3679d9b700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f367c101fe0 0x7f367c197020 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3670009990 tx=0x7f367000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f366b7fe700 1 -- 172.21.15.22:0/2818906702 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367001d070 con 0x7f367c101fe0 2023-09-20T21:48:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f3680fff700 1 -- 172.21.15.22:0/2818906702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367c19ebb0 con 0x7f367c101fe0 2023-09-20T21:48:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f366b7fe700 1 -- 172.21.15.22:0/2818906702 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3670005030 con 0x7f367c101fe0 2023-09-20T21:48:10.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f366b7fe700 1 -- 172.21.15.22:0/2818906702 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670004a70 con 0x7f367c101fe0 2023-09-20T21:48:10.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.051+0000 7f3680fff700 1 -- 172.21.15.22:0/2818906702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367c19f030 con 0x7f367c101fe0 2023-09-20T21:48:10.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.052+0000 7f3680fff700 1 -- 172.21.15.22:0/2818906702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367c107d70 con 0x7f367c101fe0 2023-09-20T21:48:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.054+0000 7f366b7fe700 1 -- 172.21.15.22:0/2818906702 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f3670005410 con 0x7f367c101fe0 2023-09-20T21:48:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.056+0000 7f366b7fe700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f36640a1380 0x7f36640a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.056+0000 7f366b7fe700 1 -- 172.21.15.22:0/2818906702 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f36700cdd10 con 0x7f367c101fe0 2023-09-20T21:48:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.056+0000 7f367a59c700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f36640a1380 0x7f36640a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.057+0000 7f367a59c700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f36640a1380 0x7f36640a3840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f367c101e40 tx=0x7f366c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:10.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.060+0000 7f366b7fe700 1 -- 172.21.15.22:0/2818906702 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f36700d3020 con 0x7f367c101fe0 2023-09-20T21:48:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.270+0000 7f3680fff700 1 -- 172.21.15.22:0/2818906702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f367c02d1f0 con 0x7f367c101fe0 2023-09-20T21:48:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.271+0000 7f366b7fe700 1 -- 172.21.15.22:0/2818906702 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3670097080 con 0x7f367c101fe0 2023-09-20T21:48:10.271 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T21:48:10.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 -- 172.21.15.22:0/2818906702 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f36640a1380 msgr2=0x7f36640a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f36640a1380 0x7f36640a3840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f367c101e40 tx=0x7f366c009260 comp rx=0 tx=0).stop 2023-09-20T21:48:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 -- 172.21.15.22:0/2818906702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f367c101fe0 msgr2=0x7f367c197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f367c101fe0 0x7f367c197020 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3670009990 tx=0x7f367000f690 comp rx=0 tx=0).stop 2023-09-20T21:48:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 -- 172.21.15.22:0/2818906702 shutdown_connections 2023-09-20T21:48:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f36640a1380 0x7f36640a3840 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f367c102980 0x7f367c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f367c100df0 0x7f367c196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 --2- 172.21.15.22:0/2818906702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f367c101fe0 0x7f367c197020 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 -- 172.21.15.22:0/2818906702 >> 172.21.15.22:0/2818906702 conn(0x7f367c0fc360 msgr2=0x7f367c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 -- 172.21.15.22:0/2818906702 shutdown_connections 2023-09-20T21:48:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.273+0000 7f3680fff700 1 -- 172.21.15.22:0/2818906702 wait complete. 2023-09-20T21:48:10.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T21:48:10.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:10 smithi022 ceph-mon[97651]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:10.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:10 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2214378353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:10.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:10 smithi022 ceph-mon[97651]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-20T21:48:10.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:10 smithi022 ceph-mon[97651]: osdmap e93: 12 total, 12 up, 12 in 2023-09-20T21:48:10.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:10 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/688985377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:10.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.498+0000 7f8c109ff700 1 -- 172.21.15.22:0/732547177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c0c101fe0 msgr2=0x7f8c0c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.498+0000 7f8c109ff700 1 --2- 172.21.15.22:0/732547177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c0c101fe0 0x7f8c0c102440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8c000099c0 tx=0x7f8c00009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.498+0000 7f8c109ff700 1 -- 172.21.15.22:0/732547177 shutdown_connections 2023-09-20T21:48:10.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.498+0000 7f8c109ff700 1 --2- 172.21.15.22:0/732547177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c102980 0x7f8c0c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.498+0000 7f8c109ff700 1 --2- 172.21.15.22:0/732547177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c0c100df0 0x7f8c0c101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.498+0000 7f8c109ff700 1 --2- 172.21.15.22:0/732547177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c0c101fe0 0x7f8c0c102440 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.498+0000 7f8c109ff700 1 -- 172.21.15.22:0/732547177 >> 172.21.15.22:0/732547177 conn(0x7f8c0c0fc360 msgr2=0x7f8c0c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:10.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.498+0000 7f8c109ff700 1 -- 172.21.15.22:0/732547177 shutdown_connections 2023-09-20T21:48:10.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.498+0000 7f8c109ff700 1 -- 172.21.15.22:0/732547177 wait complete. 2023-09-20T21:48:10.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.499+0000 7f8c109ff700 1 Processor -- start 2023-09-20T21:48:10.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.499+0000 7f8c109ff700 1 -- start start 2023-09-20T21:48:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.499+0000 7f8c109ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c0c100df0 0x7f8c0c198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.499+0000 7f8c109ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c0c101fe0 0x7f8c0c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.499+0000 7f8c109ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c102980 0x7f8c0c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.499+0000 7f8c109ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c1a0830 con 0x7f8c0c101fe0 2023-09-20T21:48:10.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.499+0000 7f8c109ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c1a09a0 con 0x7f8c0c102980 2023-09-20T21:48:10.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.499+0000 7f8c109ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c1a0b60 con 0x7f8c0c100df0 2023-09-20T21:48:10.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c0a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c0c100df0 0x7f8c0c198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c0a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c0c100df0 0x7f8c0c198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:40698/0 (socket says 172.21.15.22:40698) 2023-09-20T21:48:10.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c0a59c700 1 -- 172.21.15.22:0/2505072109 learned_addr learned my addr 172.21.15.22:0/2505072109 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:10.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c09d9b700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c0c101fe0 0x7f8c0c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c0ad9d700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c102980 0x7f8c0c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c0a59c700 1 -- 172.21.15.22:0/2505072109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c102980 msgr2=0x7f8c0c1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c0a59c700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c102980 0x7f8c0c1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c0a59c700 1 -- 172.21.15.22:0/2505072109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c0c101fe0 msgr2=0x7f8c0c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c0a59c700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c0c101fe0 0x7f8c0c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c0a59c700 1 -- 172.21.15.22:0/2505072109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000095f0 con 0x7f8c0c100df0 2023-09-20T21:48:10.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8c0a59c700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c0c100df0 0x7f8c0c198cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00e970 tx=0x7f8bfc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.500+0000 7f8bfb7fe700 1 -- 172.21.15.22:0/2505072109 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc009790 con 0x7f8c0c100df0 2023-09-20T21:48:10.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.501+0000 7f8bfb7fe700 1 -- 172.21.15.22:0/2505072109 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bfc00c7e0 con 0x7f8c0c100df0 2023-09-20T21:48:10.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.501+0000 7f8bfb7fe700 1 -- 172.21.15.22:0/2505072109 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc003c70 con 0x7f8c0c100df0 2023-09-20T21:48:10.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.501+0000 7f8c109ff700 1 -- 172.21.15.22:0/2505072109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c1a0e40 con 0x7f8c0c100df0 2023-09-20T21:48:10.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.501+0000 7f8c109ff700 1 -- 172.21.15.22:0/2505072109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0c1a1210 con 0x7f8c0c100df0 2023-09-20T21:48:10.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.502+0000 7f8c109ff700 1 -- 172.21.15.22:0/2505072109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0c10c600 con 0x7f8c0c100df0 2023-09-20T21:48:10.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.504+0000 7f8bfb7fe700 1 -- 172.21.15.22:0/2505072109 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f8bfc00cc50 con 0x7f8c0c100df0 2023-09-20T21:48:10.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.505+0000 7f8bfb7fe700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8bf40a1300 0x7f8bf40a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.505+0000 7f8bfb7fe700 1 -- 172.21.15.22:0/2505072109 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f8bfc014070 con 0x7f8c0c100df0 2023-09-20T21:48:10.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.505+0000 7f8c09d9b700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8bf40a1300 0x7f8bf40a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.506+0000 7f8c09d9b700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8bf40a1300 0x7f8bf40a37c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000b580 tx=0x7f8c00005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:10.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.507+0000 7f8bfb7fe700 1 -- 172.21.15.22:0/2505072109 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8bfc0964d0 con 0x7f8c0c100df0 2023-09-20T21:48:10.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:10 smithi123 ceph-mon[134433]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:10.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:10 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2214378353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:10.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:10 smithi123 ceph-mon[134433]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-20T21:48:10.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:10 smithi123 ceph-mon[134433]: osdmap e93: 12 total, 12 up, 12 in 2023-09-20T21:48:10.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:10 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/688985377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:10.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:10 smithi089 ceph-mon[134090]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:10.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:10 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2214378353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:10.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:10 smithi089 ceph-mon[134090]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-09-20T21:48:10.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:10 smithi089 ceph-mon[134090]: osdmap e93: 12 total, 12 up, 12 in 2023-09-20T21:48:10.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:10 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/688985377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.720+0000 7f8c109ff700 1 -- 172.21.15.22:0/2505072109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8c0c1a1460 con 0x7f8c0c100df0 2023-09-20T21:48:10.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.721+0000 7f8bfb7fe700 1 -- 172.21.15.22:0/2505072109 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8bfc003dd0 con 0x7f8c0c100df0 2023-09-20T21:48:10.722 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:10.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 -- 172.21.15.22:0/2505072109 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8bf40a1300 msgr2=0x7f8bf40a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8bf40a1300 0x7f8bf40a37c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000b580 tx=0x7f8c00005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 -- 172.21.15.22:0/2505072109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c0c100df0 msgr2=0x7f8c0c198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c0c100df0 0x7f8c0c198cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00e970 tx=0x7f8bfc00ec80 comp rx=0 tx=0).stop 2023-09-20T21:48:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 -- 172.21.15.22:0/2505072109 shutdown_connections 2023-09-20T21:48:10.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8bf40a1300 0x7f8bf40a37c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c0c102980 0x7f8c0c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c0c100df0 0x7f8c0c198cc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 --2- 172.21.15.22:0/2505072109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c0c101fe0 0x7f8c0c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 -- 172.21.15.22:0/2505072109 >> 172.21.15.22:0/2505072109 conn(0x7f8c0c0fc360 msgr2=0x7f8c0c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:10.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 -- 172.21.15.22:0/2505072109 shutdown_connections 2023-09-20T21:48:10.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.724+0000 7f8c109ff700 1 -- 172.21.15.22:0/2505072109 wait complete. 2023-09-20T21:48:10.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:48:10.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.949+0000 7f20887ff700 1 -- 172.21.15.22:0/3601899837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2080102970 msgr2=0x7f2080109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.949+0000 7f20887ff700 1 --2- 172.21.15.22:0/3601899837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2080102970 0x7f2080109220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f207800b0a0 tx=0x7f207800b3b0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.949+0000 7f20887ff700 1 -- 172.21.15.22:0/3601899837 shutdown_connections 2023-09-20T21:48:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.949+0000 7f20887ff700 1 --2- 172.21.15.22:0/3601899837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2080102970 0x7f2080109220 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.949+0000 7f20887ff700 1 --2- 172.21.15.22:0/3601899837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2080100de0 0x7f2080101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.949+0000 7f20887ff700 1 --2- 172.21.15.22:0/3601899837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2080101fd0 0x7f2080102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.949+0000 7f20887ff700 1 -- 172.21.15.22:0/3601899837 >> 172.21.15.22:0/3601899837 conn(0x7f20800fc310 msgr2=0x7f20800fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.949+0000 7f20887ff700 1 -- 172.21.15.22:0/3601899837 shutdown_connections 2023-09-20T21:48:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.949+0000 7f20887ff700 1 -- 172.21.15.22:0/3601899837 wait complete. 2023-09-20T21:48:10.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.950+0000 7f20887ff700 1 Processor -- start 2023-09-20T21:48:10.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.950+0000 7f20887ff700 1 -- start start 2023-09-20T21:48:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.950+0000 7f20887ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2080100de0 0x7f2080198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.950+0000 7f20887ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2080101fd0 0x7f2080199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.950+0000 7f20887ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2080102970 0x7f20801a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.950+0000 7f20887ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20801a08c0 con 0x7f2080100de0 2023-09-20T21:48:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.950+0000 7f20887ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20801a0a00 con 0x7f2080101fd0 2023-09-20T21:48:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.950+0000 7f20887ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20801a0b90 con 0x7f2080102970 2023-09-20T21:48:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.950+0000 7f208659b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2080100de0 0x7f2080198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f208659b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2080100de0 0x7f2080198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48022/0 (socket says 172.21.15.22:48022) 2023-09-20T21:48:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f208659b700 1 -- 172.21.15.22:0/584433712 learned_addr learned my addr 172.21.15.22:0/584433712 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f2086d9c700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2080102970 0x7f20801a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f2085d9a700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2080101fd0 0x7f2080199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f208659b700 1 -- 172.21.15.22:0/584433712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2080102970 msgr2=0x7f20801a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f208659b700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2080102970 0x7f20801a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f208659b700 1 -- 172.21.15.22:0/584433712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2080101fd0 msgr2=0x7f2080199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f208659b700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2080101fd0 0x7f2080199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f208659b700 1 -- 172.21.15.22:0/584433712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078009d00 con 0x7f2080100de0 2023-09-20T21:48:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f208659b700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2080100de0 0x7f2080198d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f207c00e970 tx=0x7f207c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f20777fe700 1 -- 172.21.15.22:0/584433712 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c009790 con 0x7f2080100de0 2023-09-20T21:48:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f20777fe700 1 -- 172.21.15.22:0/584433712 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f207c00c7e0 con 0x7f2080100de0 2023-09-20T21:48:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.951+0000 7f20887ff700 1 -- 172.21.15.22:0/584433712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20801a0e70 con 0x7f2080100de0 2023-09-20T21:48:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.952+0000 7f20777fe700 1 -- 172.21.15.22:0/584433712 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c003c70 con 0x7f2080100de0 2023-09-20T21:48:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.952+0000 7f20887ff700 1 -- 172.21.15.22:0/584433712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20801a1290 con 0x7f2080100de0 2023-09-20T21:48:10.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.953+0000 7f20887ff700 1 -- 172.21.15.22:0/584433712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208010c670 con 0x7f2080100de0 2023-09-20T21:48:10.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.955+0000 7f20777fe700 1 -- 172.21.15.22:0/584433712 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f207c00cc50 con 0x7f2080100de0 2023-09-20T21:48:10.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.956+0000 7f20777fe700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f206c0a1350 0x7f206c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:10.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.956+0000 7f20777fe700 1 -- 172.21.15.22:0/584433712 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f207c014070 con 0x7f2080100de0 2023-09-20T21:48:10.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.956+0000 7f2085d9a700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f206c0a1350 0x7f206c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:10.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.957+0000 7f2085d9a700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f206c0a1350 0x7f206c0a3810 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2070009c60 tx=0x7f2070009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:10.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:10.958+0000 7f20777fe700 1 -- 172.21.15.22:0/584433712 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f207c0965b0 con 0x7f2080100de0 2023-09-20T21:48:11.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.216+0000 7f20887ff700 1 -- 172.21.15.22:0/584433712 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f208010c850 con 0x7f2080100de0 2023-09-20T21:48:11.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.217+0000 7f20777fe700 1 -- 172.21.15.22:0/584433712 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f207c003dd0 con 0x7f2080100de0 2023-09-20T21:48:11.217 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:11.218 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:47:09.607119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T21:48:11.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 -- 172.21.15.22:0/584433712 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f206c0a1350 msgr2=0x7f206c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:11.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f206c0a1350 0x7f206c0a3810 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2070009c60 tx=0x7f2070009260 comp rx=0 tx=0).stop 2023-09-20T21:48:11.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 -- 172.21.15.22:0/584433712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2080100de0 msgr2=0x7f2080198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:11.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2080100de0 0x7f2080198d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f207c00e970 tx=0x7f207c00ec80 comp rx=0 tx=0).stop 2023-09-20T21:48:11.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 -- 172.21.15.22:0/584433712 shutdown_connections 2023-09-20T21:48:11.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2080102970 0x7f20801a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2080100de0 0x7f2080198d40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f206c0a1350 0x7f206c0a3810 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 --2- 172.21.15.22:0/584433712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2080101fd0 0x7f2080199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 -- 172.21.15.22:0/584433712 >> 172.21.15.22:0/584433712 conn(0x7f20800fc310 msgr2=0x7f2080107810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:11.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 -- 172.21.15.22:0/584433712 shutdown_connections 2023-09-20T21:48:11.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.219+0000 7f20887ff700 1 -- 172.21.15.22:0/584433712 wait complete. 2023-09-20T21:48:11.222 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2023-09-20T21:48:11.239 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:48:11.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.442+0000 7fe72f5ff700 1 -- 172.21.15.22:0/586429823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe728101fd0 msgr2=0x7fe728102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:11.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.442+0000 7fe72f5ff700 1 --2- 172.21.15.22:0/586429823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe728101fd0 0x7fe728102430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe7180099c0 tx=0x7fe718009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.442+0000 7fe72f5ff700 1 -- 172.21.15.22:0/586429823 shutdown_connections 2023-09-20T21:48:11.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.442+0000 7fe72f5ff700 1 --2- 172.21.15.22:0/586429823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe728102970 0x7fe728109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.442+0000 7fe72f5ff700 1 --2- 172.21.15.22:0/586429823 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe728100de0 0x7fe728101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.442+0000 7fe72f5ff700 1 --2- 172.21.15.22:0/586429823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe728101fd0 0x7fe728102430 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.442+0000 7fe72f5ff700 1 -- 172.21.15.22:0/586429823 >> 172.21.15.22:0/586429823 conn(0x7fe7280fc310 msgr2=0x7fe7280fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:11.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.442+0000 7fe72f5ff700 1 -- 172.21.15.22:0/586429823 shutdown_connections 2023-09-20T21:48:11.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.443+0000 7fe72f5ff700 1 -- 172.21.15.22:0/586429823 wait complete. 2023-09-20T21:48:11.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.443+0000 7fe72f5ff700 1 Processor -- start 2023-09-20T21:48:11.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72f5ff700 1 -- start start 2023-09-20T21:48:11.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72f5ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe728100de0 0x7fe728196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:11.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72f5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe728101fd0 0x7fe728197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:11.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72f5ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe728102970 0x7fe72819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:11.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72f5ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72819e6a0 con 0x7fe728102970 2023-09-20T21:48:11.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72f5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72819e7e0 con 0x7fe728101fd0 2023-09-20T21:48:11.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72f5ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72819e970 con 0x7fe728100de0 2023-09-20T21:48:11.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72db9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe728102970 0x7fe72819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:11.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72db9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe728102970 0x7fe72819e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48040/0 (socket says 172.21.15.22:48040) 2023-09-20T21:48:11.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72db9c700 1 -- 172.21.15.22:0/1137784337 learned_addr learned my addr 172.21.15.22:0/1137784337 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:11.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72cb9a700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe728101fd0 0x7fe728197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:11.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.444+0000 7fe72d39b700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe728100de0 0x7fe728196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:11.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe72db9c700 1 -- 172.21.15.22:0/1137784337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe728100de0 msgr2=0x7fe728196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:11.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe72db9c700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe728100de0 0x7fe728196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe72db9c700 1 -- 172.21.15.22:0/1137784337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe728101fd0 msgr2=0x7fe728197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:11.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe72db9c700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe728101fd0 0x7fe728197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe72db9c700 1 -- 172.21.15.22:0/1137784337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7180095f0 con 0x7fe728102970 2023-09-20T21:48:11.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe72db9c700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe728102970 0x7fe72819e0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe72000ef30 tx=0x7fe72000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:11.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe71e7fc700 1 -- 172.21.15.22:0/1137784337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720018070 con 0x7fe728102970 2023-09-20T21:48:11.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe71e7fc700 1 -- 172.21.15.22:0/1137784337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe72000f040 con 0x7fe728102970 2023-09-20T21:48:11.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe72f5ff700 1 -- 172.21.15.22:0/1137784337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72819ebf0 con 0x7fe728102970 2023-09-20T21:48:11.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe71e7fc700 1 -- 172.21.15.22:0/1137784337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72000cd00 con 0x7fe728102970 2023-09-20T21:48:11.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.445+0000 7fe72f5ff700 1 -- 172.21.15.22:0/1137784337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72819f100 con 0x7fe728102970 2023-09-20T21:48:11.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.446+0000 7fe72f5ff700 1 -- 172.21.15.22:0/1137784337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe728108380 con 0x7fe728102970 2023-09-20T21:48:11.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.448+0000 7fe71e7fc700 1 -- 172.21.15.22:0/1137784337 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe720003a00 con 0x7fe728102970 2023-09-20T21:48:11.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.451+0000 7fe71e7fc700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe7140a1350 0x7fe7140a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:11.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.451+0000 7fe71e7fc700 1 -- 172.21.15.22:0/1137784337 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe7200ccc30 con 0x7fe728102970 2023-09-20T21:48:11.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.451+0000 7fe72d39b700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe7140a1350 0x7fe7140a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:11.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.452+0000 7fe72d39b700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe7140a1350 0x7fe7140a3810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe728101e30 tx=0x7fe724009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:11.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.453+0000 7fe71e7fc700 1 -- 172.21.15.22:0/1137784337 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe720095fa0 con 0x7fe728102970 2023-09-20T21:48:11.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:11 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2818906702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:11.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:11 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2505072109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:11.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:11 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2818906702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:11.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:11 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2505072109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:11.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:11 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2818906702' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:11.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:11 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2505072109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:11.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.669+0000 7fe72f5ff700 1 -- 172.21.15.22:0/1137784337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe72810c640 con 0x7fe728102970 2023-09-20T21:48:11.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.671+0000 7fe71e7fc700 1 -- 172.21.15.22:0/1137784337 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7fe72004e060 con 0x7fe728102970 2023-09-20T21:48:11.672 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:11.672 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:48:11.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.673+0000 7fe72f5ff700 1 -- 172.21.15.22:0/1137784337 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe7140a1350 msgr2=0x7fe7140a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:11.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.673+0000 7fe72f5ff700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe7140a1350 0x7fe7140a3810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe728101e30 tx=0x7fe724009260 comp rx=0 tx=0).stop 2023-09-20T21:48:11.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.673+0000 7fe72f5ff700 1 -- 172.21.15.22:0/1137784337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe728102970 msgr2=0x7fe72819e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:11.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.673+0000 7fe72f5ff700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe728102970 0x7fe72819e0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe72000ef30 tx=0x7fe72000c3a0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.673+0000 7fe72f5ff700 1 -- 172.21.15.22:0/1137784337 shutdown_connections 2023-09-20T21:48:11.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.673+0000 7fe72f5ff700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe728102970 0x7fe72819e0d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.673+0000 7fe72f5ff700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe728100de0 0x7fe728196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.673+0000 7fe72f5ff700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe7140a1350 0x7fe7140a3810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.673+0000 7fe72f5ff700 1 --2- 172.21.15.22:0/1137784337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe728101fd0 0x7fe728197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.673+0000 7fe72f5ff700 1 -- 172.21.15.22:0/1137784337 >> 172.21.15.22:0/1137784337 conn(0x7fe7280fc310 msgr2=0x7fe7281054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:11.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.674+0000 7fe72f5ff700 1 -- 172.21.15.22:0/1137784337 shutdown_connections 2023-09-20T21:48:11.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.674+0000 7fe72f5ff700 1 -- 172.21.15.22:0/1137784337 wait complete. 2023-09-20T21:48:11.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:48:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.896+0000 7ff9207ff700 1 -- 172.21.15.22:0/1297597238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918101fd0 msgr2=0x7ff918102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.896+0000 7ff9207ff700 1 --2- 172.21.15.22:0/1297597238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918101fd0 0x7ff918102430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9080099c0 tx=0x7ff908009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.896+0000 7ff9207ff700 1 -- 172.21.15.22:0/1297597238 shutdown_connections 2023-09-20T21:48:11.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.896+0000 7ff9207ff700 1 --2- 172.21.15.22:0/1297597238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff918102970 0x7ff918109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.896+0000 7ff9207ff700 1 --2- 172.21.15.22:0/1297597238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff918100de0 0x7ff918101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.896+0000 7ff9207ff700 1 --2- 172.21.15.22:0/1297597238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918101fd0 0x7ff918102430 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.896+0000 7ff9207ff700 1 -- 172.21.15.22:0/1297597238 >> 172.21.15.22:0/1297597238 conn(0x7ff9180fc330 msgr2=0x7ff9180fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:11.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.897+0000 7ff9207ff700 1 -- 172.21.15.22:0/1297597238 shutdown_connections 2023-09-20T21:48:11.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.897+0000 7ff9207ff700 1 -- 172.21.15.22:0/1297597238 wait complete. 2023-09-20T21:48:11.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff9207ff700 1 Processor -- start 2023-09-20T21:48:11.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff9207ff700 1 -- start start 2023-09-20T21:48:11.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff9207ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff918100de0 0x7ff918196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:11.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff9207ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff918101fd0 0x7ff918197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:11.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff9207ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918102970 0x7ff91819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:11.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff9207ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91819e670 con 0x7ff918101fd0 2023-09-20T21:48:11.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff9207ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91819e7b0 con 0x7ff918102970 2023-09-20T21:48:11.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff9207ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91819e940 con 0x7ff918100de0 2023-09-20T21:48:11.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff91e59b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff918100de0 0x7ff918196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:11.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff91dd9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff918101fd0 0x7ff918197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:11.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff91e59b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff918100de0 0x7ff918196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:40730/0 (socket says 172.21.15.22:40730) 2023-09-20T21:48:11.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.898+0000 7ff91e59b700 1 -- 172.21.15.22:0/1260926311 learned_addr learned my addr 172.21.15.22:0/1260926311 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:11.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff91ed9c700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918102970 0x7ff91819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:11.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff91e59b700 1 -- 172.21.15.22:0/1260926311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918102970 msgr2=0x7ff91819e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:11.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff91e59b700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918102970 0x7ff91819e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff91e59b700 1 -- 172.21.15.22:0/1260926311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff918101fd0 msgr2=0x7ff918197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:11.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff91e59b700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff918101fd0 0x7ff918197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:11.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff91e59b700 1 -- 172.21.15.22:0/1260926311 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080095f0 con 0x7ff918100de0 2023-09-20T21:48:11.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff91dd9a700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff918101fd0 0x7ff918197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:11.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff91e59b700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff918100de0 0x7ff918196af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff91400e970 tx=0x7ff91400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:11.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff90f7fe700 1 -- 172.21.15.22:0/1260926311 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914009790 con 0x7ff918100de0 2023-09-20T21:48:11.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff90f7fe700 1 -- 172.21.15.22:0/1260926311 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff91400c7e0 con 0x7ff918100de0 2023-09-20T21:48:11.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff9207ff700 1 -- 172.21.15.22:0/1260926311 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91819ec20 con 0x7ff918100de0 2023-09-20T21:48:11.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.899+0000 7ff90f7fe700 1 -- 172.21.15.22:0/1260926311 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914003c70 con 0x7ff918100de0 2023-09-20T21:48:11.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.900+0000 7ff9207ff700 1 -- 172.21.15.22:0/1260926311 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91819f130 con 0x7ff918100de0 2023-09-20T21:48:11.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.900+0000 7ff9207ff700 1 -- 172.21.15.22:0/1260926311 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff918107d10 con 0x7ff918100de0 2023-09-20T21:48:11.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.903+0000 7ff90f7fe700 1 -- 172.21.15.22:0/1260926311 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ff91400cc50 con 0x7ff918100de0 2023-09-20T21:48:11.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.904+0000 7ff90f7fe700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9040a1350 0x7ff9040a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:11.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.904+0000 7ff90f7fe700 1 -- 172.21.15.22:0/1260926311 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ff914014070 con 0x7ff918100de0 2023-09-20T21:48:11.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.904+0000 7ff91dd9a700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9040a1350 0x7ff9040a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:11.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.905+0000 7ff91dd9a700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9040a1350 0x7ff9040a3810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b580 tx=0x7ff908005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:11.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:11.906+0000 7ff90f7fe700 1 -- 172.21.15.22:0/1260926311 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff9140965e0 con 0x7ff918100de0 2023-09-20T21:48:12.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.161+0000 7ff9207ff700 1 -- 172.21.15.22:0/1260926311 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff91802d1b0 con 0x7ff918100de0 2023-09-20T21:48:12.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.162+0000 7ff90f7fe700 1 -- 172.21.15.22:0/1260926311 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11361 (secure 0 0 0) 0x7ff914003dd0 con 0x7ff918100de0 2023-09-20T21:48:12.163 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:12.163 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:47:09.607119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T21:48:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.164+0000 7ff9207ff700 1 -- 172.21.15.22:0/1260926311 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9040a1350 msgr2=0x7ff9040a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.164+0000 7ff9207ff700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9040a1350 0x7ff9040a3810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff90800b580 tx=0x7ff908005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.165+0000 7ff9207ff700 1 -- 172.21.15.22:0/1260926311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff918100de0 msgr2=0x7ff918196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.165+0000 7ff9207ff700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff918100de0 0x7ff918196af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff91400e970 tx=0x7ff91400ec80 comp rx=0 tx=0).stop 2023-09-20T21:48:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.165+0000 7ff9207ff700 1 -- 172.21.15.22:0/1260926311 shutdown_connections 2023-09-20T21:48:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.165+0000 7ff9207ff700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff9040a1350 0x7ff9040a3810 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.165+0000 7ff9207ff700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff918102970 0x7ff91819e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.165+0000 7ff9207ff700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff918100de0 0x7ff918196af0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.165+0000 7ff9207ff700 1 --2- 172.21.15.22:0/1260926311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff918101fd0 0x7ff918197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.165+0000 7ff9207ff700 1 -- 172.21.15.22:0/1260926311 >> 172.21.15.22:0/1260926311 conn(0x7ff9180fc330 msgr2=0x7ff9180fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.165+0000 7ff9207ff700 1 -- 172.21.15.22:0/1260926311 shutdown_connections 2023-09-20T21:48:12.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.165+0000 7ff9207ff700 1 -- 172.21.15.22:0/1260926311 wait complete. 2023-09-20T21:48:12.168 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2023-09-20T21:48:12.184 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-20T21:46:41.044280+0000', 'modified': '2023-09-20T21:47:09.607119+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24578, 'mds_1': 24566, 'mds_2': 24530, 'mds_3': 24487, 'mds_4': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24461': {'gid': 24461, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6833/3516975495', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6832', 'nonce': 3516975495}, {'type': 'v1', 'addr': '172.21.15.89:6833', 'nonce': 3516975495}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'd', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.22:6837/4039103472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6836', 'nonce': 4039103472}, {'type': 'v1', 'addr': '172.21.15.22:6837', 'nonce': 4039103472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24473': {'gid': 24473, 'name': 'j', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.22:6835/56798476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6834', 'nonce': 56798476}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 56798476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.123:6835/835271994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 835271994}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 835271994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24497': {'gid': 24497, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6835/794452586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6834', 'nonce': 794452586}, {'type': 'v1', 'addr': '172.21.15.89:6835', 'nonce': 794452586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6837/1168160038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 1168160038}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 1168160038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6839/2368757728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 2368757728}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 2368757728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.89:6837/3887702690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 3887702690}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 3887702690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.22:6841/1806616417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6840', 'nonce': 1806616417}, {'type': 'v1', 'addr': '172.21.15.22:6841', 'nonce': 1806616417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6839/2165264535', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 2165264535}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 2165264535}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-20T21:48:12.185 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-09-20T21:48:12.185 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.j status 2023-09-20T21:48:12.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.390+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/2572945506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc98101fc0 msgr2=0x7fbc98102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.390+0000 7fbc9f5ff700 1 --2- 172.21.15.22:0/2572945506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc98101fc0 0x7fbc98102420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbc880099c0 tx=0x7fbc88009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.390+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/2572945506 shutdown_connections 2023-09-20T21:48:12.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.390+0000 7fbc9f5ff700 1 --2- 172.21.15.22:0/2572945506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc98102960 0x7fbc98109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.390+0000 7fbc9f5ff700 1 --2- 172.21.15.22:0/2572945506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc98100dd0 0x7fbc981011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.390+0000 7fbc9f5ff700 1 --2- 172.21.15.22:0/2572945506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc98101fc0 0x7fbc98102420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.390+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/2572945506 >> 172.21.15.22:0/2572945506 conn(0x7fbc980fc320 msgr2=0x7fbc980fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.390+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/2572945506 shutdown_connections 2023-09-20T21:48:12.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.390+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/2572945506 wait complete. 2023-09-20T21:48:12.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.391+0000 7fbc9f5ff700 1 Processor -- start 2023-09-20T21:48:12.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.391+0000 7fbc9f5ff700 1 -- start start 2023-09-20T21:48:12.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9f5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc98100dd0 0x7fbc98196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9f5ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc98101fc0 0x7fbc98197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9f5ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc98102960 0x7fbc9819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9f5ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9819e670 con 0x7fbc98102960 2023-09-20T21:48:12.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9f5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9819e7b0 con 0x7fbc98100dd0 2023-09-20T21:48:12.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9f5ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9819e940 con 0x7fbc98101fc0 2023-09-20T21:48:12.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9db9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc98102960 0x7fbc9819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9d39b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc98100dd0 0x7fbc98196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9d39b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc98100dd0 0x7fbc98196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46704/0 (socket says 172.21.15.22:46704) 2023-09-20T21:48:12.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9db9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc98102960 0x7fbc9819e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48084/0 (socket says 172.21.15.22:48084) 2023-09-20T21:48:12.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9d39b700 1 -- 172.21.15.22:0/3736050870 learned_addr learned my addr 172.21.15.22:0/3736050870 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.392+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc98101fc0 0x7fbc98197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc9db9c700 1 -- 172.21.15.22:0/3736050870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc98101fc0 msgr2=0x7fbc98197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc9db9c700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc98101fc0 0x7fbc98197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc9db9c700 1 -- 172.21.15.22:0/3736050870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc98100dd0 msgr2=0x7fbc98196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc9db9c700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc98100dd0 0x7fbc98196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc9db9c700 1 -- 172.21.15.22:0/3736050870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880095f0 con 0x7fbc98102960 2023-09-20T21:48:12.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc9db9c700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc98102960 0x7fbc9819e0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000efc0 tx=0x7fbc9000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc8e7fc700 1 -- 172.21.15.22:0/3736050870 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90018070 con 0x7fbc98102960 2023-09-20T21:48:12.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/3736050870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9819ebc0 con 0x7fbc98102960 2023-09-20T21:48:12.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc8e7fc700 1 -- 172.21.15.22:0/3736050870 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9000f040 con 0x7fbc98102960 2023-09-20T21:48:12.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc8e7fc700 1 -- 172.21.15.22:0/3736050870 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9000cd40 con 0x7fbc98102960 2023-09-20T21:48:12.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.393+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/3736050870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9819f120 con 0x7fbc98102960 2023-09-20T21:48:12.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.397+0000 7fbc8e7fc700 1 -- 172.21.15.22:0/3736050870 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fbc90003a00 con 0x7fbc98102960 2023-09-20T21:48:12.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.398+0000 7fbcb2f64b80 1 Processor -- start 2023-09-20T21:48:12.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.399+0000 7fbcb2f64b80 1 -- start start 2023-09-20T21:48:12.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.399+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c62ab0 0x558102c62ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.399+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.399+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63fb0 0x558102c64430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.399+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102bd2c50 con 0x558102c63410 2023-09-20T21:48:12.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.399+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102bb6c20 con 0x558102c63fb0 2023-09-20T21:48:12.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.399+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102bb4540 con 0x558102c62ab0 2023-09-20T21:48:12.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.399+0000 7fbc9d39b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc8e7fc700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbc840a1380 0x7fbc840a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9d39b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c639d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48088/0 (socket says 172.21.15.22:48088) 2023-09-20T21:48:12.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9d39b700 1 -- 172.21.15.22:0/393684034 learned_addr learned my addr 172.21.15.22:0/393684034 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/393684034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c62ab0 0x558102c62ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/393684034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63fb0 0x558102c64430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc8e7fc700 1 -- 172.21.15.22:0/3736050870 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fbc900ccb20 con 0x7fbc98102960 2023-09-20T21:48:12.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9db9c700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbc840a1380 0x7fbc840a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9d39b700 1 -- 172.21.15.22:0/393684034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c62ab0 msgr2=0x558102c62ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/393684034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c62ab0 0x558102c62ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9d39b700 1 -- 172.21.15.22:0/393684034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63fb0 msgr2=0x558102c64430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/393684034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63fb0 0x558102c64430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9d39b700 1 -- 172.21.15.22:0/393684034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581029dad90 con 0x558102c63410 2023-09-20T21:48:12.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/393684034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63fb0 0x558102c64430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:12.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.400+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/393684034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c639d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009df0 tx=0x7fbc94011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2da2da9f8728980 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbc9db9c700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbc840a1380 0x7fbc840a3840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009320 tx=0x7fbc90004dc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbc837fe700 1 -- 172.21.15.22:0/393684034 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94004230 con 0x558102c63410 2023-09-20T21:48:12.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbc837fe700 1 -- 172.21.15.22:0/393684034 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc94004390 con 0x558102c63410 2023-09-20T21:48:12.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbc837fe700 1 -- 172.21.15.22:0/393684034 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc940198c0 con 0x558102c63410 2023-09-20T21:48:12.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/393684034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 msgr2=0x558102c639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/393684034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c639d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009df0 tx=0x7fbc94011720 comp rx=0 tx=0).stop 2023-09-20T21:48:12.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/393684034 shutdown_connections 2023-09-20T21:48:12.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/393684034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63fb0 0x558102c64430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/393684034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c62ab0 0x558102c62ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/393684034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c639d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.401+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/393684034 >> 172.21.15.22:0/393684034 conn(0x558102b8afb0 msgr2=0x558102c59180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.402+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/393684034 shutdown_connections 2023-09-20T21:48:12.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.402+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/393684034 wait complete. 2023-09-20T21:48:12.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.402+0000 7fbcb2f64b80 1 -- start start 2023-09-20T21:48:12.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.402+0000 7fbcb2f64b80 1 Processor -- start 2023-09-20T21:48:12.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c62ab0 0x558102c6c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63410 0x558102c6ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63fb0 0x558102c73a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102bafe30 con 0x558102c63fb0 2023-09-20T21:48:12.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102b4c6f0 con 0x558102c63410 2023-09-20T21:48:12.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102b4c390 con 0x558102c62ab0 2023-09-20T21:48:12.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbcb2f64b80 10 client.0 _ll_register_callbacks cb 0x558102a96a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:12.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9d39b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63410 0x558102c6ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9d39b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63410 0x558102c6ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46734/0 (socket says 172.21.15.22:46734) 2023-09-20T21:48:12.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9d39b700 1 -- 172.21.15.22:0/1236056811 learned_addr learned my addr 172.21.15.22:0/1236056811 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63fb0 0x558102c73a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c62ab0 0x558102c6c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9d39b700 1 -- 172.21.15.22:0/1236056811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c62ab0 msgr2=0x558102c6c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c62ab0 0x558102c6c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9d39b700 1 -- 172.21.15.22:0/1236056811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63fb0 msgr2=0x558102c73a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63fb0 0x558102c73a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9d39b700 1 -- 172.21.15.22:0/1236056811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc94009a50 con 0x558102c63410 2023-09-20T21:48:12.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.403+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63fb0 0x558102c73a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:12.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.404+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63410 0x558102c6ca30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbc94005950 tx=0x7fbc94003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=261edffbd18b1b99 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.404+0000 7fbc80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:48:12.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.404+0000 7fbc80ff9700 1 -- 172.21.15.22:0/1236056811 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94005460 con 0x558102c63410 2023-09-20T21:48:12.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.404+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/1236056811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558102b20b60 con 0x558102c63410 2023-09-20T21:48:12.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.404+0000 7fbc80ff9700 1 -- 172.21.15.22:0/1236056811 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fbc940055c0 con 0x558102c63410 2023-09-20T21:48:12.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.404+0000 7fbc80ff9700 1 -- 172.21.15.22:0/1236056811 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc94005d50 con 0x558102c63410 2023-09-20T21:48:12.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.404+0000 7fbcb2f64b80 10 client.25115 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:12.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.404+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/1236056811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581029c3c10 con 0x558102c63410 2023-09-20T21:48:12.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.404+0000 7fbc80ff9700 1 -- 172.21.15.22:0/1236056811 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558102b20b60 con 0x558102c63410 2023-09-20T21:48:12.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.405+0000 7fbc80ff9700 1 -- 172.21.15.22:0/1236056811 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbc94021040 con 0x558102c63410 2023-09-20T21:48:12.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.405+0000 7fbcb2f64b80 10 client.25115 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:12.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.405+0000 7fbcb2f64b80 10 client.25115 resolve_mds: resolved name 'j' to daemon mds.j 2023-09-20T21:48:12.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.405+0000 7fbcb2f64b80 20 client.25115 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:12.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.405+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x558102c78060 0x558102c76b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.405+0000 7fbcb2f64b80 4 client.25115 mds_command: new command op to 24473 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:12.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.405+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/1236056811 --> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558102c2cfd0 con 0x558102c78060 2023-09-20T21:48:12.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.405+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x558102c78060 0x558102c76b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.406+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x558102c78060 0x558102c76b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=592369b06353408d server_cookie=e95cfab90930e5fc in_seq=0 out_seq=0 2023-09-20T21:48:12.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.406+0000 7fbc80ff9700 10 client.25115 ms_handle_connect on v2:172.21.15.22:6834/56798476 2023-09-20T21:48:12.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbc80ff9700 1 -- 172.21.15.22:0/1236056811 <== mds.4 v2:172.21.15.22:6834/56798476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558102c2cfd0 con 0x558102c78060 2023-09-20T21:48:12.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbc80ff9700 10 client.25115 handle_command_reply: tid=0 2023-09-20T21:48:12.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 client.25115 shutdown 2023-09-20T21:48:12.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/1236056811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63410 msgr2=0x558102c6ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63410 0x558102c6ca30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbc94005950 tx=0x7fbc94003960 comp rx=0 tx=0).stop 2023-09-20T21:48:12.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/1236056811 shutdown_connections 2023-09-20T21:48:12.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x558102c78060 0x558102c76b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63fb0 0x558102c73a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c62ab0 0x558102c6c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbc80ff9700 0 client.25115 ms_handle_reset on v2:172.21.15.22:6834/56798476 2023-09-20T21:48:12.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/1236056811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c63410 0x558102c6ca30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/1236056811 >> 172.21.15.22:0/1236056811 conn(0x558102b8afb0 msgr2=0x558102c58830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/1236056811 shutdown_connections 2023-09-20T21:48:12.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/1236056811 wait complete. 2023-09-20T21:48:12.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.407+0000 7fbcb2f64b80 20 client.25115 trim_cache size 0 max 16384 2023-09-20T21:48:12.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.425+0000 7fbcb2f64b80 1 Processor -- start 2023-09-20T21:48:12.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.425+0000 7fbcb2f64b80 1 -- start start 2023-09-20T21:48:12.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.425+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c58830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.425+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c73990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.425+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c63fb0 0x558102c6a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.425+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102a308d0 con 0x558102c63410 2023-09-20T21:48:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.425+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102b48aa0 con 0x558102c62ab0 2023-09-20T21:48:12.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.425+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102babf20 con 0x558102c63fb0 2023-09-20T21:48:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9d39b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c73990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9cb9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c58830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9d39b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c73990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48100/0 (socket says 172.21.15.22:48100) 2023-09-20T21:48:12.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9d39b700 1 -- 172.21.15.22:0/3565261641 learned_addr learned my addr 172.21.15.22:0/3565261641 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9cb9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c58830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46742/0 (socket says 172.21.15.22:46742) 2023-09-20T21:48:12.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/3565261641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c63fb0 0x558102c6a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9d39b700 1 -- 172.21.15.22:0/3565261641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c63fb0 msgr2=0x558102c6a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/3565261641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c63fb0 0x558102c6a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9d39b700 1 -- 172.21.15.22:0/3565261641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 msgr2=0x558102c58830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/3565261641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c58830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9d39b700 1 -- 172.21.15.22:0/3565261641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558102a01200 con 0x558102c63410 2023-09-20T21:48:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.426+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/3565261641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c73990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbc94004850 tx=0x7fbc94003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724191b8148704a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbc83fff700 1 -- 172.21.15.22:0/3565261641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94015030 con 0x558102c63410 2023-09-20T21:48:12.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbc83fff700 1 -- 172.21.15.22:0/3565261641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc94005d50 con 0x558102c63410 2023-09-20T21:48:12.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbc83fff700 1 -- 172.21.15.22:0/3565261641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94019490 con 0x558102c63410 2023-09-20T21:48:12.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/3565261641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 msgr2=0x558102c73990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/3565261641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c73990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbc94004850 tx=0x7fbc94003f20 comp rx=0 tx=0).stop 2023-09-20T21:48:12.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/3565261641 shutdown_connections 2023-09-20T21:48:12.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/3565261641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c63fb0 0x558102c6a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/3565261641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c58830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/3565261641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c73990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/3565261641 >> 172.21.15.22:0/3565261641 conn(0x558102b8afb0 msgr2=0x558102c6b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.427+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/3565261641 shutdown_connections 2023-09-20T21:48:12.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.428+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/3565261641 wait complete. 2023-09-20T21:48:12.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.428+0000 7fbcb2f64b80 1 -- start start 2023-09-20T21:48:12.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.428+0000 7fbcb2f64b80 1 Processor -- start 2023-09-20T21:48:12.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c72f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbcb2f64b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c63fb0 0x558102c73590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102c73cd0 con 0x558102c63410 2023-09-20T21:48:12.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102c73e40 con 0x558102c62ab0 2023-09-20T21:48:12.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbcb2f64b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558102ccba50 con 0x558102c63fb0 2023-09-20T21:48:12.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbcb2f64b80 10 client.0 _ll_register_callbacks cb 0x558102a96a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:12.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9cb9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9cb9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46756/0 (socket says 172.21.15.22:46756) 2023-09-20T21:48:12.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9cb9a700 1 -- 172.21.15.22:0/4155330602 learned_addr learned my addr 172.21.15.22:0/4155330602 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c72f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c63fb0 0x558102c73590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9cb9a700 1 -- 172.21.15.22:0/4155330602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c63fb0 msgr2=0x558102c73590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c63fb0 0x558102c73590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9cb9a700 1 -- 172.21.15.22:0/4155330602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 msgr2=0x558102c72f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c72f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9cb9a700 1 -- 172.21.15.22:0/4155330602 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581029c3c10 con 0x558102c62ab0 2023-09-20T21:48:12.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.429+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c72f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.430+0000 7fbc9cb9a700 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c709c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800bf40 tx=0x7fbc88005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aece79a6aff21e20 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.430+0000 7fbc80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:48:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.430+0000 7fbc80ff9700 1 -- 172.21.15.22:0/4155330602 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88033070 con 0x558102c62ab0 2023-09-20T21:48:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.430+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/4155330602 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc94004390 con 0x558102c62ab0 2023-09-20T21:48:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.430+0000 7fbc80ff9700 1 -- 172.21.15.22:0/4155330602 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fbc8802b430 con 0x558102c62ab0 2023-09-20T21:48:12.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.430+0000 7fbc80ff9700 1 -- 172.21.15.22:0/4155330602 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc88005080 con 0x558102c62ab0 2023-09-20T21:48:12.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.430+0000 7fbcb2f64b80 10 client.25121 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:12.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.430+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/4155330602 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5581029d9260 con 0x558102c62ab0 2023-09-20T21:48:12.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.430+0000 7fbc80ff9700 1 -- 172.21.15.22:0/4155330602 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc94004390 con 0x558102c62ab0 2023-09-20T21:48:12.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.431+0000 7fbc80ff9700 1 -- 172.21.15.22:0/4155330602 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbc88036020 con 0x558102c62ab0 2023-09-20T21:48:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.431+0000 7fbcb2f64b80 10 client.25121 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.431+0000 7fbcb2f64b80 10 client.25121 resolve_mds: resolved name 'j' to daemon mds.j 2023-09-20T21:48:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.431+0000 7fbcb2f64b80 20 client.25121 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.431+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x558102cd0ce0 0x558102ccd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.431+0000 7fbcb2f64b80 4 client.25121 mds_command: new command op to 24473 tid=0[{"prefix": "status"}] 2023-09-20T21:48:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.431+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/4155330602 --> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558102ba8e10 con 0x558102cd0ce0 2023-09-20T21:48:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.431+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x558102cd0ce0 0x558102ccd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.431+0000 7fbc9d39b700 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x558102cd0ce0 0x558102ccd140 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b9325ad84d16d0f server_cookie=a6caa394684afeb4 in_seq=0 out_seq=0 2023-09-20T21:48:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.431+0000 7fbc80ff9700 10 client.25121 ms_handle_connect on v2:172.21.15.22:6834/56798476 2023-09-20T21:48:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbc80ff9700 1 -- 172.21.15.22:0/4155330602 <== mds.4 v2:172.21.15.22:6834/56798476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558102ba8e10 con 0x558102cd0ce0 2023-09-20T21:48:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbc80ff9700 10 client.25121 handle_command_reply: tid=0 2023-09-20T21:48:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 client.25121 shutdown 2023-09-20T21:48:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/4155330602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 msgr2=0x558102c709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c709c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800bf40 tx=0x7fbc88005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/4155330602 shutdown_connections 2023-09-20T21:48:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558102c63fb0 0x558102c73590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558102c62ab0 0x558102c709c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x558102cd0ce0 0x558102ccd140 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 --2- 172.21.15.22:0/4155330602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558102c63410 0x558102c72f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/4155330602 >> 172.21.15.22:0/4155330602 conn(0x558102b8afb0 msgr2=0x558102c6b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbc80ff9700 0 client.25121 ms_handle_reset on v2:172.21.15.22:6834/56798476 2023-09-20T21:48:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/4155330602 shutdown_connections 2023-09-20T21:48:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 1 -- 172.21.15.22:0/4155330602 wait complete. 2023-09-20T21:48:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.432+0000 7fbcb2f64b80 20 client.25121 trim_cache size 0 max 16384 2023-09-20T21:48:12.457 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:12.458 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:48:12.458 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 4, 2023-09-20T21:48:12.458 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24473, 2023-09-20T21:48:12.458 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:48:12.459 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:48:12.459 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:48:12.459 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 63.865966102999998, 2023-09-20T21:48:12.459 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 28, 2023-09-20T21:48:12.459 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2023-09-20T21:48:12.459 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 91, 2023-09-20T21:48:12.460 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 120.394919496 2023-09-20T21:48:12.460 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/3736050870 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbc840a1380 msgr2=0x7fbc840a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbc840a1380 0x7fbc840a3840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009320 tx=0x7fbc90004dc0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/3736050870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc98102960 msgr2=0x7fbc9819e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc98102960 0x7fbc9819e0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000efc0 tx=0x7fbc9000c3a0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/3736050870 shutdown_connections 2023-09-20T21:48:12.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbc98102960 0x7fbc9819e0a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbc840a1380 0x7fbc840a3840 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbc98100dd0 0x7fbc98196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 --2- 172.21.15.22:0/3736050870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc98101fc0 0x7fbc98197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/3736050870 >> 172.21.15.22:0/3736050870 conn(0x7fbc980fc320 msgr2=0x7fbc980fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/3736050870 shutdown_connections 2023-09-20T21:48:12.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.433+0000 7fbc9f5ff700 1 -- 172.21.15.22:0/3736050870 wait complete. 2023-09-20T21:48:12.463 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:12 smithi022 ceph-mon[97651]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-09-20T21:48:12.464 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:12 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/584433712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:12.464 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:12 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1137784337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:12.464 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:12 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1260926311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:12.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-09-20T21:48:12.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:12 smithi123 ceph-mon[134433]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-09-20T21:48:12.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:12 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/584433712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:12.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:12 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1137784337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:12.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:12 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1260926311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:12.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:12 smithi089 ceph-mon[134090]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-09-20T21:48:12.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:12 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/584433712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:12.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:12 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1137784337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:12.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:12 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1260926311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.671+0000 7f6baefff700 1 -- 172.21.15.22:0/3116830793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 msgr2=0x7f6ba8101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.671+0000 7f6baefff700 1 --2- 172.21.15.22:0/3116830793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 0x7f6ba8101220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009920 tx=0x7f6b9c009c30 comp rx=0 tx=0).stop 2023-09-20T21:48:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.671+0000 7f6baefff700 1 -- 172.21.15.22:0/3116830793 shutdown_connections 2023-09-20T21:48:12.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.671+0000 7f6baefff700 1 --2- 172.21.15.22:0/3116830793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8102990 0x7f6ba8109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.671+0000 7f6baefff700 1 --2- 172.21.15.22:0/3116830793 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8100e00 0x7f6ba8101220 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.671+0000 7f6baefff700 1 --2- 172.21.15.22:0/3116830793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba8101ff0 0x7f6ba8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.671+0000 7f6baefff700 1 -- 172.21.15.22:0/3116830793 >> 172.21.15.22:0/3116830793 conn(0x7f6ba80fc390 msgr2=0x7f6ba80fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.671+0000 7f6baefff700 1 -- 172.21.15.22:0/3116830793 shutdown_connections 2023-09-20T21:48:12.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.671+0000 7f6baefff700 1 -- 172.21.15.22:0/3116830793 wait complete. 2023-09-20T21:48:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.672+0000 7f6baefff700 1 Processor -- start 2023-09-20T21:48:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.672+0000 7f6baefff700 1 -- start start 2023-09-20T21:48:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6baefff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba8100e00 0x7f6ba8196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6baefff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8101ff0 0x7f6ba8197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6baefff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8102990 0x7f6ba819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6baefff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba819e760 con 0x7f6ba8102990 2023-09-20T21:48:12.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6baefff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba819e8a0 con 0x7f6ba8101ff0 2023-09-20T21:48:12.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6baefff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba819ea30 con 0x7f6ba8100e00 2023-09-20T21:48:12.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6bacd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba8100e00 0x7f6ba8196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6bad59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8102990 0x7f6ba819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6bacd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba8100e00 0x7f6ba8196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:40792/0 (socket says 172.21.15.22:40792) 2023-09-20T21:48:12.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6bacd9b700 1 -- 172.21.15.22:0/2755068331 learned_addr learned my addr 172.21.15.22:0/2755068331 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8101ff0 0x7f6ba8197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6bad59c700 1 -- 172.21.15.22:0/2755068331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba8100e00 msgr2=0x7f6ba8196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6bad59c700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba8100e00 0x7f6ba8196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6bad59c700 1 -- 172.21.15.22:0/2755068331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8101ff0 msgr2=0x7f6ba8197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6bad59c700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8101ff0 0x7f6ba8197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.673+0000 7f6bad59c700 1 -- 172.21.15.22:0/2755068331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b94009520 con 0x7f6ba8102990 2023-09-20T21:48:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.674+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba8100e00 0x7f6ba8196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.674+0000 7f6bad59c700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8102990 0x7f6ba819e190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009cd0 tx=0x7f6ba000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.674+0000 7f6ba5ffb700 1 -- 172.21.15.22:0/2755068331 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0017070 con 0x7f6ba8102990 2023-09-20T21:48:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.674+0000 7f6baefff700 1 -- 172.21.15.22:0/2755068331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba0009d00 con 0x7f6ba8102990 2023-09-20T21:48:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.674+0000 7f6ba5ffb700 1 -- 172.21.15.22:0/2755068331 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ba000f040 con 0x7f6ba8102990 2023-09-20T21:48:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.674+0000 7f6ba5ffb700 1 -- 172.21.15.22:0/2755068331 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000e050 con 0x7f6ba8102990 2023-09-20T21:48:12.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.674+0000 7f6baefff700 1 -- 172.21.15.22:0/2755068331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9c009520 con 0x7f6ba8102990 2023-09-20T21:48:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.677+0000 7f6ba5ffb700 1 -- 172.21.15.22:0/2755068331 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f6ba00090d0 con 0x7f6ba8102990 2023-09-20T21:48:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6bc2998b80 1 Processor -- start 2023-09-20T21:48:12.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6bc2998b80 1 -- start start 2023-09-20T21:48:12.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6ba5ffb700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6b980a1300 0x7f6b980a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823fe6a0 0x562f823feac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6ba5ffb700 1 -- 172.21.15.22:0/2755068331 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6ba00cc8d0 con 0x7f6ba8102990 2023-09-20T21:48:12.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f823ff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ffb90 0x562f82400010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f82372c50 con 0x562f823ffb90 2023-09-20T21:48:12.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f82356c20 con 0x562f823ff000 2023-09-20T21:48:12.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f82354540 con 0x562f823fe6a0 2023-09-20T21:48:12.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.679+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6b980a1300 0x7f6b980a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bacd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823fe6a0 0x562f823feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bacd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823fe6a0 0x562f823feac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:40800/0 (socket says 172.21.15.22:40800) 2023-09-20T21:48:12.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bacd9b700 1 -- 172.21.15.22:0/2910640549 learned_addr learned my addr 172.21.15.22:0/2910640549 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/2910640549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f823ff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bad59c700 1 --2- 172.21.15.22:0/2910640549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ffb90 0x562f82400010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bacd9b700 1 -- 172.21.15.22:0/2910640549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 msgr2=0x562f823ff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/2910640549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f823ff590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bacd9b700 1 -- 172.21.15.22:0/2910640549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ffb90 msgr2=0x562f82400010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/2910640549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ffb90 0x562f82400010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bacd9b700 1 -- 172.21.15.22:0/2910640549 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f8217ad90 con 0x562f823fe6a0 2023-09-20T21:48:12.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bad59c700 1 --2- 172.21.15.22:0/2910640549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ffb90 0x562f82400010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:12.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/2910640549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823fe6a0 0x562f823feac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0094f0 tx=0x7f6b9c009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4574be4a51202d3b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.680+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6b980a1300 0x7f6b980a37c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x562f823fa100 tx=0x7f6b94009a60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.681+0000 7f6b92ffd700 1 -- 172.21.15.22:0/2910640549 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c01c050 con 0x562f823fe6a0 2023-09-20T21:48:12.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.681+0000 7f6b92ffd700 1 -- 172.21.15.22:0/2910640549 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b9c01d040 con 0x562f823fe6a0 2023-09-20T21:48:12.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.681+0000 7f6b92ffd700 1 -- 172.21.15.22:0/2910640549 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c005440 con 0x562f823fe6a0 2023-09-20T21:48:12.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.681+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2910640549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823fe6a0 msgr2=0x562f823feac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.681+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/2910640549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823fe6a0 0x562f823feac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0094f0 tx=0x7f6b9c009fa0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.681+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2910640549 shutdown_connections 2023-09-20T21:48:12.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.681+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/2910640549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ffb90 0x562f82400010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.681+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/2910640549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823fe6a0 0x562f823feac0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.681+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/2910640549 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f823ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.681+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2910640549 >> 172.21.15.22:0/2910640549 conn(0x562f8232afb0 msgr2=0x562f823f9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.682+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2910640549 shutdown_connections 2023-09-20T21:48:12.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.682+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2910640549 wait complete. 2023-09-20T21:48:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.682+0000 7f6bc2998b80 1 -- start start 2023-09-20T21:48:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.682+0000 7f6bc2998b80 1 Processor -- start 2023-09-20T21:48:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f824080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f824085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8234fe30 con 0x562f823fe6a0 2023-09-20T21:48:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f822ec6f0 con 0x562f823ff000 2023-09-20T21:48:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f822ec390 con 0x562f823ffb90 2023-09-20T21:48:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bc2998b80 10 client.0 _ll_register_callbacks cb 0x562f82236a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bacd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f824080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bacd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f824080b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48150/0 (socket says 172.21.15.22:48150) 2023-09-20T21:48:12.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bacd9b700 1 -- 172.21.15.22:0/1666942998 learned_addr learned my addr 172.21.15.22:0/1666942998 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6bad59c700 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.683+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f824085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6bacd9b700 1 -- 172.21.15.22:0/1666942998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 msgr2=0x562f8240f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6bacd9b700 1 -- 172.21.15.22:0/1666942998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 msgr2=0x562f824085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f824085f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6bacd9b700 1 -- 172.21.15.22:0/1666942998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba8109b90 con 0x562f823fe6a0 2023-09-20T21:48:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6bad59c700 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f824080b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c011010 tx=0x7f6b9c005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a8338694ff7fe7 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6b87fff700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:12.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6b87fff700 1 -- 172.21.15.22:0/1666942998 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c01c030 con 0x562f823fe6a0 2023-09-20T21:48:12.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6bc2998b80 1 -- 172.21.15.22:0/1666942998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f822c0b60 con 0x562f823fe6a0 2023-09-20T21:48:12.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6b87fff700 1 -- 172.21.15.22:0/1666942998 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6b9c00b970 con 0x562f823fe6a0 2023-09-20T21:48:12.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6b87fff700 1 -- 172.21.15.22:0/1666942998 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b9c0056f0 con 0x562f823fe6a0 2023-09-20T21:48:12.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6bc2998b80 10 client.15099 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:12.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.684+0000 7f6bc2998b80 1 -- 172.21.15.22:0/1666942998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562f82163c10 con 0x562f823fe6a0 2023-09-20T21:48:12.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.685+0000 7f6b87fff700 1 -- 172.21.15.22:0/1666942998 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f822c0b60 con 0x562f823fe6a0 2023-09-20T21:48:12.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.685+0000 7f6b87fff700 1 -- 172.21.15.22:0/1666942998 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b9c009ca0 con 0x562f823fe6a0 2023-09-20T21:48:12.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.685+0000 7f6bc2998b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:12.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.685+0000 7f6bc2998b80 10 client.15099 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-20T21:48:12.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.685+0000 7f6bc2998b80 20 client.15099 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:12.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.685+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x562f824137e0 0x562f82412560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.685+0000 7f6bc2998b80 4 client.15099 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:12.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.685+0000 7f6bc2998b80 1 -- 172.21.15.22:0/1666942998 --> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f823ccfd0 con 0x562f824137e0 2023-09-20T21:48:12.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.686+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x562f824137e0 0x562f82412560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.686+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x562f824137e0 0x562f82412560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e48ea18b8de8d167 server_cookie=9d0e77994d648b84 in_seq=0 out_seq=0 2023-09-20T21:48:12.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.686+0000 7f6b87fff700 10 client.15099 ms_handle_connect on v2:172.21.15.123:6834/835271994 2023-09-20T21:48:12.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6b87fff700 1 -- 172.21.15.22:0/1666942998 <== mds.3 v2:172.21.15.123:6834/835271994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562f823ccfd0 con 0x562f824137e0 2023-09-20T21:48:12.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6b87fff700 10 client.15099 handle_command_reply: tid=0 2023-09-20T21:48:12.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 client.15099 shutdown 2023-09-20T21:48:12.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 -- 172.21.15.22:0/1666942998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 msgr2=0x562f824080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f824080b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c011010 tx=0x7f6b9c005440 comp rx=0 tx=0).stop 2023-09-20T21:48:12.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 -- 172.21.15.22:0/1666942998 shutdown_connections 2023-09-20T21:48:12.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x562f824137e0 0x562f82412560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f824080b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6b87fff700 0 client.15099 ms_handle_reset on v2:172.21.15.123:6834/835271994 2023-09-20T21:48:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/1666942998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f824085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 -- 172.21.15.22:0/1666942998 >> 172.21.15.22:0/1666942998 conn(0x562f8232afb0 msgr2=0x562f823f8830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 -- 172.21.15.22:0/1666942998 shutdown_connections 2023-09-20T21:48:12.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.687+0000 7f6bc2998b80 1 -- 172.21.15.22:0/1666942998 wait complete. 2023-09-20T21:48:12.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.688+0000 7f6bc2998b80 20 client.15099 trim_cache size 0 max 16384 2023-09-20T21:48:12.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bc2998b80 1 Processor -- start 2023-09-20T21:48:12.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bc2998b80 1 -- start start 2023-09-20T21:48:12.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f82412560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f82412aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f821d08d0 con 0x562f823fe6a0 2023-09-20T21:48:12.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f82408fb0 con 0x562f823ff000 2023-09-20T21:48:12.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f823ccfd0 con 0x562f823ffb90 2023-09-20T21:48:12.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bacd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f82412560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bacd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f82412560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48162/0 (socket says 172.21.15.22:48162) 2023-09-20T21:48:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bacd9b700 1 -- 172.21.15.22:0/3804086259 learned_addr learned my addr 172.21.15.22:0/3804086259 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/3804086259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f82412aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.706+0000 7f6bad59c700 1 --2- 172.21.15.22:0/3804086259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.707+0000 7f6bacd9b700 1 -- 172.21.15.22:0/3804086259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 msgr2=0x562f8240f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.707+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/3804086259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.707+0000 7f6bacd9b700 1 -- 172.21.15.22:0/3804086259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 msgr2=0x562f82412aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.707+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/3804086259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f82412aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.707+0000 7f6bacd9b700 1 -- 172.21.15.22:0/3804086259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f821a1200 con 0x562f823fe6a0 2023-09-20T21:48:12.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.707+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/3804086259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f82412560 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00bf70 tx=0x7f6b9c005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e991a3270cd4f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.707+0000 7f6b937fe700 1 -- 172.21.15.22:0/3804086259 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c01c040 con 0x562f823fe6a0 2023-09-20T21:48:12.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.707+0000 7f6b937fe700 1 -- 172.21.15.22:0/3804086259 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b9c0054b0 con 0x562f823fe6a0 2023-09-20T21:48:12.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.707+0000 7f6b937fe700 1 -- 172.21.15.22:0/3804086259 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0038e0 con 0x562f823fe6a0 2023-09-20T21:48:12.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 -- 172.21.15.22:0/3804086259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 msgr2=0x562f82412560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/3804086259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f82412560 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00bf70 tx=0x7f6b9c005bf0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 -- 172.21.15.22:0/3804086259 shutdown_connections 2023-09-20T21:48:12.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/3804086259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/3804086259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823fe6a0 0x562f82412560 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/3804086259 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823ff000 0x562f82412aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 -- 172.21.15.22:0/3804086259 >> 172.21.15.22:0/3804086259 conn(0x562f8232afb0 msgr2=0x562f82406080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 -- 172.21.15.22:0/3804086259 shutdown_connections 2023-09-20T21:48:12.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 -- 172.21.15.22:0/3804086259 wait complete. 2023-09-20T21:48:12.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 -- start start 2023-09-20T21:48:12.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.708+0000 7f6bc2998b80 1 Processor -- start 2023-09-20T21:48:12.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823fe6a0 0x562f8240f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ff000 0x562f8240f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6bc2998b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8240c360 con 0x562f823ff000 2023-09-20T21:48:12.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8240c4d0 con 0x562f823fe6a0 2023-09-20T21:48:12.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6bc2998b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8240c690 con 0x562f823ffb90 2023-09-20T21:48:12.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6bc2998b80 10 client.0 _ll_register_callbacks cb 0x562f82236a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:12.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6ba7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ff000 0x562f8240f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6ba7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ff000 0x562f8240f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48178/0 (socket says 172.21.15.22:48178) 2023-09-20T21:48:12.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6ba7fff700 1 -- 172.21.15.22:0/2791102679 learned_addr learned my addr 172.21.15.22:0/2791102679 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823fe6a0 0x562f8240f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6bad59c700 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6ba7fff700 1 -- 172.21.15.22:0/2791102679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 msgr2=0x562f8240bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6ba7fff700 1 -- 172.21.15.22:0/2791102679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823fe6a0 msgr2=0x562f8240f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823fe6a0 0x562f8240f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6ba7fff700 1 -- 172.21.15.22:0/2791102679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f82163c10 con 0x562f823ff000 2023-09-20T21:48:12.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.709+0000 7f6bad59c700 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:12.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.710+0000 7f6ba7fff700 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ff000 0x562f8240f630 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400a010 tx=0x7f6b94009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef5dbd1829196b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.710+0000 7f6b87fff700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:12.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.710+0000 7f6b87fff700 1 -- 172.21.15.22:0/2791102679 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b94026040 con 0x562f823ff000 2023-09-20T21:48:12.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.710+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2791102679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f82348e10 con 0x562f823ff000 2023-09-20T21:48:12.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.710+0000 7f6b87fff700 1 -- 172.21.15.22:0/2791102679 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6b94011a00 con 0x562f823ff000 2023-09-20T21:48:12.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.710+0000 7f6b87fff700 1 -- 172.21.15.22:0/2791102679 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b9402d5b0 con 0x562f823ff000 2023-09-20T21:48:12.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.710+0000 7f6bc2998b80 10 client.15105 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:12.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.710+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2791102679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562f82179260 con 0x562f823ff000 2023-09-20T21:48:12.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.711+0000 7f6b87fff700 1 -- 172.21.15.22:0/2791102679 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562f82348e10 con 0x562f823ff000 2023-09-20T21:48:12.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.711+0000 7f6b87fff700 1 -- 172.21.15.22:0/2791102679 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6b94021b90 con 0x562f823ff000 2023-09-20T21:48:12.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.711+0000 7f6bc2998b80 10 client.15105 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:12.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.711+0000 7f6bc2998b80 10 client.15105 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-20T21:48:12.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.711+0000 7f6bc2998b80 20 client.15105 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:12.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.711+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x562f8246c2e0 0x562f82468720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.711+0000 7f6bc2998b80 4 client.15105 mds_command: new command op to 24487 tid=0[{"prefix": "status"}] 2023-09-20T21:48:12.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.711+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2791102679 --> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562f82348b30 con 0x562f8246c2e0 2023-09-20T21:48:12.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.711+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x562f8246c2e0 0x562f82468720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bacd9b700 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x562f8246c2e0 0x562f82468720 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=714abb57fd79ebe4 server_cookie=3188188ef5e8fc71 in_seq=0 out_seq=0 2023-09-20T21:48:12.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6b87fff700 10 client.15105 ms_handle_connect on v2:172.21.15.123:6834/835271994 2023-09-20T21:48:12.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6b87fff700 1 -- 172.21.15.22:0/2791102679 <== mds.3 v2:172.21.15.123:6834/835271994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x562f82348b30 con 0x562f8246c2e0 2023-09-20T21:48:12.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6b87fff700 10 client.15105 handle_command_reply: tid=0 2023-09-20T21:48:12.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 client.15105 shutdown 2023-09-20T21:48:12.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2791102679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ff000 msgr2=0x562f8240f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ff000 0x562f8240f630 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400a010 tx=0x7f6b94009260 comp rx=0 tx=0).stop 2023-09-20T21:48:12.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2791102679 shutdown_connections 2023-09-20T21:48:12.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x562f8246c2e0 0x562f82468720 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562f823ffb90 0x562f8240bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6b87fff700 0 client.15105 ms_handle_reset on v2:172.21.15.123:6834/835271994 2023-09-20T21:48:12.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f823fe6a0 0x562f8240f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 --2- 172.21.15.22:0/2791102679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562f823ff000 0x562f8240f630 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2791102679 >> 172.21.15.22:0/2791102679 conn(0x562f8232afb0 msgr2=0x562f82402bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2791102679 shutdown_connections 2023-09-20T21:48:12.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.712+0000 7f6bc2998b80 1 -- 172.21.15.22:0/2791102679 wait complete. 2023-09-20T21:48:12.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6bc2998b80 20 client.15105 trim_cache size 0 max 16384 2023-09-20T21:48:12.737 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:12.737 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:48:12.738 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 3, 2023-09-20T21:48:12.738 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24487, 2023-09-20T21:48:12.738 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:48:12.738 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:48:12.739 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:48:12.739 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 65.151380763000006, 2023-09-20T21:48:12.739 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 28, 2023-09-20T21:48:12.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2023-09-20T21:48:12.740 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 91, 2023-09-20T21:48:12.740 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 113.099227145 2023-09-20T21:48:12.741 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:12.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 -- 172.21.15.22:0/2755068331 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6b980a1300 msgr2=0x7f6b980a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6b980a1300 0x7f6b980a37c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x562f823fa100 tx=0x7f6b94009a60 comp rx=0 tx=0).stop 2023-09-20T21:48:12.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 -- 172.21.15.22:0/2755068331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8102990 msgr2=0x7f6ba819e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8102990 0x7f6ba819e190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009cd0 tx=0x7f6ba000c620 comp rx=0 tx=0).stop 2023-09-20T21:48:12.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 -- 172.21.15.22:0/2755068331 shutdown_connections 2023-09-20T21:48:12.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ba8102990 0x7f6ba819e190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba8100e00 0x7f6ba8196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6b980a1300 0x7f6b980a37c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 --2- 172.21.15.22:0/2755068331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8101ff0 0x7f6ba8197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 -- 172.21.15.22:0/2755068331 >> 172.21.15.22:0/2755068331 conn(0x7f6ba80fc390 msgr2=0x7f6ba80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 -- 172.21.15.22:0/2755068331 shutdown_connections 2023-09-20T21:48:12.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.713+0000 7f6baefff700 1 -- 172.21.15.22:0/2755068331 wait complete. 2023-09-20T21:48:12.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-20T21:48:12.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.950+0000 7f16b8fff700 1 -- 172.21.15.22:0/1216490016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b4100df0 msgr2=0x7f16b4101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.950+0000 7f16b8fff700 1 --2- 172.21.15.22:0/1216490016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b4100df0 0x7f16b4101210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f16a40099c0 tx=0x7f16a4009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.950+0000 7f16b8fff700 1 -- 172.21.15.22:0/1216490016 shutdown_connections 2023-09-20T21:48:12.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.950+0000 7f16b8fff700 1 --2- 172.21.15.22:0/1216490016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b4102980 0x7f16b4109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.950+0000 7f16b8fff700 1 --2- 172.21.15.22:0/1216490016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b4100df0 0x7f16b4101210 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.950+0000 7f16b8fff700 1 --2- 172.21.15.22:0/1216490016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b4101fe0 0x7f16b4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.950+0000 7f16b8fff700 1 -- 172.21.15.22:0/1216490016 >> 172.21.15.22:0/1216490016 conn(0x7f16b40fc360 msgr2=0x7f16b40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.950+0000 7f16b8fff700 1 -- 172.21.15.22:0/1216490016 shutdown_connections 2023-09-20T21:48:12.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.950+0000 7f16b8fff700 1 -- 172.21.15.22:0/1216490016 wait complete. 2023-09-20T21:48:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.951+0000 7f16b8fff700 1 Processor -- start 2023-09-20T21:48:12.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.951+0000 7f16b8fff700 1 -- start start 2023-09-20T21:48:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.951+0000 7f16b8fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b4100df0 0x7f16b4198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.951+0000 7f16b8fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b4101fe0 0x7f16b4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.951+0000 7f16b8fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b4102980 0x7f16b41a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.951+0000 7f16b8fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b41a0870 con 0x7f16b4101fe0 2023-09-20T21:48:12.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.951+0000 7f16b8fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b41a09e0 con 0x7f16b4100df0 2023-09-20T21:48:12.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.951+0000 7f16b8fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b41a0ba0 con 0x7f16b4102980 2023-09-20T21:48:12.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b4102980 0x7f16b41a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b4101fe0 0x7f16b4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b4102980 0x7f16b41a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:40854/0 (socket says 172.21.15.22:40854) 2023-09-20T21:48:12.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b2d9d700 1 -- 172.21.15.22:0/2077050716 learned_addr learned my addr 172.21.15.22:0/2077050716 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b259c700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b4100df0 0x7f16b4198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b2d9d700 1 -- 172.21.15.22:0/2077050716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b4100df0 msgr2=0x7f16b4198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b2d9d700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b4100df0 0x7f16b4198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b2d9d700 1 -- 172.21.15.22:0/2077050716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b4101fe0 msgr2=0x7f16b4199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b2d9d700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b4101fe0 0x7f16b4199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b2d9d700 1 -- 172.21.15.22:0/2077050716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a40095f0 con 0x7f16b4102980 2023-09-20T21:48:12.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16b2d9d700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b4102980 0x7f16b41a02a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00ef30 tx=0x7f16ac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16a37fe700 1 -- 172.21.15.22:0/2077050716 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac018070 con 0x7f16b4102980 2023-09-20T21:48:12.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.953+0000 7f16b8fff700 1 -- 172.21.15.22:0/2077050716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b41a0e20 con 0x7f16b4102980 2023-09-20T21:48:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.952+0000 7f16a37fe700 1 -- 172.21.15.22:0/2077050716 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16ac00f040 con 0x7f16b4102980 2023-09-20T21:48:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.953+0000 7f16a37fe700 1 -- 172.21.15.22:0/2077050716 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac00cd00 con 0x7f16b4102980 2023-09-20T21:48:12.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.953+0000 7f16b8fff700 1 -- 172.21.15.22:0/2077050716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b41a1240 con 0x7f16b4102980 2023-09-20T21:48:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.956+0000 7f16a37fe700 1 -- 172.21.15.22:0/2077050716 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f16ac003a00 con 0x7f16b4102980 2023-09-20T21:48:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.957+0000 7f16cc94db80 1 Processor -- start 2023-09-20T21:48:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16cc94db80 1 -- start start 2023-09-20T21:48:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e36a0 0x564bc31e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4000 0x564bc31e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4b90 0x564bc31e5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16cc94db80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc3157c50 con 0x564bc31e4000 2023-09-20T21:48:12.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16cc94db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc313bc20 con 0x564bc31e4b90 2023-09-20T21:48:12.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16cc94db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc3139540 con 0x564bc31e36a0 2023-09-20T21:48:12.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4000 0x564bc31e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4000 0x564bc31e4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48206/0 (socket says 172.21.15.22:48206) 2023-09-20T21:48:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16b259c700 1 -- 172.21.15.22:0/1220567543 learned_addr learned my addr 172.21.15.22:0/1220567543 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.958+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1220567543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e36a0 0x564bc31e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1220567543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4b90 0x564bc31e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16a37fe700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f169c0a1300 0x7f169c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16b259c700 1 -- 172.21.15.22:0/1220567543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e36a0 msgr2=0x564bc31e3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16b259c700 1 --2- 172.21.15.22:0/1220567543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e36a0 0x564bc31e3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16b259c700 1 -- 172.21.15.22:0/1220567543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4b90 msgr2=0x564bc31e5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16b259c700 1 --2- 172.21.15.22:0/1220567543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4b90 0x564bc31e5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16b259c700 1 -- 172.21.15.22:0/1220567543 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bc2f5fd90 con 0x564bc31e4000 2023-09-20T21:48:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1220567543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4b90 0x564bc31e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16a37fe700 1 -- 172.21.15.22:0/2077050716 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f16ac0ccb30 con 0x7f16b4102980 2023-09-20T21:48:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16b2d9d700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f169c0a1300 0x7f169c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.959+0000 7f16b259c700 1 --2- 172.21.15.22:0/1220567543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4000 0x564bc31e4590 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f16a40099c0 tx=0x7f16a400bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829158141e180341 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16a0ff9700 1 -- 172.21.15.22:0/1220567543 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a401c050 con 0x564bc31e4000 2023-09-20T21:48:12.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16a0ff9700 1 -- 172.21.15.22:0/1220567543 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a401d040 con 0x564bc31e4000 2023-09-20T21:48:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16a0ff9700 1 -- 172.21.15.22:0/1220567543 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a400fcf0 con 0x564bc31e4000 2023-09-20T21:48:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16cc94db80 1 -- 172.21.15.22:0/1220567543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4000 msgr2=0x564bc31e4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1220567543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4000 0x564bc31e4590 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f16a40099c0 tx=0x7f16a400bfb0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16b2d9d700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f169c0a1300 0x7f169c0a37c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000f80 tx=0x7f16ac004dc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16cc94db80 1 -- 172.21.15.22:0/1220567543 shutdown_connections 2023-09-20T21:48:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1220567543 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4b90 0x564bc31e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1220567543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e36a0 0x564bc31e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1220567543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4000 0x564bc31e4590 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16cc94db80 1 -- 172.21.15.22:0/1220567543 >> 172.21.15.22:0/1220567543 conn(0x564bc310ffb0 msgr2=0x564bc31de180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.960+0000 7f16cc94db80 1 -- 172.21.15.22:0/1220567543 shutdown_connections 2023-09-20T21:48:12.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.961+0000 7f16cc94db80 1 -- 172.21.15.22:0/1220567543 wait complete. 2023-09-20T21:48:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.961+0000 7f16cc94db80 1 -- start start 2023-09-20T21:48:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.961+0000 7f16cc94db80 1 Processor -- start 2023-09-20T21:48:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e36a0 0x564bc31ed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 0x564bc31ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4b90 0x564bc31f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16cc94db80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc3134e30 con 0x564bc31e4b90 2023-09-20T21:48:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16cc94db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc30d16f0 con 0x564bc31e4000 2023-09-20T21:48:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16cc94db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc30d1390 con 0x564bc31e36a0 2023-09-20T21:48:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16cc94db80 10 client.0 _ll_register_callbacks cb 0x564bc301ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:12.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16b259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 0x564bc31ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16b259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 0x564bc31ed630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46848/0 (socket says 172.21.15.22:46848) 2023-09-20T21:48:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16b259c700 1 -- 172.21.15.22:0/2315211636 learned_addr learned my addr 172.21.15.22:0/2315211636 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4b90 0x564bc31f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e36a0 0x564bc31ed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16b259c700 1 -- 172.21.15.22:0/2315211636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e36a0 msgr2=0x564bc31ed0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16b259c700 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e36a0 0x564bc31ed0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16b259c700 1 -- 172.21.15.22:0/2315211636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4b90 msgr2=0x564bc31f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16b259c700 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4b90 0x564bc31f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.962+0000 7f16b259c700 1 -- 172.21.15.22:0/2315211636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b4109b80 con 0x564bc31e4000 2023-09-20T21:48:12.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.963+0000 7f16b259c700 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 0x564bc31ed630 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009990 tx=0x7f16a4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45b49e9a93603642 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.963+0000 7f1699ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:48:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.963+0000 7f1699ffb700 1 -- 172.21.15.22:0/2315211636 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a400bba0 con 0x564bc31e4000 2023-09-20T21:48:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.963+0000 7f16cc94db80 1 -- 172.21.15.22:0/2315211636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bc30a5b60 con 0x564bc31e4000 2023-09-20T21:48:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.963+0000 7f1699ffb700 1 -- 172.21.15.22:0/2315211636 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f16a400bd00 con 0x564bc31e4000 2023-09-20T21:48:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.963+0000 7f1699ffb700 1 -- 172.21.15.22:0/2315211636 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a40050d0 con 0x564bc31e4000 2023-09-20T21:48:12.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.963+0000 7f16cc94db80 10 client.25151 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.963+0000 7f16cc94db80 1 -- 172.21.15.22:0/2315211636 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564bc2f48c10 con 0x564bc31e4000 2023-09-20T21:48:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.964+0000 7f1699ffb700 1 -- 172.21.15.22:0/2315211636 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564bc30a5b60 con 0x564bc31e4000 2023-09-20T21:48:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.964+0000 7f1699ffb700 1 -- 172.21.15.22:0/2315211636 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f16a40239a0 con 0x564bc31e4000 2023-09-20T21:48:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.964+0000 7f16cc94db80 10 client.25151 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.964+0000 7f16cc94db80 10 client.25151 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-20T21:48:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.964+0000 7f16cc94db80 20 client.25151 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.964+0000 7f16cc94db80 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x564bc31f8780 0x564bc31f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.964+0000 7f16cc94db80 4 client.25151 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.964+0000 7f16cc94db80 1 -- 172.21.15.22:0/2315211636 --> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bc31b1fd0 con 0x564bc31f8780 2023-09-20T21:48:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.964+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x564bc31f8780 0x564bc31f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.965+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x564bc31f8780 0x564bc31f75a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3eec15d27f69ca30 server_cookie=c702622fbef20cda in_seq=0 out_seq=0 2023-09-20T21:48:12.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.965+0000 7f1699ffb700 10 client.25151 ms_handle_connect on v2:172.21.15.89:6836/3887702690 2023-09-20T21:48:12.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f1699ffb700 1 -- 172.21.15.22:0/2315211636 <== mds.2 v2:172.21.15.89:6836/3887702690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564bc31b1fd0 con 0x564bc31f8780 2023-09-20T21:48:12.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f1699ffb700 10 client.25151 handle_command_reply: tid=0 2023-09-20T21:48:12.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 client.25151 shutdown 2023-09-20T21:48:12.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 -- 172.21.15.22:0/2315211636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 msgr2=0x564bc31ed630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 0x564bc31ed630 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009990 tx=0x7f16a4009580 comp rx=0 tx=0).stop 2023-09-20T21:48:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 -- 172.21.15.22:0/2315211636 shutdown_connections 2023-09-20T21:48:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x564bc31f8780 0x564bc31f75a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e4b90 0x564bc31f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e36a0 0x564bc31ed0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 --2- 172.21.15.22:0/2315211636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 0x564bc31ed630 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f1699ffb700 0 client.25151 ms_handle_reset on v2:172.21.15.89:6836/3887702690 2023-09-20T21:48:12.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 -- 172.21.15.22:0/2315211636 >> 172.21.15.22:0/2315211636 conn(0x564bc310ffb0 msgr2=0x564bc31dd830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 -- 172.21.15.22:0/2315211636 shutdown_connections 2023-09-20T21:48:12.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 1 -- 172.21.15.22:0/2315211636 wait complete. 2023-09-20T21:48:12.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.966+0000 7f16cc94db80 20 client.25151 trim_cache size 0 max 16384 2023-09-20T21:48:12.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16cc94db80 1 Processor -- start 2023-09-20T21:48:12.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16cc94db80 1 -- start start 2023-09-20T21:48:12.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc310ffb0 0x564bc31e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e4000 0x564bc31f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16cc94db80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc31eccc0 con 0x564bc31e36a0 2023-09-20T21:48:12.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16cc94db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc2fb58d0 con 0x564bc310ffb0 2023-09-20T21:48:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16cc94db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc31df090 con 0x564bc31e4000 2023-09-20T21:48:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48214/0 (socket says 172.21.15.22:48214) 2023-09-20T21:48:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.985+0000 7f16b259c700 1 -- 172.21.15.22:0/1365451828 learned_addr learned my addr 172.21.15.22:0/1365451828 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1365451828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc310ffb0 0x564bc31e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1365451828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e4000 0x564bc31f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16b259c700 1 -- 172.21.15.22:0/1365451828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e4000 msgr2=0x564bc31f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16b259c700 1 --2- 172.21.15.22:0/1365451828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e4000 0x564bc31f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16b259c700 1 -- 172.21.15.22:0/1365451828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc310ffb0 msgr2=0x564bc31e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16b259c700 1 --2- 172.21.15.22:0/1365451828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc310ffb0 0x564bc31e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16b259c700 1 -- 172.21.15.22:0/1365451828 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bc2f86200 con 0x564bc31e36a0 2023-09-20T21:48:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1365451828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc310ffb0 0x564bc31e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16b259c700 1 --2- 172.21.15.22:0/1365451828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f4590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f16a4006010 tx=0x7f16a4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691b6d8eeaff8c65 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16a17fa700 1 -- 172.21.15.22:0/1365451828 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a401c040 con 0x564bc31e36a0 2023-09-20T21:48:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16a17fa700 1 -- 172.21.15.22:0/1365451828 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a400f770 con 0x564bc31e36a0 2023-09-20T21:48:12.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.986+0000 7f16a17fa700 1 -- 172.21.15.22:0/1365451828 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4005570 con 0x564bc31e36a0 2023-09-20T21:48:12.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.987+0000 7f16cc94db80 1 -- 172.21.15.22:0/1365451828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 msgr2=0x564bc31f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:12.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.987+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1365451828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f4590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f16a4006010 tx=0x7f16a4001e50 comp rx=0 tx=0).stop 2023-09-20T21:48:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.987+0000 7f16cc94db80 1 -- 172.21.15.22:0/1365451828 shutdown_connections 2023-09-20T21:48:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.987+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1365451828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc31e4000 0x564bc31f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.987+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1365451828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc310ffb0 0x564bc31e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.987+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1365451828 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f4590 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.987+0000 7f16cc94db80 1 -- 172.21.15.22:0/1365451828 >> 172.21.15.22:0/1365451828 conn(0x564bc31de4b0 msgr2=0x564bc31dd830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.987+0000 7f16cc94db80 1 -- 172.21.15.22:0/1365451828 shutdown_connections 2023-09-20T21:48:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.987+0000 7f16cc94db80 1 -- 172.21.15.22:0/1365451828 wait complete. 2023-09-20T21:48:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.987+0000 7f16cc94db80 1 -- start start 2023-09-20T21:48:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16cc94db80 1 Processor -- start 2023-09-20T21:48:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc310ffb0 0x564bc31f3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16cc94db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 0x564bc31f4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:12.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16cc94db80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc31df5a0 con 0x564bc31e36a0 2023-09-20T21:48:12.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16cc94db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc31dec70 con 0x564bc31e4000 2023-09-20T21:48:12.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16cc94db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bc31f62e0 con 0x564bc310ffb0 2023-09-20T21:48:12.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16cc94db80 10 client.0 _ll_register_callbacks cb 0x564bc301ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:12.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48228/0 (socket says 172.21.15.22:48228) 2023-09-20T21:48:12.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16b259c700 1 -- 172.21.15.22:0/1464866824 learned_addr learned my addr 172.21.15.22:0/1464866824 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:12.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc310ffb0 0x564bc31f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.988+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 0x564bc31f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:12.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f16b259c700 1 -- 172.21.15.22:0/1464866824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc310ffb0 msgr2=0x564bc31f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f16b259c700 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc310ffb0 0x564bc31f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f16b259c700 1 -- 172.21.15.22:0/1464866824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 msgr2=0x564bc31f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:12.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f16b259c700 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 0x564bc31f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:12.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f16b259c700 1 -- 172.21.15.22:0/1464866824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bc2f48c10 con 0x564bc31e36a0 2023-09-20T21:48:13.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc310ffb0 0x564bc31f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:13.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f16b259c700 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f3d60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f16a400a010 tx=0x7f16a4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b282191c2c87c3f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f1699ffb700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:13.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f1699ffb700 1 -- 172.21.15.22:0/1464866824 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4005700 con 0x564bc31e36a0 2023-09-20T21:48:13.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f16cc94db80 1 -- 172.21.15.22:0/1464866824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16a401d040 con 0x564bc31e36a0 2023-09-20T21:48:13.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f1699ffb700 1 -- 172.21.15.22:0/1464866824 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f16a400b9a0 con 0x564bc31e36a0 2023-09-20T21:48:13.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f1699ffb700 1 -- 172.21.15.22:0/1464866824 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a402c480 con 0x564bc31e36a0 2023-09-20T21:48:13.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f16cc94db80 10 client.15123 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:13.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.989+0000 7f16cc94db80 1 -- 172.21.15.22:0/1464866824 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564bc2f5e260 con 0x564bc31e36a0 2023-09-20T21:48:13.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.990+0000 7f1699ffb700 1 -- 172.21.15.22:0/1464866824 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a401d040 con 0x564bc31e36a0 2023-09-20T21:48:13.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.990+0000 7f1699ffb700 1 -- 172.21.15.22:0/1464866824 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f16a4048070 con 0x564bc31e36a0 2023-09-20T21:48:13.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.990+0000 7f16cc94db80 10 client.15123 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:13.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.990+0000 7f16cc94db80 10 client.15123 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-20T21:48:13.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.990+0000 7f16cc94db80 20 client.15123 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:13.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.990+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x564bc31e4b90 0x564bc324d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.990+0000 7f16cc94db80 4 client.15123 mds_command: new command op to 24530 tid=0[{"prefix": "status"}] 2023-09-20T21:48:13.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.990+0000 7f16cc94db80 1 -- 172.21.15.22:0/1464866824 --> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564bc312de10 con 0x564bc31e4b90 2023-09-20T21:48:13.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.990+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x564bc31e4b90 0x564bc324d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16b1d9b700 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x564bc31e4b90 0x564bc324d0e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8eacd449aed34c3 server_cookie=84e76049d129c59d in_seq=0 out_seq=0 2023-09-20T21:48:13.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f1699ffb700 10 client.15123 ms_handle_connect on v2:172.21.15.89:6836/3887702690 2023-09-20T21:48:13.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f1699ffb700 1 -- 172.21.15.22:0/1464866824 <== mds.2 v2:172.21.15.89:6836/3887702690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564bc312de10 con 0x564bc31e4b90 2023-09-20T21:48:13.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f1699ffb700 10 client.15123 handle_command_reply: tid=0 2023-09-20T21:48:13.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 client.15123 shutdown 2023-09-20T21:48:13.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 -- 172.21.15.22:0/1464866824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 msgr2=0x564bc31f3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f3d60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f16a400a010 tx=0x7f16a4009580 comp rx=0 tx=0).stop 2023-09-20T21:48:13.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 -- 172.21.15.22:0/1464866824 shutdown_connections 2023-09-20T21:48:13.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x564bc31e4b90 0x564bc324d0e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564bc31e4000 0x564bc31f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564bc310ffb0 0x564bc31f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f1699ffb700 0 client.15123 ms_handle_reset on v2:172.21.15.89:6836/3887702690 2023-09-20T21:48:13.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 --2- 172.21.15.22:0/1464866824 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564bc31e36a0 0x564bc31f3d60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 -- 172.21.15.22:0/1464866824 >> 172.21.15.22:0/1464866824 conn(0x564bc31de4b0 msgr2=0x564bc31f75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 -- 172.21.15.22:0/1464866824 shutdown_connections 2023-09-20T21:48:13.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.991+0000 7f16cc94db80 1 -- 172.21.15.22:0/1464866824 wait complete. 2023-09-20T21:48:13.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16cc94db80 20 client.15123 trim_cache size 0 max 16384 2023-09-20T21:48:13.006 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:13.007 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:48:13.007 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 2, 2023-09-20T21:48:13.007 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24530, 2023-09-20T21:48:13.007 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:48:13.007 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:48:13.008 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:48:13.008 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 66.404929150000001, 2023-09-20T21:48:13.008 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 28, 2023-09-20T21:48:13.008 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2023-09-20T21:48:13.008 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 91, 2023-09-20T21:48:13.008 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 108.924894778 2023-09-20T21:48:13.009 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:13.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 -- 172.21.15.22:0/2077050716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f169c0a1300 msgr2=0x7f169c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f169c0a1300 0x7f169c0a37c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000f80 tx=0x7f16ac004dc0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 -- 172.21.15.22:0/2077050716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b4102980 msgr2=0x7f16b41a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b4102980 0x7f16b41a02a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00ef30 tx=0x7f16ac00c3a0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 -- 172.21.15.22:0/2077050716 shutdown_connections 2023-09-20T21:48:13.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b4102980 0x7f16b41a02a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f169c0a1300 0x7f169c0a37c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b4100df0 0x7f16b4198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 --2- 172.21.15.22:0/2077050716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b4101fe0 0x7f16b4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 -- 172.21.15.22:0/2077050716 >> 172.21.15.22:0/2077050716 conn(0x7f16b40fc360 msgr2=0x7f16b4107910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 -- 172.21.15.22:0/2077050716 shutdown_connections 2023-09-20T21:48:13.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:12.992+0000 7f16b8fff700 1 -- 172.21.15.22:0/2077050716 wait complete. 2023-09-20T21:48:13.013 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-20T21:48:13.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.218+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/3185460149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f98101fc0 msgr2=0x7f6f98102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.218+0000 7f6f9f5ff700 1 --2- 172.21.15.22:0/3185460149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f98101fc0 0x7f6f98102420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009920 tx=0x7f6f88009c30 comp rx=0 tx=0).stop 2023-09-20T21:48:13.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.218+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/3185460149 shutdown_connections 2023-09-20T21:48:13.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.218+0000 7f6f9f5ff700 1 --2- 172.21.15.22:0/3185460149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f98102960 0x7f6f98109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.218+0000 7f6f9f5ff700 1 --2- 172.21.15.22:0/3185460149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f98100dd0 0x7f6f981011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.218+0000 7f6f9f5ff700 1 --2- 172.21.15.22:0/3185460149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f98101fc0 0x7f6f98102420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.218+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/3185460149 >> 172.21.15.22:0/3185460149 conn(0x7f6f980fc320 msgr2=0x7f6f980fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.219+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/3185460149 shutdown_connections 2023-09-20T21:48:13.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.219+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/3185460149 wait complete. 2023-09-20T21:48:13.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9f5ff700 1 Processor -- start 2023-09-20T21:48:13.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9f5ff700 1 -- start start 2023-09-20T21:48:13.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9f5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f98100dd0 0x7f6f98198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9f5ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f98101fc0 0x7f6f98199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9f5ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f98102960 0x7f6f981a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9f5ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981a0940 con 0x7f6f98101fc0 2023-09-20T21:48:13.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9f5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981a0a80 con 0x7f6f98100dd0 2023-09-20T21:48:13.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9f5ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981a0c10 con 0x7f6f98102960 2023-09-20T21:48:13.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9cb9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f98101fc0 0x7f6f98199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9cb9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f98101fc0 0x7f6f98199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48232/0 (socket says 172.21.15.22:48232) 2023-09-20T21:48:13.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9cb9a700 1 -- 172.21.15.22:0/4204273887 learned_addr learned my addr 172.21.15.22:0/4204273887 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:13.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.220+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f98100dd0 0x7f6f98198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f98102960 0x7f6f981a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f9cb9a700 1 -- 172.21.15.22:0/4204273887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f98102960 msgr2=0x7f6f981a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f98102960 0x7f6f981a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f9cb9a700 1 -- 172.21.15.22:0/4204273887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f98100dd0 msgr2=0x7f6f98198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f98100dd0 0x7f6f98198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f9cb9a700 1 -- 172.21.15.22:0/4204273887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f88009520 con 0x7f6f98101fc0 2023-09-20T21:48:13.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f98101fc0 0x7f6f98199300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009230 tx=0x7f6f8800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f8e7fc700 1 -- 172.21.15.22:0/4204273887 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8801c070 con 0x7f6f98101fc0 2023-09-20T21:48:13.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f8e7fc700 1 -- 172.21.15.22:0/4204273887 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f8800f7d0 con 0x7f6f98101fc0 2023-09-20T21:48:13.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/4204273887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f981a0e90 con 0x7f6f98101fc0 2023-09-20T21:48:13.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.221+0000 7f6f8e7fc700 1 -- 172.21.15.22:0/4204273887 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88003a20 con 0x7f6f98101fc0 2023-09-20T21:48:13.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.222+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/4204273887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f981a1200 con 0x7f6f98101fc0 2023-09-20T21:48:13.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.225+0000 7f6f8e7fc700 1 -- 172.21.15.22:0/4204273887 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f6f88020030 con 0x7f6f98101fc0 2023-09-20T21:48:13.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6fb2fe8b80 1 Processor -- start 2023-09-20T21:48:13.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6f8e7fc700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f840a1300 0x7f6f840a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6fb2fe8b80 1 -- start start 2023-09-20T21:48:13.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6f8e7fc700 1 -- 172.21.15.22:0/4204273887 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6f880ce060 con 0x7f6f98101fc0 2023-09-20T21:48:13.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f840a1300 0x7f6f840a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 0x555d4a7c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7c9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a73bc50 con 0x555d4a7c8000 2023-09-20T21:48:13.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a71fc20 con 0x555d4a7c8b90 2023-09-20T21:48:13.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.226+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a71d540 con 0x555d4a7c76a0 2023-09-20T21:48:13.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9cb9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9cb9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48246/0 (socket says 172.21.15.22:48246) 2023-09-20T21:48:13.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9cb9a700 1 -- 172.21.15.22:0/2290714795 learned_addr learned my addr 172.21.15.22:0/2290714795 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:13.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 0x555d4a7c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f840a1300 0x7f6f840a37c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6f90003f60 tx=0x7f6f90009100 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9cb9a700 1 -- 172.21.15.22:0/2290714795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 msgr2=0x555d4a7c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 0x555d4a7c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9cb9a700 1 -- 172.21.15.22:0/2290714795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 msgr2=0x555d4a7c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9cb9a700 1 -- 172.21.15.22:0/2290714795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d4a543d90 con 0x555d4a7c8000 2023-09-20T21:48:13.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 0x555d4a7c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:13.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:13.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.227+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7c8590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f880a2b40 tx=0x7f6f88009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a2773217f8ca30 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.228+0000 7f6f837fe700 1 -- 172.21.15.22:0/2290714795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88021380 con 0x555d4a7c8000 2023-09-20T21:48:13.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.228+0000 7f6f837fe700 1 -- 172.21.15.22:0/2290714795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f8809e070 con 0x555d4a7c8000 2023-09-20T21:48:13.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.228+0000 7f6f837fe700 1 -- 172.21.15.22:0/2290714795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880a30e0 con 0x555d4a7c8000 2023-09-20T21:48:13.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.228+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/2290714795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 msgr2=0x555d4a7c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.228+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7c8590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f880a2b40 tx=0x7f6f88009c30 comp rx=0 tx=0).stop 2023-09-20T21:48:13.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.228+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/2290714795 shutdown_connections 2023-09-20T21:48:13.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.228+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.228+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 0x555d4a7c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.228+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/2290714795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7c8590 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.228+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/2290714795 >> 172.21.15.22:0/2290714795 conn(0x555d4a6f3fb0 msgr2=0x555d4a7c2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.229+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/2290714795 shutdown_connections 2023-09-20T21:48:13.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.229+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/2290714795 wait complete. 2023-09-20T21:48:13.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.229+0000 7f6fb2fe8b80 1 -- start start 2023-09-20T21:48:13.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.229+0000 7f6fb2fe8b80 1 Processor -- start 2023-09-20T21:48:13.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 0x555d4a7d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7d8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a718e30 con 0x555d4a7c8000 2023-09-20T21:48:13.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a6b56f0 con 0x555d4a7c8b90 2023-09-20T21:48:13.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a6b5390 con 0x555d4a7c76a0 2023-09-20T21:48:13.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6fb2fe8b80 10 client.0 _ll_register_callbacks cb 0x555d4a5ffa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:13.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6f9cb9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6f9db9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7d8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6f9db9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7d8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46874/0 (socket says 172.21.15.22:46874) 2023-09-20T21:48:13.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6f9db9c700 1 -- 172.21.15.22:0/3100251787 learned_addr learned my addr 172.21.15.22:0/3100251787 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:13.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 0x555d4a7d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6f9db9c700 1 -- 172.21.15.22:0/3100251787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 msgr2=0x555d4a7d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.230+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 0x555d4a7d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6f9db9c700 1 -- 172.21.15.22:0/3100251787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 msgr2=0x555d4a7d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6f9db9c700 1 -- 172.21.15.22:0/3100251787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f94009920 con 0x555d4a7c8b90 2023-09-20T21:48:13.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 0x555d4a7d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:13.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:13.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7d8380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f90008b40 tx=0x7f6f90004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8eacdba1b29773a server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6f80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:48:13.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6f80ff9700 1 -- 172.21.15.22:0/3100251787 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90004360 con 0x555d4a7c8b90 2023-09-20T21:48:13.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/3100251787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d4a689b60 con 0x555d4a7c8b90 2023-09-20T21:48:13.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6f80ff9700 1 -- 172.21.15.22:0/3100251787 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6f90015cc0 con 0x555d4a7c8b90 2023-09-20T21:48:13.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6f80ff9700 1 -- 172.21.15.22:0/3100251787 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f90004910 con 0x555d4a7c8b90 2023-09-20T21:48:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6fb2fe8b80 10 client.25169 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.231+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/3100251787 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f6f981a1200 con 0x555d4a7c8b90 2023-09-20T21:48:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.232+0000 7f6f80ff9700 1 -- 172.21.15.22:0/3100251787 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d4a689b60 con 0x555d4a7c8b90 2023-09-20T21:48:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.232+0000 7f6f80ff9700 1 -- 172.21.15.22:0/3100251787 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f900163f0 con 0x555d4a7c8b90 2023-09-20T21:48:13.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.232+0000 7f6fb2fe8b80 10 client.25169 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:13.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.232+0000 7f6fb2fe8b80 10 client.25169 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-20T21:48:13.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.232+0000 7f6fb2fe8b80 20 client.25169 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:13.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.232+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x555d4a7dca10 0x555d4a7db490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.232+0000 7f6fb2fe8b80 4 client.25169 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:13.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.232+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/3100251787 --> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d4a795fd0 con 0x555d4a7dca10 2023-09-20T21:48:13.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.232+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x555d4a7dca10 0x555d4a7db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.233+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x555d4a7dca10 0x555d4a7db490 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=507ccad11fdcec92 server_cookie=32135d0a6235fb33 in_seq=0 out_seq=0 2023-09-20T21:48:13.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.233+0000 7f6f80ff9700 10 client.25169 ms_handle_connect on v2:172.21.15.22:6840/1806616417 2023-09-20T21:48:13.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6f80ff9700 1 -- 172.21.15.22:0/3100251787 <== mds.1 v2:172.21.15.22:6840/1806616417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555d4a795fd0 con 0x555d4a7dca10 2023-09-20T21:48:13.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6f80ff9700 10 client.25169 handle_command_reply: tid=0 2023-09-20T21:48:13.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6fb2fe8b80 1 client.25169 shutdown 2023-09-20T21:48:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/3100251787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 msgr2=0x555d4a7d8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7d8380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f90008b40 tx=0x7f6f90004b40 comp rx=0 tx=0).stop 2023-09-20T21:48:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/3100251787 shutdown_connections 2023-09-20T21:48:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7d8380 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c76a0 0x555d4a7d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x555d4a7dca10 0x555d4a7db490 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/3100251787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c8000 0x555d4a7d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6f80ff9700 0 client.25169 ms_handle_reset on v2:172.21.15.22:6840/1806616417 2023-09-20T21:48:13.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.234+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/3100251787 >> 172.21.15.22:0/3100251787 conn(0x555d4a6f3fb0 msgr2=0x555d4a7c1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.235+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/3100251787 shutdown_connections 2023-09-20T21:48:13.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.235+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/3100251787 wait complete. 2023-09-20T21:48:13.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.235+0000 7f6fb2fe8b80 20 client.25169 trim_cache size 0 max 16384 2023-09-20T21:48:13.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.253+0000 7f6fb2fe8b80 1 Processor -- start 2023-09-20T21:48:13.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.253+0000 7f6fb2fe8b80 1 -- start start 2023-09-20T21:48:13.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.253+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.253+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 0x555d4a7d8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.253+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.253+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a5998d0 con 0x555d4a7c76a0 2023-09-20T21:48:13.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.253+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a7d1ff0 con 0x555d4a7c8b90 2023-09-20T21:48:13.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.253+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a6bf470 con 0x555d4a7c8000 2023-09-20T21:48:13.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9d39b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9d39b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48272/0 (socket says 172.21.15.22:48272) 2023-09-20T21:48:13.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9d39b700 1 -- 172.21.15.22:0/4255649608 learned_addr learned my addr 172.21.15.22:0/4255649608 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:13.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 0x555d4a7d8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9d39b700 1 -- 172.21.15.22:0/4255649608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 msgr2=0x555d4a7d8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 0x555d4a7d8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9d39b700 1 -- 172.21.15.22:0/4255649608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 msgr2=0x555d4a7ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9d39b700 1 -- 172.21.15.22:0/4255649608 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d4a52cc10 con 0x555d4a7c76a0 2023-09-20T21:48:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 0x555d4a7d8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:13.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:13.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.254+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7c8f70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f94000c00 tx=0x7f6f94009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc7be9d684a0dfa server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6f83fff700 1 -- 172.21.15.22:0/4255649608 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94011c00 con 0x555d4a7c76a0 2023-09-20T21:48:13.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6f83fff700 1 -- 172.21.15.22:0/4255649608 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f94018040 con 0x555d4a7c76a0 2023-09-20T21:48:13.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6f83fff700 1 -- 172.21.15.22:0/4255649608 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94005020 con 0x555d4a7c76a0 2023-09-20T21:48:13.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/4255649608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 msgr2=0x555d4a7c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7c8f70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f94000c00 tx=0x7f6f94009310 comp rx=0 tx=0).stop 2023-09-20T21:48:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/4255649608 shutdown_connections 2023-09-20T21:48:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7c8f70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/4255649608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 0x555d4a7d8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/4255649608 >> 172.21.15.22:0/4255649608 conn(0x555d4a6f3fb0 msgr2=0x555d4a7be290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/4255649608 shutdown_connections 2023-09-20T21:48:13.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.255+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/4255649608 wait complete. 2023-09-20T21:48:13.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.256+0000 7f6fb2fe8b80 1 -- start start 2023-09-20T21:48:13.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.256+0000 7f6fb2fe8b80 1 Processor -- start 2023-09-20T21:48:13.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.256+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.256+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 0x555d4a7d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.256+0000 7f6fb2fe8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7d7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.256+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a7d8250 con 0x555d4a7c76a0 2023-09-20T21:48:13.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.256+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a7d83c0 con 0x555d4a7c8b90 2023-09-20T21:48:13.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.256+0000 7f6fb2fe8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4a7d8580 con 0x555d4a7c8000 2023-09-20T21:48:13.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.256+0000 7f6fb2fe8b80 10 client.0 _ll_register_callbacks cb 0x555d4a5ffa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:13.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9d39b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9d39b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48280/0 (socket says 172.21.15.22:48280) 2023-09-20T21:48:13.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9cb9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 0x555d4a7d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9d39b700 1 -- 172.21.15.22:0/1406889625 learned_addr learned my addr 172.21.15.22:0/1406889625 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:13.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9db9c700 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7d7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9d39b700 1 -- 172.21.15.22:0/1406889625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 msgr2=0x555d4a7d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 0x555d4a7d2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9d39b700 1 -- 172.21.15.22:0/1406889625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 msgr2=0x555d4a7d7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7d7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9d39b700 1 -- 172.21.15.22:0/1406889625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9401f9f0 con 0x555d4a7c76a0 2023-09-20T21:48:13.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f9d39b700 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7d27a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6f940095c0 tx=0x7f6f94004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fb437618283184 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.257+0000 7f6f80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:13.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6f80ff9700 1 -- 172.21.15.22:0/1406889625 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94011c00 con 0x555d4a7c76a0 2023-09-20T21:48:13.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/1406889625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f8809e070 con 0x555d4a7c76a0 2023-09-20T21:48:13.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6f80ff9700 1 -- 172.21.15.22:0/1406889625 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6f94011d60 con 0x555d4a7c76a0 2023-09-20T21:48:13.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6f80ff9700 1 -- 172.21.15.22:0/1406889625 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f9401f020 con 0x555d4a7c76a0 2023-09-20T21:48:13.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6fb2fe8b80 10 client.15141 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:13.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/1406889625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555d4a56a200 con 0x555d4a7c76a0 2023-09-20T21:48:13.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6f80ff9700 1 -- 172.21.15.22:0/1406889625 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f8809e070 con 0x555d4a7c76a0 2023-09-20T21:48:13.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6f80ff9700 1 -- 172.21.15.22:0/1406889625 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f9401e420 con 0x555d4a7c76a0 2023-09-20T21:48:13.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6fb2fe8b80 10 client.15141 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:13.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6fb2fe8b80 10 client.15141 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-20T21:48:13.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6fb2fe8b80 20 client.15141 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:13.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.258+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x555d4a8351d0 0x555d4a831650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.259+0000 7f6fb2fe8b80 4 client.15141 mds_command: new command op to 24566 tid=0[{"prefix": "status"}] 2023-09-20T21:48:13.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.259+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/1406889625 --> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555d4a711e10 con 0x555d4a8351d0 2023-09-20T21:48:13.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.259+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x555d4a8351d0 0x555d4a831650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.259+0000 7f6f80ff9700 10 client.15141 ms_handle_connect on v2:172.21.15.22:6840/1806616417 2023-09-20T21:48:13.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.259+0000 7f6f9cb9a700 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x555d4a8351d0 0x555d4a831650 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df70200f76d4a8a9 server_cookie=6dfc79d2fd53e90e in_seq=0 out_seq=0 2023-09-20T21:48:13.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.259+0000 7f6f80ff9700 1 -- 172.21.15.22:0/1406889625 <== mds.1 v2:172.21.15.22:6840/1806616417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555d4a711e10 con 0x555d4a8351d0 2023-09-20T21:48:13.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.259+0000 7f6f80ff9700 10 client.15141 handle_command_reply: tid=0 2023-09-20T21:48:13.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.259+0000 7f6fb2fe8b80 1 client.15141 shutdown 2023-09-20T21:48:13.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/1406889625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 msgr2=0x555d4a7d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7d27a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6f940095c0 tx=0x7f6f94004a60 comp rx=0 tx=0).stop 2023-09-20T21:48:13.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/1406889625 shutdown_connections 2023-09-20T21:48:13.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555d4a7c8b90 0x555d4a7d7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x555d4a8351d0 0x555d4a831650 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555d4a7c76a0 0x555d4a7d27a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 1 --2- 172.21.15.22:0/1406889625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555d4a7c8000 0x555d4a7d2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6f80ff9700 0 client.15141 ms_handle_reset on v2:172.21.15.22:6840/1806616417 2023-09-20T21:48:13.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/1406889625 >> 172.21.15.22:0/1406889625 conn(0x555d4a6f3fb0 msgr2=0x555d4a7ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/1406889625 shutdown_connections 2023-09-20T21:48:13.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 1 -- 172.21.15.22:0/1406889625 wait complete. 2023-09-20T21:48:13.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.260+0000 7f6fb2fe8b80 20 client.15141 trim_cache size 0 max 16384 2023-09-20T21:48:13.276 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:13.276 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:48:13.276 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 1, 2023-09-20T21:48:13.276 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24566, 2023-09-20T21:48:13.277 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:48:13.277 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:48:13.277 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:48:13.277 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 67.680527388000002, 2023-09-20T21:48:13.277 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 28, 2023-09-20T21:48:13.277 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2023-09-20T21:48:13.278 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 91, 2023-09-20T21:48:13.278 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 100.43427468500001 2023-09-20T21:48:13.278 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:13.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/4204273887 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f840a1300 msgr2=0x7f6f840a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f840a1300 0x7f6f840a37c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6f90003f60 tx=0x7f6f90009100 comp rx=0 tx=0).stop 2023-09-20T21:48:13.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/4204273887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f98101fc0 msgr2=0x7f6f98199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f98101fc0 0x7f6f98199300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009230 tx=0x7f6f8800b790 comp rx=0 tx=0).stop 2023-09-20T21:48:13.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/4204273887 shutdown_connections 2023-09-20T21:48:13.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f98102960 0x7f6f981a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f6f840a1300 0x7f6f840a37c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f98100dd0 0x7f6f98198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 --2- 172.21.15.22:0/4204273887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f98101fc0 0x7f6f98199300 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/4204273887 >> 172.21.15.22:0/4204273887 conn(0x7f6f980fc320 msgr2=0x7f6f981079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/4204273887 shutdown_connections 2023-09-20T21:48:13.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.261+0000 7f6f9f5ff700 1 -- 172.21.15.22:0/4204273887 wait complete. 2023-09-20T21:48:13.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-20T21:48:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.486+0000 7fc715bff700 1 -- 172.21.15.22:0/3764680075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710101850 msgr2=0x7fc710103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.486+0000 7fc715bff700 1 --2- 172.21.15.22:0/3764680075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710101850 0x7fc710103c40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc700009990 tx=0x7fc700009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.486+0000 7fc715bff700 1 -- 172.21.15.22:0/3764680075 shutdown_connections 2023-09-20T21:48:13.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.486+0000 7fc715bff700 1 --2- 172.21.15.22:0/3764680075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc710106ab0 0x7fc710108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.486+0000 7fc715bff700 1 --2- 172.21.15.22:0/3764680075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710101850 0x7fc710103c40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.486+0000 7fc715bff700 1 --2- 172.21.15.22:0/3764680075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc710104180 0x7fc710106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.486+0000 7fc715bff700 1 -- 172.21.15.22:0/3764680075 >> 172.21.15.22:0/3764680075 conn(0x7fc7100fb240 msgr2=0x7fc7100fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.486+0000 7fc715bff700 1 -- 172.21.15.22:0/3764680075 shutdown_connections 2023-09-20T21:48:13.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.486+0000 7fc715bff700 1 -- 172.21.15.22:0/3764680075 wait complete. 2023-09-20T21:48:13.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.487+0000 7fc715bff700 1 Processor -- start 2023-09-20T21:48:13.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.487+0000 7fc715bff700 1 -- start start 2023-09-20T21:48:13.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc715bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710101850 0x7fc710198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc715bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc710104180 0x7fc7101992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc715bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc710106ab0 0x7fc7101a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc715bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7101a08e0 con 0x7fc710101850 2023-09-20T21:48:13.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc715bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7101a0a20 con 0x7fc710106ab0 2023-09-20T21:48:13.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc715bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7101a0bb0 con 0x7fc710104180 2023-09-20T21:48:13.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc70f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710101850 0x7fc710198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc70f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710101850 0x7fc710198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48302/0 (socket says 172.21.15.22:48302) 2023-09-20T21:48:13.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc70f7fe700 1 -- 172.21.15.22:0/3741688749 learned_addr learned my addr 172.21.15.22:0/3741688749 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:13.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc70effd700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc710104180 0x7fc7101992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.488+0000 7fc70ffff700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc710106ab0 0x7fc7101a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc70f7fe700 1 -- 172.21.15.22:0/3741688749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc710104180 msgr2=0x7fc7101992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc710104180 0x7fc7101992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc70f7fe700 1 -- 172.21.15.22:0/3741688749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc710106ab0 msgr2=0x7fc7101a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc710106ab0 0x7fc7101a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc70f7fe700 1 -- 172.21.15.22:0/3741688749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7000095f0 con 0x7fc710101850 2023-09-20T21:48:13.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710101850 0x7fc710198d60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc70000b680 tx=0x7fc700005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc70cff9700 1 -- 172.21.15.22:0/3741688749 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70001d070 con 0x7fc710101850 2023-09-20T21:48:13.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc715bff700 1 -- 172.21.15.22:0/3741688749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7101a0e30 con 0x7fc710101850 2023-09-20T21:48:13.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc70cff9700 1 -- 172.21.15.22:0/3741688749 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc700022470 con 0x7fc710101850 2023-09-20T21:48:13.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc70cff9700 1 -- 172.21.15.22:0/3741688749 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700017650 con 0x7fc710101850 2023-09-20T21:48:13.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.489+0000 7fc715bff700 1 -- 172.21.15.22:0/3741688749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7101a11a0 con 0x7fc710101850 2023-09-20T21:48:13.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.492+0000 7fc70cff9700 1 -- 172.21.15.22:0/3741688749 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fc7000225e0 con 0x7fc710101850 2023-09-20T21:48:13.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc72960db80 1 Processor -- start 2023-09-20T21:48:13.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc70cff9700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc6f80a1300 0x7fc6f80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc72960db80 1 -- start start 2023-09-20T21:48:13.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc70cff9700 1 -- 172.21.15.22:0/3741688749 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc7000ce3f0 con 0x7fc710101850 2023-09-20T21:48:13.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc70ffff700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc6f80a1300 0x7fc6f80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc72960db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 0x5644a5583ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc72960db80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584000 0x5644a5584590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc72960db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 0x5644a5585010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc72960db80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a54f7c50 con 0x5644a5584000 2023-09-20T21:48:13.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc72960db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a54dbc20 con 0x5644a5584b90 2023-09-20T21:48:13.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc72960db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a54d9540 con 0x5644a55836a0 2023-09-20T21:48:13.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.494+0000 7fc70effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584000 0x5644a5584590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 0x5644a5585010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 0x5644a5585010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46920/0 (socket says 172.21.15.22:46920) 2023-09-20T21:48:13.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70ffff700 1 -- 172.21.15.22:0/226509826 learned_addr learned my addr 172.21.15.22:0/226509826 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:13.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/226509826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 0x5644a5583ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70ffff700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc6f80a1300 0x7fc6f80a37c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc70800f930 tx=0x7fc70800fcf0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70f7fe700 1 -- 172.21.15.22:0/226509826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 msgr2=0x5644a5585010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/226509826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 0x5644a5585010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70f7fe700 1 -- 172.21.15.22:0/226509826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584000 msgr2=0x5644a5584590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/226509826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584000 0x5644a5584590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70f7fe700 1 -- 172.21.15.22:0/226509826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644a52ffd90 con 0x5644a55836a0 2023-09-20T21:48:13.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/226509826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 0x5644a5583ac0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc7000cbb60 tx=0x7fc70009b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97893c7a07a0503f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.495+0000 7fc6f5ffb700 1 -- 172.21.15.22:0/226509826 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70009af00 con 0x5644a55836a0 2023-09-20T21:48:13.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc6f5ffb700 1 -- 172.21.15.22:0/226509826 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc70009bca0 con 0x5644a55836a0 2023-09-20T21:48:13.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc6f5ffb700 1 -- 172.21.15.22:0/226509826 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7000ce130 con 0x5644a55836a0 2023-09-20T21:48:13.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc72960db80 1 -- 172.21.15.22:0/226509826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 msgr2=0x5644a5583ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc72960db80 1 --2- 172.21.15.22:0/226509826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 0x5644a5583ac0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc7000cbb60 tx=0x7fc70009b370 comp rx=0 tx=0).stop 2023-09-20T21:48:13.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc72960db80 1 -- 172.21.15.22:0/226509826 shutdown_connections 2023-09-20T21:48:13.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc72960db80 1 --2- 172.21.15.22:0/226509826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 0x5644a5585010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc72960db80 1 --2- 172.21.15.22:0/226509826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 0x5644a5583ac0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc72960db80 1 --2- 172.21.15.22:0/226509826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584000 0x5644a5584590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc72960db80 1 -- 172.21.15.22:0/226509826 >> 172.21.15.22:0/226509826 conn(0x5644a54affb0 msgr2=0x5644a557e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc72960db80 1 -- 172.21.15.22:0/226509826 shutdown_connections 2023-09-20T21:48:13.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.496+0000 7fc72960db80 1 -- 172.21.15.22:0/226509826 wait complete. 2023-09-20T21:48:13.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.497+0000 7fc72960db80 1 -- start start 2023-09-20T21:48:13.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.497+0000 7fc72960db80 1 Processor -- start 2023-09-20T21:48:13.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc72960db80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a558cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc72960db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a558d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc72960db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a5584b90 0x5644a5594420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc72960db80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a54d4e30 con 0x5644a55836a0 2023-09-20T21:48:13.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc72960db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a54716f0 con 0x5644a5584000 2023-09-20T21:48:13.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc72960db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a5471390 con 0x5644a5584b90 2023-09-20T21:48:13.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc72960db80 10 client.0 _ll_register_callbacks cb 0x5644a53bba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:13.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a558cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a558d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a558d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:46936/0 (socket says 172.21.15.22:46936) 2023-09-20T21:48:13.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70effd700 1 -- 172.21.15.22:0/1167931046 learned_addr learned my addr 172.21.15.22:0/1167931046 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:13.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70ffff700 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a5584b90 0x5644a5594420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70effd700 1 -- 172.21.15.22:0/1167931046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a5584b90 msgr2=0x5644a5594420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70effd700 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a5584b90 0x5644a5594420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70effd700 1 -- 172.21.15.22:0/1167931046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 msgr2=0x5644a558cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70effd700 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a558cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70effd700 1 -- 172.21.15.22:0/1167931046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7101a11a0 con 0x5644a5584000 2023-09-20T21:48:13.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.498+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a558cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:13.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.499+0000 7fc70effd700 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a558d3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc704012770 tx=0x7fc704012a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a675ea50752349e server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.499+0000 7fc6eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:48:13.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.499+0000 7fc6eeffd700 1 -- 172.21.15.22:0/1167931046 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc704017070 con 0x5644a5584000 2023-09-20T21:48:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.499+0000 7fc72960db80 1 -- 172.21.15.22:0/1167931046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644a5445b60 con 0x5644a5584000 2023-09-20T21:48:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.499+0000 7fc6eeffd700 1 -- 172.21.15.22:0/1167931046 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc70400f540 con 0x5644a5584000 2023-09-20T21:48:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.499+0000 7fc6eeffd700 1 -- 172.21.15.22:0/1167931046 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc70400f700 con 0x5644a5584000 2023-09-20T21:48:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.499+0000 7fc72960db80 10 client.25187 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.499+0000 7fc72960db80 1 -- 172.21.15.22:0/1167931046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5644a52e8c10 con 0x5644a5584000 2023-09-20T21:48:13.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.499+0000 7fc6eeffd700 1 -- 172.21.15.22:0/1167931046 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5644a5445b60 con 0x5644a5584000 2023-09-20T21:48:13.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.500+0000 7fc6eeffd700 1 -- 172.21.15.22:0/1167931046 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc704010880 con 0x5644a5584000 2023-09-20T21:48:13.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.500+0000 7fc72960db80 10 client.25187 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:13.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.500+0000 7fc72960db80 10 client.25187 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-20T21:48:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.500+0000 7fc72960db80 20 client.25187 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.500+0000 7fc72960db80 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5644a5598ab0 0x5644a5597580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.500+0000 7fc72960db80 4 client.25187 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.500+0000 7fc72960db80 1 -- 172.21.15.22:0/1167931046 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644a5551fd0 con 0x5644a5598ab0 2023-09-20T21:48:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.500+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5644a5598ab0 0x5644a5597580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.501+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5644a5598ab0 0x5644a5597580 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996d49ffc677ccec server_cookie=ab7bf328ff4501f1 in_seq=0 out_seq=0 2023-09-20T21:48:13.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.501+0000 7fc6eeffd700 10 client.25187 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:13.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.501+0000 7fc6eeffd700 1 -- 172.21.15.22:0/1167931046 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5644a5551fd0 con 0x5644a5598ab0 2023-09-20T21:48:13.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.501+0000 7fc6eeffd700 10 client.25187 handle_command_reply: tid=0 2023-09-20T21:48:13.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 client.25187 shutdown 2023-09-20T21:48:13.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 -- 172.21.15.22:0/1167931046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 msgr2=0x5644a558d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a558d3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc704012770 tx=0x7fc704012a80 comp rx=0 tx=0).stop 2023-09-20T21:48:13.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 -- 172.21.15.22:0/1167931046 shutdown_connections 2023-09-20T21:48:13.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5644a5598ab0 0x5644a5597580 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a5584b90 0x5644a5594420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a558cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 --2- 172.21.15.22:0/1167931046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a558d3e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 -- 172.21.15.22:0/1167931046 >> 172.21.15.22:0/1167931046 conn(0x5644a54affb0 msgr2=0x5644a557d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc6eeffd700 0 client.25187 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:13.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 -- 172.21.15.22:0/1167931046 shutdown_connections 2023-09-20T21:48:13.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 1 -- 172.21.15.22:0/1167931046 wait complete. 2023-09-20T21:48:13.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.502+0000 7fc72960db80 20 client.25187 trim_cache size 0 max 16384 2023-09-20T21:48:13.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.520+0000 7fc72960db80 1 Processor -- start 2023-09-20T21:48:13.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc72960db80 1 -- start start 2023-09-20T21:48:13.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc72960db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 0x5644a557e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc72960db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a55943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc72960db80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584b90 0x5644a558bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc72960db80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a53558d0 con 0x5644a5584b90 2023-09-20T21:48:13.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc72960db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a558e090 con 0x5644a5584000 2023-09-20T21:48:13.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc72960db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a547b470 con 0x5644a55836a0 2023-09-20T21:48:13.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584b90 0x5644a558bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc70effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a55943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc70ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584b90 0x5644a558bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48332/0 (socket says 172.21.15.22:48332) 2023-09-20T21:48:13.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc70ffff700 1 -- 172.21.15.22:0/1154600516 learned_addr learned my addr 172.21.15.22:0/1154600516 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:13.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.521+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/1154600516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 0x5644a557e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.522+0000 7fc70ffff700 1 -- 172.21.15.22:0/1154600516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 msgr2=0x5644a557e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.522+0000 7fc70ffff700 1 --2- 172.21.15.22:0/1154600516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 0x5644a557e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.522+0000 7fc70ffff700 1 -- 172.21.15.22:0/1154600516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 msgr2=0x5644a55943e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.522+0000 7fc70ffff700 1 --2- 172.21.15.22:0/1154600516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a55943e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.522+0000 7fc70ffff700 1 -- 172.21.15.22:0/1154600516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644a5326200 con 0x5644a5584b90 2023-09-20T21:48:13.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.522+0000 7fc70effd700 1 --2- 172.21.15.22:0/1154600516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a55943e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:13.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.522+0000 7fc70ffff700 1 --2- 172.21.15.22:0/1154600516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584b90 0x5644a558bb70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc708009c10 tx=0x7fc70801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7b6e70e33480db server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.522+0000 7fc6f67fc700 1 -- 172.21.15.22:0/1154600516 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70800e040 con 0x5644a5584b90 2023-09-20T21:48:13.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.522+0000 7fc6f67fc700 1 -- 172.21.15.22:0/1154600516 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc70800f040 con 0x5644a5584b90 2023-09-20T21:48:13.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.522+0000 7fc6f67fc700 1 -- 172.21.15.22:0/1154600516 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708008e80 con 0x5644a5584b90 2023-09-20T21:48:13.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 -- 172.21.15.22:0/1154600516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584b90 msgr2=0x5644a558bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 --2- 172.21.15.22:0/1154600516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584b90 0x5644a558bb70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc708009c10 tx=0x7fc70801f000 comp rx=0 tx=0).stop 2023-09-20T21:48:13.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 -- 172.21.15.22:0/1154600516 shutdown_connections 2023-09-20T21:48:13.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 --2- 172.21.15.22:0/1154600516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a5584b90 0x5644a558bb70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 --2- 172.21.15.22:0/1154600516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a55836a0 0x5644a557e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 --2- 172.21.15.22:0/1154600516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584000 0x5644a55943e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 -- 172.21.15.22:0/1154600516 >> 172.21.15.22:0/1154600516 conn(0x5644a54affb0 msgr2=0x5644a557a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 -- 172.21.15.22:0/1154600516 shutdown_connections 2023-09-20T21:48:13.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 -- 172.21.15.22:0/1154600516 wait complete. 2023-09-20T21:48:13.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 -- start start 2023-09-20T21:48:13.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.523+0000 7fc72960db80 1 Processor -- start 2023-09-20T21:48:13.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc72960db80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a5592660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc72960db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a5584000 0x5644a5592ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc72960db80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 0x5644a55930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc72960db80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a5593700 con 0x5644a55836a0 2023-09-20T21:48:13.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc72960db80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a5593840 con 0x5644a5584b90 2023-09-20T21:48:13.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc72960db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a5593a00 con 0x5644a5584000 2023-09-20T21:48:13.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc72960db80 10 client.0 _ll_register_callbacks cb 0x5644a53bba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:13.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc70f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a5592660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc70f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a5592660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48344/0 (socket says 172.21.15.22:48344) 2023-09-20T21:48:13.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc70f7fe700 1 -- 172.21.15.22:0/2310396974 learned_addr learned my addr 172.21.15.22:0/2310396974 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:13.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc70ffff700 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 0x5644a55930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.524+0000 7fc70effd700 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a5584000 0x5644a5592ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc70f7fe700 1 -- 172.21.15.22:0/2310396974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a5584000 msgr2=0x5644a5592ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a5584000 0x5644a5592ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc70f7fe700 1 -- 172.21.15.22:0/2310396974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 msgr2=0x5644a55930e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:13.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 0x5644a55930e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc70f7fe700 1 -- 172.21.15.22:0/2310396974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644a52e8c10 con 0x5644a55836a0 2023-09-20T21:48:13.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc70ffff700 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 0x5644a55930e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:13.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc70f7fe700 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a5592660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc700098010 tx=0x7fc700009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbc49d3c8375002 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:13.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc6eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:13.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc6eeffd700 1 -- 172.21.15.22:0/2310396974 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7000996e0 con 0x5644a55836a0 2023-09-20T21:48:13.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc6eeffd700 1 -- 172.21.15.22:0/2310396974 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc7000b8070 con 0x5644a55836a0 2023-09-20T21:48:13.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc72960db80 1 -- 172.21.15.22:0/2310396974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc70009bca0 con 0x5644a55836a0 2023-09-20T21:48:13.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc6eeffd700 1 -- 172.21.15.22:0/2310396974 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc70009b5a0 con 0x5644a55836a0 2023-09-20T21:48:13.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc72960db80 10 client.15159 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:13.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc72960db80 1 -- 172.21.15.22:0/2310396974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fc70800fe70 con 0x5644a55836a0 2023-09-20T21:48:13.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.525+0000 7fc6eeffd700 1 -- 172.21.15.22:0/2310396974 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc70009bca0 con 0x5644a55836a0 2023-09-20T21:48:13.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.526+0000 7fc6eeffd700 1 -- 172.21.15.22:0/2310396974 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc70009aa90 con 0x5644a55836a0 2023-09-20T21:48:13.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.526+0000 7fc72960db80 10 client.15159 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:13.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.526+0000 7fc72960db80 10 client.15159 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-20T21:48:13.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.526+0000 7fc72960db80 20 client.15159 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:13.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.526+0000 7fc72960db80 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5644a55f0340 0x5644a55ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:13.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.526+0000 7fc72960db80 4 client.15159 mds_command: new command op to 24578 tid=0[{"prefix": "status"}] 2023-09-20T21:48:13.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.526+0000 7fc72960db80 1 -- 172.21.15.22:0/2310396974 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5644a54cde10 con 0x5644a55f0340 2023-09-20T21:48:13.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.526+0000 7fc70effd700 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5644a55f0340 0x5644a55ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:13.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.526+0000 7fc70effd700 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5644a55f0340 0x5644a55ec7e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c570143f6cc5b3de server_cookie=da316647d9a1d91d in_seq=0 out_seq=0 2023-09-20T21:48:13.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.526+0000 7fc6eeffd700 10 client.15159 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:13.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc6eeffd700 1 -- 172.21.15.22:0/2310396974 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5644a54cde10 con 0x5644a55f0340 2023-09-20T21:48:13.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc6eeffd700 10 client.15159 handle_command_reply: tid=0 2023-09-20T21:48:13.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 client.15159 shutdown 2023-09-20T21:48:13.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 -- 172.21.15.22:0/2310396974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 msgr2=0x5644a5592660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a5592660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc700098010 tx=0x7fc700009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 -- 172.21.15.22:0/2310396974 shutdown_connections 2023-09-20T21:48:13.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5644a55f0340 0x5644a55ec7e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5644a5584b90 0x5644a55930e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5644a55836a0 0x5644a5592660 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc6eeffd700 0 client.15159 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:13.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 --2- 172.21.15.22:0/2310396974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644a5584000 0x5644a5592ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 -- 172.21.15.22:0/2310396974 >> 172.21.15.22:0/2310396974 conn(0x5644a54affb0 msgr2=0x5644a557e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 -- 172.21.15.22:0/2310396974 shutdown_connections 2023-09-20T21:48:13.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 1 -- 172.21.15.22:0/2310396974 wait complete. 2023-09-20T21:48:13.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.527+0000 7fc72960db80 20 client.15159 trim_cache size 0 max 16384 2023-09-20T21:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:13.545 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2023-09-20T21:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24578, 2023-09-20T21:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:48:13.546 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 92.013639405999996, 2023-09-20T21:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 28, 2023-09-20T21:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2023-09-20T21:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 91, 2023-09-20T21:48:13.547 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 98.278485715000002 2023-09-20T21:48:13.548 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:13.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 -- 172.21.15.22:0/3741688749 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc6f80a1300 msgr2=0x7fc6f80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc6f80a1300 0x7fc6f80a37c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc70800f930 tx=0x7fc70800fcf0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 -- 172.21.15.22:0/3741688749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710101850 msgr2=0x7fc710198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:13.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710101850 0x7fc710198d60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc70000b680 tx=0x7fc700005740 comp rx=0 tx=0).stop 2023-09-20T21:48:13.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 -- 172.21.15.22:0/3741688749 shutdown_connections 2023-09-20T21:48:13.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc6f80a1300 0x7fc6f80a37c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc710106ab0 0x7fc7101a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc710101850 0x7fc710198d60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 --2- 172.21.15.22:0/3741688749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc710104180 0x7fc7101992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:13.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 -- 172.21.15.22:0/3741688749 >> 172.21.15.22:0/3741688749 conn(0x7fc7100fb240 msgr2=0x7fc7101024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:13.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 -- 172.21.15.22:0/3741688749 shutdown_connections 2023-09-20T21:48:13.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:13.528+0000 7fc715bff700 1 -- 172.21.15.22:0/3741688749 wait complete. 2023-09-20T21:48:13.553 INFO:tasks.cephadm:Setup complete, yielding 2023-09-20T21:48:13.553 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-20T21:48:13.565 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2023-09-20T21:48:13.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph orch status' 2023-09-20T21:48:13.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:13 smithi123 ceph-mon[134433]: pgmap v326: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2023-09-20T21:48:13.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:13 smithi089 ceph-mon[134090]: pgmap v326: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2023-09-20T21:48:13.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:13 smithi022 ceph-mon[97651]: pgmap v326: 129 pgs: 2 peering, 127 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2023-09-20T21:48:14.300 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:48:15.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:15 smithi022 ceph-mon[97651]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2023-09-20T21:48:15.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:15 smithi022 ceph-mon[97651]: Cluster is now healthy 2023-09-20T21:48:15.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:15 smithi123 ceph-mon[134433]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2023-09-20T21:48:15.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:15 smithi123 ceph-mon[134433]: Cluster is now healthy 2023-09-20T21:48:15.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:15 smithi089 ceph-mon[134090]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2023-09-20T21:48:15.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:15 smithi089 ceph-mon[134090]: Cluster is now healthy 2023-09-20T21:48:15.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.980+0000 7f228c1d8700 1 -- 172.21.15.22:0/2488020128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2284102990 msgr2=0x7f22841092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:15.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.980+0000 7f228c1d8700 1 --2- 172.21.15.22:0/2488020128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2284102990 0x7f22841092b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f227c00b0a0 tx=0x7f227c00b3b0 comp rx=0 tx=0).stop 2023-09-20T21:48:15.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.980+0000 7f228c1d8700 1 -- 172.21.15.22:0/2488020128 shutdown_connections 2023-09-20T21:48:15.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.980+0000 7f228c1d8700 1 --2- 172.21.15.22:0/2488020128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2284102990 0x7f22841092b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:15.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.980+0000 7f228c1d8700 1 --2- 172.21.15.22:0/2488020128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2284100e70 0x7f2284101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:15.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.980+0000 7f228c1d8700 1 --2- 172.21.15.22:0/2488020128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284101fd0 0x7f2284102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:15.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.980+0000 7f228c1d8700 1 -- 172.21.15.22:0/2488020128 >> 172.21.15.22:0/2488020128 conn(0x7f22840fc3d0 msgr2=0x7f22840fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:15.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.980+0000 7f228c1d8700 1 -- 172.21.15.22:0/2488020128 shutdown_connections 2023-09-20T21:48:15.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.980+0000 7f228c1d8700 1 -- 172.21.15.22:0/2488020128 wait complete. 2023-09-20T21:48:15.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.981+0000 7f228c1d8700 1 Processor -- start 2023-09-20T21:48:15.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f228c1d8700 1 -- start start 2023-09-20T21:48:15.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f228c1d8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284100e70 0x7f2284074b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:15.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f228c1d8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2284101fd0 0x7f22840731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:15.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f228c1d8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2284102990 0x7f2284073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:15.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f228c1d8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2284073e90 con 0x7f2284102990 2023-09-20T21:48:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f228c1d8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2284074000 con 0x7f2284101fd0 2023-09-20T21:48:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f228c1d8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22841a0cf0 con 0x7f2284100e70 2023-09-20T21:48:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f228a775700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2284102990 0x7f2284073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f228a775700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2284102990 0x7f2284073710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38896/0 (socket says 172.21.15.22:38896) 2023-09-20T21:48:15.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f228a775700 1 -- 172.21.15.22:0/2386043269 learned_addr learned my addr 172.21.15.22:0/2386043269 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:15.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f2289f74700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284100e70 0x7f2284074b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:15.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.982+0000 7f2289773700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2284101fd0 0x7f22840731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:15.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f228a775700 1 -- 172.21.15.22:0/2386043269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284100e70 msgr2=0x7f2284074b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:15.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f228a775700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284100e70 0x7f2284074b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:15.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f228a775700 1 -- 172.21.15.22:0/2386043269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2284101fd0 msgr2=0x7f22840731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:15.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f228a775700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2284101fd0 0x7f22840731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:15.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f228a775700 1 -- 172.21.15.22:0/2386043269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c009d00 con 0x7f2284102990 2023-09-20T21:48:15.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f2289f74700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284100e70 0x7f2284074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:15.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f228a775700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2284102990 0x7f2284073710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f227c00d010 tx=0x7f227c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:15.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f227affd700 1 -- 172.21.15.22:0/2386043269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00e040 con 0x7f2284102990 2023-09-20T21:48:15.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f228c1d8700 1 -- 172.21.15.22:0/2386043269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22841a0f70 con 0x7f2284102990 2023-09-20T21:48:15.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f227affd700 1 -- 172.21.15.22:0/2386043269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f227c003e10 con 0x7f2284102990 2023-09-20T21:48:15.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f227affd700 1 -- 172.21.15.22:0/2386043269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c0052a0 con 0x7f2284102990 2023-09-20T21:48:15.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.983+0000 7f228c1d8700 1 -- 172.21.15.22:0/2386043269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22841a1320 con 0x7f2284102990 2023-09-20T21:48:15.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.985+0000 7f228c1d8700 1 -- 172.21.15.22:0/2386043269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2284108400 con 0x7f2284102990 2023-09-20T21:48:15.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.986+0000 7f227affd700 1 -- 172.21.15.22:0/2386043269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f227c01d480 con 0x7f2284102990 2023-09-20T21:48:15.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.988+0000 7f227affd700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22700a1300 0x7f22700a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:15.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.988+0000 7f227affd700 1 -- 172.21.15.22:0/2386043269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f227c0ce0e0 con 0x7f2284102990 2023-09-20T21:48:15.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.988+0000 7f2289f74700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22700a1300 0x7f22700a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:15.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.989+0000 7f2289f74700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22700a1300 0x7f22700a37c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2280011490 tx=0x7f2280009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:15.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:15.992+0000 7f227affd700 1 -- 172.21.15.22:0/2386043269 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f227c097450 con 0x7f2284102990 2023-09-20T21:48:16.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.220+0000 7f228c1d8700 1 -- 172.21.15.22:0/2386043269 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f228402d230 con 0x7f22700a1300 2023-09-20T21:48:16.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.221+0000 7f227affd700 1 -- 172.21.15.22:0/2386043269 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f228402d230 con 0x7f22700a1300 2023-09-20T21:48:16.226 INFO:teuthology.orchestra.run.smithi022.stdout:Backend: cephadm 2023-09-20T21:48:16.226 INFO:teuthology.orchestra.run.smithi022.stdout:Available: Yes 2023-09-20T21:48:16.227 INFO:teuthology.orchestra.run.smithi022.stdout:Paused: No 2023-09-20T21:48:16.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 -- 172.21.15.22:0/2386043269 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22700a1300 msgr2=0x7f22700a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:16.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22700a1300 0x7f22700a37c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2280011490 tx=0x7f2280009260 comp rx=0 tx=0).stop 2023-09-20T21:48:16.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 -- 172.21.15.22:0/2386043269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2284102990 msgr2=0x7f2284073710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:16.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2284102990 0x7f2284073710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f227c00d010 tx=0x7f227c00bc80 comp rx=0 tx=0).stop 2023-09-20T21:48:16.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 -- 172.21.15.22:0/2386043269 shutdown_connections 2023-09-20T21:48:16.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2284102990 0x7f2284073710 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2284100e70 0x7f2284074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22700a1300 0x7f22700a37c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 --2- 172.21.15.22:0/2386043269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2284101fd0 0x7f22840731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 -- 172.21.15.22:0/2386043269 >> 172.21.15.22:0/2386043269 conn(0x7f22840fc3d0 msgr2=0x7f2284105540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 -- 172.21.15.22:0/2386043269 shutdown_connections 2023-09-20T21:48:16.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:16.224+0000 7f228c1d8700 1 -- 172.21.15.22:0/2386043269 wait complete. 2023-09-20T21:48:16.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:16 smithi022 ceph-mon[97651]: pgmap v327: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:16 smithi123 ceph-mon[134433]: pgmap v327: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:16.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:16 smithi089 ceph-mon[134090]: pgmap v327: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:16.800 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph orch ps' 2023-09-20T21:48:17.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:17 smithi022 ceph-mon[97651]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:17.541 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:48:17.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:17 smithi123 ceph-mon[134433]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:17.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:17 smithi089 ceph-mon[134090]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:18.368 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:18 smithi022 ceph-mon[97651]: pgmap v328: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:18.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:18 smithi123 ceph-mon[134433]: pgmap v328: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:18.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:18 smithi089 ceph-mon[134090]: pgmap v328: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:19.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.079+0000 7f5b203fb700 1 -- 172.21.15.22:0/771626627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b18104c60 msgr2=0x7f5b1810b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:19.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.079+0000 7f5b203fb700 1 --2- 172.21.15.22:0/771626627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b18104c60 0x7f5b1810b510 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000b0d0 tx=0x7f5b1000b3e0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.079+0000 7f5b203fb700 1 -- 172.21.15.22:0/771626627 shutdown_connections 2023-09-20T21:48:19.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.079+0000 7f5b203fb700 1 --2- 172.21.15.22:0/771626627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b18104c60 0x7f5b1810b510 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.079+0000 7f5b203fb700 1 --2- 172.21.15.22:0/771626627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b181030c0 0x7f5b181034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.079+0000 7f5b203fb700 1 --2- 172.21.15.22:0/771626627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b181042c0 0x7f5b18104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.079+0000 7f5b203fb700 1 -- 172.21.15.22:0/771626627 >> 172.21.15.22:0/771626627 conn(0x7f5b180fe640 msgr2=0x7f5b18100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:19.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.079+0000 7f5b203fb700 1 -- 172.21.15.22:0/771626627 shutdown_connections 2023-09-20T21:48:19.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.079+0000 7f5b203fb700 1 -- 172.21.15.22:0/771626627 wait complete. 2023-09-20T21:48:19.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.081+0000 7f5b203fb700 1 Processor -- start 2023-09-20T21:48:19.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.081+0000 7f5b203fb700 1 -- start start 2023-09-20T21:48:19.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b203fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b181030c0 0x7f5b18198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:19.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b203fb700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b181042c0 0x7f5b181993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:19.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b203fb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b18104c60 0x7f5b181a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:19.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b203fb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b181a09f0 con 0x7f5b18104c60 2023-09-20T21:48:19.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b203fb700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b181a0b30 con 0x7f5b181042c0 2023-09-20T21:48:19.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b203fb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b181a0cc0 con 0x7f5b181030c0 2023-09-20T21:48:19.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1d996700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b181042c0 0x7f5b181993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:19.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1e998700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b18104c60 0x7f5b181a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:19.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1d996700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b181042c0 0x7f5b181993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:57846/0 (socket says 172.21.15.22:57846) 2023-09-20T21:48:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1d996700 1 -- 172.21.15.22:0/661755316 learned_addr learned my addr 172.21.15.22:0/661755316 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1e197700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b181030c0 0x7f5b18198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1d996700 1 -- 172.21.15.22:0/661755316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b181030c0 msgr2=0x7f5b18198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:19.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1d996700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b181030c0 0x7f5b18198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1d996700 1 -- 172.21.15.22:0/661755316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b18104c60 msgr2=0x7f5b181a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:19.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1d996700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b18104c60 0x7f5b181a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1d996700 1 -- 172.21.15.22:0/661755316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b10009d00 con 0x7f5b181042c0 2023-09-20T21:48:19.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.082+0000 7f5b1e998700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b18104c60 0x7f5b181a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:19.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.083+0000 7f5b1d996700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b181042c0 0x7f5b181993a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800e9a0 tx=0x7f5b0800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:19.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.083+0000 7f5b0f7fe700 1 -- 172.21.15.22:0/661755316 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08009790 con 0x7f5b181042c0 2023-09-20T21:48:19.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.083+0000 7f5b203fb700 1 -- 172.21.15.22:0/661755316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b181a0fa0 con 0x7f5b181042c0 2023-09-20T21:48:19.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.083+0000 7f5b0f7fe700 1 -- 172.21.15.22:0/661755316 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b08010430 con 0x7f5b181042c0 2023-09-20T21:48:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.083+0000 7f5b0f7fe700 1 -- 172.21.15.22:0/661755316 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08018680 con 0x7f5b181042c0 2023-09-20T21:48:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.083+0000 7f5b203fb700 1 -- 172.21.15.22:0/661755316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b181a1460 con 0x7f5b181042c0 2023-09-20T21:48:19.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.084+0000 7f5b203fb700 1 -- 172.21.15.22:0/661755316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1810a5c0 con 0x7f5b181042c0 2023-09-20T21:48:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.086+0000 7f5b0f7fe700 1 -- 172.21.15.22:0/661755316 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5b080105a0 con 0x7f5b181042c0 2023-09-20T21:48:19.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.088+0000 7f5b0f7fe700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5b040a1300 0x7f5b040a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.088+0000 7f5b0f7fe700 1 -- 172.21.15.22:0/661755316 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f5b08014070 con 0x7f5b181042c0 2023-09-20T21:48:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.088+0000 7f5b1e197700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5b040a1300 0x7f5b040a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.088+0000 7f5b1e197700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5b040a1300 0x7f5b040a37c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b18104120 tx=0x7f5b14009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.090+0000 7f5b0f7fe700 1 -- 172.21.15.22:0/661755316 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5b08096640 con 0x7f5b181042c0 2023-09-20T21:48:19.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.313+0000 7f5b203fb700 1 -- 172.21.15.22:0/661755316 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b1802d260 con 0x7f5b040a1300 2023-09-20T21:48:19.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.326+0000 7f5b0f7fe700 1 -- 172.21.15.22:0/661755316 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f5b1802d260 con 0x7f5b040a1300 2023-09-20T21:48:19.327 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-09-20T21:48:19.327 INFO:teuthology.orchestra.run.smithi022.stdout:mds.a smithi022 running (106s) - 106s 17.3M - 18.0.0-6213-gbf8a061f 6308322be975 e77b6e976816 2023-09-20T21:48:19.328 INFO:teuthology.orchestra.run.smithi022.stdout:mds.b smithi089 running (104s) - 104s 23.3M - 18.0.0-6213-gbf8a061f 6308322be975 52eb3a32f968 2023-09-20T21:48:19.328 INFO:teuthology.orchestra.run.smithi022.stdout:mds.c smithi123 running (117s) - 117s 19.6M - 18.0.0-6213-gbf8a061f 6308322be975 b4bce249e0a9 2023-09-20T21:48:19.328 INFO:teuthology.orchestra.run.smithi022.stdout:mds.d smithi022 running (2m) - 2m 16.1M - 18.0.0-6213-gbf8a061f 6308322be975 e72068a2f4fd 2023-09-20T21:48:19.329 INFO:teuthology.orchestra.run.smithi022.stdout:mds.e smithi089 running (2m) - 2m 16.4M - 18.0.0-6213-gbf8a061f 6308322be975 37415591aa59 2023-09-20T21:48:19.329 INFO:teuthology.orchestra.run.smithi022.stdout:mds.f smithi123 running (2m) - 119s 20.7M - 18.0.0-6213-gbf8a061f 6308322be975 c640e4b2ac2c 2023-09-20T21:48:19.329 INFO:teuthology.orchestra.run.smithi022.stdout:mds.g smithi022 running (110s) - 109s 13.5M - 18.0.0-6213-gbf8a061f 6308322be975 af018144da6c 2023-09-20T21:48:19.329 INFO:teuthology.orchestra.run.smithi022.stdout:mds.h smithi089 running (115s) - 115s 17.2M - 18.0.0-6213-gbf8a061f 6308322be975 2ecc198b1095 2023-09-20T21:48:19.330 INFO:teuthology.orchestra.run.smithi022.stdout:mds.i smithi123 running (113s) - 113s 18.9M - 18.0.0-6213-gbf8a061f 6308322be975 5c07339c5af7 2023-09-20T21:48:19.330 INFO:teuthology.orchestra.run.smithi022.stdout:mds.j smithi022 running (2m) - 2m 17.0M - 18.0.0-6213-gbf8a061f 6308322be975 80c202ec73c8 2023-09-20T21:48:19.330 INFO:teuthology.orchestra.run.smithi022.stdout:mds.k smithi089 running (2m) - 2m 19.7M - 18.0.0-6213-gbf8a061f 6308322be975 511d7fb0e699 2023-09-20T21:48:19.331 INFO:teuthology.orchestra.run.smithi022.stdout:mds.l smithi123 running (2m) - 2m 16.1M - 18.0.0-6213-gbf8a061f 6308322be975 74b0c9fb763c 2023-09-20T21:48:19.331 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.x smithi022 *:9283,8765 running (9m) - 9m 525M - 18.0.0-6213-gbf8a061f 6308322be975 0f9ad732a219 2023-09-20T21:48:19.331 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.y smithi089 *:8443,8765 running (7m) - 7m 452M - 18.0.0-6213-gbf8a061f 6308322be975 963b52447891 2023-09-20T21:48:19.332 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.z smithi123 *:8443,8765 running (7m) - 7m 452M - 18.0.0-6213-gbf8a061f 6308322be975 2c7eb7e1ca69 2023-09-20T21:48:19.332 INFO:teuthology.orchestra.run.smithi022.stdout:mon.a smithi022 running (9m) - 9m 93.8M 2048M 18.0.0-6213-gbf8a061f 6308322be975 0ee46ea65b53 2023-09-20T21:48:19.332 INFO:teuthology.orchestra.run.smithi022.stdout:mon.b smithi089 running (7m) - 7m 63.7M 2048M 18.0.0-6213-gbf8a061f 6308322be975 125e4a072e72 2023-09-20T21:48:19.332 INFO:teuthology.orchestra.run.smithi022.stdout:mon.c smithi123 running (7m) - 7m 63.3M 2048M 18.0.0-6213-gbf8a061f 6308322be975 cf9392da2285 2023-09-20T21:48:19.332 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (6m) - 6m 71.7M 4318M 18.0.0-6213-gbf8a061f 6308322be975 11e7d1aaf594 2023-09-20T21:48:19.333 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi089 running (6m) - 6m 76.0M 4292M 18.0.0-6213-gbf8a061f 6308322be975 a50c0a161281 2023-09-20T21:48:19.333 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi123 running (6m) - 6m 75.6M 4292M 18.0.0-6213-gbf8a061f 6308322be975 7695ff4637bd 2023-09-20T21:48:19.333 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 smithi022 running (5m) - 5m 68.7M 4318M 18.0.0-6213-gbf8a061f 6308322be975 4acc52d70053 2023-09-20T21:48:19.333 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 smithi089 running (5m) - 5m 64.9M 4292M 18.0.0-6213-gbf8a061f 6308322be975 5181ea26fe25 2023-09-20T21:48:19.333 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 smithi123 running (4m) - 4m 69.6M 4292M 18.0.0-6213-gbf8a061f 6308322be975 864f33291596 2023-09-20T21:48:19.334 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 smithi022 running (4m) - 4m 61.7M 4318M 18.0.0-6213-gbf8a061f 6308322be975 4c2985e7fc60 2023-09-20T21:48:19.334 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 smithi089 running (4m) - 4m 66.5M 4292M 18.0.0-6213-gbf8a061f 6308322be975 8ef4fa72701c 2023-09-20T21:48:19.334 INFO:teuthology.orchestra.run.smithi022.stdout:osd.8 smithi123 running (3m) - 3m 63.6M 4292M 18.0.0-6213-gbf8a061f 6308322be975 988d907c99d8 2023-09-20T21:48:19.334 INFO:teuthology.orchestra.run.smithi022.stdout:osd.9 smithi022 running (3m) - 3m 63.2M 4318M 18.0.0-6213-gbf8a061f 6308322be975 3d7b4aabd0ba 2023-09-20T21:48:19.334 INFO:teuthology.orchestra.run.smithi022.stdout:osd.10 smithi089 running (2m) - 2m 63.2M 4292M 18.0.0-6213-gbf8a061f 6308322be975 30bda93095ee 2023-09-20T21:48:19.335 INFO:teuthology.orchestra.run.smithi022.stdout:osd.11 smithi123 running (2m) - 2m 55.6M 4292M 18.0.0-6213-gbf8a061f 6308322be975 5eabc38b791e 2023-09-20T21:48:19.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.328+0000 7f5b203fb700 1 -- 172.21.15.22:0/661755316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5b040a1300 msgr2=0x7f5b040a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:19.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.328+0000 7f5b203fb700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5b040a1300 0x7f5b040a37c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5b18104120 tx=0x7f5b14009260 comp rx=0 tx=0).stop 2023-09-20T21:48:19.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.328+0000 7f5b203fb700 1 -- 172.21.15.22:0/661755316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b181042c0 msgr2=0x7f5b181993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:19.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.328+0000 7f5b203fb700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b181042c0 0x7f5b181993a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800e9a0 tx=0x7f5b0800ecb0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.329+0000 7f5b203fb700 1 -- 172.21.15.22:0/661755316 shutdown_connections 2023-09-20T21:48:19.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.329+0000 7f5b203fb700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5b18104c60 0x7f5b181a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.329+0000 7f5b203fb700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b181030c0 0x7f5b18198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.329+0000 7f5b203fb700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5b040a1300 0x7f5b040a37c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.329+0000 7f5b203fb700 1 --2- 172.21.15.22:0/661755316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b181042c0 0x7f5b181993a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:19.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.329+0000 7f5b203fb700 1 -- 172.21.15.22:0/661755316 >> 172.21.15.22:0/661755316 conn(0x7f5b180fe640 msgr2=0x7f5b18107700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:19.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.329+0000 7f5b203fb700 1 -- 172.21.15.22:0/661755316 shutdown_connections 2023-09-20T21:48:19.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:19.329+0000 7f5b203fb700 1 -- 172.21.15.22:0/661755316 wait complete. 2023-09-20T21:48:19.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph orch ls' 2023-09-20T21:48:20.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:20 smithi022 ceph-mon[97651]: pgmap v329: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:20.568 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:48:20.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:20 smithi123 ceph-mon[134433]: pgmap v329: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:20.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:20 smithi089 ceph-mon[134090]: pgmap v329: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:21.493 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:21 smithi022 ceph-mon[97651]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:21.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:21 smithi123 ceph-mon[134433]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:21.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:21 smithi089 ceph-mon[134090]: from='client.25205 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:22.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.136+0000 7ff806a75700 1 -- 172.21.15.22:0/4148683150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8001030c0 msgr2=0x7ff8001034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:22.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.136+0000 7ff806a75700 1 --2- 172.21.15.22:0/4148683150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8001030c0 0x7ff8001034e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40099c0 tx=0x7ff7f4009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:22.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.136+0000 7ff806a75700 1 -- 172.21.15.22:0/4148683150 shutdown_connections 2023-09-20T21:48:22.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.136+0000 7ff806a75700 1 --2- 172.21.15.22:0/4148683150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff800104c60 0x7ff80010b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:22.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.136+0000 7ff806a75700 1 --2- 172.21.15.22:0/4148683150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8001030c0 0x7ff8001034e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:22.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.136+0000 7ff806a75700 1 --2- 172.21.15.22:0/4148683150 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8001042c0 0x7ff800104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:22.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.136+0000 7ff806a75700 1 -- 172.21.15.22:0/4148683150 >> 172.21.15.22:0/4148683150 conn(0x7ff8000fe640 msgr2=0x7ff800100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:22.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.136+0000 7ff806a75700 1 -- 172.21.15.22:0/4148683150 shutdown_connections 2023-09-20T21:48:22.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.136+0000 7ff806a75700 1 -- 172.21.15.22:0/4148683150 wait complete. 2023-09-20T21:48:22.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.137+0000 7ff806a75700 1 Processor -- start 2023-09-20T21:48:22.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.137+0000 7ff806a75700 1 -- start start 2023-09-20T21:48:22.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.137+0000 7ff806a75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8001030c0 0x7ff800198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:22.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.137+0000 7ff806a75700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8001042c0 0x7ff800199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:22.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.137+0000 7ff806a75700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800104c60 0x7ff8001a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:22.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.137+0000 7ff806a75700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001a0990 con 0x7ff8001042c0 2023-09-20T21:48:22.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff806a75700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001a0ad0 con 0x7ff800104c60 2023-09-20T21:48:22.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff806a75700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001a0c60 con 0x7ff8001030c0 2023-09-20T21:48:22.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff804811700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8001030c0 0x7ff800198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff804811700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8001030c0 0x7ff800198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:35002/0 (socket says 172.21.15.22:35002) 2023-09-20T21:48:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff804811700 1 -- 172.21.15.22:0/1995434988 learned_addr learned my addr 172.21.15.22:0/1995434988 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff805012700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800104c60 0x7ff8001a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff7fffff700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8001042c0 0x7ff800199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff804811700 1 -- 172.21.15.22:0/1995434988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800104c60 msgr2=0x7ff8001a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff804811700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800104c60 0x7ff8001a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff804811700 1 -- 172.21.15.22:0/1995434988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8001042c0 msgr2=0x7ff800199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff804811700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8001042c0 0x7ff800199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff804811700 1 -- 172.21.15.22:0/1995434988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f40095f0 con 0x7ff8001030c0 2023-09-20T21:48:22.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.138+0000 7ff804811700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8001030c0 0x7ff800198e00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40098e0 tx=0x7ff7f400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:22.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.139+0000 7ff7fdffb700 1 -- 172.21.15.22:0/1995434988 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f401d070 con 0x7ff8001030c0 2023-09-20T21:48:22.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.139+0000 7ff7fdffb700 1 -- 172.21.15.22:0/1995434988 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f4005030 con 0x7ff8001030c0 2023-09-20T21:48:22.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.139+0000 7ff806a75700 1 -- 172.21.15.22:0/1995434988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8001a0ee0 con 0x7ff8001030c0 2023-09-20T21:48:22.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.139+0000 7ff7fdffb700 1 -- 172.21.15.22:0/1995434988 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4004a90 con 0x7ff8001030c0 2023-09-20T21:48:22.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.139+0000 7ff806a75700 1 -- 172.21.15.22:0/1995434988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8001a1240 con 0x7ff8001030c0 2023-09-20T21:48:22.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.140+0000 7ff806a75700 1 -- 172.21.15.22:0/1995434988 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80010a6b0 con 0x7ff8001030c0 2023-09-20T21:48:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.142+0000 7ff7fdffb700 1 -- 172.21.15.22:0/1995434988 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ff7f40051a0 con 0x7ff8001030c0 2023-09-20T21:48:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.143+0000 7ff7fdffb700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff7f00a1300 0x7ff7f00a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:22.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.144+0000 7ff7fdffb700 1 -- 172.21.15.22:0/1995434988 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ff7f40ce1a0 con 0x7ff8001030c0 2023-09-20T21:48:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.144+0000 7ff7fffff700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff7f00a1300 0x7ff7f00a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.144+0000 7ff7fffff700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff7f00a1300 0x7ff7f00a37c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009c60 tx=0x7ff7ec009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:22.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.146+0000 7ff7fdffb700 1 -- 172.21.15.22:0/1995434988 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff7f4097510 con 0x7ff8001030c0 2023-09-20T21:48:22.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.367+0000 7ff806a75700 1 -- 172.21.15.22:0/1995434988 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff80010e920 con 0x7ff7f00a1300 2023-09-20T21:48:22.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.372+0000 7ff7fdffb700 1 -- 172.21.15.22:0/1995434988 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7ff80010e920 con 0x7ff7f00a1300 2023-09-20T21:48:22.373 INFO:teuthology.orchestra.run.smithi022.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-09-20T21:48:22.373 INFO:teuthology.orchestra.run.smithi022.stdout:mds.all 12/12 - 2m smithi022=a;smithi022=d;smithi022=g;smithi022=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2023-09-20T21:48:22.373 INFO:teuthology.orchestra.run.smithi022.stdout:mgr 3/3 - 7m smithi022=x;smithi089=y;smithi123=z;count:3 2023-09-20T21:48:22.374 INFO:teuthology.orchestra.run.smithi022.stdout:mon 3/3 - 7m smithi022:172.21.15.22=a;smithi089:172.21.15.89=b;smithi123:172.21.15.123=c;count:3 2023-09-20T21:48:22.374 INFO:teuthology.orchestra.run.smithi022.stdout:osd 12 - - 2023-09-20T21:48:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 -- 172.21.15.22:0/1995434988 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff7f00a1300 msgr2=0x7ff7f00a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff7f00a1300 0x7ff7f00a37c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009c60 tx=0x7ff7ec009260 comp rx=0 tx=0).stop 2023-09-20T21:48:22.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 -- 172.21.15.22:0/1995434988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8001030c0 msgr2=0x7ff800198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:22.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8001030c0 0x7ff800198e00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40098e0 tx=0x7ff7f400f690 comp rx=0 tx=0).stop 2023-09-20T21:48:22.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 -- 172.21.15.22:0/1995434988 shutdown_connections 2023-09-20T21:48:22.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff7f00a1300 0x7ff7f00a37c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:22.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff800104c60 0x7ff8001a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:22.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8001030c0 0x7ff800198e00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:22.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 --2- 172.21.15.22:0/1995434988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8001042c0 0x7ff800199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:22.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 -- 172.21.15.22:0/1995434988 >> 172.21.15.22:0/1995434988 conn(0x7ff8000fe640 msgr2=0x7ff8001077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:22.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 -- 172.21.15.22:0/1995434988 shutdown_connections 2023-09-20T21:48:22.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:22.375+0000 7ff806a75700 1 -- 172.21.15.22:0/1995434988 wait complete. 2023-09-20T21:48:22.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:22 smithi022 ceph-mon[97651]: pgmap v330: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:22.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:22 smithi123 ceph-mon[134433]: pgmap v330: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:22.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:22 smithi089 ceph-mon[134090]: pgmap v330: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:22.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph orch host ls' 2023-09-20T21:48:23.607 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:48:23.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:23 smithi123 ceph-mon[134433]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:23.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:23 smithi123 ceph-mon[134433]: pgmap v331: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:23.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:23 smithi089 ceph-mon[134090]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:23.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:23 smithi089 ceph-mon[134090]: pgmap v331: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:23.881 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:23 smithi022 ceph-mon[97651]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:23.881 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:23 smithi022 ceph-mon[97651]: pgmap v331: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:25.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.196+0000 7fb41a822700 1 -- 172.21.15.22:0/1590045974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4140ffd70 msgr2=0x7fb414100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:25.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.196+0000 7fb41a822700 1 --2- 172.21.15.22:0/1590045974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4140ffd70 0x7fb414100190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb404009920 tx=0x7fb404009c30 comp rx=0 tx=0).stop 2023-09-20T21:48:25.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.196+0000 7fb41a822700 1 -- 172.21.15.22:0/1590045974 shutdown_connections 2023-09-20T21:48:25.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.196+0000 7fb41a822700 1 --2- 172.21.15.22:0/1590045974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414101910 0x7fb4141081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:25.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.196+0000 7fb41a822700 1 --2- 172.21.15.22:0/1590045974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4140ffd70 0x7fb414100190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:25.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.196+0000 7fb41a822700 1 --2- 172.21.15.22:0/1590045974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb414100f70 0x7fb4141013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:25.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.196+0000 7fb41a822700 1 -- 172.21.15.22:0/1590045974 >> 172.21.15.22:0/1590045974 conn(0x7fb4140fb2f0 msgr2=0x7fb4140fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:25.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.196+0000 7fb41a822700 1 -- 172.21.15.22:0/1590045974 shutdown_connections 2023-09-20T21:48:25.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.196+0000 7fb41a822700 1 -- 172.21.15.22:0/1590045974 wait complete. 2023-09-20T21:48:25.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.198+0000 7fb41a822700 1 Processor -- start 2023-09-20T21:48:25.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.198+0000 7fb41a822700 1 -- start start 2023-09-20T21:48:25.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb41a822700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4140ffd70 0x7fb414198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:25.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb41a822700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414100f70 0x7fb414199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:25.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb41a822700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb414101910 0x7fb4141a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:25.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb41a822700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4141a0970 con 0x7fb414101910 2023-09-20T21:48:25.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb41a822700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4141a0ab0 con 0x7fb4140ffd70 2023-09-20T21:48:25.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb41a822700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4141a0c40 con 0x7fb414100f70 2023-09-20T21:48:25.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb413fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4140ffd70 0x7fb414198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:25.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb418dbf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb414101910 0x7fb4141a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:25.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb413fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4140ffd70 0x7fb414198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:57884/0 (socket says 172.21.15.22:57884) 2023-09-20T21:48:25.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb413fff700 1 -- 172.21.15.22:0/3674685576 learned_addr learned my addr 172.21.15.22:0/3674685576 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:25.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb4137fe700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414100f70 0x7fb414199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:25.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb413fff700 1 -- 172.21.15.22:0/3674685576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414100f70 msgr2=0x7fb414199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:25.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb413fff700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414100f70 0x7fb414199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:25.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb413fff700 1 -- 172.21.15.22:0/3674685576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb414101910 msgr2=0x7fb4141a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:25.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb413fff700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb414101910 0x7fb4141a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:25.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb413fff700 1 -- 172.21.15.22:0/3674685576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb404009520 con 0x7fb4140ffd70 2023-09-20T21:48:25.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.199+0000 7fb418dbf700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb414101910 0x7fb4141a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:25.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.200+0000 7fb413fff700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4140ffd70 0x7fb414198de0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb404009230 tx=0x7fb40400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:25.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.200+0000 7fb4117fa700 1 -- 172.21.15.22:0/3674685576 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40401c070 con 0x7fb4140ffd70 2023-09-20T21:48:25.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.200+0000 7fb4117fa700 1 -- 172.21.15.22:0/3674685576 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb40400f7d0 con 0x7fb4140ffd70 2023-09-20T21:48:25.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.200+0000 7fb4117fa700 1 -- 172.21.15.22:0/3674685576 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404004910 con 0x7fb4140ffd70 2023-09-20T21:48:25.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.200+0000 7fb41a822700 1 -- 172.21.15.22:0/3674685576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4141a0ec0 con 0x7fb4140ffd70 2023-09-20T21:48:25.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.200+0000 7fb41a822700 1 -- 172.21.15.22:0/3674685576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4141a1220 con 0x7fb4140ffd70 2023-09-20T21:48:25.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.201+0000 7fb41a822700 1 -- 172.21.15.22:0/3674685576 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41410b5c0 con 0x7fb4140ffd70 2023-09-20T21:48:25.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.203+0000 7fb4117fa700 1 -- 172.21.15.22:0/3674685576 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fb404020070 con 0x7fb4140ffd70 2023-09-20T21:48:25.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.205+0000 7fb4117fa700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3fc0a1300 0x7fb3fc0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:25.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.205+0000 7fb4117fa700 1 -- 172.21.15.22:0/3674685576 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fb4040cdd80 con 0x7fb4140ffd70 2023-09-20T21:48:25.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.205+0000 7fb4137fe700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3fc0a1300 0x7fb3fc0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:25.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.205+0000 7fb4137fe700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3fc0a1300 0x7fb3fc0a37c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb408005fd0 tx=0x7fb408009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:25.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.206+0000 7fb4117fa700 1 -- 172.21.15.22:0/3674685576 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb4040970f0 con 0x7fb4140ffd70 2023-09-20T21:48:25.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.423+0000 7fb41a822700 1 -- 172.21.15.22:0/3674685576 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb414069010 con 0x7fb3fc0a1300 2023-09-20T21:48:25.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.425+0000 7fb4117fa700 1 -- 172.21.15.22:0/3674685576 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fb414069010 con 0x7fb3fc0a1300 2023-09-20T21:48:25.425 INFO:teuthology.orchestra.run.smithi022.stdout:HOST ADDR LABELS STATUS 2023-09-20T21:48:25.426 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 172.21.15.22 2023-09-20T21:48:25.426 INFO:teuthology.orchestra.run.smithi022.stdout:smithi089 172.21.15.89 2023-09-20T21:48:25.426 INFO:teuthology.orchestra.run.smithi022.stdout:smithi123 172.21.15.123 2023-09-20T21:48:25.427 INFO:teuthology.orchestra.run.smithi022.stdout:3 hosts in cluster 2023-09-20T21:48:25.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 -- 172.21.15.22:0/3674685576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3fc0a1300 msgr2=0x7fb3fc0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:25.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3fc0a1300 0x7fb3fc0a37c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb408005fd0 tx=0x7fb408009310 comp rx=0 tx=0).stop 2023-09-20T21:48:25.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 -- 172.21.15.22:0/3674685576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4140ffd70 msgr2=0x7fb414198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4140ffd70 0x7fb414198de0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb404009230 tx=0x7fb40400b790 comp rx=0 tx=0).stop 2023-09-20T21:48:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 -- 172.21.15.22:0/3674685576 shutdown_connections 2023-09-20T21:48:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb414101910 0x7fb4141a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3fc0a1300 0x7fb3fc0a37c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:25.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb4140ffd70 0x7fb414198de0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:25.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 --2- 172.21.15.22:0/3674685576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb414100f70 0x7fb414199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:25.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 -- 172.21.15.22:0/3674685576 >> 172.21.15.22:0/3674685576 conn(0x7fb4140fb2f0 msgr2=0x7fb414068900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:25.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 -- 172.21.15.22:0/3674685576 shutdown_connections 2023-09-20T21:48:25.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:25.427+0000 7fb41a822700 1 -- 172.21.15.22:0/3674685576 wait complete. 2023-09-20T21:48:25.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph orch device ls' 2023-09-20T21:48:26.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:26 smithi022 ceph-mon[97651]: pgmap v332: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:26.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:26 smithi123 ceph-mon[134433]: pgmap v332: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:26.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:26 smithi089 ceph-mon[134090]: pgmap v332: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2023-09-20T21:48:26.638 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:48:27.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:27 smithi022 ceph-mon[97651]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:27.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:27 smithi123 ceph-mon[134433]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:27.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:27 smithi089 ceph-mon[134090]: from='client.25217 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:28.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.169+0000 7f38e631f700 1 -- 172.21.15.22:0/1962268606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38e0073840 msgr2=0x7f38e0073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:28.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.169+0000 7f38e631f700 1 --2- 172.21.15.22:0/1962268606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38e0073840 0x7f38e0073cc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009990 tx=0x7f38d4009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.170+0000 7f38e631f700 1 -- 172.21.15.22:0/1962268606 shutdown_connections 2023-09-20T21:48:28.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.170+0000 7f38e631f700 1 --2- 172.21.15.22:0/1962268606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e0108d40 0x7f38e010b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.170+0000 7f38e631f700 1 --2- 172.21.15.22:0/1962268606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e0074e10 0x7f38e0073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.170+0000 7f38e631f700 1 --2- 172.21.15.22:0/1962268606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38e0073840 0x7f38e0073cc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.170+0000 7f38e631f700 1 -- 172.21.15.22:0/1962268606 >> 172.21.15.22:0/1962268606 conn(0x7f38e00fc3d0 msgr2=0x7f38e00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:28.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.170+0000 7f38e631f700 1 -- 172.21.15.22:0/1962268606 shutdown_connections 2023-09-20T21:48:28.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.170+0000 7f38e631f700 1 -- 172.21.15.22:0/1962268606 wait complete. 2023-09-20T21:48:28.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.171+0000 7f38e631f700 1 Processor -- start 2023-09-20T21:48:28.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.171+0000 7f38e631f700 1 -- start start 2023-09-20T21:48:28.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.171+0000 7f38e631f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38e0073840 0x7f38e0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:28.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.171+0000 7f38e631f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e0074e10 0x7f38e01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:28.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.171+0000 7f38e631f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e0108d40 0x7f38e01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:28.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.171+0000 7f38e631f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e01a08f0 con 0x7f38e0073840 2023-09-20T21:48:28.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.171+0000 7f38e631f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e01a0a30 con 0x7f38e0074e10 2023-09-20T21:48:28.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.171+0000 7f38e631f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e01a0bc0 con 0x7f38e0108d40 2023-09-20T21:48:28.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38e48bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e0108d40 0x7f38e01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:28.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.171+0000 7f38dffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38e0073840 0x7f38e0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:28.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38e48bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e0108d40 0x7f38e01a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:36270/0 (socket says 172.21.15.22:36270) 2023-09-20T21:48:28.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38e48bc700 1 -- 172.21.15.22:0/1532821204 learned_addr learned my addr 172.21.15.22:0/1532821204 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:28.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38df7fe700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e0074e10 0x7f38e01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:28.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38dffff700 1 -- 172.21.15.22:0/1532821204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e0108d40 msgr2=0x7f38e01a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:28.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38dffff700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e0108d40 0x7f38e01a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38dffff700 1 -- 172.21.15.22:0/1532821204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e0074e10 msgr2=0x7f38e01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:28.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38dffff700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e0074e10 0x7f38e01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38dffff700 1 -- 172.21.15.22:0/1532821204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40095f0 con 0x7f38e0073840 2023-09-20T21:48:28.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38e48bc700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e0108d40 0x7f38e01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:28.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38df7fe700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e0074e10 0x7f38e01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:28.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.172+0000 7f38dffff700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38e0073840 0x7f38e0198d60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f38d000e9f0 tx=0x7f38d000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:28.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.173+0000 7f38dd7fa700 1 -- 172.21.15.22:0/1532821204 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000c9c0 con 0x7f38e0073840 2023-09-20T21:48:28.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.173+0000 7f38dd7fa700 1 -- 172.21.15.22:0/1532821204 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38d0010430 con 0x7f38e0073840 2023-09-20T21:48:28.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.173+0000 7f38dd7fa700 1 -- 172.21.15.22:0/1532821204 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0018680 con 0x7f38e0073840 2023-09-20T21:48:28.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.173+0000 7f38e631f700 1 -- 172.21.15.22:0/1532821204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e01a0ea0 con 0x7f38e0073840 2023-09-20T21:48:28.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.173+0000 7f38e631f700 1 -- 172.21.15.22:0/1532821204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e01a1260 con 0x7f38e0073840 2023-09-20T21:48:28.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.174+0000 7f38e631f700 1 -- 172.21.15.22:0/1532821204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e01073f0 con 0x7f38e0073840 2023-09-20T21:48:28.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.176+0000 7f38dd7fa700 1 -- 172.21.15.22:0/1532821204 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f38d00105a0 con 0x7f38e0073840 2023-09-20T21:48:28.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.178+0000 7f38dd7fa700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f38c80a1380 0x7f38c80a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:28.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.178+0000 7f38dd7fa700 1 -- 172.21.15.22:0/1532821204 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f38d0014070 con 0x7f38e0073840 2023-09-20T21:48:28.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.178+0000 7f38df7fe700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f38c80a1380 0x7f38c80a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:28.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.179+0000 7f38df7fe700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f38c80a1380 0x7f38c80a3840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f38d400b2e0 tx=0x7f38d4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:28.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.182+0000 7f38dd7fa700 1 -- 172.21.15.22:0/1532821204 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f38d0051e10 con 0x7f38e0073840 2023-09-20T21:48:28.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.404+0000 7f38e631f700 1 -- 172.21.15.22:0/1532821204 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f38e002d2f0 con 0x7f38c80a1380 2023-09-20T21:48:28.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.406+0000 7f38dd7fa700 1 -- 172.21.15.22:0/1532821204 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f38e002d2f0 con 0x7f38c80a1380 2023-09-20T21:48:28.406 INFO:teuthology.orchestra.run.smithi022.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-09-20T21:48:28.406 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402EC480DGN 447G No 3m ago Has a FileSystem, LVM detected 2023-09-20T21:48:28.406 INFO:teuthology.orchestra.run.smithi022.stdout:smithi089 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040126400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-20T21:48:28.407 INFO:teuthology.orchestra.run.smithi022.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-09-20T21:48:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 -- 172.21.15.22:0/1532821204 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f38c80a1380 msgr2=0x7f38c80a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f38c80a1380 0x7f38c80a3840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f38d400b2e0 tx=0x7f38d4005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 -- 172.21.15.22:0/1532821204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38e0073840 msgr2=0x7f38e0198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38e0073840 0x7f38e0198d60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f38d000e9f0 tx=0x7f38d000ed00 comp rx=0 tx=0).stop 2023-09-20T21:48:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 -- 172.21.15.22:0/1532821204 shutdown_connections 2023-09-20T21:48:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e0108d40 0x7f38e01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38e0073840 0x7f38e0198d60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f38c80a1380 0x7f38c80a3840 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 --2- 172.21.15.22:0/1532821204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38e0074e10 0x7f38e01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 -- 172.21.15.22:0/1532821204 >> 172.21.15.22:0/1532821204 conn(0x7f38e00fc3d0 msgr2=0x7f38e00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:28.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 -- 172.21.15.22:0/1532821204 shutdown_connections 2023-09-20T21:48:28.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:28.408+0000 7f38e631f700 1 -- 172.21.15.22:0/1532821204 wait complete. 2023-09-20T21:48:28.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:28 smithi022 ceph-mon[97651]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:28.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:28 smithi123 ceph-mon[134433]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:28.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:28 smithi089 ceph-mon[134090]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:28.837 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-09-20T21:48:28.847 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2023-09-20T21:48:28.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph fs dump' 2023-09-20T21:48:29.584 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:48:30.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:30 smithi022 ceph-mon[97651]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:30.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:30 smithi022 ceph-mon[97651]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:48:30.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:30 smithi123 ceph-mon[134433]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:30.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:30 smithi123 ceph-mon[134433]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:48:30.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:30 smithi089 ceph-mon[134090]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-09-20T21:48:30.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:30 smithi089 ceph-mon[134090]: pgmap v334: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:48:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.110+0000 7f6853b8a700 1 -- 172.21.15.22:0/781587590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f684c101fd0 msgr2=0x7f684c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.110+0000 7f6853b8a700 1 --2- 172.21.15.22:0/781587590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f684c101fd0 0x7f684c102450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f683c009990 tx=0x7f683c009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:31.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.110+0000 7f6853b8a700 1 -- 172.21.15.22:0/781587590 shutdown_connections 2023-09-20T21:48:31.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.110+0000 7f6853b8a700 1 --2- 172.21.15.22:0/781587590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f684c102990 0x7f684c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:31.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.110+0000 7f6853b8a700 1 --2- 172.21.15.22:0/781587590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f684c100e70 0x7f684c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:31.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.110+0000 7f6853b8a700 1 --2- 172.21.15.22:0/781587590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f684c101fd0 0x7f684c102450 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:31.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.110+0000 7f6853b8a700 1 -- 172.21.15.22:0/781587590 >> 172.21.15.22:0/781587590 conn(0x7f684c0fc3d0 msgr2=0x7f684c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:31.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.110+0000 7f6853b8a700 1 -- 172.21.15.22:0/781587590 shutdown_connections 2023-09-20T21:48:31.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.110+0000 7f6853b8a700 1 -- 172.21.15.22:0/781587590 wait complete. 2023-09-20T21:48:31.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.111+0000 7f6853b8a700 1 Processor -- start 2023-09-20T21:48:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.111+0000 7f6853b8a700 1 -- start start 2023-09-20T21:48:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6853b8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f684c100e70 0x7f684c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6853b8a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f684c101fd0 0x7f684c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6853b8a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f684c102990 0x7f684c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:31.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6853b8a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1a08e0 con 0x7f684c101fd0 2023-09-20T21:48:31.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6853b8a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1a0a20 con 0x7f684c102990 2023-09-20T21:48:31.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6853b8a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1a0be0 con 0x7f684c100e70 2023-09-20T21:48:31.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6851125700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f684c101fd0 0x7f684c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:31.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6851125700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f684c101fd0 0x7f684c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54616/0 (socket says 172.21.15.22:54616) 2023-09-20T21:48:31.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6851125700 1 -- 172.21.15.22:0/3170442381 learned_addr learned my addr 172.21.15.22:0/3170442381 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6851926700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f684c100e70 0x7f684c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6852127700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f684c102990 0x7f684c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6851125700 1 -- 172.21.15.22:0/3170442381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f684c100e70 msgr2=0x7f684c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6851125700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f684c100e70 0x7f684c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6851125700 1 -- 172.21.15.22:0/3170442381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f684c102990 msgr2=0x7f684c1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6851125700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f684c102990 0x7f684c1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.112+0000 7f6851125700 1 -- 172.21.15.22:0/3170442381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848009520 con 0x7f684c101fd0 2023-09-20T21:48:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.113+0000 7f6851125700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f684c101fd0 0x7f684c1992d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f683c00b620 tx=0x7f683c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.113+0000 7f6842ffd700 1 -- 172.21.15.22:0/3170442381 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c01d070 con 0x7f684c101fd0 2023-09-20T21:48:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.113+0000 7f6842ffd700 1 -- 172.21.15.22:0/3170442381 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f683c022470 con 0x7f684c101fd0 2023-09-20T21:48:31.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.113+0000 7f6842ffd700 1 -- 172.21.15.22:0/3170442381 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c017650 con 0x7f684c101fd0 2023-09-20T21:48:31.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.113+0000 7f6853b8a700 1 -- 172.21.15.22:0/3170442381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c0095f0 con 0x7f684c101fd0 2023-09-20T21:48:31.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.113+0000 7f6853b8a700 1 -- 172.21.15.22:0/3170442381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c1a10e0 con 0x7f684c101fd0 2023-09-20T21:48:31.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.114+0000 7f6853b8a700 1 -- 172.21.15.22:0/3170442381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c1934f0 con 0x7f684c101fd0 2023-09-20T21:48:31.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.116+0000 7f6842ffd700 1 -- 172.21.15.22:0/3170442381 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f683c0225e0 con 0x7f684c101fd0 2023-09-20T21:48:31.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.119+0000 7f6842ffd700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68380a1380 0x7f68380a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:31.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.119+0000 7f6842ffd700 1 -- 172.21.15.22:0/3170442381 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f683c0ce3e0 con 0x7f684c101fd0 2023-09-20T21:48:31.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.119+0000 7f6851926700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68380a1380 0x7f68380a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:31.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.119+0000 7f6851926700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68380a1380 0x7f68380a3840 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f684c19a270 tx=0x7f6848009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:31.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.123+0000 7f6842ffd700 1 -- 172.21.15.22:0/3170442381 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f683c097810 con 0x7f684c101fd0 2023-09-20T21:48:31.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.379+0000 7f6853b8a700 1 -- 172.21.15.22:0/3170442381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f684c10c700 con 0x7f684c101fd0 2023-09-20T21:48:31.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.380+0000 7f6842ffd700 1 -- 172.21.15.22:0/3170442381 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+2967 (secure 0 0 0) 0x7f683c027bb0 con 0x7f684c101fd0 2023-09-20T21:48:31.380 INFO:teuthology.orchestra.run.smithi022.stdout:e28 2023-09-20T21:48:31.381 INFO:teuthology.orchestra.run.smithi022.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-09-20T21:48:31.381 INFO:teuthology.orchestra.run.smithi022.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-20T21:48:31.381 INFO:teuthology.orchestra.run.smithi022.stdout:legacy client fscid: 1 2023-09-20T21:48:31.381 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:31.382 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem 'cephfs' (1) 2023-09-20T21:48:31.382 INFO:teuthology.orchestra.run.smithi022.stdout:fs_name cephfs 2023-09-20T21:48:31.382 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 28 2023-09-20T21:48:31.382 INFO:teuthology.orchestra.run.smithi022.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-09-20T21:48:31.382 INFO:teuthology.orchestra.run.smithi022.stdout:created 2023-09-20T21:46:41.044280+0000 2023-09-20T21:48:31.383 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2023-09-20T21:47:09.607119+0000 2023-09-20T21:48:31.383 INFO:teuthology.orchestra.run.smithi022.stdout:tableserver 0 2023-09-20T21:48:31.383 INFO:teuthology.orchestra.run.smithi022.stdout:root 0 2023-09-20T21:48:31.383 INFO:teuthology.orchestra.run.smithi022.stdout:session_timeout 300 2023-09-20T21:48:31.383 INFO:teuthology.orchestra.run.smithi022.stdout:session_autoclose 300 2023-09-20T21:48:31.383 INFO:teuthology.orchestra.run.smithi022.stdout:max_file_size 1099511627776 2023-09-20T21:48:31.384 INFO:teuthology.orchestra.run.smithi022.stdout:max_xattr_size 65536 2023-09-20T21:48:31.384 INFO:teuthology.orchestra.run.smithi022.stdout:required_client_features {} 2023-09-20T21:48:31.384 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure 0 2023-09-20T21:48:31.384 INFO:teuthology.orchestra.run.smithi022.stdout:last_failure_osd_epoch 0 2023-09-20T21:48:31.384 INFO:teuthology.orchestra.run.smithi022.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-09-20T21:48:31.385 INFO:teuthology.orchestra.run.smithi022.stdout:max_mds 5 2023-09-20T21:48:31.385 INFO:teuthology.orchestra.run.smithi022.stdout:in 0,1,2,3,4 2023-09-20T21:48:31.385 INFO:teuthology.orchestra.run.smithi022.stdout:up {0=24578,1=24566,2=24530,3=24487,4=24473} 2023-09-20T21:48:31.385 INFO:teuthology.orchestra.run.smithi022.stdout:failed 2023-09-20T21:48:31.385 INFO:teuthology.orchestra.run.smithi022.stdout:damaged 2023-09-20T21:48:31.385 INFO:teuthology.orchestra.run.smithi022.stdout:stopped 2023-09-20T21:48:31.386 INFO:teuthology.orchestra.run.smithi022.stdout:data_pools [3,4] 2023-09-20T21:48:31.386 INFO:teuthology.orchestra.run.smithi022.stdout:metadata_pool 2 2023-09-20T21:48:31.386 INFO:teuthology.orchestra.run.smithi022.stdout:inline_data disabled 2023-09-20T21:48:31.386 INFO:teuthology.orchestra.run.smithi022.stdout:balancer 2023-09-20T21:48:31.386 INFO:teuthology.orchestra.run.smithi022.stdout:bal_rank_mask -1 2023-09-20T21:48:31.387 INFO:teuthology.orchestra.run.smithi022.stdout:standby_count_wanted 0 2023-09-20T21:48:31.387 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b{0:24578} state up:active seq 3 addr [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.387 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.i{0:24520} state up:standby-replay seq 1 addr [v2:172.21.15.123:6838/2368757728,v1:172.21.15.123:6839/2368757728] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.387 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a{1:24566} state up:active seq 10 addr [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.387 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c{1:24499} state up:standby-replay seq 1 addr [v2:172.21.15.123:6836/1168160038,v1:172.21.15.123:6837/1168160038] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.387 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.h{2:24530} state up:active seq 12 addr [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.388 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.e{2:24497} state up:standby-replay seq 1 addr [v2:172.21.15.89:6834/794452586,v1:172.21.15.89:6835/794452586] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.388 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.f{3:24487} state up:active seq 13 addr [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.388 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d{3:24466} state up:standby-replay seq 1 addr [v2:172.21.15.22:6836/4039103472,v1:172.21.15.22:6837/4039103472] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.388 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.j{4:24473} state up:active seq 16 addr [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.388 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.k{4:24461} state up:standby-replay seq 1 addr [v2:172.21.15.89:6832/3516975495,v1:172.21.15.89:6833/3516975495] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:31.389 INFO:teuthology.orchestra.run.smithi022.stdout:Standby daemons: 2023-09-20T21:48:31.389 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:31.389 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.g{-1:14505} state up:standby seq 1 addr [v2:172.21.15.22:6838/3781158526,v1:172.21.15.22:6839/3781158526] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.389 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.l{-1:24433} state up:standby seq 1 addr [v2:172.21.15.123:6832/2368905204,v1:172.21.15.123:6833/2368905204] compat {c=[1],r=[1],i=[fff]}] 2023-09-20T21:48:31.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 -- 172.21.15.22:0/3170442381 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68380a1380 msgr2=0x7f68380a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:31.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68380a1380 0x7f68380a3840 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f684c19a270 tx=0x7f6848009260 comp rx=0 tx=0).stop 2023-09-20T21:48:31.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 -- 172.21.15.22:0/3170442381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f684c101fd0 msgr2=0x7f684c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:31.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f684c101fd0 0x7f684c1992d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f683c00b620 tx=0x7f683c005740 comp rx=0 tx=0).stop 2023-09-20T21:48:31.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 -- 172.21.15.22:0/3170442381 shutdown_connections 2023-09-20T21:48:31.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68380a1380 0x7f68380a3840 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:31.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f684c102990 0x7f684c1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:31.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f684c100e70 0x7f684c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:31.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 --2- 172.21.15.22:0/3170442381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f684c101fd0 0x7f684c1992d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:31.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 -- 172.21.15.22:0/3170442381 >> 172.21.15.22:0/3170442381 conn(0x7f684c0fc3d0 msgr2=0x7f684c107930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:31.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 -- 172.21.15.22:0/3170442381 shutdown_connections 2023-09-20T21:48:31.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:31.382+0000 7f6853b8a700 1 -- 172.21.15.22:0/3170442381 wait complete. 2023-09-20T21:48:31.396 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2023-09-20T21:48:31.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- bash -c 'ceph osd dump' 2023-09-20T21:48:32.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:32 smithi022 ceph-mon[97651]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:32.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:32 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3170442381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-20T21:48:32.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:32 smithi123 ceph-mon[134433]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:32.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:32 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3170442381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-20T21:48:32.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:32 smithi089 ceph-mon[134090]: pgmap v335: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:32.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:32 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3170442381' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-09-20T21:48:32.638 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:48:33.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:33 smithi089 ceph-mon[134090]: pgmap v336: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:48:33.990 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:33 smithi022 ceph-mon[97651]: pgmap v336: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:48:34.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:33 smithi123 ceph-mon[134433]: pgmap v336: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:48:34.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.253+0000 7fa98f08e700 1 -- 172.21.15.22:0/1640172460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa988101fd0 msgr2=0x7fa988102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:34.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.253+0000 7fa98f08e700 1 --2- 172.21.15.22:0/1640172460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa988101fd0 0x7fa988102450 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa974009990 tx=0x7fa974009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.253+0000 7fa98f08e700 1 -- 172.21.15.22:0/1640172460 shutdown_connections 2023-09-20T21:48:34.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.253+0000 7fa98f08e700 1 --2- 172.21.15.22:0/1640172460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa988102990 0x7fa9881092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.253+0000 7fa98f08e700 1 --2- 172.21.15.22:0/1640172460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa988100e70 0x7fa988101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.253+0000 7fa98f08e700 1 --2- 172.21.15.22:0/1640172460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa988101fd0 0x7fa988102450 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.253+0000 7fa98f08e700 1 -- 172.21.15.22:0/1640172460 >> 172.21.15.22:0/1640172460 conn(0x7fa9880fc3d0 msgr2=0x7fa9880fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:34.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.253+0000 7fa98f08e700 1 -- 172.21.15.22:0/1640172460 shutdown_connections 2023-09-20T21:48:34.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.253+0000 7fa98f08e700 1 -- 172.21.15.22:0/1640172460 wait complete. 2023-09-20T21:48:34.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.254+0000 7fa98f08e700 1 Processor -- start 2023-09-20T21:48:34.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98f08e700 1 -- start start 2023-09-20T21:48:34.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98f08e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa988100e70 0x7fa988196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:34.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98f08e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa988101fd0 0x7fa988197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:34.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98f08e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa988102990 0x7fa98819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:34.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98f08e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98819e6d0 con 0x7fa988100e70 2023-09-20T21:48:34.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98f08e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98819e810 con 0x7fa988102990 2023-09-20T21:48:34.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98f08e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98819e9a0 con 0x7fa988101fd0 2023-09-20T21:48:34.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98ce2a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa988100e70 0x7fa988196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:34.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98ce2a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa988100e70 0x7fa988196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54648/0 (socket says 172.21.15.22:54648) 2023-09-20T21:48:34.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98ce2a700 1 -- 172.21.15.22:0/2825430582 learned_addr learned my addr 172.21.15.22:0/2825430582 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:34.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa987fff700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa988101fd0 0x7fa988197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:34.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.255+0000 7fa98d62b700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa988102990 0x7fa98819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:34.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa98ce2a700 1 -- 172.21.15.22:0/2825430582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa988101fd0 msgr2=0x7fa988197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:34.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa98ce2a700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa988101fd0 0x7fa988197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa98ce2a700 1 -- 172.21.15.22:0/2825430582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa988102990 msgr2=0x7fa98819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:34.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa98ce2a700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa988102990 0x7fa98819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa98ce2a700 1 -- 172.21.15.22:0/2825430582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa980009c40 con 0x7fa988100e70 2023-09-20T21:48:34.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa98ce2a700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa988100e70 0x7fa988196b40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00e9d0 tx=0x7fa97c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:34.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa985ffb700 1 -- 172.21.15.22:0/2825430582 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c009790 con 0x7fa988100e70 2023-09-20T21:48:34.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa985ffb700 1 -- 172.21.15.22:0/2825430582 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa97c00ee60 con 0x7fa988100e70 2023-09-20T21:48:34.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa98f08e700 1 -- 172.21.15.22:0/2825430582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9740095f0 con 0x7fa988100e70 2023-09-20T21:48:34.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa985ffb700 1 -- 172.21.15.22:0/2825430582 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c018680 con 0x7fa988100e70 2023-09-20T21:48:34.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.256+0000 7fa98f08e700 1 -- 172.21.15.22:0/2825430582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98819eff0 con 0x7fa988100e70 2023-09-20T21:48:34.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.257+0000 7fa98f08e700 1 -- 172.21.15.22:0/2825430582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa988108480 con 0x7fa988100e70 2023-09-20T21:48:34.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.259+0000 7fa985ffb700 1 -- 172.21.15.22:0/2825430582 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa97c010430 con 0x7fa988100e70 2023-09-20T21:48:34.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.261+0000 7fa985ffb700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa9780a1300 0x7fa9780a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:34.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.261+0000 7fa985ffb700 1 -- 172.21.15.22:0/2825430582 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa97c014070 con 0x7fa988100e70 2023-09-20T21:48:34.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.261+0000 7fa987fff700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa9780a1300 0x7fa9780a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:34.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.262+0000 7fa987fff700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa9780a1300 0x7fa9780a37c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa97400b2e0 tx=0x7fa974005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:34.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.265+0000 7fa985ffb700 1 -- 172.21.15.22:0/2825430582 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa97c051e10 con 0x7fa988100e70 2023-09-20T21:48:34.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.476+0000 7fa98f08e700 1 -- 172.21.15.22:0/2825430582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fa98810c680 con 0x7fa988100e70 2023-09-20T21:48:34.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.477+0000 7fa985ffb700 1 -- 172.21.15.22:0/2825430582 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) v1 ==== 56+0+5693 (secure 0 0 0) 0x7fa97c096530 con 0x7fa988100e70 2023-09-20T21:48:34.476 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 93 2023-09-20T21:48:34.477 INFO:teuthology.orchestra.run.smithi022.stdout:fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T21:48:34.477 INFO:teuthology.orchestra.run.smithi022.stdout:created 2023-09-20T21:39:07.129064+0000 2023-09-20T21:48:34.477 INFO:teuthology.orchestra.run.smithi022.stdout:modified 2023-09-20T21:48:09.582452+0000 2023-09-20T21:48:34.477 INFO:teuthology.orchestra.run.smithi022.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-09-20T21:48:34.477 INFO:teuthology.orchestra.run.smithi022.stdout:crush_version 27 2023-09-20T21:48:34.477 INFO:teuthology.orchestra.run.smithi022.stdout:full_ratio 0.95 2023-09-20T21:48:34.478 INFO:teuthology.orchestra.run.smithi022.stdout:backfillfull_ratio 0.9 2023-09-20T21:48:34.478 INFO:teuthology.orchestra.run.smithi022.stdout:nearfull_ratio 0.85 2023-09-20T21:48:34.478 INFO:teuthology.orchestra.run.smithi022.stdout:require_min_compat_client luminous 2023-09-20T21:48:34.478 INFO:teuthology.orchestra.run.smithi022.stdout:min_compat_client luminous 2023-09-20T21:48:34.478 INFO:teuthology.orchestra.run.smithi022.stdout:require_osd_release reef 2023-09-20T21:48:34.479 INFO:teuthology.orchestra.run.smithi022.stdout:stretch_mode_enabled false 2023-09-20T21:48:34.479 INFO:teuthology.orchestra.run.smithi022.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-09-20T21:48:34.479 INFO:teuthology.orchestra.run.smithi022.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-09-20T21:48:34.479 INFO:teuthology.orchestra.run.smithi022.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-09-20T21:48:34.479 INFO:teuthology.orchestra.run.smithi022.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-09-20T21:48:34.479 INFO:teuthology.orchestra.run.smithi022.stdout:max_osd 12 2023-09-20T21:48:34.480 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6802/3853681188,v1:172.21.15.22:6803/3853681188] [v2:172.21.15.22:6804/3853681188,v1:172.21.15.22:6805/3853681188] exists,up 6bdd85c1-5ec1-40a9-be24-e1c0701d30a2 2023-09-20T21:48:34.480 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 up in weight 1 up_from 13 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6800/2519597559,v1:172.21.15.89:6801/2519597559] [v2:172.21.15.89:6802/2519597559,v1:172.21.15.89:6803/2519597559] exists,up acd88d1d-b602-4722-a8b3-4ef144e4da4c 2023-09-20T21:48:34.480 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 up in weight 1 up_from 18 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/877674627,v1:172.21.15.123:6801/877674627] [v2:172.21.15.123:6802/877674627,v1:172.21.15.123:6803/877674627] exists,up 28865364-a5f4-426f-98a6-19cb758151a0 2023-09-20T21:48:34.480 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6810/143368577,v1:172.21.15.22:6811/143368577] [v2:172.21.15.22:6812/143368577,v1:172.21.15.22:6813/143368577] exists,up d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6 2023-09-20T21:48:34.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6808/1802541493,v1:172.21.15.89:6809/1802541493] [v2:172.21.15.89:6810/1802541493,v1:172.21.15.89:6811/1802541493] exists,up e4f26e77-aca2-47d8-8158-4d080560576e 2023-09-20T21:48:34.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/402706185,v1:172.21.15.123:6809/402706185] [v2:172.21.15.123:6810/402706185,v1:172.21.15.123:6811/402706185] exists,up 9d8866df-2568-4a6d-83e7-1be50e327005 2023-09-20T21:48:34.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 up in weight 1 up_from 46 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6818/1850518910,v1:172.21.15.22:6819/1850518910] [v2:172.21.15.22:6820/1850518910,v1:172.21.15.22:6821/1850518910] exists,up 8a625e71-db36-4eab-a821-68672c3284e6 2023-09-20T21:48:34.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6816/4165516997,v1:172.21.15.89:6817/4165516997] [v2:172.21.15.89:6818/4165516997,v1:172.21.15.89:6819/4165516997] exists,up 7674a06e-2830-441e-9178-687e42dfce80 2023-09-20T21:48:34.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.8 up in weight 1 up_from 59 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/1023938376,v1:172.21.15.123:6817/1023938376] [v2:172.21.15.123:6818/1023938376,v1:172.21.15.123:6819/1023938376] exists,up bf42a69e-43d0-47b2-bd13-9378e765e7d1 2023-09-20T21:48:34.481 INFO:teuthology.orchestra.run.smithi022.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.22:6826/4238100695,v1:172.21.15.22:6827/4238100695] [v2:172.21.15.22:6828/4238100695,v1:172.21.15.22:6829/4238100695] exists,up 80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06 2023-09-20T21:48:34.482 INFO:teuthology.orchestra.run.smithi022.stdout:osd.10 up in weight 1 up_from 72 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6824/2445411927,v1:172.21.15.89:6825/2445411927] [v2:172.21.15.89:6826/2445411927,v1:172.21.15.89:6827/2445411927] exists,up 3d118a3d-5e31-4f50-aabd-72d9125546ee 2023-09-20T21:48:34.482 INFO:teuthology.orchestra.run.smithi022.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/2107937412,v1:172.21.15.123:6825/2107937412] [v2:172.21.15.123:6826/2107937412,v1:172.21.15.123:6827/2107937412] exists,up 41242a75-3273-4217-ad0d-c5da81c6b369 2023-09-20T21:48:34.482 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.a [2,11] 2023-09-20T21:48:34.482 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.b [8,11,2,1] 2023-09-20T21:48:34.482 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 2.1d [2,4] 2023-09-20T21:48:34.482 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.3 [4,9] 2023-09-20T21:48:34.483 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.10 [5,9] 2023-09-20T21:48:34.483 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.14 [4,7] 2023-09-20T21:48:34.483 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.17 [4,6] 2023-09-20T21:48:34.483 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.19 [3,8] 2023-09-20T21:48:34.483 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.1b [5,9] 2023-09-20T21:48:34.483 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 3.31 [4,8] 2023-09-20T21:48:34.484 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.2 [9,1] 2023-09-20T21:48:34.484 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.a [5,10] 2023-09-20T21:48:34.484 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.16 [9,8] 2023-09-20T21:48:34.484 INFO:teuthology.orchestra.run.smithi022.stdout:pg_upmap_items 4.19 [9,8] 2023-09-20T21:48:34.484 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6801/3260510802 expires 2023-09-21T21:40:07.017182+0000 2023-09-20T21:48:34.485 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/3260510802 expires 2023-09-21T21:40:07.017182+0000 2023-09-20T21:48:34.485 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6801/1128310681 expires 2023-09-21T21:39:39.363251+0000 2023-09-20T21:48:34.485 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/3419565055 expires 2023-09-21T21:40:07.017182+0000 2023-09-20T21:48:34.485 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:6800/1128310681 expires 2023-09-21T21:39:39.363251+0000 2023-09-20T21:48:34.485 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/4209005662 expires 2023-09-21T21:39:39.363251+0000 2023-09-20T21:48:34.485 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/1007451083 expires 2023-09-21T21:39:39.363251+0000 2023-09-20T21:48:34.486 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/1994934339 expires 2023-09-21T21:40:07.017182+0000 2023-09-20T21:48:34.486 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/875874054 expires 2023-09-21T21:40:07.017182+0000 2023-09-20T21:48:34.486 INFO:teuthology.orchestra.run.smithi022.stdout:blocklist 172.21.15.22:0/2863894527 expires 2023-09-21T21:39:39.363251+0000 2023-09-20T21:48:34.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.479+0000 7fa98f08e700 1 -- 172.21.15.22:0/2825430582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa9780a1300 msgr2=0x7fa9780a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:34.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.479+0000 7fa98f08e700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa9780a1300 0x7fa9780a37c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa97400b2e0 tx=0x7fa974005960 comp rx=0 tx=0).stop 2023-09-20T21:48:34.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.479+0000 7fa98f08e700 1 -- 172.21.15.22:0/2825430582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa988100e70 msgr2=0x7fa988196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:34.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.479+0000 7fa98f08e700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa988100e70 0x7fa988196b40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00e9d0 tx=0x7fa97c00ece0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.479+0000 7fa98f08e700 1 -- 172.21.15.22:0/2825430582 shutdown_connections 2023-09-20T21:48:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.479+0000 7fa98f08e700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa9780a1300 0x7fa9780a37c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.479+0000 7fa98f08e700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa988102990 0x7fa98819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.479+0000 7fa98f08e700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa988100e70 0x7fa988196b40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.479+0000 7fa98f08e700 1 --2- 172.21.15.22:0/2825430582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa988101fd0 0x7fa988197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:34.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.479+0000 7fa98f08e700 1 -- 172.21.15.22:0/2825430582 >> 172.21.15.22:0/2825430582 conn(0x7fa9880fc3d0 msgr2=0x7fa9881055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:34.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.480+0000 7fa98f08e700 1 -- 172.21.15.22:0/2825430582 shutdown_connections 2023-09-20T21:48:34.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:34.480+0000 7fa98f08e700 1 -- 172.21.15.22:0/2825430582 wait complete. 2023-09-20T21:48:34.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:34 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2825430582' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-20T21:48:34.931 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:34 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2825430582' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-20T21:48:34.938 INFO:teuthology.run_tasks:Running task fs.ready... 2023-09-20T21:48:34.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:48:35.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:34 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2825430582' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-09-20T21:48:35.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.159+0000 7fbc00bff700 1 -- 172.21.15.22:0/3741646413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbfc101fd0 msgr2=0x7fbbfc102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.159+0000 7fbc00bff700 1 --2- 172.21.15.22:0/3741646413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbfc101fd0 0x7fbbfc102430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40099c0 tx=0x7fbbe4009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.159+0000 7fbc00bff700 1 -- 172.21.15.22:0/3741646413 shutdown_connections 2023-09-20T21:48:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.159+0000 7fbc00bff700 1 --2- 172.21.15.22:0/3741646413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc102970 0x7fbbfc109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.159+0000 7fbc00bff700 1 --2- 172.21.15.22:0/3741646413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbfc100de0 0x7fbbfc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.159+0000 7fbc00bff700 1 --2- 172.21.15.22:0/3741646413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbfc101fd0 0x7fbbfc102430 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.159+0000 7fbc00bff700 1 -- 172.21.15.22:0/3741646413 >> 172.21.15.22:0/3741646413 conn(0x7fbbfc0fc310 msgr2=0x7fbbfc0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.159+0000 7fbc00bff700 1 -- 172.21.15.22:0/3741646413 shutdown_connections 2023-09-20T21:48:35.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.159+0000 7fbc00bff700 1 -- 172.21.15.22:0/3741646413 wait complete. 2023-09-20T21:48:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.160+0000 7fbc00bff700 1 Processor -- start 2023-09-20T21:48:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.160+0000 7fbc00bff700 1 -- start start 2023-09-20T21:48:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbc00bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbfc100de0 0x7fbbfc198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbc00bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbfc101fd0 0x7fbbfc1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbc00bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc102970 0x7fbbfc1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbc00bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc1a0820 con 0x7fbbfc101fd0 2023-09-20T21:48:35.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbc00bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc1a0990 con 0x7fbbfc100de0 2023-09-20T21:48:35.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbc00bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc1a0b50 con 0x7fbbfc102970 2023-09-20T21:48:35.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbfc101fd0 0x7fbbfc1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:35.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbbfa59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbfc100de0 0x7fbbfc198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:35.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbbf3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbfc101fd0 0x7fbbfc1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54668/0 (socket says 172.21.15.22:54668) 2023-09-20T21:48:35.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbbf3fff700 1 -- 172.21.15.22:0/3986332781 learned_addr learned my addr 172.21.15.22:0/3986332781 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:35.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.161+0000 7fbbfad9d700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc102970 0x7fbbfc1a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:35.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbbfa59c700 1 -- 172.21.15.22:0/3986332781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc102970 msgr2=0x7fbbfc1a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:35.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbbfa59c700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc102970 0x7fbbfc1a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbbfa59c700 1 -- 172.21.15.22:0/3986332781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbfc101fd0 msgr2=0x7fbbfc1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbbfa59c700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbfc101fd0 0x7fbbfc1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbbfa59c700 1 -- 172.21.15.22:0/3986332781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe40095f0 con 0x7fbbfc100de0 2023-09-20T21:48:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbbf3fff700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbfc101fd0 0x7fbbfc1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbbfa59c700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbfc100de0 0x7fbbfc198cb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00e9c0 tx=0x7fbbec00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbbf37fe700 1 -- 172.21.15.22:0/3986332781 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec004220 con 0x7fbbfc100de0 2023-09-20T21:48:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbc00bff700 1 -- 172.21.15.22:0/3986332781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc1a0e30 con 0x7fbbfc100de0 2023-09-20T21:48:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbbf37fe700 1 -- 172.21.15.22:0/3986332781 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbec004380 con 0x7fbbfc100de0 2023-09-20T21:48:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbbf37fe700 1 -- 172.21.15.22:0/3986332781 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec003be0 con 0x7fbbfc100de0 2023-09-20T21:48:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.162+0000 7fbc00bff700 1 -- 172.21.15.22:0/3986332781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc1a1250 con 0x7fbbfc100de0 2023-09-20T21:48:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.163+0000 7fbc00bff700 1 -- 172.21.15.22:0/3986332781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc10c640 con 0x7fbbfc100de0 2023-09-20T21:48:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.165+0000 7fbbf37fe700 1 -- 172.21.15.22:0/3986332781 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fbbec00c7c0 con 0x7fbbfc100de0 2023-09-20T21:48:35.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.167+0000 7fbbf37fe700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbbe80a1350 0x7fbbe80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:35.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.167+0000 7fbbf37fe700 1 -- 172.21.15.22:0/3986332781 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fbbec014070 con 0x7fbbfc100de0 2023-09-20T21:48:35.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.167+0000 7fbbf3fff700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbbe80a1350 0x7fbbe80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:35.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.167+0000 7fbbf3fff700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbbe80a1350 0x7fbbe80a3810 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc19a1b0 tx=0x7fbbe4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:35.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.168+0000 7fbbf37fe700 1 -- 172.21.15.22:0/3986332781 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbbec096730 con 0x7fbbfc100de0 2023-09-20T21:48:35.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.383+0000 7fbc00bff700 1 -- 172.21.15.22:0/3986332781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbfc10c7d0 con 0x7fbbfc100de0 2023-09-20T21:48:35.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.385+0000 7fbbf37fe700 1 -- 172.21.15.22:0/3986332781 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7fbbec099d50 con 0x7fbbfc100de0 2023-09-20T21:48:35.385 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:35.385 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:48:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 -- 172.21.15.22:0/3986332781 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbbe80a1350 msgr2=0x7fbbe80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbbe80a1350 0x7fbbe80a3810 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc19a1b0 tx=0x7fbbe4005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 -- 172.21.15.22:0/3986332781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbfc100de0 msgr2=0x7fbbfc198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:35.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbfc100de0 0x7fbbfc198cb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00e9c0 tx=0x7fbbec00ecd0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 -- 172.21.15.22:0/3986332781 shutdown_connections 2023-09-20T21:48:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbfc102970 0x7fbbfc1a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbbe80a1350 0x7fbbe80a3810 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbbfc100de0 0x7fbbfc198cb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 --2- 172.21.15.22:0/3986332781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbfc101fd0 0x7fbbfc1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 -- 172.21.15.22:0/3986332781 >> 172.21.15.22:0/3986332781 conn(0x7fbbfc0fc310 msgr2=0x7fbbfc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 -- 172.21.15.22:0/3986332781 shutdown_connections 2023-09-20T21:48:35.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.387+0000 7fbc00bff700 1 -- 172.21.15.22:0/3986332781 wait complete. 2023-09-20T21:48:35.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T21:48:35.406 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:48:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.620+0000 7fdba17ff700 1 -- 172.21.15.22:0/1567592487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb9c102960 msgr2=0x7fdb9c109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.620+0000 7fdba17ff700 1 --2- 172.21.15.22:0/1567592487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb9c102960 0x7fdb9c109210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400b0d0 tx=0x7fdb9400b3e0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.620+0000 7fdba17ff700 1 -- 172.21.15.22:0/1567592487 shutdown_connections 2023-09-20T21:48:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.620+0000 7fdba17ff700 1 --2- 172.21.15.22:0/1567592487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb9c102960 0x7fdb9c109210 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.620+0000 7fdba17ff700 1 --2- 172.21.15.22:0/1567592487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c100dd0 0x7fdb9c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.620+0000 7fdba17ff700 1 --2- 172.21.15.22:0/1567592487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb9c101fc0 0x7fdb9c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.620+0000 7fdba17ff700 1 -- 172.21.15.22:0/1567592487 >> 172.21.15.22:0/1567592487 conn(0x7fdb9c0fc320 msgr2=0x7fdb9c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.620+0000 7fdba17ff700 1 -- 172.21.15.22:0/1567592487 shutdown_connections 2023-09-20T21:48:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.620+0000 7fdba17ff700 1 -- 172.21.15.22:0/1567592487 wait complete. 2023-09-20T21:48:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.621+0000 7fdba17ff700 1 Processor -- start 2023-09-20T21:48:35.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.621+0000 7fdba17ff700 1 -- start start 2023-09-20T21:48:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.621+0000 7fdba17ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb9c100dd0 0x7fdb9c074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.621+0000 7fdba17ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb9c101fc0 0x7fdb9c0731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:35.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.621+0000 7fdba17ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c102960 0x7fdb9c073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:35.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.621+0000 7fdba17ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c073eb0 con 0x7fdb9c100dd0 2023-09-20T21:48:35.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.621+0000 7fdba17ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c074020 con 0x7fdb9c102960 2023-09-20T21:48:35.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.621+0000 7fdba17ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c1a0c50 con 0x7fdb9c101fc0 2023-09-20T21:48:35.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb9c100dd0 0x7fdb9c074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:35.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb9c101fc0 0x7fdb9c0731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:35.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9affd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb9c100dd0 0x7fdb9c074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54692/0 (socket says 172.21.15.22:54692) 2023-09-20T21:48:35.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9affd700 1 -- 172.21.15.22:0/3552142437 learned_addr learned my addr 172.21.15.22:0/3552142437 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9b7fe700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c102960 0x7fdb9c073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9a7fc700 1 -- 172.21.15.22:0/3552142437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c102960 msgr2=0x7fdb9c073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9a7fc700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c102960 0x7fdb9c073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9a7fc700 1 -- 172.21.15.22:0/3552142437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb9c100dd0 msgr2=0x7fdb9c074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:35.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9a7fc700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb9c100dd0 0x7fdb9c074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9a7fc700 1 -- 172.21.15.22:0/3552142437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94009d00 con 0x7fdb9c101fc0 2023-09-20T21:48:35.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9affd700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb9c100dd0 0x7fdb9c074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb9a7fc700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb9c101fc0 0x7fdb9c0731d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000e9f0 tx=0x7fdb9000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.622+0000 7fdb83fff700 1 -- 172.21.15.22:0/3552142437 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000c9c0 con 0x7fdb9c101fc0 2023-09-20T21:48:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.623+0000 7fdb83fff700 1 -- 172.21.15.22:0/3552142437 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb90010430 con 0x7fdb9c101fc0 2023-09-20T21:48:35.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.623+0000 7fdba17ff700 1 -- 172.21.15.22:0/3552142437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c1a0f30 con 0x7fdb9c101fc0 2023-09-20T21:48:35.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.623+0000 7fdb83fff700 1 -- 172.21.15.22:0/3552142437 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90018680 con 0x7fdb9c101fc0 2023-09-20T21:48:35.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.623+0000 7fdba17ff700 1 -- 172.21.15.22:0/3552142437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c1a13b0 con 0x7fdb9c101fc0 2023-09-20T21:48:35.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.624+0000 7fdba17ff700 1 -- 172.21.15.22:0/3552142437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9c107d00 con 0x7fdb9c101fc0 2023-09-20T21:48:35.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.626+0000 7fdb83fff700 1 -- 172.21.15.22:0/3552142437 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fdb90010860 con 0x7fdb9c101fc0 2023-09-20T21:48:35.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.627+0000 7fdb83fff700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdb840a1300 0x7fdb840a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:35.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.627+0000 7fdb83fff700 1 -- 172.21.15.22:0/3552142437 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fdb90014070 con 0x7fdb9c101fc0 2023-09-20T21:48:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.627+0000 7fdb9affd700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdb840a1300 0x7fdb840a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.628+0000 7fdb9affd700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdb840a1300 0x7fdb840a37c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c005950 tx=0x7fdb8c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.629+0000 7fdb83fff700 1 -- 172.21.15.22:0/3552142437 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdb900963c0 con 0x7fdb9c101fc0 2023-09-20T21:48:35.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:35 smithi022 ceph-mon[97651]: pgmap v337: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:48:35.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:35 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3986332781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:35.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.841+0000 7fdba17ff700 1 -- 172.21.15.22:0/3552142437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb9c02cef0 con 0x7fdb9c101fc0 2023-09-20T21:48:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.842+0000 7fdb83fff700 1 -- 172.21.15.22:0/3552142437 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb900999e0 con 0x7fdb9c101fc0 2023-09-20T21:48:35.841 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:48:35.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 -- 172.21.15.22:0/3552142437 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdb840a1300 msgr2=0x7fdb840a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:35.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdb840a1300 0x7fdb840a37c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c005950 tx=0x7fdb8c009260 comp rx=0 tx=0).stop 2023-09-20T21:48:35.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 -- 172.21.15.22:0/3552142437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb9c101fc0 msgr2=0x7fdb9c0731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:35.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb9c101fc0 0x7fdb9c0731d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000e9f0 tx=0x7fdb9000ed00 comp rx=0 tx=0).stop 2023-09-20T21:48:35.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 -- 172.21.15.22:0/3552142437 shutdown_connections 2023-09-20T21:48:35.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdb840a1300 0x7fdb840a37c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdb9c102960 0x7fdb9c073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb9c100dd0 0x7fdb9c074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 --2- 172.21.15.22:0/3552142437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb9c101fc0 0x7fdb9c0731d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:35.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 -- 172.21.15.22:0/3552142437 >> 172.21.15.22:0/3552142437 conn(0x7fdb9c0fc320 msgr2=0x7fdb9c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:35.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 -- 172.21.15.22:0/3552142437 shutdown_connections 2023-09-20T21:48:35.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:35.844+0000 7fdba17ff700 1 -- 172.21.15.22:0/3552142437 wait complete. 2023-09-20T21:48:35.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:35 smithi089 ceph-mon[134090]: pgmap v337: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:48:35.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:35 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3986332781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:35.862 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T21:48:36.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.071+0000 7f93699ff700 1 -- 172.21.15.22:0/3274590076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9364100dd0 msgr2=0x7f93641011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.071+0000 7f93699ff700 1 --2- 172.21.15.22:0/3274590076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9364100dd0 0x7f93641011f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9354009920 tx=0x7f9354009c30 comp rx=0 tx=0).stop 2023-09-20T21:48:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.071+0000 7f93699ff700 1 -- 172.21.15.22:0/3274590076 shutdown_connections 2023-09-20T21:48:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.071+0000 7f93699ff700 1 --2- 172.21.15.22:0/3274590076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9364102960 0x7f9364109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.071+0000 7f93699ff700 1 --2- 172.21.15.22:0/3274590076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9364100dd0 0x7f93641011f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.071+0000 7f93699ff700 1 --2- 172.21.15.22:0/3274590076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9364101fc0 0x7f9364102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.071+0000 7f93699ff700 1 -- 172.21.15.22:0/3274590076 >> 172.21.15.22:0/3274590076 conn(0x7f93640fc320 msgr2=0x7f93640fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:36.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.071+0000 7f93699ff700 1 -- 172.21.15.22:0/3274590076 shutdown_connections 2023-09-20T21:48:36.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.071+0000 7f93699ff700 1 -- 172.21.15.22:0/3274590076 wait complete. 2023-09-20T21:48:36.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.072+0000 7f93699ff700 1 Processor -- start 2023-09-20T21:48:36.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.072+0000 7f93699ff700 1 -- start start 2023-09-20T21:48:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.072+0000 7f93699ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9364100dd0 0x7f9364074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.072+0000 7f93699ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9364101fc0 0x7f93640731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.072+0000 7f93699ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9364102960 0x7f9364073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.072+0000 7f93699ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9364073eb0 con 0x7f9364100dd0 2023-09-20T21:48:36.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.072+0000 7f93699ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9364074020 con 0x7f9364102960 2023-09-20T21:48:36.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.072+0000 7f93699ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93641a0ca0 con 0x7f9364101fc0 2023-09-20T21:48:36.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f9362ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9364100dd0 0x7f9364074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f9362ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9364100dd0 0x7f9364074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47602/0 (socket says 172.21.15.22:47602) 2023-09-20T21:48:36.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f9362ffd700 1 -- 172.21.15.22:0/1930601576 learned_addr learned my addr 172.21.15.22:0/1930601576 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:36.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f93627fc700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9364101fc0 0x7f93640731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f93637fe700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9364102960 0x7f9364073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f93627fc700 1 -- 172.21.15.22:0/1930601576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9364102960 msgr2=0x7f9364073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f93627fc700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9364102960 0x7f9364073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f93627fc700 1 -- 172.21.15.22:0/1930601576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9364100dd0 msgr2=0x7f9364074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f93627fc700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9364100dd0 0x7f9364074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f93627fc700 1 -- 172.21.15.22:0/1930601576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93580095f0 con 0x7f9364101fc0 2023-09-20T21:48:36.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f9362ffd700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9364100dd0 0x7f9364074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:36.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.073+0000 7f93627fc700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9364101fc0 0x7f93640731d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f935800eb90 tx=0x7f935800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:36.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.074+0000 7f93689fd700 1 -- 172.21.15.22:0/1930601576 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358009970 con 0x7f9364101fc0 2023-09-20T21:48:36.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.074+0000 7f93699ff700 1 -- 172.21.15.22:0/1930601576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9354009520 con 0x7f9364101fc0 2023-09-20T21:48:36.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.074+0000 7f93689fd700 1 -- 172.21.15.22:0/1930601576 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93580044e0 con 0x7f9364101fc0 2023-09-20T21:48:36.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.074+0000 7f93689fd700 1 -- 172.21.15.22:0/1930601576 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358003e50 con 0x7f9364101fc0 2023-09-20T21:48:36.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.074+0000 7f93699ff700 1 -- 172.21.15.22:0/1930601576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93641a12d0 con 0x7f9364101fc0 2023-09-20T21:48:36.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.075+0000 7f93699ff700 1 -- 172.21.15.22:0/1930601576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9364108550 con 0x7f9364101fc0 2023-09-20T21:48:36.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.077+0000 7f93689fd700 1 -- 172.21.15.22:0/1930601576 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f9358004000 con 0x7f9364101fc0 2023-09-20T21:48:36.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.078+0000 7f93689fd700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f934c0a1380 0x7f934c0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.078+0000 7f93689fd700 1 -- 172.21.15.22:0/1930601576 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f9358014070 con 0x7f9364101fc0 2023-09-20T21:48:36.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.079+0000 7f9362ffd700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f934c0a1380 0x7f934c0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.079+0000 7f9362ffd700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f934c0a1380 0x7f934c0a3840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f935400c010 tx=0x7f9354019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:36.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.080+0000 7f93689fd700 1 -- 172.21.15.22:0/1930601576 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9358096890 con 0x7f9364101fc0 2023-09-20T21:48:36.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:35 smithi123 ceph-mon[134433]: pgmap v337: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:48:36.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:35 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3986332781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:36.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.293+0000 7f93699ff700 1 -- 172.21.15.22:0/1930601576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f936410c7c0 con 0x7f9364101fc0 2023-09-20T21:48:36.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.294+0000 7f93689fd700 1 -- 172.21.15.22:0/1930601576 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9358017e90 con 0x7f9364101fc0 2023-09-20T21:48:36.293 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:36.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.296+0000 7f93699ff700 1 -- 172.21.15.22:0/1930601576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f934c0a1380 msgr2=0x7f934c0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.296+0000 7f93699ff700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f934c0a1380 0x7f934c0a3840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f935400c010 tx=0x7f9354019040 comp rx=0 tx=0).stop 2023-09-20T21:48:36.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.296+0000 7f93699ff700 1 -- 172.21.15.22:0/1930601576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9364101fc0 msgr2=0x7f93640731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.296+0000 7f93699ff700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9364101fc0 0x7f93640731d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f935800eb90 tx=0x7f935800eea0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.296+0000 7f93699ff700 1 -- 172.21.15.22:0/1930601576 shutdown_connections 2023-09-20T21:48:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.296+0000 7f93699ff700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f934c0a1380 0x7f934c0a3840 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.296+0000 7f93699ff700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9364102960 0x7f9364073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.296+0000 7f93699ff700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9364100dd0 0x7f9364074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.296+0000 7f93699ff700 1 --2- 172.21.15.22:0/1930601576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9364101fc0 0x7f93640731d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.296+0000 7f93699ff700 1 -- 172.21.15.22:0/1930601576 >> 172.21.15.22:0/1930601576 conn(0x7f93640fc320 msgr2=0x7f9364105690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.297+0000 7f93699ff700 1 -- 172.21.15.22:0/1930601576 shutdown_connections 2023-09-20T21:48:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.297+0000 7f93699ff700 1 -- 172.21.15.22:0/1930601576 wait complete. 2023-09-20T21:48:36.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T21:48:36.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.523+0000 7fa7775ff700 1 -- 172.21.15.22:0/1077387611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa770102980 msgr2=0x7fa770109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.523+0000 7fa7775ff700 1 --2- 172.21.15.22:0/1077387611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa770102980 0x7fa770109230 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa76800b0d0 tx=0x7fa76800b3e0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.523+0000 7fa7775ff700 1 -- 172.21.15.22:0/1077387611 shutdown_connections 2023-09-20T21:48:36.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.523+0000 7fa7775ff700 1 --2- 172.21.15.22:0/1077387611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa770102980 0x7fa770109230 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.523+0000 7fa7775ff700 1 --2- 172.21.15.22:0/1077387611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770100df0 0x7fa770101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.523+0000 7fa7775ff700 1 --2- 172.21.15.22:0/1077387611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa770101fe0 0x7fa770102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.523+0000 7fa7775ff700 1 -- 172.21.15.22:0/1077387611 >> 172.21.15.22:0/1077387611 conn(0x7fa7700fc360 msgr2=0x7fa7700fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:36.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.523+0000 7fa7775ff700 1 -- 172.21.15.22:0/1077387611 shutdown_connections 2023-09-20T21:48:36.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.523+0000 7fa7775ff700 1 -- 172.21.15.22:0/1077387611 wait complete. 2023-09-20T21:48:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.524+0000 7fa7775ff700 1 Processor -- start 2023-09-20T21:48:36.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa7775ff700 1 -- start start 2023-09-20T21:48:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa7775ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770100df0 0x7fa770074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa7775ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa770101fe0 0x7fa7700731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa7775ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa770102980 0x7fa770073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa7775ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770073eb0 con 0x7fa770102980 2023-09-20T21:48:36.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa7775ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770074020 con 0x7fa770100df0 2023-09-20T21:48:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa7775ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7701a0c70 con 0x7fa770101fe0 2023-09-20T21:48:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa775b9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa770102980 0x7fa770073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa775b9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa770102980 0x7fa770073710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47622/0 (socket says 172.21.15.22:47622) 2023-09-20T21:48:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa775b9c700 1 -- 172.21.15.22:0/2395996763 learned_addr learned my addr 172.21.15.22:0/2395996763 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:36.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa774b9a700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa770101fe0 0x7fa7700731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.525+0000 7fa77539b700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770100df0 0x7fa770074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa775b9c700 1 -- 172.21.15.22:0/2395996763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa770101fe0 msgr2=0x7fa7700731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa775b9c700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa770101fe0 0x7fa7700731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa775b9c700 1 -- 172.21.15.22:0/2395996763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770100df0 msgr2=0x7fa770074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa775b9c700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770100df0 0x7fa770074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa775b9c700 1 -- 172.21.15.22:0/2395996763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768009d00 con 0x7fa770102980 2023-09-20T21:48:36.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa774b9a700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa770101fe0 0x7fa7700731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:36.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa775b9c700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa770102980 0x7fa770073710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa768008eb0 tx=0x7fa768008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:36.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa7667fc700 1 -- 172.21.15.22:0/2395996763 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800e040 con 0x7fa770102980 2023-09-20T21:48:36.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa7667fc700 1 -- 172.21.15.22:0/2395996763 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa76800bdb0 con 0x7fa770102980 2023-09-20T21:48:36.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa7667fc700 1 -- 172.21.15.22:0/2395996763 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76801d470 con 0x7fa770102980 2023-09-20T21:48:36.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.526+0000 7fa7775ff700 1 -- 172.21.15.22:0/2395996763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7701a0ef0 con 0x7fa770102980 2023-09-20T21:48:36.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.527+0000 7fa7775ff700 1 -- 172.21.15.22:0/2395996763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7701a12c0 con 0x7fa770102980 2023-09-20T21:48:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.527+0000 7fa7775ff700 1 -- 172.21.15.22:0/2395996763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa770108330 con 0x7fa770102980 2023-09-20T21:48:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.529+0000 7fa7667fc700 1 -- 172.21.15.22:0/2395996763 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa768005000 con 0x7fa770102980 2023-09-20T21:48:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.531+0000 7fa7667fc700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa75c0a1380 0x7fa75c0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.531+0000 7fa7667fc700 1 -- 172.21.15.22:0/2395996763 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa7680ce500 con 0x7fa770102980 2023-09-20T21:48:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.531+0000 7fa77539b700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa75c0a1380 0x7fa75c0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.532+0000 7fa77539b700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa75c0a1380 0x7fa75c0a3840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa770101e40 tx=0x7fa76c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:36.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.533+0000 7fa7667fc700 1 -- 172.21.15.22:0/2395996763 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa768097870 con 0x7fa770102980 2023-09-20T21:48:36.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.747+0000 7fa7775ff700 1 -- 172.21.15.22:0/2395996763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa77010c8b0 con 0x7fa770102980 2023-09-20T21:48:36.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.747+0000 7fa7667fc700 1 -- 172.21.15.22:0/2395996763 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa76809ae90 con 0x7fa770102980 2023-09-20T21:48:36.747 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T21:48:36.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 -- 172.21.15.22:0/2395996763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa75c0a1380 msgr2=0x7fa75c0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa75c0a1380 0x7fa75c0a3840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa770101e40 tx=0x7fa76c009260 comp rx=0 tx=0).stop 2023-09-20T21:48:36.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 -- 172.21.15.22:0/2395996763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa770102980 msgr2=0x7fa770073710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa770102980 0x7fa770073710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa768008eb0 tx=0x7fa768008ee0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 -- 172.21.15.22:0/2395996763 shutdown_connections 2023-09-20T21:48:36.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa770102980 0x7fa770073710 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa75c0a1380 0x7fa75c0a3840 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa770100df0 0x7fa770074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 --2- 172.21.15.22:0/2395996763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa770101fe0 0x7fa7700731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 -- 172.21.15.22:0/2395996763 >> 172.21.15.22:0/2395996763 conn(0x7fa7700fc360 msgr2=0x7fa770105470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:36.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 -- 172.21.15.22:0/2395996763 shutdown_connections 2023-09-20T21:48:36.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.750+0000 7fa7775ff700 1 -- 172.21.15.22:0/2395996763 wait complete. 2023-09-20T21:48:36.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T21:48:36.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:36 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3552142437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:36.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:36 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1930601576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:36.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.975+0000 7fa2277ff700 1 -- 172.21.15.22:0/3340985785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa220101fc0 msgr2=0x7fa220102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.975+0000 7fa2277ff700 1 --2- 172.21.15.22:0/3340985785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa220101fc0 0x7fa220102420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2100099c0 tx=0x7fa210009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.976+0000 7fa2277ff700 1 -- 172.21.15.22:0/3340985785 shutdown_connections 2023-09-20T21:48:36.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.976+0000 7fa2277ff700 1 --2- 172.21.15.22:0/3340985785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa220102960 0x7fa220109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.976+0000 7fa2277ff700 1 --2- 172.21.15.22:0/3340985785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa220100dd0 0x7fa2201011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.976+0000 7fa2277ff700 1 --2- 172.21.15.22:0/3340985785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa220101fc0 0x7fa220102420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.976+0000 7fa2277ff700 1 -- 172.21.15.22:0/3340985785 >> 172.21.15.22:0/3340985785 conn(0x7fa2200fc320 msgr2=0x7fa2200fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:36.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.976+0000 7fa2277ff700 1 -- 172.21.15.22:0/3340985785 shutdown_connections 2023-09-20T21:48:36.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.976+0000 7fa2277ff700 1 -- 172.21.15.22:0/3340985785 wait complete. 2023-09-20T21:48:36.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa2277ff700 1 Processor -- start 2023-09-20T21:48:36.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa2277ff700 1 -- start start 2023-09-20T21:48:36.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa2277ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa220100dd0 0x7fa220196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa2277ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa220101fc0 0x7fa220197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa2277ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa220102960 0x7fa22019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa2277ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22019e6a0 con 0x7fa220101fc0 2023-09-20T21:48:36.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa2277ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22019e7e0 con 0x7fa220102960 2023-09-20T21:48:36.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa2277ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22019e970 con 0x7fa220100dd0 2023-09-20T21:48:36.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa224d9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa220101fc0 0x7fa220197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa22559b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa220100dd0 0x7fa220196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa224d9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa220101fc0 0x7fa220197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47640/0 (socket says 172.21.15.22:47640) 2023-09-20T21:48:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa22559b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa220100dd0 0x7fa220196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60456/0 (socket says 172.21.15.22:60456) 2023-09-20T21:48:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.977+0000 7fa224d9a700 1 -- 172.21.15.22:0/932578835 learned_addr learned my addr 172.21.15.22:0/932578835 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa225d9c700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa220102960 0x7fa22019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa224d9a700 1 -- 172.21.15.22:0/932578835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa220100dd0 msgr2=0x7fa220196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa224d9a700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa220100dd0 0x7fa220196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa224d9a700 1 -- 172.21.15.22:0/932578835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa220102960 msgr2=0x7fa22019e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa224d9a700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa220102960 0x7fa22019e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa224d9a700 1 -- 172.21.15.22:0/932578835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100095f0 con 0x7fa220101fc0 2023-09-20T21:48:36.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa22559b700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa220100dd0 0x7fa220196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:36.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa224d9a700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa220101fc0 0x7fa220197060 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa210009fd0 tx=0x7fa21000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:36.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa2167fc700 1 -- 172.21.15.22:0/932578835 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21001d070 con 0x7fa220101fc0 2023-09-20T21:48:36.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa2167fc700 1 -- 172.21.15.22:0/932578835 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa210005050 con 0x7fa220101fc0 2023-09-20T21:48:36.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa2277ff700 1 -- 172.21.15.22:0/932578835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22019ebf0 con 0x7fa220101fc0 2023-09-20T21:48:36.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.978+0000 7fa2167fc700 1 -- 172.21.15.22:0/932578835 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210004b30 con 0x7fa220101fc0 2023-09-20T21:48:36.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.979+0000 7fa2277ff700 1 -- 172.21.15.22:0/932578835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22019f010 con 0x7fa220101fc0 2023-09-20T21:48:36.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.980+0000 7fa2277ff700 1 -- 172.21.15.22:0/932578835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa220107d50 con 0x7fa220101fc0 2023-09-20T21:48:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.982+0000 7fa2167fc700 1 -- 172.21.15.22:0/932578835 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa210004c90 con 0x7fa220101fc0 2023-09-20T21:48:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.983+0000 7fa2167fc700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa20c0a1300 0x7fa20c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.983+0000 7fa2167fc700 1 -- 172.21.15.22:0/932578835 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa2100cde00 con 0x7fa220101fc0 2023-09-20T21:48:36.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.983+0000 7fa22559b700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa20c0a1300 0x7fa20c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.984+0000 7fa22559b700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa20c0a1300 0x7fa20c0a37c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa220101e20 tx=0x7fa21c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:36.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:36.987+0000 7fa2167fc700 1 -- 172.21.15.22:0/932578835 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa210097170 con 0x7fa220101fc0 2023-09-20T21:48:36.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:36 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3552142437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:36.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:36 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1930601576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:37.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:36 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3552142437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:37.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:36 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1930601576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.196+0000 7fa2277ff700 1 -- 172.21.15.22:0/932578835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa22002cef0 con 0x7fa220101fc0 2023-09-20T21:48:37.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.196+0000 7fa2167fc700 1 -- 172.21.15.22:0/932578835 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa21009a790 con 0x7fa220101fc0 2023-09-20T21:48:37.197 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 -- 172.21.15.22:0/932578835 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa20c0a1300 msgr2=0x7fa20c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:37.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa20c0a1300 0x7fa20c0a37c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa220101e20 tx=0x7fa21c009260 comp rx=0 tx=0).stop 2023-09-20T21:48:37.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 -- 172.21.15.22:0/932578835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa220101fc0 msgr2=0x7fa220197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:37.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa220101fc0 0x7fa220197060 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa210009fd0 tx=0x7fa21000f690 comp rx=0 tx=0).stop 2023-09-20T21:48:37.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 -- 172.21.15.22:0/932578835 shutdown_connections 2023-09-20T21:48:37.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa20c0a1300 0x7fa20c0a37c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa220102960 0x7fa22019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa220100dd0 0x7fa220196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 --2- 172.21.15.22:0/932578835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa220101fc0 0x7fa220197060 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 -- 172.21.15.22:0/932578835 >> 172.21.15.22:0/932578835 conn(0x7fa2200fc320 msgr2=0x7fa2200fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:37.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 -- 172.21.15.22:0/932578835 shutdown_connections 2023-09-20T21:48:37.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.199+0000 7fa2277ff700 1 -- 172.21.15.22:0/932578835 wait complete. 2023-09-20T21:48:37.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:48:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.422+0000 7f57f75ff700 1 -- 172.21.15.22:0/1070408356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f0102980 msgr2=0x7f57f0109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.422+0000 7f57f75ff700 1 --2- 172.21.15.22:0/1070408356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f0102980 0x7f57f0109230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57e800b0d0 tx=0x7f57e800b3e0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.422+0000 7f57f75ff700 1 -- 172.21.15.22:0/1070408356 shutdown_connections 2023-09-20T21:48:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.422+0000 7f57f75ff700 1 --2- 172.21.15.22:0/1070408356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f0102980 0x7f57f0109230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.422+0000 7f57f75ff700 1 --2- 172.21.15.22:0/1070408356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f0100df0 0x7f57f0101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.422+0000 7f57f75ff700 1 --2- 172.21.15.22:0/1070408356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f0101fe0 0x7f57f0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.422+0000 7f57f75ff700 1 -- 172.21.15.22:0/1070408356 >> 172.21.15.22:0/1070408356 conn(0x7f57f00fc320 msgr2=0x7f57f00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.422+0000 7f57f75ff700 1 -- 172.21.15.22:0/1070408356 shutdown_connections 2023-09-20T21:48:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.422+0000 7f57f75ff700 1 -- 172.21.15.22:0/1070408356 wait complete. 2023-09-20T21:48:37.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.424+0000 7f57f75ff700 1 Processor -- start 2023-09-20T21:48:37.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.424+0000 7f57f75ff700 1 -- start start 2023-09-20T21:48:37.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.424+0000 7f57f75ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f0100df0 0x7f57f0196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:37.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.424+0000 7f57f75ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f0101fe0 0x7f57f0197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.424+0000 7f57f75ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f0102980 0x7f57f019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.424+0000 7f57f75ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f019e650 con 0x7f57f0102980 2023-09-20T21:48:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.424+0000 7f57f75ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f019e790 con 0x7f57f0100df0 2023-09-20T21:48:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.424+0000 7f57f75ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f019e920 con 0x7f57f0101fe0 2023-09-20T21:48:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f5b9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f0102980 0x7f57f019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f5b9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f0102980 0x7f57f019e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47668/0 (socket says 172.21.15.22:47668) 2023-09-20T21:48:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f5b9c700 1 -- 172.21.15.22:0/4209133463 learned_addr learned my addr 172.21.15.22:0/4209133463 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f539b700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f0100df0 0x7f57f0196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f4b9a700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f0101fe0 0x7f57f0197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f5b9c700 1 -- 172.21.15.22:0/4209133463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f0101fe0 msgr2=0x7f57f0197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f5b9c700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f0101fe0 0x7f57f0197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f5b9c700 1 -- 172.21.15.22:0/4209133463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f0100df0 msgr2=0x7f57f0196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f5b9c700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f0100df0 0x7f57f0196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f5b9c700 1 -- 172.21.15.22:0/4209133463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e8009d00 con 0x7f57f0102980 2023-09-20T21:48:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.425+0000 7f57f5b9c700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f0102980 0x7f57f019e080 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f57e8008ec0 tx=0x7f57e8008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.426+0000 7f57e67fc700 1 -- 172.21.15.22:0/4209133463 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e800e040 con 0x7f57f0102980 2023-09-20T21:48:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.426+0000 7f57f75ff700 1 -- 172.21.15.22:0/4209133463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f019eba0 con 0x7f57f0102980 2023-09-20T21:48:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.426+0000 7f57e67fc700 1 -- 172.21.15.22:0/4209133463 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57e8005070 con 0x7f57f0102980 2023-09-20T21:48:37.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.426+0000 7f57e67fc700 1 -- 172.21.15.22:0/4209133463 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e801b6c0 con 0x7f57f0102980 2023-09-20T21:48:37.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.426+0000 7f57f75ff700 1 -- 172.21.15.22:0/4209133463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f019efc0 con 0x7f57f0102980 2023-09-20T21:48:37.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.427+0000 7f57f75ff700 1 -- 172.21.15.22:0/4209133463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f010c610 con 0x7f57f0102980 2023-09-20T21:48:37.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.429+0000 7f57e67fc700 1 -- 172.21.15.22:0/4209133463 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f57e80051e0 con 0x7f57f0102980 2023-09-20T21:48:37.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.431+0000 7f57e67fc700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f57dc0a1350 0x7f57dc0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:37.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.431+0000 7f57e67fc700 1 -- 172.21.15.22:0/4209133463 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f57e80ce200 con 0x7f57f0102980 2023-09-20T21:48:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.431+0000 7f57f539b700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f57dc0a1350 0x7f57dc0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.431+0000 7f57f539b700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f57dc0a1350 0x7f57dc0a3810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f57f0101e40 tx=0x7f57ec009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.434+0000 7f57e67fc700 1 -- 172.21.15.22:0/4209133463 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f57e8097570 con 0x7f57f0102980 2023-09-20T21:48:37.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.697+0000 7f57f75ff700 1 -- 172.21.15.22:0/4209133463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57f010c800 con 0x7f57f0102980 2023-09-20T21:48:37.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.698+0000 7f57e67fc700 1 -- 172.21.15.22:0/4209133463 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f57e809ab90 con 0x7f57f0102980 2023-09-20T21:48:37.699 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:37.699 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:47:09.607119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.700+0000 7f57f75ff700 1 -- 172.21.15.22:0/4209133463 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f57dc0a1350 msgr2=0x7f57dc0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.700+0000 7f57f75ff700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f57dc0a1350 0x7f57dc0a3810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f57f0101e40 tx=0x7f57ec009260 comp rx=0 tx=0).stop 2023-09-20T21:48:37.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.700+0000 7f57f75ff700 1 -- 172.21.15.22:0/4209133463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f0102980 msgr2=0x7f57f019e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.700+0000 7f57f75ff700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f0102980 0x7f57f019e080 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f57e8008ec0 tx=0x7f57e8008ef0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.701+0000 7f57f75ff700 1 -- 172.21.15.22:0/4209133463 shutdown_connections 2023-09-20T21:48:37.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.701+0000 7f57f75ff700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57f0102980 0x7f57f019e080 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.701+0000 7f57f75ff700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f57dc0a1350 0x7f57dc0a3810 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.701+0000 7f57f75ff700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f57f0100df0 0x7f57f0196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.701+0000 7f57f75ff700 1 --2- 172.21.15.22:0/4209133463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57f0101fe0 0x7f57f0197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.701+0000 7f57f75ff700 1 -- 172.21.15.22:0/4209133463 >> 172.21.15.22:0/4209133463 conn(0x7f57f00fc320 msgr2=0x7f57f00fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.701+0000 7f57f75ff700 1 -- 172.21.15.22:0/4209133463 shutdown_connections 2023-09-20T21:48:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.701+0000 7f57f75ff700 1 -- 172.21.15.22:0/4209133463 wait complete. 2023-09-20T21:48:37.704 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2023-09-20T21:48:37.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:48:37.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:37 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2395996763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:37.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:37 smithi089 ceph-mon[134090]: pgmap v338: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:37.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:37 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/932578835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.924+0000 7fb3df9ff700 1 -- 172.21.15.22:0/3998691388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3d8101930 msgr2=0x7fb3d8103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.924+0000 7fb3df9ff700 1 --2- 172.21.15.22:0/3998691388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3d8101930 0x7fb3d8103d20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009990 tx=0x7fb3d4009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.924+0000 7fb3df9ff700 1 -- 172.21.15.22:0/3998691388 shutdown_connections 2023-09-20T21:48:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.924+0000 7fb3df9ff700 1 --2- 172.21.15.22:0/3998691388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d8106b90 0x7fb3d8108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.924+0000 7fb3df9ff700 1 --2- 172.21.15.22:0/3998691388 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3d8101930 0x7fb3d8103d20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.924+0000 7fb3df9ff700 1 --2- 172.21.15.22:0/3998691388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3d8104260 0x7fb3d8106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.924+0000 7fb3df9ff700 1 -- 172.21.15.22:0/3998691388 >> 172.21.15.22:0/3998691388 conn(0x7fb3d80fb240 msgr2=0x7fb3d80fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:37.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.924+0000 7fb3df9ff700 1 -- 172.21.15.22:0/3998691388 shutdown_connections 2023-09-20T21:48:37.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.924+0000 7fb3df9ff700 1 -- 172.21.15.22:0/3998691388 wait complete. 2023-09-20T21:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.925+0000 7fb3df9ff700 1 Processor -- start 2023-09-20T21:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.925+0000 7fb3df9ff700 1 -- start start 2023-09-20T21:48:37.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.925+0000 7fb3df9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3d8101930 0x7fb3d8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:37.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.925+0000 7fb3df9ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d8104260 0x7fb3d81991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:37.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3df9ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3d8106b90 0x7fb3d81a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:37.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3df9ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d81a0800 con 0x7fb3d8104260 2023-09-20T21:48:37.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3df9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d81a0970 con 0x7fb3d8101930 2023-09-20T21:48:37.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3df9ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d81a0b30 con 0x7fb3d8106b90 2023-09-20T21:48:37.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dd79b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3d8101930 0x7fb3d8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:37.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dd79b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3d8101930 0x7fb3d8198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43286/0 (socket says 172.21.15.22:43286) 2023-09-20T21:48:37.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dd79b700 1 -- 172.21.15.22:0/2789597084 learned_addr learned my addr 172.21.15.22:0/2789597084 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:37.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dcf9a700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d8104260 0x7fb3d81991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:37.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3ddf9c700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3d8106b90 0x7fb3d81a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:37.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dd79b700 1 -- 172.21.15.22:0/2789597084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3d8106b90 msgr2=0x7fb3d81a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:37.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dd79b700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3d8106b90 0x7fb3d81a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dd79b700 1 -- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d8104260 msgr2=0x7fb3d81991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:37.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dd79b700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d8104260 0x7fb3d81991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:37.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dd79b700 1 -- 172.21.15.22:0/2789597084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d40095f0 con 0x7fb3d8101930 2023-09-20T21:48:37.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dcf9a700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d8104260 0x7fb3d81991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:37.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.926+0000 7fb3dd79b700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3d8101930 0x7fb3d8198c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4004740 tx=0x7fb3d4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:37.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.927+0000 7fb3ce7fc700 1 -- 172.21.15.22:0/2789597084 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d401d070 con 0x7fb3d8101930 2023-09-20T21:48:37.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.927+0000 7fb3df9ff700 1 -- 172.21.15.22:0/2789597084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d81a0db0 con 0x7fb3d8101930 2023-09-20T21:48:37.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.927+0000 7fb3ce7fc700 1 -- 172.21.15.22:0/2789597084 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3d4004830 con 0x7fb3d8101930 2023-09-20T21:48:37.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.927+0000 7fb3ce7fc700 1 -- 172.21.15.22:0/2789597084 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40053c0 con 0x7fb3d8101930 2023-09-20T21:48:37.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.927+0000 7fb3df9ff700 1 -- 172.21.15.22:0/2789597084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d81a1170 con 0x7fb3d8101930 2023-09-20T21:48:37.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.928+0000 7fb3df9ff700 1 -- 172.21.15.22:0/2789597084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d8108710 con 0x7fb3d8101930 2023-09-20T21:48:37.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.930+0000 7fb3ce7fc700 1 -- 172.21.15.22:0/2789597084 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fb3d4005560 con 0x7fb3d8101930 2023-09-20T21:48:37.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.931+0000 7fb3ce7fc700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3c40a1350 0x7fb3c40a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:37.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.931+0000 7fb3ce7fc700 1 -- 172.21.15.22:0/2789597084 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fb3d40ce5c0 con 0x7fb3d8101930 2023-09-20T21:48:37.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.932+0000 7fb3dcf9a700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3c40a1350 0x7fb3c40a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:37.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.932+0000 7fb3dcf9a700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3c40a1350 0x7fb3c40a3810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb3d819a190 tx=0x7fb3c8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:37.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:37.933+0000 7fb3ce7fc700 1 -- 172.21.15.22:0/2789597084 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb3d4097990 con 0x7fb3d8101930 2023-09-20T21:48:37.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:37 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2395996763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:37.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:37 smithi022 ceph-mon[97651]: pgmap v338: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:37.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:37 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/932578835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:38.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:37 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2395996763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:38.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:37 smithi123 ceph-mon[134433]: pgmap v338: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:38.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:37 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/932578835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:38.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.144+0000 7fb3df9ff700 1 -- 172.21.15.22:0/2789597084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3d810c7a0 con 0x7fb3d8101930 2023-09-20T21:48:38.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.146+0000 7fb3ce7fc700 1 -- 172.21.15.22:0/2789597084 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7fb3d409afb0 con 0x7fb3d8101930 2023-09-20T21:48:38.147 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:38.147 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:48:38.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 -- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3c40a1350 msgr2=0x7fb3c40a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:38.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3c40a1350 0x7fb3c40a3810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb3d819a190 tx=0x7fb3c8009260 comp rx=0 tx=0).stop 2023-09-20T21:48:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 -- 172.21.15.22:0/2789597084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3d8101930 msgr2=0x7fb3d8198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3d8101930 0x7fb3d8198c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4004740 tx=0x7fb3d4009580 comp rx=0 tx=0).stop 2023-09-20T21:48:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 -- 172.21.15.22:0/2789597084 shutdown_connections 2023-09-20T21:48:38.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3d8106b90 0x7fb3d81a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3c40a1350 0x7fb3c40a3810 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3d8101930 0x7fb3d8198c90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 --2- 172.21.15.22:0/2789597084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3d8104260 0x7fb3d81991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 -- 172.21.15.22:0/2789597084 >> 172.21.15.22:0/2789597084 conn(0x7fb3d80fb240 msgr2=0x7fb3d81025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:38.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 -- 172.21.15.22:0/2789597084 shutdown_connections 2023-09-20T21:48:38.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.149+0000 7fb3df9ff700 1 -- 172.21.15.22:0/2789597084 wait complete. 2023-09-20T21:48:38.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T21:48:38.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:48:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.376+0000 7f0abd7ff700 1 -- 172.21.15.22:0/2438570837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ab8100dd0 msgr2=0x7f0ab81011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.376+0000 7f0abd7ff700 1 --2- 172.21.15.22:0/2438570837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ab8100dd0 0x7f0ab81011f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80099c0 tx=0x7f0aa8009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.376+0000 7f0abd7ff700 1 -- 172.21.15.22:0/2438570837 shutdown_connections 2023-09-20T21:48:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.376+0000 7f0abd7ff700 1 --2- 172.21.15.22:0/2438570837 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ab8102960 0x7f0ab8109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.376+0000 7f0abd7ff700 1 --2- 172.21.15.22:0/2438570837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ab8100dd0 0x7f0ab81011f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.376+0000 7f0abd7ff700 1 --2- 172.21.15.22:0/2438570837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8101fc0 0x7f0ab8102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.376+0000 7f0abd7ff700 1 -- 172.21.15.22:0/2438570837 >> 172.21.15.22:0/2438570837 conn(0x7f0ab80fc320 msgr2=0x7f0ab80fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.376+0000 7f0abd7ff700 1 -- 172.21.15.22:0/2438570837 shutdown_connections 2023-09-20T21:48:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.376+0000 7f0abd7ff700 1 -- 172.21.15.22:0/2438570837 wait complete. 2023-09-20T21:48:38.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.377+0000 7f0abd7ff700 1 Processor -- start 2023-09-20T21:48:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.377+0000 7f0abd7ff700 1 -- start start 2023-09-20T21:48:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0abd7ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ab8100dd0 0x7f0ab8198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0abd7ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8101fc0 0x7f0ab8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0abd7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ab8102960 0x7f0ab81a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0abd7ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81a08c0 con 0x7f0ab8100dd0 2023-09-20T21:48:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0abd7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81a0a00 con 0x7f0ab8102960 2023-09-20T21:48:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0abd7ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81a0b90 con 0x7f0ab8101fc0 2023-09-20T21:48:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0ab6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ab8100dd0 0x7f0ab8198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0ab6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ab8100dd0 0x7f0ab8198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47708/0 (socket says 172.21.15.22:47708) 2023-09-20T21:48:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0ab6ffd700 1 -- 172.21.15.22:0/1401699131 learned_addr learned my addr 172.21.15.22:0/1401699131 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0ab67fc700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8101fc0 0x7f0ab8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0ab77fe700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ab8102960 0x7f0ab81a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0ab6ffd700 1 -- 172.21.15.22:0/1401699131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8101fc0 msgr2=0x7f0ab8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0ab6ffd700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8101fc0 0x7f0ab8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0ab6ffd700 1 -- 172.21.15.22:0/1401699131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ab8102960 msgr2=0x7f0ab81a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0ab6ffd700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ab8102960 0x7f0ab81a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.378+0000 7f0ab6ffd700 1 -- 172.21.15.22:0/1401699131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80095f0 con 0x7f0ab8100dd0 2023-09-20T21:48:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.379+0000 7f0ab6ffd700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ab8100dd0 0x7f0ab8198d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009fd0 tx=0x7f0aa800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.379+0000 7f0a9ffff700 1 -- 172.21.15.22:0/1401699131 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa801d070 con 0x7f0ab8100dd0 2023-09-20T21:48:38.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.379+0000 7f0a9ffff700 1 -- 172.21.15.22:0/1401699131 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0aa8005050 con 0x7f0ab8100dd0 2023-09-20T21:48:38.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.379+0000 7f0abd7ff700 1 -- 172.21.15.22:0/1401699131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab81a0e10 con 0x7f0ab8100dd0 2023-09-20T21:48:38.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.379+0000 7f0a9ffff700 1 -- 172.21.15.22:0/1401699131 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8004a70 con 0x7f0ab8100dd0 2023-09-20T21:48:38.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.379+0000 7f0abd7ff700 1 -- 172.21.15.22:0/1401699131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab81a11d0 con 0x7f0ab8100dd0 2023-09-20T21:48:38.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.380+0000 7f0abd7ff700 1 -- 172.21.15.22:0/1401699131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab8193420 con 0x7f0ab8100dd0 2023-09-20T21:48:38.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.382+0000 7f0a9ffff700 1 -- 172.21.15.22:0/1401699131 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f0aa8005450 con 0x7f0ab8100dd0 2023-09-20T21:48:38.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.383+0000 7f0a9ffff700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0aa00a1380 0x7f0aa00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:38.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.384+0000 7f0a9ffff700 1 -- 172.21.15.22:0/1401699131 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f0aa80ce120 con 0x7f0ab8100dd0 2023-09-20T21:48:38.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.384+0000 7f0ab67fc700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0aa00a1380 0x7f0aa00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:38.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.384+0000 7f0ab67fc700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0aa00a1380 0x7f0aa00a3840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009c60 tx=0x7f0aac009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:38.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.386+0000 7f0a9ffff700 1 -- 172.21.15.22:0/1401699131 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f0aa8097490 con 0x7f0ab8100dd0 2023-09-20T21:48:38.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.597+0000 7f0abd7ff700 1 -- 172.21.15.22:0/1401699131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ab802cfa0 con 0x7f0ab8100dd0 2023-09-20T21:48:38.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.598+0000 7f0a9ffff700 1 -- 172.21.15.22:0/1401699131 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0aa809aab0 con 0x7f0ab8100dd0 2023-09-20T21:48:38.598 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:48:38.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 -- 172.21.15.22:0/1401699131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0aa00a1380 msgr2=0x7f0aa00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:38.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0aa00a1380 0x7f0aa00a3840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009c60 tx=0x7f0aac009260 comp rx=0 tx=0).stop 2023-09-20T21:48:38.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 -- 172.21.15.22:0/1401699131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ab8100dd0 msgr2=0x7f0ab8198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:38.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ab8100dd0 0x7f0ab8198d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009fd0 tx=0x7f0aa800f690 comp rx=0 tx=0).stop 2023-09-20T21:48:38.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 -- 172.21.15.22:0/1401699131 shutdown_connections 2023-09-20T21:48:38.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0aa00a1380 0x7f0aa00a3840 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ab8102960 0x7f0ab81a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ab8100dd0 0x7f0ab8198d40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 --2- 172.21.15.22:0/1401699131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab8101fc0 0x7f0ab8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 -- 172.21.15.22:0/1401699131 >> 172.21.15.22:0/1401699131 conn(0x7f0ab80fc320 msgr2=0x7f0ab8107940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:38.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 -- 172.21.15.22:0/1401699131 shutdown_connections 2023-09-20T21:48:38.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.600+0000 7f0abd7ff700 1 -- 172.21.15.22:0/1401699131 wait complete. 2023-09-20T21:48:38.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T21:48:38.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.827+0000 7ff32efff700 1 -- 172.21.15.22:0/3289617936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff328073840 msgr2=0x7ff328073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:38.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.827+0000 7ff32efff700 1 --2- 172.21.15.22:0/3289617936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff328073840 0x7ff328073cc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff314009990 tx=0x7ff314009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.827+0000 7ff32efff700 1 -- 172.21.15.22:0/3289617936 shutdown_connections 2023-09-20T21:48:38.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.827+0000 7ff32efff700 1 --2- 172.21.15.22:0/3289617936 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff328108c90 0x7ff32810b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.827+0000 7ff32efff700 1 --2- 172.21.15.22:0/3289617936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff328074e20 0x7ff328073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.827+0000 7ff32efff700 1 --2- 172.21.15.22:0/3289617936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff328073840 0x7ff328073cc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.827+0000 7ff32efff700 1 -- 172.21.15.22:0/3289617936 >> 172.21.15.22:0/3289617936 conn(0x7ff3280fc320 msgr2=0x7ff3280fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:38.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.828+0000 7ff32efff700 1 -- 172.21.15.22:0/3289617936 shutdown_connections 2023-09-20T21:48:38.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.828+0000 7ff32efff700 1 -- 172.21.15.22:0/3289617936 wait complete. 2023-09-20T21:48:38.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff32efff700 1 Processor -- start 2023-09-20T21:48:38.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff32efff700 1 -- start start 2023-09-20T21:48:38.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff32efff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff328073840 0x7ff328198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:38.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff32efff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff328074e20 0x7ff328199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:38.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff32efff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff328108c90 0x7ff3281a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:38.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff32efff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3281a0850 con 0x7ff328074e20 2023-09-20T21:48:38.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff32efff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3281a09c0 con 0x7ff328108c90 2023-09-20T21:48:38.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff32efff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3281a0b80 con 0x7ff328073840 2023-09-20T21:48:38.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff327fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff328074e20 0x7ff328199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:38.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff327fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff328074e20 0x7ff328199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47720/0 (socket says 172.21.15.22:47720) 2023-09-20T21:48:38.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff327fff700 1 -- 172.21.15.22:0/730477398 learned_addr learned my addr 172.21.15.22:0/730477398 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:38.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff32d59c700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff328108c90 0x7ff3281a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:38.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.829+0000 7ff32cd9b700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff328073840 0x7ff328198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:38.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff327fff700 1 -- 172.21.15.22:0/730477398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff328073840 msgr2=0x7ff328198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:38.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff327fff700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff328073840 0x7ff328198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff327fff700 1 -- 172.21.15.22:0/730477398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff328108c90 msgr2=0x7ff3281a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:38.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff327fff700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff328108c90 0x7ff3281a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:38.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff327fff700 1 -- 172.21.15.22:0/730477398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c009520 con 0x7ff328074e20 2023-09-20T21:48:38.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff32d59c700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff328108c90 0x7ff3281a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:38.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff327fff700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff328074e20 0x7ff328199250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff314000c00 tx=0x7ff314005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:38.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff325ffb700 1 -- 172.21.15.22:0/730477398 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31401d070 con 0x7ff328074e20 2023-09-20T21:48:38.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff325ffb700 1 -- 172.21.15.22:0/730477398 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff314022470 con 0x7ff328074e20 2023-09-20T21:48:38.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff325ffb700 1 -- 172.21.15.22:0/730477398 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff314017650 con 0x7ff328074e20 2023-09-20T21:48:38.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.830+0000 7ff32efff700 1 -- 172.21.15.22:0/730477398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff320009c40 con 0x7ff328074e20 2023-09-20T21:48:38.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.831+0000 7ff32efff700 1 -- 172.21.15.22:0/730477398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3140095f0 con 0x7ff328074e20 2023-09-20T21:48:38.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.832+0000 7ff32efff700 1 -- 172.21.15.22:0/730477398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff328107340 con 0x7ff328074e20 2023-09-20T21:48:38.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.834+0000 7ff325ffb700 1 -- 172.21.15.22:0/730477398 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ff3140225e0 con 0x7ff328074e20 2023-09-20T21:48:38.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.835+0000 7ff325ffb700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff3180a1300 0x7ff3180a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:38.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.835+0000 7ff325ffb700 1 -- 172.21.15.22:0/730477398 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ff3140ce2b0 con 0x7ff328074e20 2023-09-20T21:48:38.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.835+0000 7ff32cd9b700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff3180a1300 0x7ff3180a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:38.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.836+0000 7ff32cd9b700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff3180a1300 0x7ff3180a37c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff32819a1e0 tx=0x7ff31c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:38.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:38.839+0000 7ff325ffb700 1 -- 172.21.15.22:0/730477398 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff31405ce10 con 0x7ff328074e20 2023-09-20T21:48:38.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:38 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4209133463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:38.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:38 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2789597084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:38.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:38 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1401699131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:39.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.049+0000 7ff32efff700 1 -- 172.21.15.22:0/730477398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff32802d160 con 0x7ff328074e20 2023-09-20T21:48:39.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.050+0000 7ff325ffb700 1 -- 172.21.15.22:0/730477398 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff314097620 con 0x7ff328074e20 2023-09-20T21:48:39.050 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:39.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 -- 172.21.15.22:0/730477398 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff3180a1300 msgr2=0x7ff3180a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff3180a1300 0x7ff3180a37c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff32819a1e0 tx=0x7ff31c009260 comp rx=0 tx=0).stop 2023-09-20T21:48:39.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 -- 172.21.15.22:0/730477398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff328074e20 msgr2=0x7ff328199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff328074e20 0x7ff328199250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff314000c00 tx=0x7ff314005720 comp rx=0 tx=0).stop 2023-09-20T21:48:39.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 -- 172.21.15.22:0/730477398 shutdown_connections 2023-09-20T21:48:39.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff3180a1300 0x7ff3180a37c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff328108c90 0x7ff3281a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff328073840 0x7ff328198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 --2- 172.21.15.22:0/730477398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff328074e20 0x7ff328199250 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 -- 172.21.15.22:0/730477398 >> 172.21.15.22:0/730477398 conn(0x7ff3280fc320 msgr2=0x7ff3280fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:39.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 -- 172.21.15.22:0/730477398 shutdown_connections 2023-09-20T21:48:39.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.052+0000 7ff32efff700 1 -- 172.21.15.22:0/730477398 wait complete. 2023-09-20T21:48:39.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T21:48:39.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:38 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4209133463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:39.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:38 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2789597084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:39.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:38 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1401699131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:39.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:38 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4209133463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:39.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:38 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2789597084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:39.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:38 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1401699131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.278+0000 7fd1f19ff700 1 -- 172.21.15.22:0/1535571840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1ec102960 msgr2=0x7fd1ec109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.278+0000 7fd1f19ff700 1 --2- 172.21.15.22:0/1535571840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1ec102960 0x7fd1ec109210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400b0d0 tx=0x7fd1e400b3e0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.278+0000 7fd1f19ff700 1 -- 172.21.15.22:0/1535571840 shutdown_connections 2023-09-20T21:48:39.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.278+0000 7fd1f19ff700 1 --2- 172.21.15.22:0/1535571840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1ec102960 0x7fd1ec109210 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.278+0000 7fd1f19ff700 1 --2- 172.21.15.22:0/1535571840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec100dd0 0x7fd1ec1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.278+0000 7fd1f19ff700 1 --2- 172.21.15.22:0/1535571840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ec101fc0 0x7fd1ec102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.278+0000 7fd1f19ff700 1 -- 172.21.15.22:0/1535571840 >> 172.21.15.22:0/1535571840 conn(0x7fd1ec0fc320 msgr2=0x7fd1ec0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.278+0000 7fd1f19ff700 1 -- 172.21.15.22:0/1535571840 shutdown_connections 2023-09-20T21:48:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.278+0000 7fd1f19ff700 1 -- 172.21.15.22:0/1535571840 wait complete. 2023-09-20T21:48:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.279+0000 7fd1f19ff700 1 Processor -- start 2023-09-20T21:48:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1f19ff700 1 -- start start 2023-09-20T21:48:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1f19ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ec100dd0 0x7fd1ec198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1f19ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec101fc0 0x7fd1ec199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1f19ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1ec102960 0x7fd1ec1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1f19ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1a09a0 con 0x7fd1ec102960 2023-09-20T21:48:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1f19ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1a0ae0 con 0x7fd1ec101fc0 2023-09-20T21:48:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1f19ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1a0c70 con 0x7fd1ec100dd0 2023-09-20T21:48:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1eaffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ec100dd0 0x7fd1ec198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1eb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1ec102960 0x7fd1ec1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1eaffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ec100dd0 0x7fd1ec198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60574/0 (socket says 172.21.15.22:60574) 2023-09-20T21:48:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1ea7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec101fc0 0x7fd1ec199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1eb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1ec102960 0x7fd1ec1a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47734/0 (socket says 172.21.15.22:47734) 2023-09-20T21:48:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.280+0000 7fd1eaffd700 1 -- 172.21.15.22:0/3447058006 learned_addr learned my addr 172.21.15.22:0/3447058006 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1eb7fe700 1 -- 172.21.15.22:0/3447058006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ec100dd0 msgr2=0x7fd1ec198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1eb7fe700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ec100dd0 0x7fd1ec198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1eb7fe700 1 -- 172.21.15.22:0/3447058006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec101fc0 msgr2=0x7fd1ec199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1eb7fe700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec101fc0 0x7fd1ec199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1eb7fe700 1 -- 172.21.15.22:0/3447058006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc009520 con 0x7fd1ec102960 2023-09-20T21:48:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1eb7fe700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1ec102960 0x7fd1ec1a03d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4007a30 tx=0x7fd1e4008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1f09fd700 1 -- 172.21.15.22:0/3447058006 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400e040 con 0x7fd1ec102960 2023-09-20T21:48:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1f09fd700 1 -- 172.21.15.22:0/3447058006 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1e400bd30 con 0x7fd1ec102960 2023-09-20T21:48:39.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1f19ff700 1 -- 172.21.15.22:0/3447058006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e0009520 con 0x7fd1ec102960 2023-09-20T21:48:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1f09fd700 1 -- 172.21.15.22:0/3447058006 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e401d470 con 0x7fd1ec102960 2023-09-20T21:48:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.281+0000 7fd1f19ff700 1 -- 172.21.15.22:0/3447058006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e4009d00 con 0x7fd1ec102960 2023-09-20T21:48:39.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.282+0000 7fd1f19ff700 1 -- 172.21.15.22:0/3447058006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec10c640 con 0x7fd1ec102960 2023-09-20T21:48:39.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.284+0000 7fd1f09fd700 1 -- 172.21.15.22:0/3447058006 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fd1e4005020 con 0x7fd1ec102960 2023-09-20T21:48:39.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.286+0000 7fd1f09fd700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1d40a1300 0x7fd1d40a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:39.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.286+0000 7fd1f09fd700 1 -- 172.21.15.22:0/3447058006 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fd1e40ce1e0 con 0x7fd1ec102960 2023-09-20T21:48:39.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.286+0000 7fd1eaffd700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1d40a1300 0x7fd1d40a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:39.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.286+0000 7fd1eaffd700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1d40a1300 0x7fd1d40a37c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc005950 tx=0x7fd1dc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:39.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.289+0000 7fd1f09fd700 1 -- 172.21.15.22:0/3447058006 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd1e4097550 con 0x7fd1ec102960 2023-09-20T21:48:39.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.499+0000 7fd1f19ff700 1 -- 172.21.15.22:0/3447058006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd1ec1a11f0 con 0x7fd1ec102960 2023-09-20T21:48:39.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.500+0000 7fd1f09fd700 1 -- 172.21.15.22:0/3447058006 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd1e409ab70 con 0x7fd1ec102960 2023-09-20T21:48:39.501 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T21:48:39.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.502+0000 7fd1f19ff700 1 -- 172.21.15.22:0/3447058006 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1d40a1300 msgr2=0x7fd1d40a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.502+0000 7fd1f19ff700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1d40a1300 0x7fd1d40a37c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc005950 tx=0x7fd1dc009260 comp rx=0 tx=0).stop 2023-09-20T21:48:39.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.502+0000 7fd1f19ff700 1 -- 172.21.15.22:0/3447058006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1ec102960 msgr2=0x7fd1ec1a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.502+0000 7fd1f19ff700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1ec102960 0x7fd1ec1a03d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4007a30 tx=0x7fd1e4008eb0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.502+0000 7fd1f19ff700 1 -- 172.21.15.22:0/3447058006 shutdown_connections 2023-09-20T21:48:39.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.502+0000 7fd1f19ff700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd1ec102960 0x7fd1ec1a03d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.502+0000 7fd1f19ff700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ec100dd0 0x7fd1ec198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.502+0000 7fd1f19ff700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd1d40a1300 0x7fd1d40a37c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.502+0000 7fd1f19ff700 1 --2- 172.21.15.22:0/3447058006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1ec101fc0 0x7fd1ec199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.502+0000 7fd1f19ff700 1 -- 172.21.15.22:0/3447058006 >> 172.21.15.22:0/3447058006 conn(0x7fd1ec0fc320 msgr2=0x7fd1ec0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:39.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.503+0000 7fd1f19ff700 1 -- 172.21.15.22:0/3447058006 shutdown_connections 2023-09-20T21:48:39.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.503+0000 7fd1f19ff700 1 -- 172.21.15.22:0/3447058006 wait complete. 2023-09-20T21:48:39.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T21:48:39.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.729+0000 7fa631dff700 1 -- 172.21.15.22:0/3745291580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c073840 msgr2=0x7fa62c073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.729+0000 7fa631dff700 1 --2- 172.21.15.22:0/3745291580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c073840 0x7fa62c073cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa620009990 tx=0x7fa620009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.730+0000 7fa631dff700 1 -- 172.21.15.22:0/3745291580 shutdown_connections 2023-09-20T21:48:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.730+0000 7fa631dff700 1 --2- 172.21.15.22:0/3745291580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c108cb0 0x7fa62c10b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.730+0000 7fa631dff700 1 --2- 172.21.15.22:0/3745291580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c074e20 0x7fa62c073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.730+0000 7fa631dff700 1 --2- 172.21.15.22:0/3745291580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c073840 0x7fa62c073cc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.730+0000 7fa631dff700 1 -- 172.21.15.22:0/3745291580 >> 172.21.15.22:0/3745291580 conn(0x7fa62c0fc360 msgr2=0x7fa62c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:39.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.730+0000 7fa631dff700 1 -- 172.21.15.22:0/3745291580 shutdown_connections 2023-09-20T21:48:39.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.730+0000 7fa631dff700 1 -- 172.21.15.22:0/3745291580 wait complete. 2023-09-20T21:48:39.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.731+0000 7fa631dff700 1 Processor -- start 2023-09-20T21:48:39.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa631dff700 1 -- start start 2023-09-20T21:48:39.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa631dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c073840 0x7fa62c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:39.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa631dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c074e20 0x7fa62c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:39.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa631dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c108cb0 0x7fa62c1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:39.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa631dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c1a0840 con 0x7fa62c074e20 2023-09-20T21:48:39.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa631dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c1a09b0 con 0x7fa62c108cb0 2023-09-20T21:48:39.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa631dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c1a0b70 con 0x7fa62c073840 2023-09-20T21:48:39.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa62b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c073840 0x7fa62c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:39.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa62b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c073840 0x7fa62c198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60592/0 (socket says 172.21.15.22:60592) 2023-09-20T21:48:39.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa62b7fe700 1 -- 172.21.15.22:0/1521677972 learned_addr learned my addr 172.21.15.22:0/1521677972 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:39.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.732+0000 7fa62affd700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c074e20 0x7fa62c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:39.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.733+0000 7fa62bfff700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c108cb0 0x7fa62c1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:39.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.733+0000 7fa62b7fe700 1 -- 172.21.15.22:0/1521677972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c108cb0 msgr2=0x7fa62c1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.733+0000 7fa62b7fe700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c108cb0 0x7fa62c1a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.733+0000 7fa62b7fe700 1 -- 172.21.15.22:0/1521677972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c074e20 msgr2=0x7fa62c199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.733+0000 7fa62b7fe700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c074e20 0x7fa62c199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.733+0000 7fa62b7fe700 1 -- 172.21.15.22:0/1521677972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6200095f0 con 0x7fa62c073840 2023-09-20T21:48:39.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.733+0000 7fa62b7fe700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c073840 0x7fa62c198d00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00e9a0 tx=0x7fa61c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:39.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.733+0000 7fa628ff9700 1 -- 172.21.15.22:0/1521677972 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c009790 con 0x7fa62c073840 2023-09-20T21:48:39.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.733+0000 7fa628ff9700 1 -- 172.21.15.22:0/1521677972 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa61c010430 con 0x7fa62c073840 2023-09-20T21:48:39.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.734+0000 7fa631dff700 1 -- 172.21.15.22:0/1521677972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c1a0e50 con 0x7fa62c073840 2023-09-20T21:48:39.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.734+0000 7fa628ff9700 1 -- 172.21.15.22:0/1521677972 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c018680 con 0x7fa62c073840 2023-09-20T21:48:39.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.734+0000 7fa631dff700 1 -- 172.21.15.22:0/1521677972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c1a1280 con 0x7fa62c073840 2023-09-20T21:48:39.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.735+0000 7fa631dff700 1 -- 172.21.15.22:0/1521677972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62c107360 con 0x7fa62c073840 2023-09-20T21:48:39.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.737+0000 7fa628ff9700 1 -- 172.21.15.22:0/1521677972 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa61c0105a0 con 0x7fa62c073840 2023-09-20T21:48:39.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.738+0000 7fa628ff9700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6140a1380 0x7fa6140a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:39.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.738+0000 7fa628ff9700 1 -- 172.21.15.22:0/1521677972 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa61c014070 con 0x7fa62c073840 2023-09-20T21:48:39.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.738+0000 7fa62affd700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6140a1380 0x7fa6140a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:39.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.739+0000 7fa62affd700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6140a1380 0x7fa6140a3840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa62000b2e0 tx=0x7fa620005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:39.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.740+0000 7fa628ff9700 1 -- 172.21.15.22:0/1521677972 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa61c0963f0 con 0x7fa62c073840 2023-09-20T21:48:39.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.947+0000 7fa631dff700 1 -- 172.21.15.22:0/1521677972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa62c02cfa0 con 0x7fa62c073840 2023-09-20T21:48:39.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.948+0000 7fa628ff9700 1 -- 172.21.15.22:0/1521677972 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa61c099a10 con 0x7fa62c073840 2023-09-20T21:48:39.948 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 -- 172.21.15.22:0/1521677972 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6140a1380 msgr2=0x7fa6140a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6140a1380 0x7fa6140a3840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa62000b2e0 tx=0x7fa620005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 -- 172.21.15.22:0/1521677972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c073840 msgr2=0x7fa62c198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c073840 0x7fa62c198d00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00e9a0 tx=0x7fa61c00ecb0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 -- 172.21.15.22:0/1521677972 shutdown_connections 2023-09-20T21:48:39.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa6140a1380 0x7fa6140a3840 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa62c108cb0 0x7fa62c1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa62c073840 0x7fa62c198d00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 --2- 172.21.15.22:0/1521677972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa62c074e20 0x7fa62c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:39.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 -- 172.21.15.22:0/1521677972 >> 172.21.15.22:0/1521677972 conn(0x7fa62c0fc360 msgr2=0x7fa62c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:39.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 -- 172.21.15.22:0/1521677972 shutdown_connections 2023-09-20T21:48:39.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:39.950+0000 7fa631dff700 1 -- 172.21.15.22:0/1521677972 wait complete. 2023-09-20T21:48:39.967 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:39 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/730477398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:39.968 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:39 smithi022 ceph-mon[97651]: pgmap v339: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:48:39.968 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:39 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3447058006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:39.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:48:40.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:39 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/730477398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:40.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:39 smithi123 ceph-mon[134433]: pgmap v339: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:48:40.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:39 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3447058006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:40.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:39 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/730477398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:40.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:39 smithi089 ceph-mon[134090]: pgmap v339: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-09-20T21:48:40.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:39 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3447058006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:40.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.173+0000 7fa3f8dff700 1 -- 172.21.15.22:0/2098809293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4101fd0 msgr2=0x7fa3f4102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:40.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.173+0000 7fa3f8dff700 1 --2- 172.21.15.22:0/2098809293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4101fd0 0x7fa3f4102430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009920 tx=0x7fa3e8009c30 comp rx=0 tx=0).stop 2023-09-20T21:48:40.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.173+0000 7fa3f8dff700 1 -- 172.21.15.22:0/2098809293 shutdown_connections 2023-09-20T21:48:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.173+0000 7fa3f8dff700 1 --2- 172.21.15.22:0/2098809293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f4102970 0x7fa3f4109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.173+0000 7fa3f8dff700 1 --2- 172.21.15.22:0/2098809293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4100de0 0x7fa3f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.173+0000 7fa3f8dff700 1 --2- 172.21.15.22:0/2098809293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4101fd0 0x7fa3f4102430 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.173+0000 7fa3f8dff700 1 -- 172.21.15.22:0/2098809293 >> 172.21.15.22:0/2098809293 conn(0x7fa3f40fc310 msgr2=0x7fa3f40fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.174+0000 7fa3f8dff700 1 -- 172.21.15.22:0/2098809293 shutdown_connections 2023-09-20T21:48:40.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.174+0000 7fa3f8dff700 1 -- 172.21.15.22:0/2098809293 wait complete. 2023-09-20T21:48:40.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.174+0000 7fa3f8dff700 1 Processor -- start 2023-09-20T21:48:40.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f8dff700 1 -- start start 2023-09-20T21:48:40.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f8dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4100de0 0x7fa3f4196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:40.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f8dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f4101fd0 0x7fa3f4197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:40.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f8dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4102970 0x7fa3f419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:40.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f8dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f419e680 con 0x7fa3f4102970 2023-09-20T21:48:40.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f8dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f419e7c0 con 0x7fa3f4100de0 2023-09-20T21:48:40.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f8dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f419e950 con 0x7fa3f4101fd0 2023-09-20T21:48:40.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f2d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4102970 0x7fa3f419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:40.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4100de0 0x7fa3f4196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:40.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f4101fd0 0x7fa3f4197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:40.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4100de0 0x7fa3f4196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43364/0 (socket says 172.21.15.22:43364) 2023-09-20T21:48:40.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.175+0000 7fa3f259c700 1 -- 172.21.15.22:0/3761395679 learned_addr learned my addr 172.21.15.22:0/3761395679 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:40.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3f259c700 1 -- 172.21.15.22:0/3761395679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f4101fd0 msgr2=0x7fa3f4197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:40.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3f259c700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f4101fd0 0x7fa3f4197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3f259c700 1 -- 172.21.15.22:0/3761395679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4102970 msgr2=0x7fa3f419e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:40.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3f259c700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4102970 0x7fa3f419e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3f259c700 1 -- 172.21.15.22:0/3761395679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec009d00 con 0x7fa3f4100de0 2023-09-20T21:48:40.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3f1d9b700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f4101fd0 0x7fa3f4197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:40.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3f2d9d700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4102970 0x7fa3f419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3f259c700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4100de0 0x7fa3f4196b00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400e970 tx=0x7fa3e400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3e37fe700 1 -- 172.21.15.22:0/3761395679 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4009790 con 0x7fa3f4100de0 2023-09-20T21:48:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3f8dff700 1 -- 172.21.15.22:0/3761395679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e8009520 con 0x7fa3f4100de0 2023-09-20T21:48:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3e37fe700 1 -- 172.21.15.22:0/3761395679 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e400c7e0 con 0x7fa3f4100de0 2023-09-20T21:48:40.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.176+0000 7fa3e37fe700 1 -- 172.21.15.22:0/3761395679 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4003c70 con 0x7fa3f4100de0 2023-09-20T21:48:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.177+0000 7fa3f8dff700 1 -- 172.21.15.22:0/3761395679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f419efb0 con 0x7fa3f4100de0 2023-09-20T21:48:40.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.178+0000 7fa3f8dff700 1 -- 172.21.15.22:0/3761395679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f4107d60 con 0x7fa3f4100de0 2023-09-20T21:48:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.180+0000 7fa3e37fe700 1 -- 172.21.15.22:0/3761395679 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa3e400cc50 con 0x7fa3f4100de0 2023-09-20T21:48:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.181+0000 7fa3e37fe700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa3dc0a1350 0x7fa3dc0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.181+0000 7fa3e37fe700 1 -- 172.21.15.22:0/3761395679 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa3e4014070 con 0x7fa3f4100de0 2023-09-20T21:48:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.181+0000 7fa3f1d9b700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa3dc0a1350 0x7fa3dc0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:40.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.182+0000 7fa3f1d9b700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa3dc0a1350 0x7fa3dc0a3810 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800fa40 tx=0x7fa3e8019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:40.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.183+0000 7fa3e37fe700 1 -- 172.21.15.22:0/3761395679 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa3e4096630 con 0x7fa3f4100de0 2023-09-20T21:48:40.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.441+0000 7fa3f8dff700 1 -- 172.21.15.22:0/3761395679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3f410c880 con 0x7fa3f4100de0 2023-09-20T21:48:40.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.442+0000 7fa3e37fe700 1 -- 172.21.15.22:0/3761395679 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11361 (secure 0 0 0) 0x7fa3e409ee60 con 0x7fa3f4100de0 2023-09-20T21:48:40.443 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:40.443 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:47:09.607119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T21:48:40.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.444+0000 7fa3f8dff700 1 -- 172.21.15.22:0/3761395679 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa3dc0a1350 msgr2=0x7fa3dc0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:40.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.444+0000 7fa3f8dff700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa3dc0a1350 0x7fa3dc0a3810 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800fa40 tx=0x7fa3e8019040 comp rx=0 tx=0).stop 2023-09-20T21:48:40.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.444+0000 7fa3f8dff700 1 -- 172.21.15.22:0/3761395679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4100de0 msgr2=0x7fa3f4196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:40.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.444+0000 7fa3f8dff700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4100de0 0x7fa3f4196b00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400e970 tx=0x7fa3e400ec80 comp rx=0 tx=0).stop 2023-09-20T21:48:40.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.444+0000 7fa3f8dff700 1 -- 172.21.15.22:0/3761395679 shutdown_connections 2023-09-20T21:48:40.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.444+0000 7fa3f8dff700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa3f4102970 0x7fa3f419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.444+0000 7fa3f8dff700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa3dc0a1350 0x7fa3dc0a3810 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.444+0000 7fa3f8dff700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4100de0 0x7fa3f4196b00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.444+0000 7fa3f8dff700 1 --2- 172.21.15.22:0/3761395679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f4101fd0 0x7fa3f4197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.444+0000 7fa3f8dff700 1 -- 172.21.15.22:0/3761395679 >> 172.21.15.22:0/3761395679 conn(0x7fa3f40fc310 msgr2=0x7fa3f40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:40.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.445+0000 7fa3f8dff700 1 -- 172.21.15.22:0/3761395679 shutdown_connections 2023-09-20T21:48:40.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.445+0000 7fa3f8dff700 1 -- 172.21.15.22:0/3761395679 wait complete. 2023-09-20T21:48:40.447 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2023-09-20T21:48:40.464 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:48:40.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.670+0000 7f68711ff700 1 -- 172.21.15.22:0/3390709167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f686c100df0 msgr2=0x7f686c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:40.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.670+0000 7f68711ff700 1 --2- 172.21.15.22:0/3390709167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f686c100df0 0x7f686c101210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f685c009990 tx=0x7f685c009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.670+0000 7f68711ff700 1 -- 172.21.15.22:0/3390709167 shutdown_connections 2023-09-20T21:48:40.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.670+0000 7f68711ff700 1 --2- 172.21.15.22:0/3390709167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f686c102980 0x7f686c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.670+0000 7f68711ff700 1 --2- 172.21.15.22:0/3390709167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f686c100df0 0x7f686c101210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.670+0000 7f68711ff700 1 --2- 172.21.15.22:0/3390709167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f686c101fe0 0x7f686c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.670+0000 7f68711ff700 1 -- 172.21.15.22:0/3390709167 >> 172.21.15.22:0/3390709167 conn(0x7f686c0fc320 msgr2=0x7f686c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:40.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.671+0000 7f68711ff700 1 -- 172.21.15.22:0/3390709167 shutdown_connections 2023-09-20T21:48:40.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.671+0000 7f68711ff700 1 -- 172.21.15.22:0/3390709167 wait complete. 2023-09-20T21:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f68711ff700 1 Processor -- start 2023-09-20T21:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f68711ff700 1 -- start start 2023-09-20T21:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f68711ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f686c100df0 0x7f686c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f68711ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f686c101fe0 0x7f686c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:40.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f68711ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f686c102980 0x7f686c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:40.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f68711ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1a07c0 con 0x7f686c101fe0 2023-09-20T21:48:40.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f68711ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1a0930 con 0x7f686c102980 2023-09-20T21:48:40.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f68711ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1a0af0 con 0x7f686c100df0 2023-09-20T21:48:40.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f686a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f686c101fe0 0x7f686c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:40.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f686a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f686c101fe0 0x7f686c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47784/0 (socket says 172.21.15.22:47784) 2023-09-20T21:48:40.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f686a59c700 1 -- 172.21.15.22:0/2517722332 learned_addr learned my addr 172.21.15.22:0/2517722332 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:40.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f686ad9d700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f686c100df0 0x7f686c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:40.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.672+0000 7f686b59e700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f686c102980 0x7f686c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:40.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f686a59c700 1 -- 172.21.15.22:0/2517722332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f686c100df0 msgr2=0x7f686c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:40.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f686a59c700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f686c100df0 0x7f686c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f686a59c700 1 -- 172.21.15.22:0/2517722332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f686c102980 msgr2=0x7f686c1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:40.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f686a59c700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f686c102980 0x7f686c1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f686a59c700 1 -- 172.21.15.22:0/2517722332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0095f0 con 0x7f686c101fe0 2023-09-20T21:48:40.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f686ad9d700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f686c100df0 0x7f686c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:40.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f686a59c700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f686c101fe0 0x7f686c1991c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f686000e9a0 tx=0x7f686000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:40.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f6853fff700 1 -- 172.21.15.22:0/2517722332 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860009790 con 0x7f686c101fe0 2023-09-20T21:48:40.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f68711ff700 1 -- 172.21.15.22:0/2517722332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c1a0dd0 con 0x7f686c101fe0 2023-09-20T21:48:40.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f6853fff700 1 -- 172.21.15.22:0/2517722332 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6860010430 con 0x7f686c101fe0 2023-09-20T21:48:40.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.673+0000 7f6853fff700 1 -- 172.21.15.22:0/2517722332 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860018680 con 0x7f686c101fe0 2023-09-20T21:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.674+0000 7f68711ff700 1 -- 172.21.15.22:0/2517722332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c1a1220 con 0x7f686c101fe0 2023-09-20T21:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.675+0000 7f68711ff700 1 -- 172.21.15.22:0/2517722332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c1933f0 con 0x7f686c101fe0 2023-09-20T21:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.677+0000 7f6853fff700 1 -- 172.21.15.22:0/2517722332 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f68600105a0 con 0x7f686c101fe0 2023-09-20T21:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.678+0000 7f6853fff700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68540a1350 0x7f68540a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.678+0000 7f6853fff700 1 -- 172.21.15.22:0/2517722332 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6860014070 con 0x7f686c101fe0 2023-09-20T21:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.678+0000 7f686ad9d700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68540a1350 0x7f68540a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.679+0000 7f686ad9d700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68540a1350 0x7f68540a3810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f686c19a150 tx=0x7f685c005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:40.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.682+0000 7f6853fff700 1 -- 172.21.15.22:0/2517722332 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f6860051e10 con 0x7f686c101fe0 2023-09-20T21:48:40.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:40 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1521677972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:40.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:40 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3761395679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:40.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.897+0000 7f68711ff700 1 -- 172.21.15.22:0/2517722332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f686c10c600 con 0x7f686c101fe0 2023-09-20T21:48:40.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.898+0000 7f6853fff700 1 -- 172.21.15.22:0/2517722332 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7f6860096470 con 0x7f686c101fe0 2023-09-20T21:48:40.899 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:40.900 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:48:40.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 -- 172.21.15.22:0/2517722332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68540a1350 msgr2=0x7f68540a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:40.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68540a1350 0x7f68540a3810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f686c19a150 tx=0x7f685c005fd0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 -- 172.21.15.22:0/2517722332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f686c101fe0 msgr2=0x7f686c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:40.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f686c101fe0 0x7f686c1991c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f686000e9a0 tx=0x7f686000ecb0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 -- 172.21.15.22:0/2517722332 shutdown_connections 2023-09-20T21:48:40.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68540a1350 0x7f68540a3810 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f686c102980 0x7f686c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f686c100df0 0x7f686c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 --2- 172.21.15.22:0/2517722332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f686c101fe0 0x7f686c1991c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:40.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 -- 172.21.15.22:0/2517722332 >> 172.21.15.22:0/2517722332 conn(0x7f686c0fc320 msgr2=0x7f686c107890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:40.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 -- 172.21.15.22:0/2517722332 shutdown_connections 2023-09-20T21:48:40.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:40.901+0000 7f68711ff700 1 -- 172.21.15.22:0/2517722332 wait complete. 2023-09-20T21:48:40.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-09-20T21:46:41.044280+0000', 'modified': '2023-09-20T21:47:09.607119+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24578, 'mds_1': 24566, 'mds_2': 24530, 'mds_3': 24487, 'mds_4': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24461': {'gid': 24461, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6833/3516975495', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6832', 'nonce': 3516975495}, {'type': 'v1', 'addr': '172.21.15.89:6833', 'nonce': 3516975495}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'd', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.22:6837/4039103472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6836', 'nonce': 4039103472}, {'type': 'v1', 'addr': '172.21.15.22:6837', 'nonce': 4039103472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24473': {'gid': 24473, 'name': 'j', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.22:6835/56798476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6834', 'nonce': 56798476}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 56798476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.123:6835/835271994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 835271994}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 835271994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24497': {'gid': 24497, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6835/794452586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6834', 'nonce': 794452586}, {'type': 'v1', 'addr': '172.21.15.89:6835', 'nonce': 794452586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6837/1168160038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 1168160038}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 1168160038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6839/2368757728', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 2368757728}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 2368757728}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.89:6837/3887702690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 3887702690}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 3887702690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24566': {'gid': 24566, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.22:6841/1806616417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6840', 'nonce': 1806616417}, {'type': 'v1', 'addr': '172.21.15.22:6841', 'nonce': 1806616417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.89:6839/2165264535', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 2165264535}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 2165264535}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-20T21:48:40.922 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-09-20T21:48:40.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.j status 2023-09-20T21:48:41.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:40 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1521677972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:41.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:40 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3761395679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:41.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:40 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1521677972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:41.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:40 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3761395679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:41.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.126+0000 7f65623ff700 1 -- 172.21.15.22:0/350143921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f655c101850 msgr2=0x7f655c103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.126+0000 7f65623ff700 1 --2- 172.21.15.22:0/350143921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f655c101850 0x7f655c103c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f654c0098f0 tx=0x7f654c009c00 comp rx=0 tx=0).stop 2023-09-20T21:48:41.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.126+0000 7f65623ff700 1 -- 172.21.15.22:0/350143921 shutdown_connections 2023-09-20T21:48:41.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.126+0000 7f65623ff700 1 --2- 172.21.15.22:0/350143921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f655c106ab0 0x7f655c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.126+0000 7f65623ff700 1 --2- 172.21.15.22:0/350143921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f655c101850 0x7f655c103c40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.126+0000 7f65623ff700 1 --2- 172.21.15.22:0/350143921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c104180 0x7f655c106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.126+0000 7f65623ff700 1 -- 172.21.15.22:0/350143921 >> 172.21.15.22:0/350143921 conn(0x7f655c0fb240 msgr2=0x7f655c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.126+0000 7f65623ff700 1 -- 172.21.15.22:0/350143921 shutdown_connections 2023-09-20T21:48:41.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.126+0000 7f65623ff700 1 -- 172.21.15.22:0/350143921 wait complete. 2023-09-20T21:48:41.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.127+0000 7f65623ff700 1 Processor -- start 2023-09-20T21:48:41.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f65623ff700 1 -- start start 2023-09-20T21:48:41.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f65623ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c101850 0x7f655c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f65623ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f655c104180 0x7f655c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f65623ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f655c106ab0 0x7f655c1a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f65623ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c1a0890 con 0x7f655c106ab0 2023-09-20T21:48:41.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f65623ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c1a0a00 con 0x7f655c101850 2023-09-20T21:48:41.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f65623ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c1a0bc0 con 0x7f655c104180 2023-09-20T21:48:41.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f656099c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f655c106ab0 0x7f655c1a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f655bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c101850 0x7f655c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f656099c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f655c106ab0 0x7f655c1a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47800/0 (socket says 172.21.15.22:47800) 2023-09-20T21:48:41.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f655bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c101850 0x7f655c198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43400/0 (socket says 172.21.15.22:43400) 2023-09-20T21:48:41.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f656099c700 1 -- 172.21.15.22:0/1488055175 learned_addr learned my addr 172.21.15.22:0/1488055175 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.128+0000 7f655b7fe700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f655c104180 0x7f655c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.129+0000 7f655bfff700 1 -- 172.21.15.22:0/1488055175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f655c104180 msgr2=0x7f655c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.129+0000 7f655bfff700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f655c104180 0x7f655c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.129+0000 7f655bfff700 1 -- 172.21.15.22:0/1488055175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f655c106ab0 msgr2=0x7f655c1a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.129+0000 7f655bfff700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f655c106ab0 0x7f655c1a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.129+0000 7f655bfff700 1 -- 172.21.15.22:0/1488055175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500095f0 con 0x7f655c101850 2023-09-20T21:48:41.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.129+0000 7f655bfff700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c101850 0x7f655c198d50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f654c009230 tx=0x7f654c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.129+0000 7f65597fa700 1 -- 172.21.15.22:0/1488055175 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c01c070 con 0x7f655c101850 2023-09-20T21:48:41.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.129+0000 7f65623ff700 1 -- 172.21.15.22:0/1488055175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654c009520 con 0x7f655c101850 2023-09-20T21:48:41.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.129+0000 7f65597fa700 1 -- 172.21.15.22:0/1488055175 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f654c00f7d0 con 0x7f655c101850 2023-09-20T21:48:41.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.129+0000 7f65597fa700 1 -- 172.21.15.22:0/1488055175 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c003a00 con 0x7f655c101850 2023-09-20T21:48:41.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.130+0000 7f65623ff700 1 -- 172.21.15.22:0/1488055175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655c1a10d0 con 0x7f655c101850 2023-09-20T21:48:41.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.134+0000 7f65597fa700 1 -- 172.21.15.22:0/1488055175 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f654c020030 con 0x7f655c101850 2023-09-20T21:48:41.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.135+0000 7f6575ddcb80 1 Processor -- start 2023-09-20T21:48:41.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.135+0000 7f6575ddcb80 1 -- start start 2023-09-20T21:48:41.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d0f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 0x562e1d0f8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8b90 0x562e1d0f9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1d06bc50 con 0x562e1d0f8000 2023-09-20T21:48:41.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1d04fc20 con 0x562e1d0f8b90 2023-09-20T21:48:41.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1d04d540 con 0x562e1d0f76a0 2023-09-20T21:48:41.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f65597fa700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65440a1300 0x7f65440a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f655bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 0x562e1d0f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f656099c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d0f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f655b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8b90 0x562e1d0f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f65597fa700 1 -- 172.21.15.22:0/1488055175 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f654c0cde50 con 0x7f655c101850 2023-09-20T21:48:41.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f656099c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d0f7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60648/0 (socket says 172.21.15.22:60648) 2023-09-20T21:48:41.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f656099c700 1 -- 172.21.15.22:0/3528702497 learned_addr learned my addr 172.21.15.22:0/3528702497 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f656099c700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65440a1300 0x7f65440a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f655b7fe700 1 -- 172.21.15.22:0/3528702497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 msgr2=0x562e1d0f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.136+0000 7f655b7fe700 1 --2- 172.21.15.22:0/3528702497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d0f7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.137+0000 7f655b7fe700 1 -- 172.21.15.22:0/3528702497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 msgr2=0x562e1d0f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.137+0000 7f655b7fe700 1 --2- 172.21.15.22:0/3528702497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 0x562e1d0f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.137+0000 7f655b7fe700 1 -- 172.21.15.22:0/3528702497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e1ce73d90 con 0x562e1d0f8b90 2023-09-20T21:48:41.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.137+0000 7f655b7fe700 1 --2- 172.21.15.22:0/3528702497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8b90 0x562e1d0f9010 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6550009f20 tx=0x7f6550011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58d362ffd8c4ffb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.137+0000 7f656099c700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65440a1300 0x7f65440a37c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6554009d30 tx=0x7f655400b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.137+0000 7f65427fc700 1 -- 172.21.15.22:0/3528702497 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550004310 con 0x562e1d0f8b90 2023-09-20T21:48:41.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.137+0000 7f65427fc700 1 -- 172.21.15.22:0/3528702497 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6550004470 con 0x562e1d0f8b90 2023-09-20T21:48:41.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.137+0000 7f65427fc700 1 -- 172.21.15.22:0/3528702497 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550019910 con 0x562e1d0f8b90 2023-09-20T21:48:41.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.138+0000 7f6575ddcb80 1 -- 172.21.15.22:0/3528702497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8b90 msgr2=0x562e1d0f9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.138+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/3528702497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8b90 0x562e1d0f9010 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6550009f20 tx=0x7f6550011720 comp rx=0 tx=0).stop 2023-09-20T21:48:41.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.138+0000 7f6575ddcb80 1 -- 172.21.15.22:0/3528702497 shutdown_connections 2023-09-20T21:48:41.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.138+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/3528702497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8b90 0x562e1d0f9010 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.138+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/3528702497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d0f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.138+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/3528702497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 0x562e1d0f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.138+0000 7f6575ddcb80 1 -- 172.21.15.22:0/3528702497 >> 172.21.15.22:0/3528702497 conn(0x562e1d023fb0 msgr2=0x562e1d0f2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.138+0000 7f6575ddcb80 1 -- 172.21.15.22:0/3528702497 shutdown_connections 2023-09-20T21:48:41.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.138+0000 7f6575ddcb80 1 -- 172.21.15.22:0/3528702497 wait complete. 2023-09-20T21:48:41.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.139+0000 7f6575ddcb80 1 -- start start 2023-09-20T21:48:41.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.139+0000 7f6575ddcb80 1 Processor -- start 2023-09-20T21:48:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d101170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 0x562e1d1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8b90 0x562e1d1086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1d048e30 con 0x562e1d0f8b90 2023-09-20T21:48:41.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1cfe56f0 con 0x562e1d0f8000 2023-09-20T21:48:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1cfe5390 con 0x562e1d0f76a0 2023-09-20T21:48:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f6575ddcb80 10 client.0 _ll_register_callbacks cb 0x562e1cf2fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f655bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f656099c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8b90 0x562e1d1086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f655b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 0x562e1d1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f655bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d101170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60660/0 (socket says 172.21.15.22:60660) 2023-09-20T21:48:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f656099c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8b90 0x562e1d1086f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47828/0 (socket says 172.21.15.22:47828) 2023-09-20T21:48:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f656099c700 1 -- 172.21.15.22:0/529875886 learned_addr learned my addr 172.21.15.22:0/529875886 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f655b7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 0x562e1d1016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43418/0 (socket says 172.21.15.22:43418) 2023-09-20T21:48:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f656099c700 1 -- 172.21.15.22:0/529875886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 msgr2=0x562e1d101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f656099c700 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f656099c700 1 -- 172.21.15.22:0/529875886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 msgr2=0x562e1d1016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f656099c700 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 0x562e1d1016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f656099c700 1 -- 172.21.15.22:0/529875886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6550009b80 con 0x562e1d0f8b90 2023-09-20T21:48:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.140+0000 7f655bfff700 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.141+0000 7f656099c700 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8b90 0x562e1d1086f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6554009be0 tx=0x7f6554004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3058b5d75eb28baf server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.141+0000 7f653b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:41.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.141+0000 7f653b7fe700 1 -- 172.21.15.22:0/529875886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554029070 con 0x562e1d0f8b90 2023-09-20T21:48:41.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.141+0000 7f6575ddcb80 1 -- 172.21.15.22:0/529875886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e1cfb9b60 con 0x562e1d0f8b90 2023-09-20T21:48:41.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.141+0000 7f653b7fe700 1 -- 172.21.15.22:0/529875886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f655401be10 con 0x562e1d0f8b90 2023-09-20T21:48:41.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.141+0000 7f653b7fe700 1 -- 172.21.15.22:0/529875886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f655400e070 con 0x562e1d0f8b90 2023-09-20T21:48:41.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.141+0000 7f6575ddcb80 10 client.15291 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:41.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.141+0000 7f6575ddcb80 1 -- 172.21.15.22:0/529875886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562e1ce5cc10 con 0x562e1d0f8b90 2023-09-20T21:48:41.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.142+0000 7f653b7fe700 1 -- 172.21.15.22:0/529875886 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e1cfb9b60 con 0x562e1d0f8b90 2023-09-20T21:48:41.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.142+0000 7f653b7fe700 1 -- 172.21.15.22:0/529875886 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f655402c030 con 0x562e1d0f8b90 2023-09-20T21:48:41.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.142+0000 7f6575ddcb80 10 client.15291 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:41.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.142+0000 7f6575ddcb80 10 client.15291 resolve_mds: resolved name 'j' to daemon mds.j 2023-09-20T21:48:41.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.142+0000 7f6575ddcb80 20 client.15291 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.142+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x562e1d10cda0 0x562e1d10b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.142+0000 7f6575ddcb80 4 client.15291 mds_command: new command op to 24473 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.142+0000 7f6575ddcb80 1 -- 172.21.15.22:0/529875886 --> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e1d0c5fd0 con 0x562e1d10cda0 2023-09-20T21:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.142+0000 7f655bfff700 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x562e1d10cda0 0x562e1d10b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.143+0000 7f653b7fe700 10 client.15291 ms_handle_connect on v2:172.21.15.22:6834/56798476 2023-09-20T21:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.143+0000 7f655bfff700 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x562e1d10cda0 0x562e1d10b7f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a83adeacc0f95d1d server_cookie=67b9f3e5a390a37b in_seq=0 out_seq=0 2023-09-20T21:48:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.143+0000 7f653b7fe700 1 -- 172.21.15.22:0/529875886 <== mds.4 v2:172.21.15.22:6834/56798476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562e1d0c5fd0 con 0x562e1d10cda0 2023-09-20T21:48:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.143+0000 7f653b7fe700 10 client.15291 handle_command_reply: tid=0 2023-09-20T21:48:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.143+0000 7f6575ddcb80 1 client.15291 shutdown 2023-09-20T21:48:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 1 -- 172.21.15.22:0/529875886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8b90 msgr2=0x562e1d1086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8b90 0x562e1d1086f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6554009be0 tx=0x7f6554004ad0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 1 -- 172.21.15.22:0/529875886 shutdown_connections 2023-09-20T21:48:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x562e1d10cda0 0x562e1d10b7f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8b90 0x562e1d1086f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f76a0 0x562e1d101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/529875886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 0x562e1d1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f653b7fe700 0 client.15291 ms_handle_reset on v2:172.21.15.22:6834/56798476 2023-09-20T21:48:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 1 -- 172.21.15.22:0/529875886 >> 172.21.15.22:0/529875886 conn(0x562e1d023fb0 msgr2=0x562e1d0f1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 1 -- 172.21.15.22:0/529875886 shutdown_connections 2023-09-20T21:48:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 1 -- 172.21.15.22:0/529875886 wait complete. 2023-09-20T21:48:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.144+0000 7f6575ddcb80 20 client.15291 trim_cache size 0 max 16384 2023-09-20T21:48:41.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.162+0000 7f6575ddcb80 1 Processor -- start 2023-09-20T21:48:41.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.162+0000 7f6575ddcb80 1 -- start start 2023-09-20T21:48:41.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f76a0 0x562e1d0fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 0x562e1d0fc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f8b90 0x562e1d108610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1cec98d0 con 0x562e1d0f8000 2023-09-20T21:48:41.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1d102360 con 0x562e1d0f76a0 2023-09-20T21:48:41.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1cfef470 con 0x562e1d0f8b90 2023-09-20T21:48:41.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f655b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 0x562e1d0fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f655b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 0x562e1d0fc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47834/0 (socket says 172.21.15.22:47834) 2023-09-20T21:48:41.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f655b7fe700 1 -- 172.21.15.22:0/2773924668 learned_addr learned my addr 172.21.15.22:0/2773924668 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f655bfff700 1 --2- 172.21.15.22:0/2773924668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f76a0 0x562e1d0fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f656099c700 1 --2- 172.21.15.22:0/2773924668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f8b90 0x562e1d108610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f655b7fe700 1 -- 172.21.15.22:0/2773924668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f8b90 msgr2=0x562e1d108610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.163+0000 7f655b7fe700 1 --2- 172.21.15.22:0/2773924668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f8b90 0x562e1d108610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.164+0000 7f655b7fe700 1 -- 172.21.15.22:0/2773924668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f76a0 msgr2=0x562e1d0fbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.164+0000 7f655b7fe700 1 --2- 172.21.15.22:0/2773924668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f76a0 0x562e1d0fbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.164+0000 7f655b7fe700 1 -- 172.21.15.22:0/2773924668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e1ce9a200 con 0x562e1d0f8000 2023-09-20T21:48:41.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.164+0000 7f655bfff700 1 --2- 172.21.15.22:0/2773924668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f76a0 0x562e1d0fbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:41.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.164+0000 7f655b7fe700 1 --2- 172.21.15.22:0/2773924668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 0x562e1d0fc010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6550009f20 tx=0x7f6550005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bd10249fea0e5e server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.164+0000 7f6542ffd700 1 -- 172.21.15.22:0/2773924668 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550004110 con 0x562e1d0f8000 2023-09-20T21:48:41.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.164+0000 7f6542ffd700 1 -- 172.21.15.22:0/2773924668 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6550004270 con 0x562e1d0f8000 2023-09-20T21:48:41.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.164+0000 7f6542ffd700 1 -- 172.21.15.22:0/2773924668 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550024a50 con 0x562e1d0f8000 2023-09-20T21:48:41.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 -- 172.21.15.22:0/2773924668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 msgr2=0x562e1d0fc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/2773924668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 0x562e1d0fc010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6550009f20 tx=0x7f6550005ee0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 -- 172.21.15.22:0/2773924668 shutdown_connections 2023-09-20T21:48:41.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/2773924668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f8b90 0x562e1d108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/2773924668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f76a0 0x562e1d0fbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/2773924668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f8000 0x562e1d0fc010 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 -- 172.21.15.22:0/2773924668 >> 172.21.15.22:0/2773924668 conn(0x562e1d023fb0 msgr2=0x562e1d0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 -- 172.21.15.22:0/2773924668 shutdown_connections 2023-09-20T21:48:41.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 -- 172.21.15.22:0/2773924668 wait complete. 2023-09-20T21:48:41.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 -- start start 2023-09-20T21:48:41.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.165+0000 7f6575ddcb80 1 Processor -- start 2023-09-20T21:48:41.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f76a0 0x562e1d1084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 0x562e1d104f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f6575ddcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f8b90 0x562e1d105480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1d1059c0 con 0x562e1d0f76a0 2023-09-20T21:48:41.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1d105b30 con 0x562e1d0f8000 2023-09-20T21:48:41.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f6575ddcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e1d160790 con 0x562e1d0f8b90 2023-09-20T21:48:41.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f6575ddcb80 10 client.0 _ll_register_callbacks cb 0x562e1cf2fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:41.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f655bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f76a0 0x562e1d1084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f655bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f76a0 0x562e1d1084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47840/0 (socket says 172.21.15.22:47840) 2023-09-20T21:48:41.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f655bfff700 1 -- 172.21.15.22:0/4145536944 learned_addr learned my addr 172.21.15.22:0/4145536944 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f655b7fe700 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 0x562e1d104f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f656099c700 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f8b90 0x562e1d105480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f655bfff700 1 -- 172.21.15.22:0/4145536944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f8b90 msgr2=0x562e1d105480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f655bfff700 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f8b90 0x562e1d105480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f655bfff700 1 -- 172.21.15.22:0/4145536944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 msgr2=0x562e1d104f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f655bfff700 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 0x562e1d104f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.166+0000 7f655bfff700 1 -- 172.21.15.22:0/4145536944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655001e040 con 0x562e1d0f76a0 2023-09-20T21:48:41.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.167+0000 7f655bfff700 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f76a0 0x562e1d1084a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f654c00c010 tx=0x7f654c00bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad543badb4e5c7a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.167+0000 7f653b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:41.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.167+0000 7f653b7fe700 1 -- 172.21.15.22:0/4145536944 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0a2e60 con 0x562e1d0f76a0 2023-09-20T21:48:41.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.167+0000 7f653b7fe700 1 -- 172.21.15.22:0/4145536944 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f654c0cb670 con 0x562e1d0f76a0 2023-09-20T21:48:41.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.167+0000 7f6575ddcb80 1 -- 172.21.15.22:0/4145536944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6550004470 con 0x562e1d0f76a0 2023-09-20T21:48:41.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.167+0000 7f653b7fe700 1 -- 172.21.15.22:0/4145536944 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f654c00b900 con 0x562e1d0f76a0 2023-09-20T21:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.167+0000 7f6575ddcb80 10 client.15297 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.167+0000 7f6575ddcb80 1 -- 172.21.15.22:0/4145536944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562e1ce72260 con 0x562e1d0f76a0 2023-09-20T21:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.167+0000 7f653b7fe700 1 -- 172.21.15.22:0/4145536944 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6550004470 con 0x562e1d0f76a0 2023-09-20T21:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.168+0000 7f653b7fe700 1 -- 172.21.15.22:0/4145536944 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f654c09b170 con 0x562e1d0f76a0 2023-09-20T21:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.168+0000 7f6575ddcb80 10 client.15297 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.168+0000 7f6575ddcb80 10 client.15297 resolve_mds: resolved name 'j' to daemon mds.j 2023-09-20T21:48:41.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.168+0000 7f6575ddcb80 20 client.15297 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:41.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.168+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x562e1d1656b0 0x562e1d161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.168+0000 7f6575ddcb80 4 client.15297 mds_command: new command op to 24473 tid=0[{"prefix": "status"}] 2023-09-20T21:48:41.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.168+0000 7f6575ddcb80 1 -- 172.21.15.22:0/4145536944 --> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562e1d041e10 con 0x562e1d1656b0 2023-09-20T21:48:41.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.168+0000 7f655b7fe700 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x562e1d1656b0 0x562e1d161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f655b7fe700 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x562e1d1656b0 0x562e1d161b50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=307f8d5bca35a5a4 server_cookie=e073c74120719428 in_seq=0 out_seq=0 2023-09-20T21:48:41.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f653b7fe700 10 client.15297 ms_handle_connect on v2:172.21.15.22:6834/56798476 2023-09-20T21:48:41.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f653b7fe700 1 -- 172.21.15.22:0/4145536944 <== mds.4 v2:172.21.15.22:6834/56798476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562e1d041e10 con 0x562e1d1656b0 2023-09-20T21:48:41.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f653b7fe700 10 client.15297 handle_command_reply: tid=0 2023-09-20T21:48:41.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 client.15297 shutdown 2023-09-20T21:48:41.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 -- 172.21.15.22:0/4145536944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f76a0 msgr2=0x562e1d1084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f76a0 0x562e1d1084a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f654c00c010 tx=0x7f654c00bbd0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 -- 172.21.15.22:0/4145536944 shutdown_connections 2023-09-20T21:48:41.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562e1d0f8b90 0x562e1d105480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x562e1d1656b0 0x562e1d161b50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562e1d0f76a0 0x562e1d1084a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 --2- 172.21.15.22:0/4145536944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562e1d0f8000 0x562e1d104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 -- 172.21.15.22:0/4145536944 >> 172.21.15.22:0/4145536944 conn(0x562e1d023fb0 msgr2=0x562e1d0fbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f653b7fe700 0 client.15297 ms_handle_reset on v2:172.21.15.22:6834/56798476 2023-09-20T21:48:41.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 -- 172.21.15.22:0/4145536944 shutdown_connections 2023-09-20T21:48:41.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.169+0000 7f6575ddcb80 1 -- 172.21.15.22:0/4145536944 wait complete. 2023-09-20T21:48:41.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f6575ddcb80 20 client.15297 trim_cache size 0 max 16384 2023-09-20T21:48:41.184 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:48:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 4, 2023-09-20T21:48:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24473, 2023-09-20T21:48:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:48:41.185 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:48:41.185 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:48:41.185 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 92.602985359000002, 2023-09-20T21:48:41.185 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 28, 2023-09-20T21:48:41.185 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2023-09-20T21:48:41.185 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 91, 2023-09-20T21:48:41.186 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 149.13193869200001 2023-09-20T21:48:41.186 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:41.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 -- 172.21.15.22:0/1488055175 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65440a1300 msgr2=0x7f65440a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65440a1300 0x7f65440a37c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6554009d30 tx=0x7f655400b040 comp rx=0 tx=0).stop 2023-09-20T21:48:41.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 -- 172.21.15.22:0/1488055175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c101850 msgr2=0x7f655c198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c101850 0x7f655c198d50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f654c009230 tx=0x7f654c00b790 comp rx=0 tx=0).stop 2023-09-20T21:48:41.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 -- 172.21.15.22:0/1488055175 shutdown_connections 2023-09-20T21:48:41.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f655c106ab0 0x7f655c1a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f65440a1300 0x7f65440a37c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f655c101850 0x7f655c198d50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 --2- 172.21.15.22:0/1488055175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f655c104180 0x7f655c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 -- 172.21.15.22:0/1488055175 >> 172.21.15.22:0/1488055175 conn(0x7f655c0fb240 msgr2=0x7f655c1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 -- 172.21.15.22:0/1488055175 shutdown_connections 2023-09-20T21:48:41.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.170+0000 7f65623ff700 1 -- 172.21.15.22:0/1488055175 wait complete. 2023-09-20T21:48:41.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-09-20T21:48:41.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.393+0000 7f3692bff700 1 -- 172.21.15.22:0/2878696293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c074e20 msgr2=0x7f368c073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.393+0000 7f3692bff700 1 --2- 172.21.15.22:0/2878696293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c074e20 0x7f368c073270 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3680009990 tx=0x7f3680009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.393+0000 7f3692bff700 1 -- 172.21.15.22:0/2878696293 shutdown_connections 2023-09-20T21:48:41.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.393+0000 7f3692bff700 1 --2- 172.21.15.22:0/2878696293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c108cb0 0x7f368c10b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.393+0000 7f3692bff700 1 --2- 172.21.15.22:0/2878696293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c074e20 0x7f368c073270 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.393+0000 7f3692bff700 1 --2- 172.21.15.22:0/2878696293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368c073840 0x7f368c073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.393+0000 7f3692bff700 1 -- 172.21.15.22:0/2878696293 >> 172.21.15.22:0/2878696293 conn(0x7f368c0fc360 msgr2=0x7f368c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.393+0000 7f3692bff700 1 -- 172.21.15.22:0/2878696293 shutdown_connections 2023-09-20T21:48:41.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.393+0000 7f3692bff700 1 -- 172.21.15.22:0/2878696293 wait complete. 2023-09-20T21:48:41.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.394+0000 7f3692bff700 1 Processor -- start 2023-09-20T21:48:41.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f3692bff700 1 -- start start 2023-09-20T21:48:41.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f3692bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368c073840 0x7f368c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f3692bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c074e20 0x7f368c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f3692bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c108cb0 0x7f368c1a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f3692bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1a08d0 con 0x7f368c073840 2023-09-20T21:48:41.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f3692bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1a0a10 con 0x7f368c108cb0 2023-09-20T21:48:41.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f3692bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1a0ba0 con 0x7f368c074e20 2023-09-20T21:48:41.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f369099b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368c073840 0x7f368c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f368bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c074e20 0x7f368c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f369119c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c108cb0 0x7f368c1a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f369099b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368c073840 0x7f368c198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47870/0 (socket says 172.21.15.22:47870) 2023-09-20T21:48:41.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f369119c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c108cb0 0x7f368c1a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43466/0 (socket says 172.21.15.22:43466) 2023-09-20T21:48:41.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f369099b700 1 -- 172.21.15.22:0/2996680862 learned_addr learned my addr 172.21.15.22:0/2996680862 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.395+0000 7f368bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c074e20 0x7f368c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60706/0 (socket says 172.21.15.22:60706) 2023-09-20T21:48:41.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f368bfff700 1 -- 172.21.15.22:0/2996680862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c108cb0 msgr2=0x7f368c1a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f368bfff700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c108cb0 0x7f368c1a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f368bfff700 1 -- 172.21.15.22:0/2996680862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368c073840 msgr2=0x7f368c198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f368bfff700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368c073840 0x7f368c198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f368bfff700 1 -- 172.21.15.22:0/2996680862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800095f0 con 0x7f368c074e20 2023-09-20T21:48:41.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f369099b700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368c073840 0x7f368c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:41.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f368bfff700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c074e20 0x7f368c199290 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f367800e9c0 tx=0x7f367800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f3689ffb700 1 -- 172.21.15.22:0/2996680862 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678004220 con 0x7f368c074e20 2023-09-20T21:48:41.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f3689ffb700 1 -- 172.21.15.22:0/2996680862 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3678004380 con 0x7f368c074e20 2023-09-20T21:48:41.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f3692bff700 1 -- 172.21.15.22:0/2996680862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c1a0e80 con 0x7f368c074e20 2023-09-20T21:48:41.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.396+0000 7f3689ffb700 1 -- 172.21.15.22:0/2996680862 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678003be0 con 0x7f368c074e20 2023-09-20T21:48:41.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.397+0000 7f3692bff700 1 -- 172.21.15.22:0/2996680862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c1a1350 con 0x7f368c074e20 2023-09-20T21:48:41.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.400+0000 7f3689ffb700 1 -- 172.21.15.22:0/2996680862 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f3678003d40 con 0x7f368c074e20 2023-09-20T21:48:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.401+0000 7f36a64d3b80 1 Processor -- start 2023-09-20T21:48:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.401+0000 7f36a64d3b80 1 -- start start 2023-09-20T21:48:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.401+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe641ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.401+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe642590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.401+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 0x55b4fe643010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.401+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe5b5c50 con 0x55b4fe6416a0 2023-09-20T21:48:41.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.401+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe599c20 con 0x55b4fe642000 2023-09-20T21:48:41.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.401+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe597540 con 0x55b4fe642b90 2023-09-20T21:48:41.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.401+0000 7f3689ffb700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f367c0a1380 0x7f367c0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369119c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe641ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f368bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 0x55b4fe643010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369099b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe642590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369119c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe641ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47874/0 (socket says 172.21.15.22:47874) 2023-09-20T21:48:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369119c700 1 -- 172.21.15.22:0/1639595160 learned_addr learned my addr 172.21.15.22:0/1639595160 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369119c700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f367c0a1380 0x7f367c0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f3689ffb700 1 -- 172.21.15.22:0/2996680862 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f3678014070 con 0x7f368c074e20 2023-09-20T21:48:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369099b700 1 -- 172.21.15.22:0/1639595160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 msgr2=0x55b4fe643010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369099b700 1 --2- 172.21.15.22:0/1639595160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 0x55b4fe643010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369099b700 1 -- 172.21.15.22:0/1639595160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 msgr2=0x55b4fe641ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369099b700 1 --2- 172.21.15.22:0/1639595160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe641ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369099b700 1 -- 172.21.15.22:0/1639595160 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4fe3bdd90 con 0x55b4fe642000 2023-09-20T21:48:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f368bfff700 1 --2- 172.21.15.22:0/1639595160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 0x55b4fe643010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.402+0000 7f369099b700 1 --2- 172.21.15.22:0/1639595160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe642590 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3680000c00 tx=0x7f3680005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260695005c26bc4d server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.403+0000 7f369119c700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f367c0a1380 0x7f367c0a3840 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3684008d40 tx=0x7f3684009100 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.403+0000 7f3676ffd700 1 -- 172.21.15.22:0/1639595160 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368001d050 con 0x55b4fe642000 2023-09-20T21:48:41.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.403+0000 7f3676ffd700 1 -- 172.21.15.22:0/1639595160 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f368001e040 con 0x55b4fe642000 2023-09-20T21:48:41.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.403+0000 7f3676ffd700 1 -- 172.21.15.22:0/1639595160 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680005460 con 0x55b4fe642000 2023-09-20T21:48:41.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.403+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1639595160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 msgr2=0x55b4fe642590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.403+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/1639595160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe642590 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3680000c00 tx=0x7f3680005ea0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.404+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1639595160 shutdown_connections 2023-09-20T21:48:41.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.404+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/1639595160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 0x55b4fe643010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.404+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/1639595160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe641ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.404+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/1639595160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe642590 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.404+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1639595160 >> 172.21.15.22:0/1639595160 conn(0x55b4fe56dfb0 msgr2=0x55b4fe63c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.404+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1639595160 shutdown_connections 2023-09-20T21:48:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.404+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1639595160 wait complete. 2023-09-20T21:48:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.404+0000 7f36a64d3b80 1 -- start start 2023-09-20T21:48:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.404+0000 7f36a64d3b80 1 Processor -- start 2023-09-20T21:48:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe64b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe64b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 0x55b4fe6526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe592e30 con 0x55b4fe6416a0 2023-09-20T21:48:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe52f6f0 con 0x55b4fe642000 2023-09-20T21:48:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe52f390 con 0x55b4fe642b90 2023-09-20T21:48:41.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f36a64d3b80 10 client.0 _ll_register_callbacks cb 0x55b4fe479a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:41.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f369099b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe64b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f369099b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe64b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47882/0 (socket says 172.21.15.22:47882) 2023-09-20T21:48:41.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f369099b700 1 -- 172.21.15.22:0/1352928480 learned_addr learned my addr 172.21.15.22:0/1352928480 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f368bfff700 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe64b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.405+0000 7f369119c700 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 0x55b4fe6526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f369099b700 1 -- 172.21.15.22:0/1352928480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 msgr2=0x55b4fe6526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f369099b700 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 0x55b4fe6526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f369099b700 1 -- 172.21.15.22:0/1352928480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 msgr2=0x55b4fe64b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f369099b700 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe64b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f369099b700 1 -- 172.21.15.22:0/1352928480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c1a1350 con 0x55b4fe6416a0 2023-09-20T21:48:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f369119c700 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 0x55b4fe6526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f369099b700 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe64b170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3680005000 tx=0x7f3680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b260bca89083865e server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f366bfff700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f366bfff700 1 -- 172.21.15.22:0/1352928480 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36800055b0 con 0x55b4fe6416a0 2023-09-20T21:48:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f366bfff700 1 -- 172.21.15.22:0/1352928480 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f3680023460 con 0x55b4fe6416a0 2023-09-20T21:48:41.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1352928480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4fe503b60 con 0x55b4fe6416a0 2023-09-20T21:48:41.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.406+0000 7f366bfff700 1 -- 172.21.15.22:0/1352928480 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3680005710 con 0x55b4fe6416a0 2023-09-20T21:48:41.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f36a64d3b80 10 client.15309 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:41.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1352928480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f368c10bd60 con 0x55b4fe6416a0 2023-09-20T21:48:41.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f366bfff700 1 -- 172.21.15.22:0/1352928480 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b4fe503b60 con 0x55b4fe6416a0 2023-09-20T21:48:41.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f366bfff700 1 -- 172.21.15.22:0/1352928480 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f368000f620 con 0x55b4fe6416a0 2023-09-20T21:48:41.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f36a64d3b80 10 client.15309 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:41.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f36a64d3b80 10 client.15309 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-20T21:48:41.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f36a64d3b80 20 client.15309 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:41.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55b4fe6567e0 0x55b4fe655620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f36a64d3b80 4 client.15309 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:41.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1352928480 --> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4fe60ffd0 con 0x55b4fe6567e0 2023-09-20T21:48:41.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.407+0000 7f368bfff700 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55b4fe6567e0 0x55b4fe655620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.408+0000 7f368bfff700 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55b4fe6567e0 0x55b4fe655620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a49d4e00147de05 server_cookie=213ab7f1271a240c in_seq=0 out_seq=0 2023-09-20T21:48:41.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.408+0000 7f366bfff700 10 client.15309 ms_handle_connect on v2:172.21.15.123:6834/835271994 2023-09-20T21:48:41.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f366bfff700 1 -- 172.21.15.22:0/1352928480 <== mds.3 v2:172.21.15.123:6834/835271994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b4fe60ffd0 con 0x55b4fe6567e0 2023-09-20T21:48:41.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f366bfff700 10 client.15309 handle_command_reply: tid=0 2023-09-20T21:48:41.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 client.15309 shutdown 2023-09-20T21:48:41.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1352928480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 msgr2=0x55b4fe64b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe64b170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3680005000 tx=0x7f3680004a40 comp rx=0 tx=0).stop 2023-09-20T21:48:41.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1352928480 shutdown_connections 2023-09-20T21:48:41.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55b4fe6567e0 0x55b4fe655620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642b90 0x55b4fe6526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe6416a0 0x55b4fe64b170 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f366bfff700 0 client.15309 ms_handle_reset on v2:172.21.15.123:6834/835271994 2023-09-20T21:48:41.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/1352928480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe64b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1352928480 >> 172.21.15.22:0/1352928480 conn(0x55b4fe56dfb0 msgr2=0x55b4fe63b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1352928480 shutdown_connections 2023-09-20T21:48:41.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.409+0000 7f36a64d3b80 1 -- 172.21.15.22:0/1352928480 wait complete. 2023-09-20T21:48:41.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.410+0000 7f36a64d3b80 20 client.15309 trim_cache size 0 max 16384 2023-09-20T21:48:41.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.428+0000 7f36a64d3b80 1 Processor -- start 2023-09-20T21:48:41.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.428+0000 7f36a64d3b80 1 -- start start 2023-09-20T21:48:41.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.428+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe6416a0 0x55b4fe655620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.428+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe6526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.428+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 0x55b4fe645b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.428+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe4138d0 con 0x55b4fe642b90 2023-09-20T21:48:41.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.428+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe503b60 con 0x55b4fe642000 2023-09-20T21:48:41.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.428+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe657570 con 0x55b4fe6416a0 2023-09-20T21:48:41.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f368bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe6526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f368bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe6526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43492/0 (socket says 172.21.15.22:43492) 2023-09-20T21:48:41.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f368bfff700 1 -- 172.21.15.22:0/3657501240 learned_addr learned my addr 172.21.15.22:0/3657501240 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f369099b700 1 --2- 172.21.15.22:0/3657501240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe6416a0 0x55b4fe655620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f369119c700 1 --2- 172.21.15.22:0/3657501240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 0x55b4fe645b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f368bfff700 1 -- 172.21.15.22:0/3657501240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe6416a0 msgr2=0x55b4fe655620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f368bfff700 1 --2- 172.21.15.22:0/3657501240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe6416a0 0x55b4fe655620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f368bfff700 1 -- 172.21.15.22:0/3657501240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 msgr2=0x55b4fe645b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f368bfff700 1 --2- 172.21.15.22:0/3657501240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 0x55b4fe645b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f368bfff700 1 -- 172.21.15.22:0/3657501240 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4fe3a6c10 con 0x55b4fe642000 2023-09-20T21:48:41.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f368bfff700 1 --2- 172.21.15.22:0/3657501240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe6526b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3678003ff0 tx=0x7f3678007550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b817aa1b530c3e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.429+0000 7f36777fe700 1 -- 172.21.15.22:0/3657501240 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36780044f0 con 0x55b4fe642000 2023-09-20T21:48:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36777fe700 1 -- 172.21.15.22:0/3657501240 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f367809a8c0 con 0x55b4fe642000 2023-09-20T21:48:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36777fe700 1 -- 172.21.15.22:0/3657501240 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36780caa40 con 0x55b4fe642000 2023-09-20T21:48:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3657501240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 msgr2=0x55b4fe6526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/3657501240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe6526b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3678003ff0 tx=0x7f3678007550 comp rx=0 tx=0).stop 2023-09-20T21:48:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3657501240 shutdown_connections 2023-09-20T21:48:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/3657501240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 0x55b4fe645b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/3657501240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe6416a0 0x55b4fe655620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/3657501240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe642000 0x55b4fe6526b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3657501240 >> 172.21.15.22:0/3657501240 conn(0x55b4fe56dfb0 msgr2=0x55b4fe649e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3657501240 shutdown_connections 2023-09-20T21:48:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.430+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3657501240 wait complete. 2023-09-20T21:48:41.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.431+0000 7f36a64d3b80 1 -- start start 2023-09-20T21:48:41.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.431+0000 7f36a64d3b80 1 Processor -- start 2023-09-20T21:48:41.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.431+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe6416a0 0x55b4fe64f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.431+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642000 0x55b4fe64f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.431+0000 7f36a64d3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 0x55b4fe6aa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.431+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe652800 con 0x55b4fe642b90 2023-09-20T21:48:41.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.431+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe652970 con 0x55b4fe6416a0 2023-09-20T21:48:41.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.431+0000 7f36a64d3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe6aa710 con 0x55b4fe642000 2023-09-20T21:48:41.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.431+0000 7f36a64d3b80 10 client.0 _ll_register_callbacks cb 0x55b4fe479a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:41.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f369099b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe6416a0 0x55b4fe64f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f368bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642000 0x55b4fe64f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f368bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642000 0x55b4fe64f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60734/0 (socket says 172.21.15.22:60734) 2023-09-20T21:48:41.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f368bfff700 1 -- 172.21.15.22:0/3986829729 learned_addr learned my addr 172.21.15.22:0/3986829729 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f369119c700 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 0x55b4fe6aa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f369099b700 1 -- 172.21.15.22:0/3986829729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642000 msgr2=0x55b4fe64f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f369099b700 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642000 0x55b4fe64f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f369099b700 1 -- 172.21.15.22:0/3986829729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 msgr2=0x55b4fe6aa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f369099b700 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 0x55b4fe6aa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f369099b700 1 -- 172.21.15.22:0/3986829729 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c1a0e80 con 0x55b4fe6416a0 2023-09-20T21:48:41.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f369119c700 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 0x55b4fe6aa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:41.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f369099b700 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe6416a0 0x55b4fe64f2a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3680005ac0 tx=0x7f3680004d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75de70e9e61622ca server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f366bfff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:48:41.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.432+0000 7f366bfff700 1 -- 172.21.15.22:0/3986829729 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368001d040 con 0x55b4fe6416a0 2023-09-20T21:48:41.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3986829729 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f368001e040 con 0x55b4fe6416a0 2023-09-20T21:48:41.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f366bfff700 1 -- 172.21.15.22:0/3986829729 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f3680000e40 con 0x55b4fe6416a0 2023-09-20T21:48:41.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f366bfff700 1 -- 172.21.15.22:0/3986829729 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3680005030 con 0x55b4fe6416a0 2023-09-20T21:48:41.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f36a64d3b80 10 client.25349 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:41.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3986829729 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b4fe3e4200 con 0x55b4fe6416a0 2023-09-20T21:48:41.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f366bfff700 1 -- 172.21.15.22:0/3986829729 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f368001e040 con 0x55b4fe6416a0 2023-09-20T21:48:41.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f366bfff700 1 -- 172.21.15.22:0/3986829729 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3680023760 con 0x55b4fe6416a0 2023-09-20T21:48:41.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f36a64d3b80 10 client.25349 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:41.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f36a64d3b80 10 client.25349 resolve_mds: resolved name 'f' to daemon mds.f 2023-09-20T21:48:41.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f36a64d3b80 20 client.25349 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:41.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55b4fe6af380 0x55b4fe6ab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f36a64d3b80 4 client.25349 mds_command: new command op to 24487 tid=0[{"prefix": "status"}] 2023-09-20T21:48:41.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.433+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3986829729 --> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b4fe58be10 con 0x55b4fe6af380 2023-09-20T21:48:41.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.434+0000 7f368bfff700 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55b4fe6af380 0x55b4fe6ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.434+0000 7f368bfff700 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55b4fe6af380 0x55b4fe6ab850 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b8666fcbb8d9a05 server_cookie=a85580b56996ced3 in_seq=0 out_seq=0 2023-09-20T21:48:41.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.434+0000 7f366bfff700 10 client.25349 ms_handle_connect on v2:172.21.15.123:6834/835271994 2023-09-20T21:48:41.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.434+0000 7f366bfff700 1 -- 172.21.15.22:0/3986829729 <== mds.3 v2:172.21.15.123:6834/835271994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b4fe58be10 con 0x55b4fe6af380 2023-09-20T21:48:41.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.434+0000 7f366bfff700 10 client.25349 handle_command_reply: tid=0 2023-09-20T21:48:41.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 client.25349 shutdown 2023-09-20T21:48:41.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3986829729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe6416a0 msgr2=0x55b4fe64f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe6416a0 0x55b4fe64f2a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3680005ac0 tx=0x7f3680004d90 comp rx=0 tx=0).stop 2023-09-20T21:48:41.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3986829729 shutdown_connections 2023-09-20T21:48:41.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b4fe642b90 0x55b4fe6aa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b4fe6416a0 0x55b4fe64f2a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55b4fe6af380 0x55b4fe6ab850 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 --2- 172.21.15.22:0/3986829729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4fe642000 0x55b4fe64f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f366bfff700 0 client.25349 ms_handle_reset on v2:172.21.15.123:6834/835271994 2023-09-20T21:48:41.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3986829729 >> 172.21.15.22:0/3986829729 conn(0x55b4fe56dfb0 msgr2=0x55b4fe649e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3986829729 shutdown_connections 2023-09-20T21:48:41.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 1 -- 172.21.15.22:0/3986829729 wait complete. 2023-09-20T21:48:41.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f36a64d3b80 20 client.25349 trim_cache size 0 max 16384 2023-09-20T21:48:41.452 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:41.452 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:48:41.452 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 3, 2023-09-20T21:48:41.452 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24487, 2023-09-20T21:48:41.453 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:48:41.453 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:48:41.453 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:48:41.453 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 93.873933043999997, 2023-09-20T21:48:41.453 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 28, 2023-09-20T21:48:41.454 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2023-09-20T21:48:41.454 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 91, 2023-09-20T21:48:41.454 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 141.82177731799999 2023-09-20T21:48:41.454 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:41.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f3692bff700 1 -- 172.21.15.22:0/2996680862 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f367c0a1380 msgr2=0x7f367c0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f3692bff700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f367c0a1380 0x7f367c0a3840 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3684008d40 tx=0x7f3684009100 comp rx=0 tx=0).stop 2023-09-20T21:48:41.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.435+0000 7f3692bff700 1 -- 172.21.15.22:0/2996680862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c074e20 msgr2=0x7f368c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.436+0000 7f3692bff700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c074e20 0x7f368c199290 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f367800e9c0 tx=0x7f367800ecd0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.436+0000 7f3692bff700 1 -- 172.21.15.22:0/2996680862 shutdown_connections 2023-09-20T21:48:41.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.436+0000 7f3692bff700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f367c0a1380 0x7f367c0a3840 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.436+0000 7f3692bff700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f368c108cb0 0x7f368c1a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.436+0000 7f3692bff700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f368c073840 0x7f368c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.436+0000 7f3692bff700 1 --2- 172.21.15.22:0/2996680862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c074e20 0x7f368c199290 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.436+0000 7f3692bff700 1 -- 172.21.15.22:0/2996680862 >> 172.21.15.22:0/2996680862 conn(0x7f368c0fc360 msgr2=0x7f368c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.436+0000 7f3692bff700 1 -- 172.21.15.22:0/2996680862 shutdown_connections 2023-09-20T21:48:41.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.436+0000 7f3692bff700 1 -- 172.21.15.22:0/2996680862 wait complete. 2023-09-20T21:48:41.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-09-20T21:48:41.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.668+0000 7fec85bff700 1 -- 172.21.15.22:0/3259361217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec80106ab0 msgr2=0x7fec80108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.668+0000 7fec85bff700 1 --2- 172.21.15.22:0/3259361217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec80106ab0 0x7fec80108f20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fec7800b0a0 tx=0x7fec7800b3b0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.668+0000 7fec85bff700 1 -- 172.21.15.22:0/3259361217 shutdown_connections 2023-09-20T21:48:41.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.668+0000 7fec85bff700 1 --2- 172.21.15.22:0/3259361217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec80106ab0 0x7fec80108f20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.668+0000 7fec85bff700 1 --2- 172.21.15.22:0/3259361217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec80101850 0x7fec80103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.668+0000 7fec85bff700 1 --2- 172.21.15.22:0/3259361217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec80104180 0x7fec80106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.668+0000 7fec85bff700 1 -- 172.21.15.22:0/3259361217 >> 172.21.15.22:0/3259361217 conn(0x7fec800fb240 msgr2=0x7fec800fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.668+0000 7fec85bff700 1 -- 172.21.15.22:0/3259361217 shutdown_connections 2023-09-20T21:48:41.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.668+0000 7fec85bff700 1 -- 172.21.15.22:0/3259361217 wait complete. 2023-09-20T21:48:41.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.669+0000 7fec85bff700 1 Processor -- start 2023-09-20T21:48:41.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.669+0000 7fec85bff700 1 -- start start 2023-09-20T21:48:41.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec85bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec80101850 0x7fec801011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec85bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec80104180 0x7fec800ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec85bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec80106ab0 0x7fec800ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec85bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec801004e0 con 0x7fec80106ab0 2023-09-20T21:48:41.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec85bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec80100650 con 0x7fec80104180 2023-09-20T21:48:41.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec85bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec801a0c60 con 0x7fec80101850 2023-09-20T21:48:41.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec80106ab0 0x7fec800ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec80104180 0x7fec800ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec80106ab0 0x7fec800ffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47912/0 (socket says 172.21.15.22:47912) 2023-09-20T21:48:41.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7ffff700 1 -- 172.21.15.22:0/926844192 learned_addr learned my addr 172.21.15.22:0/926844192 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7f7fe700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec80101850 0x7fec801011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7effd700 1 -- 172.21.15.22:0/926844192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec80101850 msgr2=0x7fec801011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7effd700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec80101850 0x7fec801011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7effd700 1 -- 172.21.15.22:0/926844192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec80106ab0 msgr2=0x7fec800ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7effd700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec80106ab0 0x7fec800ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7effd700 1 -- 172.21.15.22:0/926844192 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec78009d00 con 0x7fec80104180 2023-09-20T21:48:41.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.670+0000 7fec7ffff700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec80106ab0 0x7fec800ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:41.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.671+0000 7fec7effd700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec80104180 0x7fec800ff800 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec7400e9f0 tx=0x7fec7400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.671+0000 7fec7cff9700 1 -- 172.21.15.22:0/926844192 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400c9c0 con 0x7fec80104180 2023-09-20T21:48:41.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.671+0000 7fec7cff9700 1 -- 172.21.15.22:0/926844192 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec74010430 con 0x7fec80104180 2023-09-20T21:48:41.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.671+0000 7fec85bff700 1 -- 172.21.15.22:0/926844192 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec801a0f40 con 0x7fec80104180 2023-09-20T21:48:41.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.671+0000 7fec7cff9700 1 -- 172.21.15.22:0/926844192 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74018680 con 0x7fec80104180 2023-09-20T21:48:41.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.671+0000 7fec85bff700 1 -- 172.21.15.22:0/926844192 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec801a13c0 con 0x7fec80104180 2023-09-20T21:48:41.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.676+0000 7fec7cff9700 1 -- 172.21.15.22:0/926844192 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fec74010860 con 0x7fec80104180 2023-09-20T21:48:41.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec99471b80 1 Processor -- start 2023-09-20T21:48:41.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec99471b80 1 -- start start 2023-09-20T21:48:41.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec7cff9700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec680a1300 0x7fec680a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec99471b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626377ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec99471b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626378590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec99471b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 0x558626379010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec99471b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586262ebc50 con 0x558626378b90 2023-09-20T21:48:41.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec99471b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586262cfc20 con 0x5586263776a0 2023-09-20T21:48:41.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec99471b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586262cd540 con 0x558626378000 2023-09-20T21:48:41.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec7cff9700 1 -- 172.21.15.22:0/926844192 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fec74014070 con 0x7fec80104180 2023-09-20T21:48:41.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.677+0000 7fec7ffff700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec680a1300 0x7fec680a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626378590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626377ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626378590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60750/0 (socket says 172.21.15.22:60750) 2023-09-20T21:48:41.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626377ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43540/0 (socket says 172.21.15.22:43540) 2023-09-20T21:48:41.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7effd700 1 -- 172.21.15.22:0/3651115422 learned_addr learned my addr 172.21.15.22:0/3651115422 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7ffff700 1 --2- 172.21.15.22:0/3651115422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 0x558626379010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7effd700 1 -- 172.21.15.22:0/3651115422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 msgr2=0x558626377ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7effd700 1 --2- 172.21.15.22:0/3651115422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626377ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7effd700 1 -- 172.21.15.22:0/3651115422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 msgr2=0x558626379010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7effd700 1 --2- 172.21.15.22:0/3651115422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 0x558626379010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7effd700 1 -- 172.21.15.22:0/3651115422 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586260f3d90 con 0x558626378000 2023-09-20T21:48:41.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7effd700 1 --2- 172.21.15.22:0/3651115422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626378590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec740a1c00 tx=0x7fec74096530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71191abb09f1274f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec7ffff700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec680a1300 0x7fec680a37c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fec80101820 tx=0x7fec78009510 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.678+0000 7fec65ffb700 1 -- 172.21.15.22:0/3651115422 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7409a7c0 con 0x558626378000 2023-09-20T21:48:41.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec65ffb700 1 -- 172.21.15.22:0/3651115422 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec74065c80 con 0x558626378000 2023-09-20T21:48:41.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec65ffb700 1 -- 172.21.15.22:0/3651115422 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740a2130 con 0x558626378000 2023-09-20T21:48:41.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec99471b80 1 -- 172.21.15.22:0/3651115422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 msgr2=0x558626378590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec99471b80 1 --2- 172.21.15.22:0/3651115422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626378590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec740a1c00 tx=0x7fec74096530 comp rx=0 tx=0).stop 2023-09-20T21:48:41.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec99471b80 1 -- 172.21.15.22:0/3651115422 shutdown_connections 2023-09-20T21:48:41.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec99471b80 1 --2- 172.21.15.22:0/3651115422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 0x558626379010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec99471b80 1 --2- 172.21.15.22:0/3651115422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626377ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec99471b80 1 --2- 172.21.15.22:0/3651115422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626378590 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec99471b80 1 -- 172.21.15.22:0/3651115422 >> 172.21.15.22:0/3651115422 conn(0x5586262a3fb0 msgr2=0x558626372180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec99471b80 1 -- 172.21.15.22:0/3651115422 shutdown_connections 2023-09-20T21:48:41.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.679+0000 7fec99471b80 1 -- 172.21.15.22:0/3651115422 wait complete. 2023-09-20T21:48:41.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.680+0000 7fec99471b80 1 -- start start 2023-09-20T21:48:41.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.680+0000 7fec99471b80 1 Processor -- start 2023-09-20T21:48:41.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.680+0000 7fec99471b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626381030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec99471b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626381570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec99471b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 0x5586263885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec99471b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586262c8e30 con 0x558626378b90 2023-09-20T21:48:41.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec99471b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586262656f0 con 0x5586263776a0 2023-09-20T21:48:41.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec99471b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558626265390 con 0x558626378000 2023-09-20T21:48:41.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec99471b80 10 client.0 _ll_register_callbacks cb 0x5586261afa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:41.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 0x5586263885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626381570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 0x5586263885b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47934/0 (socket says 172.21.15.22:47934) 2023-09-20T21:48:41.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7ffff700 1 -- 172.21.15.22:0/3626439162 learned_addr learned my addr 172.21.15.22:0/3626439162 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7f7fe700 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626381030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7effd700 1 -- 172.21.15.22:0/3626439162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 msgr2=0x558626381030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7effd700 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626381030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7effd700 1 -- 172.21.15.22:0/3626439162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 msgr2=0x5586263885b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7effd700 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 0x5586263885b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7effd700 1 -- 172.21.15.22:0/3626439162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec801a13c0 con 0x558626378000 2023-09-20T21:48:41.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.681+0000 7fec7ffff700 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 0x5586263885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:41.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.682+0000 7fec7effd700 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626381570 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fec7409a280 tx=0x7fec7409a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f96cf5eedb7492b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.682+0000 7fec5effd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:48:41.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.682+0000 7fec5effd700 1 -- 172.21.15.22:0/3626439162 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740105a0 con 0x558626378000 2023-09-20T21:48:41.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.682+0000 7fec99471b80 1 -- 172.21.15.22:0/3626439162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558626239b60 con 0x558626378000 2023-09-20T21:48:41.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.682+0000 7fec5effd700 1 -- 172.21.15.22:0/3626439162 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fec7409a320 con 0x558626378000 2023-09-20T21:48:41.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.682+0000 7fec5effd700 1 -- 172.21.15.22:0/3626439162 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec7409a4e0 con 0x558626378000 2023-09-20T21:48:41.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.682+0000 7fec99471b80 10 client.25354 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:41.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.682+0000 7fec99471b80 1 -- 172.21.15.22:0/3626439162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5586260dcc10 con 0x558626378000 2023-09-20T21:48:41.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.682+0000 7fec5effd700 1 -- 172.21.15.22:0/3626439162 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558626239b60 con 0x558626378000 2023-09-20T21:48:41.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.683+0000 7fec5effd700 1 -- 172.21.15.22:0/3626439162 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fec740a1dc0 con 0x558626378000 2023-09-20T21:48:41.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.683+0000 7fec99471b80 10 client.25354 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:41.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.683+0000 7fec99471b80 10 client.25354 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-20T21:48:41.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.683+0000 7fec99471b80 20 client.25354 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:41.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.683+0000 7fec99471b80 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55862638c770 0x55862638b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.683+0000 7fec99471b80 4 client.25354 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:41.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.683+0000 7fec99471b80 1 -- 172.21.15.22:0/3626439162 --> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558626345fd0 con 0x55862638c770 2023-09-20T21:48:41.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.683+0000 7fec7f7fe700 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55862638c770 0x55862638b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.684+0000 7fec7f7fe700 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55862638c770 0x55862638b4e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d13479ced38389eb server_cookie=f84b9bbbf690092b in_seq=0 out_seq=0 2023-09-20T21:48:41.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.684+0000 7fec5effd700 10 client.25354 ms_handle_connect on v2:172.21.15.89:6836/3887702690 2023-09-20T21:48:41.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.684+0000 7fec5effd700 1 -- 172.21.15.22:0/3626439162 <== mds.2 v2:172.21.15.89:6836/3887702690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558626345fd0 con 0x55862638c770 2023-09-20T21:48:41.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.684+0000 7fec5effd700 10 client.25354 handle_command_reply: tid=0 2023-09-20T21:48:41.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.684+0000 7fec99471b80 1 client.25354 shutdown 2023-09-20T21:48:41.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 1 -- 172.21.15.22:0/3626439162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 msgr2=0x558626381570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626381570 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fec7409a280 tx=0x7fec7409a130 comp rx=0 tx=0).stop 2023-09-20T21:48:41.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 1 -- 172.21.15.22:0/3626439162 shutdown_connections 2023-09-20T21:48:41.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55862638c770 0x55862638b4e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378b90 0x5586263885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec5effd700 0 client.25354 ms_handle_reset on v2:172.21.15.89:6836/3887702690 2023-09-20T21:48:41.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626381030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 1 --2- 172.21.15.22:0/3626439162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378000 0x558626381570 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 1 -- 172.21.15.22:0/3626439162 >> 172.21.15.22:0/3626439162 conn(0x5586262a3fb0 msgr2=0x558626371830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 1 -- 172.21.15.22:0/3626439162 shutdown_connections 2023-09-20T21:48:41.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 1 -- 172.21.15.22:0/3626439162 wait complete. 2023-09-20T21:48:41.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.685+0000 7fec99471b80 20 client.25354 trim_cache size 0 max 16384 2023-09-20T21:48:41.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.703+0000 7fec99471b80 1 Processor -- start 2023-09-20T21:48:41.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.703+0000 7fec99471b80 1 -- start start 2023-09-20T21:48:41.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.703+0000 7fec99471b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626378f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.703+0000 7fec99471b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626388570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.703+0000 7fec99471b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378b90 0x55862638b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.703+0000 7fec99471b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586261498d0 con 0x558626378000 2023-09-20T21:48:41.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.703+0000 7fec99471b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558626265390 con 0x5586263776a0 2023-09-20T21:48:41.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.703+0000 7fec99471b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558626345fd0 con 0x558626378b90 2023-09-20T21:48:41.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626388570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626388570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47942/0 (socket says 172.21.15.22:47942) 2023-09-20T21:48:41.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7effd700 1 -- 172.21.15.22:0/717283471 learned_addr learned my addr 172.21.15.22:0/717283471 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7f7fe700 1 --2- 172.21.15.22:0/717283471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626378f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7ffff700 1 --2- 172.21.15.22:0/717283471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378b90 0x55862638b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7effd700 1 -- 172.21.15.22:0/717283471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378b90 msgr2=0x55862638b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7effd700 1 --2- 172.21.15.22:0/717283471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378b90 0x55862638b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7effd700 1 -- 172.21.15.22:0/717283471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 msgr2=0x558626378f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7effd700 1 --2- 172.21.15.22:0/717283471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626378f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7effd700 1 -- 172.21.15.22:0/717283471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55862611a200 con 0x558626378000 2023-09-20T21:48:41.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.704+0000 7fec7effd700 1 --2- 172.21.15.22:0/717283471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626388570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fec7409a050 tx=0x7fec74096530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e4b01146374366 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec667fc700 1 -- 172.21.15.22:0/717283471 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7409df60 con 0x558626378000 2023-09-20T21:48:41.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec667fc700 1 -- 172.21.15.22:0/717283471 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec740a2130 con 0x558626378000 2023-09-20T21:48:41.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec667fc700 1 -- 172.21.15.22:0/717283471 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740997f0 con 0x558626378000 2023-09-20T21:48:41.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec99471b80 1 -- 172.21.15.22:0/717283471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 msgr2=0x558626388570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec99471b80 1 --2- 172.21.15.22:0/717283471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626388570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fec7409a050 tx=0x7fec74096530 comp rx=0 tx=0).stop 2023-09-20T21:48:41.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec99471b80 1 -- 172.21.15.22:0/717283471 shutdown_connections 2023-09-20T21:48:41.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec99471b80 1 --2- 172.21.15.22:0/717283471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378b90 0x55862638b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec99471b80 1 --2- 172.21.15.22:0/717283471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626378f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec99471b80 1 --2- 172.21.15.22:0/717283471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626388570 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec99471b80 1 -- 172.21.15.22:0/717283471 >> 172.21.15.22:0/717283471 conn(0x5586262a3fb0 msgr2=0x55862637fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec99471b80 1 -- 172.21.15.22:0/717283471 shutdown_connections 2023-09-20T21:48:41.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.705+0000 7fec99471b80 1 -- 172.21.15.22:0/717283471 wait complete. 2023-09-20T21:48:41.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.706+0000 7fec99471b80 1 -- start start 2023-09-20T21:48:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.706+0000 7fec99471b80 1 Processor -- start 2023-09-20T21:48:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec99471b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626384a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec99471b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626384fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec99471b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378b90 0x558626387b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec99471b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586263880a0 con 0x558626378000 2023-09-20T21:48:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec99471b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558626388210 con 0x5586263776a0 2023-09-20T21:48:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec99471b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586263883d0 con 0x558626378b90 2023-09-20T21:48:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec99471b80 10 client.0 _ll_register_callbacks cb 0x5586261afa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:41.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec7f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626384a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec7effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626384fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec7f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626384a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43562/0 (socket says 172.21.15.22:43562) 2023-09-20T21:48:41.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec7f7fe700 1 -- 172.21.15.22:0/3600245267 learned_addr learned my addr 172.21.15.22:0/3600245267 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec7ffff700 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378b90 0x558626387b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec7f7fe700 1 -- 172.21.15.22:0/3600245267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378b90 msgr2=0x558626387b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec7f7fe700 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378b90 0x558626387b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.707+0000 7fec7f7fe700 1 -- 172.21.15.22:0/3600245267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 msgr2=0x558626384fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.708+0000 7fec7f7fe700 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626384fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.708+0000 7fec7f7fe700 1 -- 172.21.15.22:0/3600245267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586260dcc10 con 0x5586263776a0 2023-09-20T21:48:41.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.708+0000 7fec7effd700 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626384fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.708+0000 7fec7f7fe700 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626384a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55862638d4a0 tx=0x7fec7001dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd72872efe4b83d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.708+0000 7fec5effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:48:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.708+0000 7fec5effd700 1 -- 172.21.15.22:0/3600245267 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70016040 con 0x5586263776a0 2023-09-20T21:48:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.708+0000 7fec99471b80 1 -- 172.21.15.22:0/3600245267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec74065c80 con 0x5586263776a0 2023-09-20T21:48:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.708+0000 7fec5effd700 1 -- 172.21.15.22:0/3600245267 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fec70004830 con 0x5586263776a0 2023-09-20T21:48:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.708+0000 7fec5effd700 1 -- 172.21.15.22:0/3600245267 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec700049f0 con 0x5586263776a0 2023-09-20T21:48:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.708+0000 7fec99471b80 10 client.25367 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.709+0000 7fec99471b80 1 -- 172.21.15.22:0/3600245267 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5586260f2260 con 0x5586263776a0 2023-09-20T21:48:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.709+0000 7fec5effd700 1 -- 172.21.15.22:0/3600245267 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec74065c80 con 0x5586263776a0 2023-09-20T21:48:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.709+0000 7fec5effd700 1 -- 172.21.15.22:0/3600245267 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fec70010670 con 0x5586263776a0 2023-09-20T21:48:41.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.709+0000 7fec99471b80 10 client.25367 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:41.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.709+0000 7fec99471b80 10 client.25367 resolve_mds: resolved name 'h' to daemon mds.h 2023-09-20T21:48:41.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.709+0000 7fec99471b80 20 client.25367 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:41.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.709+0000 7fec99471b80 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x5586263e4cc0 0x5586263e1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.709+0000 7fec99471b80 4 client.25367 mds_command: new command op to 24530 tid=0[{"prefix": "status"}] 2023-09-20T21:48:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.709+0000 7fec99471b80 1 -- 172.21.15.22:0/3600245267 --> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5586262c1e10 con 0x5586263e4cc0 2023-09-20T21:48:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.709+0000 7fec7effd700 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x5586263e4cc0 0x5586263e1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec7effd700 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x5586263e4cc0 0x5586263e1120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa10cecbc2b0a4f5 server_cookie=67ac8938421904c6 in_seq=0 out_seq=0 2023-09-20T21:48:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec5effd700 10 client.25367 ms_handle_connect on v2:172.21.15.89:6836/3887702690 2023-09-20T21:48:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec5effd700 1 -- 172.21.15.22:0/3600245267 <== mds.2 v2:172.21.15.89:6836/3887702690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5586262c1e10 con 0x5586263e4cc0 2023-09-20T21:48:41.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec5effd700 10 client.25367 handle_command_reply: tid=0 2023-09-20T21:48:41.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 client.25367 shutdown 2023-09-20T21:48:41.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 -- 172.21.15.22:0/3600245267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 msgr2=0x558626384a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626384a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55862638d4a0 tx=0x7fec7001dd50 comp rx=0 tx=0).stop 2023-09-20T21:48:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 -- 172.21.15.22:0/3600245267 shutdown_connections 2023-09-20T21:48:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x5586263e4cc0 0x5586263e1120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558626378b90 0x558626387b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5586263776a0 0x558626384a60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec5effd700 0 client.25367 ms_handle_reset on v2:172.21.15.89:6836/3887702690 2023-09-20T21:48:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 --2- 172.21.15.22:0/3600245267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558626378000 0x558626384fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 -- 172.21.15.22:0/3600245267 >> 172.21.15.22:0/3600245267 conn(0x5586262a3fb0 msgr2=0x55862638b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 -- 172.21.15.22:0/3600245267 shutdown_connections 2023-09-20T21:48:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.710+0000 7fec99471b80 1 -- 172.21.15.22:0/3600245267 wait complete. 2023-09-20T21:48:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec99471b80 20 client.25367 trim_cache size 0 max 16384 2023-09-20T21:48:41.727 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:41.727 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:48:41.727 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 2, 2023-09-20T21:48:41.727 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24530, 2023-09-20T21:48:41.728 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:48:41.728 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:48:41.728 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:48:41.728 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 95.123945599999999, 2023-09-20T21:48:41.729 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 28, 2023-09-20T21:48:41.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2023-09-20T21:48:41.729 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 91, 2023-09-20T21:48:41.729 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 137.643911301 2023-09-20T21:48:41.729 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:41.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 -- 172.21.15.22:0/926844192 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec680a1300 msgr2=0x7fec680a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec680a1300 0x7fec680a37c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fec80101820 tx=0x7fec78009510 comp rx=0 tx=0).stop 2023-09-20T21:48:41.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 -- 172.21.15.22:0/926844192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec80104180 msgr2=0x7fec800ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec80104180 0x7fec800ff800 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec7400e9f0 tx=0x7fec7400ed00 comp rx=0 tx=0).stop 2023-09-20T21:48:41.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 -- 172.21.15.22:0/926844192 shutdown_connections 2023-09-20T21:48:41.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec80106ab0 0x7fec800ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec80101850 0x7fec801011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fec680a1300 0x7fec680a37c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 --2- 172.21.15.22:0/926844192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec80104180 0x7fec800ff800 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 -- 172.21.15.22:0/926844192 >> 172.21.15.22:0/926844192 conn(0x7fec800fb240 msgr2=0x7fec801024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 -- 172.21.15.22:0/926844192 shutdown_connections 2023-09-20T21:48:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.711+0000 7fec85bff700 1 -- 172.21.15.22:0/926844192 wait complete. 2023-09-20T21:48:41.733 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:41 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2517722332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:41.733 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:41 smithi022 ceph-mon[97651]: pgmap v340: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:41.734 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-20T21:48:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.939+0000 7f22aa5ff700 1 -- 172.21.15.22:0/596349931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a4101fc0 msgr2=0x7f22a4102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.939+0000 7f22aa5ff700 1 --2- 172.21.15.22:0/596349931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a4101fc0 0x7f22a4102420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f22980099c0 tx=0x7f2298009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.939+0000 7f22aa5ff700 1 -- 172.21.15.22:0/596349931 shutdown_connections 2023-09-20T21:48:41.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.939+0000 7f22aa5ff700 1 --2- 172.21.15.22:0/596349931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a4102960 0x7f22a4109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.939+0000 7f22aa5ff700 1 --2- 172.21.15.22:0/596349931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4100dd0 0x7f22a41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.939+0000 7f22aa5ff700 1 --2- 172.21.15.22:0/596349931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a4101fc0 0x7f22a4102420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.939+0000 7f22aa5ff700 1 -- 172.21.15.22:0/596349931 >> 172.21.15.22:0/596349931 conn(0x7f22a40fc320 msgr2=0x7f22a40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.940+0000 7f22aa5ff700 1 -- 172.21.15.22:0/596349931 shutdown_connections 2023-09-20T21:48:41.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.940+0000 7f22aa5ff700 1 -- 172.21.15.22:0/596349931 wait complete. 2023-09-20T21:48:41.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22aa5ff700 1 Processor -- start 2023-09-20T21:48:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22aa5ff700 1 -- start start 2023-09-20T21:48:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22aa5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a4100dd0 0x7f22a4196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22aa5ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101fc0 0x7f22a4197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22aa5ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a4102960 0x7f22a419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22aa5ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419e6a0 con 0x7f22a4102960 2023-09-20T21:48:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22aa5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419e7e0 con 0x7f22a4100dd0 2023-09-20T21:48:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22aa5ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419e970 con 0x7f22a4101fc0 2023-09-20T21:48:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101fc0 0x7f22a4197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22a8b9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a4102960 0x7f22a419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101fc0 0x7f22a4197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60794/0 (socket says 172.21.15.22:60794) 2023-09-20T21:48:41.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.941+0000 7f22a37fe700 1 -- 172.21.15.22:0/2621237660 learned_addr learned my addr 172.21.15.22:0/2621237660 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a3fff700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a4100dd0 0x7f22a4196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a37fe700 1 -- 172.21.15.22:0/2621237660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a4100dd0 msgr2=0x7f22a4196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a37fe700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a4100dd0 0x7f22a4196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a37fe700 1 -- 172.21.15.22:0/2621237660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a4102960 msgr2=0x7f22a419e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a37fe700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a4102960 0x7f22a419e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a37fe700 1 -- 172.21.15.22:0/2621237660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22980095f0 con 0x7f22a4101fc0 2023-09-20T21:48:41.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a8b9c700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a4102960 0x7f22a419e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:41.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a37fe700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101fc0 0x7f22a4197060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2298009990 tx=0x7f229800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a17fa700 1 -- 172.21.15.22:0/2621237660 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229801d070 con 0x7f22a4101fc0 2023-09-20T21:48:41.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a17fa700 1 -- 172.21.15.22:0/2621237660 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2298005030 con 0x7f22a4101fc0 2023-09-20T21:48:41.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22aa5ff700 1 -- 172.21.15.22:0/2621237660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a419ebf0 con 0x7f22a4101fc0 2023-09-20T21:48:41.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.942+0000 7f22a17fa700 1 -- 172.21.15.22:0/2621237660 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298004a70 con 0x7f22a4101fc0 2023-09-20T21:48:41.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.943+0000 7f22aa5ff700 1 -- 172.21.15.22:0/2621237660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a419f010 con 0x7f22a4101fc0 2023-09-20T21:48:41.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.947+0000 7f22a17fa700 1 -- 172.21.15.22:0/2621237660 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2298005410 con 0x7f22a4101fc0 2023-09-20T21:48:41.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.948+0000 7f22bdffcb80 1 Processor -- start 2023-09-20T21:48:41.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.948+0000 7f22bdffcb80 1 -- start start 2023-09-20T21:48:41.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 0x5558fb17aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 0x5558fb17b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb17c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb0eac50 con 0x5558fb17bfb0 2023-09-20T21:48:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb0cec20 con 0x5558fb17aab0 2023-09-20T21:48:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb0cc540 con 0x5558fb17b410 2023-09-20T21:48:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a17fa700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f228c0a1300 0x7f228c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb17c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb17c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47994/0 (socket says 172.21.15.22:47994) 2023-09-20T21:48:41.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a37fe700 1 -- 172.21.15.22:0/60152691 learned_addr learned my addr 172.21.15.22:0/60152691 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a17fa700 1 -- 172.21.15.22:0/2621237660 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f22980cddc0 con 0x7f22a4101fc0 2023-09-20T21:48:41.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a3fff700 1 --2- 172.21.15.22:0/60152691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 0x5558fb17b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a8b9c700 1 --2- 172.21.15.22:0/60152691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 0x5558fb17aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a8b9c700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f228c0a1300 0x7f228c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a3fff700 1 -- 172.21.15.22:0/60152691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 msgr2=0x5558fb17aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a3fff700 1 --2- 172.21.15.22:0/60152691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 0x5558fb17aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a3fff700 1 -- 172.21.15.22:0/60152691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 msgr2=0x5558fb17c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a3fff700 1 --2- 172.21.15.22:0/60152691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb17c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.949+0000 7f22a3fff700 1 -- 172.21.15.22:0/60152691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558faef2d90 con 0x5558fb17b410 2023-09-20T21:48:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.950+0000 7f22a3fff700 1 --2- 172.21.15.22:0/60152691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 0x5558fb17b9d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2294009c80 tx=0x7f2294009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b04a9eba24cdd1c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.950+0000 7f22a8b9c700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f228c0a1300 0x7f228c0a37c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f229c009fd0 tx=0x7f229c00b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.950+0000 7f228a7fc700 1 -- 172.21.15.22:0/60152691 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294011cb0 con 0x5558fb17b410 2023-09-20T21:48:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.950+0000 7f228a7fc700 1 -- 172.21.15.22:0/60152691 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2294011e10 con 0x5558fb17b410 2023-09-20T21:48:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.950+0000 7f228a7fc700 1 -- 172.21.15.22:0/60152691 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294005220 con 0x5558fb17b410 2023-09-20T21:48:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.951+0000 7f22bdffcb80 1 -- 172.21.15.22:0/60152691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 msgr2=0x5558fb17b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.951+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/60152691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 0x5558fb17b9d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2294009c80 tx=0x7f2294009f90 comp rx=0 tx=0).stop 2023-09-20T21:48:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.951+0000 7f22bdffcb80 1 -- 172.21.15.22:0/60152691 shutdown_connections 2023-09-20T21:48:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.951+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/60152691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb17c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.951+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/60152691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 0x5558fb17aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.951+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/60152691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 0x5558fb17b9d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.951+0000 7f22bdffcb80 1 -- 172.21.15.22:0/60152691 >> 172.21.15.22:0/60152691 conn(0x5558fb0a2fb0 msgr2=0x5558fb171180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.951+0000 7f22bdffcb80 1 -- 172.21.15.22:0/60152691 shutdown_connections 2023-09-20T21:48:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.951+0000 7f22bdffcb80 1 -- 172.21.15.22:0/60152691 wait complete. 2023-09-20T21:48:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.951+0000 7f22bdffcb80 1 -- start start 2023-09-20T21:48:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.952+0000 7f22bdffcb80 1 Processor -- start 2023-09-20T21:48:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.952+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb184480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17b410 0x5558fb1849c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17bfb0 0x5558fb18ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb0c7e30 con 0x5558fb17b410 2023-09-20T21:48:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb0646f0 con 0x5558fb17bfb0 2023-09-20T21:48:41.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb064390 con 0x5558fb17aab0 2023-09-20T21:48:41.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22bdffcb80 10 client.0 _ll_register_callbacks cb 0x5558fafaea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:41.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a8b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17bfb0 0x5558fb18ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17b410 0x5558fb1849c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a8b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17bfb0 0x5558fb18ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43596/0 (socket says 172.21.15.22:43596) 2023-09-20T21:48:41.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17b410 0x5558fb1849c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48008/0 (socket says 172.21.15.22:48008) 2023-09-20T21:48:41.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb184480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a8b9c700 1 -- 172.21.15.22:0/4193482881 learned_addr learned my addr 172.21.15.22:0/4193482881 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a3fff700 1 -- 172.21.15.22:0/4193482881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17bfb0 msgr2=0x5558fb18ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a3fff700 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17bfb0 0x5558fb18ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a3fff700 1 -- 172.21.15.22:0/4193482881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17b410 msgr2=0x5558fb1849c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a3fff700 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17b410 0x5558fb1849c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a3fff700 1 -- 172.21.15.22:0/4193482881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22940098e0 con 0x5558fb17aab0 2023-09-20T21:48:41.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a37fe700 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17b410 0x5558fb1849c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:41.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.953+0000 7f22a8b9c700 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17bfb0 0x5558fb18ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:41.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.954+0000 7f22a3fff700 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb184480 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2294000c00 tx=0x7f2294004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c8793f8bfeaa6e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.954+0000 7f22837fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:48:41.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.954+0000 7f22837fe700 1 -- 172.21.15.22:0/4193482881 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22940044f0 con 0x5558fb17aab0 2023-09-20T21:48:41.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.954+0000 7f22bdffcb80 1 -- 172.21.15.22:0/4193482881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558fb038b60 con 0x5558fb17aab0 2023-09-20T21:48:41.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.954+0000 7f22837fe700 1 -- 172.21.15.22:0/4193482881 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f2294003680 con 0x5558fb17aab0 2023-09-20T21:48:41.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.954+0000 7f22837fe700 1 -- 172.21.15.22:0/4193482881 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2294003840 con 0x5558fb17aab0 2023-09-20T21:48:41.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.954+0000 7f22bdffcb80 10 client.25372 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:41.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.954+0000 7f22bdffcb80 1 -- 172.21.15.22:0/4193482881 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5558faedbc10 con 0x5558fb17aab0 2023-09-20T21:48:41.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.955+0000 7f22837fe700 1 -- 172.21.15.22:0/4193482881 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558fb038b60 con 0x5558fb17aab0 2023-09-20T21:48:41.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.955+0000 7f22837fe700 1 -- 172.21.15.22:0/4193482881 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2294020070 con 0x5558fb17aab0 2023-09-20T21:48:41.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.955+0000 7f22bdffcb80 10 client.25372 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:41.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.955+0000 7f22bdffcb80 10 client.25372 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-20T21:48:41.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.955+0000 7f22bdffcb80 20 client.25372 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:41.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.955+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5558fb18ffb0 0x5558fb18ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.955+0000 7f22bdffcb80 4 client.25372 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:41.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.955+0000 7f22bdffcb80 1 -- 172.21.15.22:0/4193482881 --> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558fb144fd0 con 0x5558fb18ffb0 2023-09-20T21:48:41.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.955+0000 7f22a37fe700 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5558fb18ffb0 0x5558fb18ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.956+0000 7f22a37fe700 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5558fb18ffb0 0x5558fb18ea50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdd22d01ebccdf12 server_cookie=57c3d5f7951ae0aa in_seq=0 out_seq=0 2023-09-20T21:48:41.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.956+0000 7f22837fe700 10 client.25372 ms_handle_connect on v2:172.21.15.22:6840/1806616417 2023-09-20T21:48:41.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.956+0000 7f22837fe700 1 -- 172.21.15.22:0/4193482881 <== mds.1 v2:172.21.15.22:6840/1806616417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5558fb144fd0 con 0x5558fb18ffb0 2023-09-20T21:48:41.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.956+0000 7f22837fe700 10 client.25372 handle_command_reply: tid=0 2023-09-20T21:48:41.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.956+0000 7f22bdffcb80 1 client.25372 shutdown 2023-09-20T21:48:41.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 1 -- 172.21.15.22:0/4193482881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 msgr2=0x5558fb184480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb184480 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2294000c00 tx=0x7f2294004040 comp rx=0 tx=0).stop 2023-09-20T21:48:41.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 1 -- 172.21.15.22:0/4193482881 shutdown_connections 2023-09-20T21:48:41.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17bfb0 0x5558fb18ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb184480 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5558fb18ffb0 0x5558fb18ea50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/4193482881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17b410 0x5558fb1849c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22837fe700 0 client.25372 ms_handle_reset on v2:172.21.15.22:6840/1806616417 2023-09-20T21:48:41.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 1 -- 172.21.15.22:0/4193482881 >> 172.21.15.22:0/4193482881 conn(0x5558fb0a2fb0 msgr2=0x5558fb170830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 1 -- 172.21.15.22:0/4193482881 shutdown_connections 2023-09-20T21:48:41.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 1 -- 172.21.15.22:0/4193482881 wait complete. 2023-09-20T21:48:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.957+0000 7f22bdffcb80 20 client.25372 trim_cache size 0 max 16384 2023-09-20T21:48:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22bdffcb80 1 Processor -- start 2023-09-20T21:48:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22bdffcb80 1 -- start start 2023-09-20T21:48:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 0x5558fb183b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 0x5558fb18b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb1822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558faf488d0 con 0x5558fb17bfb0 2023-09-20T21:48:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb082820 con 0x5558fb17aab0 2023-09-20T21:48:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb060aa0 con 0x5558fb17b410 2023-09-20T21:48:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22a3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 0x5558fb183b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22a3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 0x5558fb183b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43600/0 (socket says 172.21.15.22:43600) 2023-09-20T21:48:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22a8b9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb1822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.976+0000 7f22a3fff700 1 -- 172.21.15.22:0/3948979069 learned_addr learned my addr 172.21.15.22:0/3948979069 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.977+0000 7f22a37fe700 1 --2- 172.21.15.22:0/3948979069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 0x5558fb18b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.977+0000 7f22a3fff700 1 -- 172.21.15.22:0/3948979069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 msgr2=0x5558fb18b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.977+0000 7f22a3fff700 1 --2- 172.21.15.22:0/3948979069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 0x5558fb18b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.977+0000 7f22a3fff700 1 -- 172.21.15.22:0/3948979069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 msgr2=0x5558fb1822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.977+0000 7f22a3fff700 1 --2- 172.21.15.22:0/3948979069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb1822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.977+0000 7f22a3fff700 1 -- 172.21.15.22:0/3948979069 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558faf19200 con 0x5558fb17aab0 2023-09-20T21:48:41.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.977+0000 7f22a3fff700 1 --2- 172.21.15.22:0/3948979069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 0x5558fb183b30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2294004ae0 tx=0x7f22940096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=194d2521ca504fa server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.977+0000 7f228affd700 1 -- 172.21.15.22:0/3948979069 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22940117d0 con 0x5558fb17aab0 2023-09-20T21:48:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.977+0000 7f228affd700 1 -- 172.21.15.22:0/3948979069 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2294004160 con 0x5558fb17aab0 2023-09-20T21:48:41.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.977+0000 7f228affd700 1 -- 172.21.15.22:0/3948979069 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294019a50 con 0x5558fb17aab0 2023-09-20T21:48:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.978+0000 7f22bdffcb80 1 -- 172.21.15.22:0/3948979069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 msgr2=0x5558fb183b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.978+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/3948979069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 0x5558fb183b30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2294004ae0 tx=0x7f22940096f0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.978+0000 7f22bdffcb80 1 -- 172.21.15.22:0/3948979069 shutdown_connections 2023-09-20T21:48:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.978+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/3948979069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb1822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.978+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/3948979069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17aab0 0x5558fb183b30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.978+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/3948979069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17b410 0x5558fb18b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.978+0000 7f22bdffcb80 1 -- 172.21.15.22:0/3948979069 >> 172.21.15.22:0/3948979069 conn(0x5558fb0a2fb0 msgr2=0x5558fb17edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.978+0000 7f22bdffcb80 1 -- 172.21.15.22:0/3948979069 shutdown_connections 2023-09-20T21:48:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.978+0000 7f22bdffcb80 1 -- 172.21.15.22:0/3948979069 wait complete. 2023-09-20T21:48:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.978+0000 7f22bdffcb80 1 -- start start 2023-09-20T21:48:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.979+0000 7f22bdffcb80 1 Processor -- start 2023-09-20T21:48:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.979+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb1889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17b410 0x5558fb18af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22bdffcb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb18b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb18bb60 con 0x5558fb17bfb0 2023-09-20T21:48:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb18bcd0 con 0x5558fb17b410 2023-09-20T21:48:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22bdffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558fb1e39a0 con 0x5558fb17aab0 2023-09-20T21:48:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22bdffcb80 10 client.0 _ll_register_callbacks cb 0x5558fafaea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb1889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22a8b9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb18b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17b410 0x5558fb18af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb1889f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60840/0 (socket says 172.21.15.22:60840) 2023-09-20T21:48:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22a3fff700 1 -- 172.21.15.22:0/942976218 learned_addr learned my addr 172.21.15.22:0/942976218 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22a3fff700 1 -- 172.21.15.22:0/942976218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17b410 msgr2=0x5558fb18af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22a3fff700 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17b410 0x5558fb18af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22a3fff700 1 -- 172.21.15.22:0/942976218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 msgr2=0x5558fb18b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22a3fff700 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb18b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.980+0000 7f22a3fff700 1 -- 172.21.15.22:0/942976218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558faedbc10 con 0x5558fb17aab0 2023-09-20T21:48:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.981+0000 7f22a3fff700 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb1889f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2294005e70 tx=0x7f2294005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aba6351eab3cb393 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.981+0000 7f22837fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:48:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.981+0000 7f22837fe700 1 -- 172.21.15.22:0/942976218 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22940044f0 con 0x5558fb17aab0 2023-09-20T21:48:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.981+0000 7f22837fe700 1 -- 172.21.15.22:0/942976218 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f2294000e40 con 0x5558fb17aab0 2023-09-20T21:48:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.981+0000 7f22bdffcb80 1 -- 172.21.15.22:0/942976218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558fb0c0e10 con 0x5558fb17aab0 2023-09-20T21:48:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.981+0000 7f22837fe700 1 -- 172.21.15.22:0/942976218 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2294004f50 con 0x5558fb17aab0 2023-09-20T21:48:41.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.981+0000 7f22bdffcb80 10 client.25378 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:41.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.981+0000 7f22bdffcb80 1 -- 172.21.15.22:0/942976218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5558faef1260 con 0x5558fb17aab0 2023-09-20T21:48:41.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.981+0000 7f22837fe700 1 -- 172.21.15.22:0/942976218 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558fb0c0e10 con 0x5558fb17aab0 2023-09-20T21:48:41.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.981+0000 7f22837fe700 1 -- 172.21.15.22:0/942976218 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2294011440 con 0x5558fb17aab0 2023-09-20T21:48:41.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.982+0000 7f22bdffcb80 10 client.25378 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.982+0000 7f22bdffcb80 10 client.25378 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-20T21:48:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.982+0000 7f22bdffcb80 20 client.25378 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.982+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5558fb1e8630 0x5558fb1e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.982+0000 7f22bdffcb80 4 client.25378 mds_command: new command op to 24566 tid=0[{"prefix": "status"}] 2023-09-20T21:48:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.982+0000 7f22bdffcb80 1 -- 172.21.15.22:0/942976218 --> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5558fb0c0b30 con 0x5558fb1e8630 2023-09-20T21:48:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.982+0000 7f22a37fe700 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5558fb1e8630 0x5558fb1e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.982+0000 7f22837fe700 10 client.25378 ms_handle_connect on v2:172.21.15.22:6840/1806616417 2023-09-20T21:48:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.982+0000 7f22a37fe700 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5558fb1e8630 0x5558fb1e4ab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3d01a6af7b03028 server_cookie=78eba3a788ddc932 in_seq=0 out_seq=0 2023-09-20T21:48:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22837fe700 1 -- 172.21.15.22:0/942976218 <== mds.1 v2:172.21.15.22:6840/1806616417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5558fb0c0b30 con 0x5558fb1e8630 2023-09-20T21:48:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22837fe700 10 client.25378 handle_command_reply: tid=0 2023-09-20T21:48:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 client.25378 shutdown 2023-09-20T21:48:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 -- 172.21.15.22:0/942976218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 msgr2=0x5558fb1889f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb1889f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2294005e70 tx=0x7f2294005de0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 -- 172.21.15.22:0/942976218 shutdown_connections 2023-09-20T21:48:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5558fb1e8630 0x5558fb1e4ab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5558fb17bfb0 0x5558fb18b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558fb17aab0 0x5558fb1889f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 --2- 172.21.15.22:0/942976218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5558fb17b410 0x5558fb18af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22837fe700 0 client.25378 ms_handle_reset on v2:172.21.15.22:6840/1806616417 2023-09-20T21:48:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 -- 172.21.15.22:0/942976218 >> 172.21.15.22:0/942976218 conn(0x5558fb0a2fb0 msgr2=0x5558fb17edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 -- 172.21.15.22:0/942976218 shutdown_connections 2023-09-20T21:48:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 1 -- 172.21.15.22:0/942976218 wait complete. 2023-09-20T21:48:41.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.983+0000 7f22bdffcb80 20 client.25378 trim_cache size 0 max 16384 2023-09-20T21:48:41.998 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:41.998 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:48:41.998 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 1, 2023-09-20T21:48:41.999 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24566, 2023-09-20T21:48:41.999 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:48:41.999 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:48:41.999 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:48:41.999 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 96.403691422999998, 2023-09-20T21:48:42.000 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 28, 2023-09-20T21:48:42.000 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2023-09-20T21:48:42.000 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 91, 2023-09-20T21:48:42.000 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 129.15744327300001 2023-09-20T21:48:42.000 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:42.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 -- 172.21.15.22:0/2621237660 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f228c0a1300 msgr2=0x7f228c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f228c0a1300 0x7f228c0a37c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f229c009fd0 tx=0x7f229c00b040 comp rx=0 tx=0).stop 2023-09-20T21:48:42.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 -- 172.21.15.22:0/2621237660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101fc0 msgr2=0x7f22a4197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101fc0 0x7f22a4197060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2298009990 tx=0x7f229800f690 comp rx=0 tx=0).stop 2023-09-20T21:48:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 -- 172.21.15.22:0/2621237660 shutdown_connections 2023-09-20T21:48:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a4102960 0x7f22a419e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f228c0a1300 0x7f228c0a37c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22a4100dd0 0x7f22a4196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 --2- 172.21.15.22:0/2621237660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22a4101fc0 0x7f22a4197060 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 -- 172.21.15.22:0/2621237660 >> 172.21.15.22:0/2621237660 conn(0x7f22a40fc320 msgr2=0x7f22a40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:42.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 -- 172.21.15.22:0/2621237660 shutdown_connections 2023-09-20T21:48:42.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:41.984+0000 7f22aa5ff700 1 -- 172.21.15.22:0/2621237660 wait complete. 2023-09-20T21:48:42.004 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-20T21:48:42.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:41 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2517722332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:42.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:41 smithi123 ceph-mon[134433]: pgmap v340: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:42.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:41 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2517722332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:42.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:41 smithi089 ceph-mon[134090]: pgmap v340: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-09-20T21:48:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.209+0000 7ff03b7ff700 1 -- 172.21.15.22:0/692228925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff034100dd0 msgr2=0x7ff0341011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.209+0000 7ff03b7ff700 1 --2- 172.21.15.22:0/692228925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff034100dd0 0x7ff0341011f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff0300099c0 tx=0x7ff030009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.209+0000 7ff03b7ff700 1 -- 172.21.15.22:0/692228925 shutdown_connections 2023-09-20T21:48:42.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.209+0000 7ff03b7ff700 1 --2- 172.21.15.22:0/692228925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff034102960 0x7ff034109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.209+0000 7ff03b7ff700 1 --2- 172.21.15.22:0/692228925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff034100dd0 0x7ff0341011f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.209+0000 7ff03b7ff700 1 --2- 172.21.15.22:0/692228925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff034101fc0 0x7ff034102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.209+0000 7ff03b7ff700 1 -- 172.21.15.22:0/692228925 >> 172.21.15.22:0/692228925 conn(0x7ff0340fc320 msgr2=0x7ff0340fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.209+0000 7ff03b7ff700 1 -- 172.21.15.22:0/692228925 shutdown_connections 2023-09-20T21:48:42.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.209+0000 7ff03b7ff700 1 -- 172.21.15.22:0/692228925 wait complete. 2023-09-20T21:48:42.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.210+0000 7ff03b7ff700 1 Processor -- start 2023-09-20T21:48:42.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.210+0000 7ff03b7ff700 1 -- start start 2023-09-20T21:48:42.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff03b7ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff034100dd0 0x7ff034196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff03b7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff034101fc0 0x7ff034197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff03b7ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff034102960 0x7ff03419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff03b7ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03419e690 con 0x7ff034102960 2023-09-20T21:48:42.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff03b7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03419e7d0 con 0x7ff034101fc0 2023-09-20T21:48:42.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff03b7ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03419e960 con 0x7ff034100dd0 2023-09-20T21:48:42.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff039d9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff034102960 0x7ff03419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff038d9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff034101fc0 0x7ff034197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff039d9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff034102960 0x7ff03419e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48034/0 (socket says 172.21.15.22:48034) 2023-09-20T21:48:42.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff039d9c700 1 -- 172.21.15.22:0/802545908 learned_addr learned my addr 172.21.15.22:0/802545908 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:42.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.211+0000 7ff03959b700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff034100dd0 0x7ff034196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff039d9c700 1 -- 172.21.15.22:0/802545908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff034100dd0 msgr2=0x7ff034196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff039d9c700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff034100dd0 0x7ff034196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff039d9c700 1 -- 172.21.15.22:0/802545908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff034101fc0 msgr2=0x7ff034197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff039d9c700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff034101fc0 0x7ff034197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff039d9c700 1 -- 172.21.15.22:0/802545908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0300095f0 con 0x7ff034102960 2023-09-20T21:48:42.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff03959b700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff034100dd0 0x7ff034196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:42.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff039d9c700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff034102960 0x7ff03419e0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00ef30 tx=0x7ff02c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:42.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff02a7fc700 1 -- 172.21.15.22:0/802545908 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c018070 con 0x7ff034102960 2023-09-20T21:48:42.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff03b7ff700 1 -- 172.21.15.22:0/802545908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03419ebe0 con 0x7ff034102960 2023-09-20T21:48:42.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff02a7fc700 1 -- 172.21.15.22:0/802545908 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff02c00f040 con 0x7ff034102960 2023-09-20T21:48:42.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff02a7fc700 1 -- 172.21.15.22:0/802545908 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c00cd00 con 0x7ff034102960 2023-09-20T21:48:42.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.212+0000 7ff03b7ff700 1 -- 172.21.15.22:0/802545908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03419f140 con 0x7ff034102960 2023-09-20T21:48:42.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.215+0000 7ff02a7fc700 1 -- 172.21.15.22:0/802545908 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ff02c003a00 con 0x7ff034102960 2023-09-20T21:48:42.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.217+0000 7ff04f0f0b80 1 Processor -- start 2023-09-20T21:48:42.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.217+0000 7ff02a7fc700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0200a1300 0x7ff0200a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.217+0000 7ff04f0f0b80 1 -- start start 2023-09-20T21:48:42.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.217+0000 7ff02a7fc700 1 -- 172.21.15.22:0/802545908 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ff02c0ccb00 con 0x7ff034102960 2023-09-20T21:48:42.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.217+0000 7ff038d9a700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0200a1300 0x7ff0200a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.217+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb2ab0 0x557e85eb2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.217+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 0x557e85eb39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.217+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85eb4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.217+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85e22c50 con 0x557e85eb3fb0 2023-09-20T21:48:42.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85e06c20 con 0x557e85eb3410 2023-09-20T21:48:42.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85e04540 con 0x557e85eb2ab0 2023-09-20T21:48:42.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff03959b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb2ab0 0x557e85eb2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff03959b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb2ab0 0x557e85eb2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60870/0 (socket says 172.21.15.22:60870) 2023-09-20T21:48:42.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff03959b700 1 -- 172.21.15.22:0/1796589941 learned_addr learned my addr 172.21.15.22:0/1796589941 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:42.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff038d9a700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0200a1300 0x7ff0200a37c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff024009cb0 tx=0x7ff024009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:42.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff038d9a700 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 0x557e85eb39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff039d9c700 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85eb4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff038d9a700 1 -- 172.21.15.22:0/1796589941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb2ab0 msgr2=0x557e85eb2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:42.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff038d9a700 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb2ab0 0x557e85eb2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff038d9a700 1 -- 172.21.15.22:0/1796589941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 msgr2=0x557e85eb4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:42.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff038d9a700 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85eb4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff038d9a700 1 -- 172.21.15.22:0/1796589941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e85c2ad90 con 0x557e85eb3410 2023-09-20T21:48:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff03959b700 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb2ab0 0x557e85eb2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.218+0000 7ff039d9c700 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85eb4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:42.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.219+0000 7ff038d9a700 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 0x557e85eb39d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff02400f3c0 tx=0x7ff024014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a235d3986efb85c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:42.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.219+0000 7ff01f7fe700 1 -- 172.21.15.22:0/1796589941 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02400fe20 con 0x557e85eb3410 2023-09-20T21:48:42.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.219+0000 7ff01f7fe700 1 -- 172.21.15.22:0/1796589941 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff024021070 con 0x557e85eb3410 2023-09-20T21:48:42.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.219+0000 7ff01f7fe700 1 -- 172.21.15.22:0/1796589941 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024005280 con 0x557e85eb3410 2023-09-20T21:48:42.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.220+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1796589941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 msgr2=0x557e85eb39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.220+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 0x557e85eb39d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff02400f3c0 tx=0x7ff024014000 comp rx=0 tx=0).stop 2023-09-20T21:48:42.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.220+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1796589941 shutdown_connections 2023-09-20T21:48:42.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.220+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85eb4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.220+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb2ab0 0x557e85eb2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.220+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1796589941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 0x557e85eb39d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.220+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1796589941 >> 172.21.15.22:0/1796589941 conn(0x557e85ddafb0 msgr2=0x557e85ea9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:42.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.220+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1796589941 shutdown_connections 2023-09-20T21:48:42.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.220+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1796589941 wait complete. 2023-09-20T21:48:42.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.220+0000 7ff04f0f0b80 1 -- start start 2023-09-20T21:48:42.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff04f0f0b80 1 Processor -- start 2023-09-20T21:48:42.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb2ab0 0x557e85ebc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3410 0x557e85ebca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85ec3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85dffe30 con 0x557e85eb3fb0 2023-09-20T21:48:42.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85d9c6f0 con 0x557e85eb2ab0 2023-09-20T21:48:42.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85d9c390 con 0x557e85eb3410 2023-09-20T21:48:42.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff04f0f0b80 10 client.0 _ll_register_callbacks cb 0x557e85ce6a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:42.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff039d9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85ec3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff039d9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85ec3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48044/0 (socket says 172.21.15.22:48044) 2023-09-20T21:48:42.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff039d9c700 1 -- 172.21.15.22:0/66057423 learned_addr learned my addr 172.21.15.22:0/66057423 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:42.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff038d9a700 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3410 0x557e85ebca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.221+0000 7ff03959b700 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb2ab0 0x557e85ebc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff039d9c700 1 -- 172.21.15.22:0/66057423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3410 msgr2=0x557e85ebca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:42.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff039d9c700 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3410 0x557e85ebca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff039d9c700 1 -- 172.21.15.22:0/66057423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb2ab0 msgr2=0x557e85ebc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:42.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff039d9c700 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb2ab0 0x557e85ebc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff039d9c700 1 -- 172.21.15.22:0/66057423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff034109b60 con 0x557e85eb3fb0 2023-09-20T21:48:42.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff039d9c700 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85ec3a70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00ef00 tx=0x7ff02c099560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddf9ad4eee1243c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:42.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff01cff9700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:42.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff01cff9700 1 -- 172.21.15.22:0/66057423 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c09a230 con 0x557e85eb3fb0 2023-09-20T21:48:42.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/66057423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e85d70b60 con 0x557e85eb3fb0 2023-09-20T21:48:42.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff01cff9700 1 -- 172.21.15.22:0/66057423 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ff02c00ce60 con 0x557e85eb3fb0 2023-09-20T21:48:42.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff01cff9700 1 -- 172.21.15.22:0/66057423 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff02c099b30 con 0x557e85eb3fb0 2023-09-20T21:48:42.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff04f0f0b80 10 client.15363 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:42.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.222+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/66057423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7ff03419f140 con 0x557e85eb3fb0 2023-09-20T21:48:42.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.223+0000 7ff01cff9700 1 -- 172.21.15.22:0/66057423 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e85d70b60 con 0x557e85eb3fb0 2023-09-20T21:48:42.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.223+0000 7ff01cff9700 1 -- 172.21.15.22:0/66057423 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff02c099020 con 0x557e85eb3fb0 2023-09-20T21:48:42.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.223+0000 7ff04f0f0b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:42.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.223+0000 7ff04f0f0b80 10 client.15363 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-20T21:48:42.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.223+0000 7ff04f0f0b80 20 client.15363 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:42.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.223+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e85ec80c0 0x557e85ec6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.223+0000 7ff04f0f0b80 4 client.15363 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:42.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.223+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/66057423 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e85e7cfd0 con 0x557e85ec80c0 2023-09-20T21:48:42.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.223+0000 7ff03959b700 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e85ec80c0 0x557e85ec6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.224+0000 7ff03959b700 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e85ec80c0 0x557e85ec6c20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9529088d1efcf7 server_cookie=a849943e4c26619 in_seq=0 out_seq=0 2023-09-20T21:48:42.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.224+0000 7ff01cff9700 10 client.15363 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:42.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff01cff9700 1 -- 172.21.15.22:0/66057423 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e85e7cfd0 con 0x557e85ec80c0 2023-09-20T21:48:42.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff01cff9700 10 client.15363 handle_command_reply: tid=0 2023-09-20T21:48:42.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 client.15363 shutdown 2023-09-20T21:48:42.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/66057423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 msgr2=0x557e85ec3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85ec3a70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00ef00 tx=0x7ff02c099560 comp rx=0 tx=0).stop 2023-09-20T21:48:42.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/66057423 shutdown_connections 2023-09-20T21:48:42.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e85ec80c0 0x557e85ec6c20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb3fb0 0x557e85ec3a70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb2ab0 0x557e85ebc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/66057423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3410 0x557e85ebca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff01cff9700 0 client.15363 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:42.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/66057423 >> 172.21.15.22:0/66057423 conn(0x557e85ddafb0 msgr2=0x557e85ea8830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:42.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/66057423 shutdown_connections 2023-09-20T21:48:42.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/66057423 wait complete. 2023-09-20T21:48:42.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.225+0000 7ff04f0f0b80 20 client.15363 trim_cache size 0 max 16384 2023-09-20T21:48:42.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.243+0000 7ff04f0f0b80 1 Processor -- start 2023-09-20T21:48:42.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.243+0000 7ff04f0f0b80 1 -- start start 2023-09-20T21:48:42.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85eba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3410 0x557e85ec3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3fb0 0x557e85eb6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85c808d0 con 0x557e85eb2ab0 2023-09-20T21:48:42.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85d98aa0 con 0x557e85eb3fb0 2023-09-20T21:48:42.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85ea3bd0 con 0x557e85eb3410 2023-09-20T21:48:42.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff03959b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85eba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff03959b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85eba3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48054/0 (socket says 172.21.15.22:48054) 2023-09-20T21:48:42.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff03959b700 1 -- 172.21.15.22:0/1264525780 learned_addr learned my addr 172.21.15.22:0/1264525780 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:42.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff039d9c700 1 --2- 172.21.15.22:0/1264525780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3fb0 0x557e85eb6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff038d9a700 1 --2- 172.21.15.22:0/1264525780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3410 0x557e85ec3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff03959b700 1 -- 172.21.15.22:0/1264525780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3410 msgr2=0x557e85ec3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:42.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff03959b700 1 --2- 172.21.15.22:0/1264525780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3410 0x557e85ec3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff03959b700 1 -- 172.21.15.22:0/1264525780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3fb0 msgr2=0x557e85eb6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:42.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff03959b700 1 --2- 172.21.15.22:0/1264525780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3fb0 0x557e85eb6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.244+0000 7ff03959b700 1 -- 172.21.15.22:0/1264525780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff024017050 con 0x557e85eb2ab0 2023-09-20T21:48:42.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.245+0000 7ff03959b700 1 --2- 172.21.15.22:0/1264525780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85eba3d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff03000a010 tx=0x7ff030005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ccadc4d41976c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:42.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.245+0000 7ff01ffff700 1 -- 172.21.15.22:0/1264525780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03002bd00 con 0x557e85eb2ab0 2023-09-20T21:48:42.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.245+0000 7ff01ffff700 1 -- 172.21.15.22:0/1264525780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff03001e040 con 0x557e85eb2ab0 2023-09-20T21:48:42.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.245+0000 7ff01ffff700 1 -- 172.21.15.22:0/1264525780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000fdb0 con 0x557e85eb2ab0 2023-09-20T21:48:42.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.245+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1264525780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 msgr2=0x557e85eba3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.245+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1264525780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85eba3d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff03000a010 tx=0x7ff030005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.246+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1264525780 shutdown_connections 2023-09-20T21:48:42.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.246+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1264525780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3fb0 0x557e85eb6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.246+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1264525780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85eba3d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.246+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1264525780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3410 0x557e85ec3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.246+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1264525780 >> 172.21.15.22:0/1264525780 conn(0x557e85ddafb0 msgr2=0x557e85ebb210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:42.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.246+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1264525780 shutdown_connections 2023-09-20T21:48:42.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.246+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1264525780 wait complete. 2023-09-20T21:48:42.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.246+0000 7ff04f0f0b80 1 -- start start 2023-09-20T21:48:42.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.246+0000 7ff04f0f0b80 1 Processor -- start 2023-09-20T21:48:42.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85ec35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 0x557e85ec0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff04f0f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3fb0 0x557e85ec05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85ec0cb0 con 0x557e85eb2ab0 2023-09-20T21:48:42.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85ec3c70 con 0x557e85eb3410 2023-09-20T21:48:42.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff04f0f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e85ec3e00 con 0x557e85eb3fb0 2023-09-20T21:48:42.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff04f0f0b80 10 client.0 _ll_register_callbacks cb 0x557e85ce6a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:42.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff03959b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85ec35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff03959b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85ec35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48066/0 (socket says 172.21.15.22:48066) 2023-09-20T21:48:42.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff03959b700 1 -- 172.21.15.22:0/1332251195 learned_addr learned my addr 172.21.15.22:0/1332251195 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:42.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff038d9a700 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 0x557e85ec0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff039d9c700 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3fb0 0x557e85ec05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff03959b700 1 -- 172.21.15.22:0/1332251195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3fb0 msgr2=0x557e85ec05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:42.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff03959b700 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3fb0 0x557e85ec05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff03959b700 1 -- 172.21.15.22:0/1332251195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 msgr2=0x557e85ec0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:42.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff03959b700 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 0x557e85ec0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.247+0000 7ff03959b700 1 -- 172.21.15.22:0/1332251195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03000f460 con 0x557e85eb2ab0 2023-09-20T21:48:42.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.248+0000 7ff03959b700 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85ec35f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff0300098e0 tx=0x7ff03000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35aa108f89a2f253 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:42.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.248+0000 7ff01cff9700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:42.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.248+0000 7ff01cff9700 1 -- 172.21.15.22:0/1332251195 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03002bb00 con 0x557e85eb2ab0 2023-09-20T21:48:42.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.248+0000 7ff01cff9700 1 -- 172.21.15.22:0/1332251195 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ff03002bc60 con 0x557e85eb2ab0 2023-09-20T21:48:42.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.248+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1332251195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff024021070 con 0x557e85eb2ab0 2023-09-20T21:48:42.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.248+0000 7ff01cff9700 1 -- 172.21.15.22:0/1332251195 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff03002be20 con 0x557e85eb2ab0 2023-09-20T21:48:42.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.248+0000 7ff04f0f0b80 10 client.15369 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:42.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.248+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1332251195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557e85c13c10 con 0x557e85eb2ab0 2023-09-20T21:48:42.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.248+0000 7ff01cff9700 1 -- 172.21.15.22:0/1332251195 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff024021070 con 0x557e85eb2ab0 2023-09-20T21:48:42.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.249+0000 7ff01cff9700 1 -- 172.21.15.22:0/1332251195 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff0300099f0 con 0x557e85eb2ab0 2023-09-20T21:48:42.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.249+0000 7ff04f0f0b80 10 client.15369 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:42.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.249+0000 7ff04f0f0b80 10 client.15369 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-20T21:48:42.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.249+0000 7ff04f0f0b80 20 client.15369 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:42.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.249+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e85f206f0 0x557e85f1cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:42.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.249+0000 7ff04f0f0b80 4 client.15369 mds_command: new command op to 24578 tid=0[{"prefix": "status"}] 2023-09-20T21:48:42.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.249+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1332251195 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557e85df8e10 con 0x557e85f206f0 2023-09-20T21:48:42.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.249+0000 7ff038d9a700 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e85f206f0 0x557e85f1cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:42.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff038d9a700 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e85f206f0 0x557e85f1cb90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421ccebeda2088fa server_cookie=8949fbc68b3bb33f in_seq=0 out_seq=0 2023-09-20T21:48:42.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff01cff9700 10 client.15369 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:42.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff01cff9700 1 -- 172.21.15.22:0/1332251195 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x557e85df8e10 con 0x557e85f206f0 2023-09-20T21:48:42.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff01cff9700 10 client.15369 handle_command_reply: tid=0 2023-09-20T21:48:42.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 client.15369 shutdown 2023-09-20T21:48:42.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1332251195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 msgr2=0x557e85ec35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85ec35f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff0300098e0 tx=0x7ff03000b6b0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1332251195 shutdown_connections 2023-09-20T21:48:42.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e85f206f0 0x557e85f1cb90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e85eb3fb0 0x557e85ec05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e85eb2ab0 0x557e85ec35f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff01cff9700 0 client.15369 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:42.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 --2- 172.21.15.22:0/1332251195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e85eb3410 0x557e85ec0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1332251195 >> 172.21.15.22:0/1332251195 conn(0x557e85ddafb0 msgr2=0x557e85eba3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:42.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1332251195 shutdown_connections 2023-09-20T21:48:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 1 -- 172.21.15.22:0/1332251195 wait complete. 2023-09-20T21:48:42.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.250+0000 7ff04f0f0b80 20 client.15369 trim_cache size 0 max 16384 2023-09-20T21:48:42.277 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:42.277 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da", 2023-09-20T21:48:42.278 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2023-09-20T21:48:42.278 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 24578, 2023-09-20T21:48:42.278 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-09-20T21:48:42.278 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-09-20T21:48:42.278 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-09-20T21:48:42.279 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 120.736651088, 2023-09-20T21:48:42.279 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 28, 2023-09-20T21:48:42.279 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 93, 2023-09-20T21:48:42.279 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 91, 2023-09-20T21:48:42.279 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 127.001497683 2023-09-20T21:48:42.279 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:42.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 -- 172.21.15.22:0/802545908 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0200a1300 msgr2=0x7ff0200a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0200a1300 0x7ff0200a37c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff024009cb0 tx=0x7ff024009260 comp rx=0 tx=0).stop 2023-09-20T21:48:42.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 -- 172.21.15.22:0/802545908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff034102960 msgr2=0x7ff03419e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:42.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff034102960 0x7ff03419e0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00ef30 tx=0x7ff02c00c3a0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 -- 172.21.15.22:0/802545908 shutdown_connections 2023-09-20T21:48:42.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff034102960 0x7ff03419e0c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff034100dd0 0x7ff034196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0200a1300 0x7ff0200a37c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 --2- 172.21.15.22:0/802545908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff034101fc0 0x7ff034197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:42.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 -- 172.21.15.22:0/802545908 >> 172.21.15.22:0/802545908 conn(0x7ff0340fc320 msgr2=0x7ff0340fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:42.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 -- 172.21.15.22:0/802545908 shutdown_connections 2023-09-20T21:48:42.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:42.251+0000 7ff03b7ff700 1 -- 172.21.15.22:0/802545908 wait complete. 2023-09-20T21:48:42.283 INFO:teuthology.run_tasks:Running task kclient... 2023-09-20T21:48:42.299 INFO:tasks.kclient:Mounting kernel clients... 2023-09-20T21:48:42.300 INFO:tasks.kclient:config is {'client.0': {}} 2023-09-20T21:48:42.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:42.301 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-09-20T21:48:42.345 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-09-20T21:48:42.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:42.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-20T21:48:42.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:42.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2023-09-20T21:48:42.564 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mountpoint': '/tmp/cephfs'} 2023-09-20T21:48:42.565 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-20T21:48:42.565 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-09-20T21:48:42.565 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-09-20T21:48:42.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph fs ls 2023-09-20T21:48:43.302 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T21:48:43.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:43 smithi089 ceph-mon[134090]: pgmap v341: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:48:43.873 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:43 smithi022 ceph-mon[97651]: pgmap v341: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:48:44.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:43 smithi123 ceph-mon[134433]: pgmap v341: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-09-20T21:48:44.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.872+0000 7f2766b60700 1 -- 172.21.15.22:0/3032810328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2760101060 msgr2=0x7f2760101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.872+0000 7f2766b60700 1 --2- 172.21.15.22:0/3032810328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2760101060 0x7f2760101440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2754009990 tx=0x7f2754009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.872+0000 7f2766b60700 1 -- 172.21.15.22:0/3032810328 shutdown_connections 2023-09-20T21:48:44.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.872+0000 7f2766b60700 1 --2- 172.21.15.22:0/3032810328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2760106140 0x7f276010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:44.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.872+0000 7f2766b60700 1 --2- 172.21.15.22:0/3032810328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2760101060 0x7f2760101440 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:44.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.872+0000 7f2766b60700 1 --2- 172.21.15.22:0/3032810328 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2760101a10 0x7f2760105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:44.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.872+0000 7f2766b60700 1 -- 172.21.15.22:0/3032810328 >> 172.21.15.22:0/3032810328 conn(0x7f27600fc8d0 msgr2=0x7f27600fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:44.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.872+0000 7f2766b60700 1 -- 172.21.15.22:0/3032810328 shutdown_connections 2023-09-20T21:48:44.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.872+0000 7f2766b60700 1 -- 172.21.15.22:0/3032810328 wait complete. 2023-09-20T21:48:44.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.873+0000 7f2766b60700 1 Processor -- start 2023-09-20T21:48:44.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.873+0000 7f2766b60700 1 -- start start 2023-09-20T21:48:44.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.873+0000 7f2766b60700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2760101060 0x7f27600730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:44.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.873+0000 7f2766b60700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2760101a10 0x7f2760073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:44.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.873+0000 7f2766b60700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2760106140 0x7f2760073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:44.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f2766b60700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601a53d0 con 0x7f2760106140 2023-09-20T21:48:44.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f2766b60700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601a5540 con 0x7f2760101060 2023-09-20T21:48:44.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f2766b60700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601a5700 con 0x7f2760101a10 2023-09-20T21:48:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f275ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2760101a10 0x7f2760073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f275ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2760101a10 0x7f2760073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:60904/0 (socket says 172.21.15.22:60904) 2023-09-20T21:48:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f27648fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2760101060 0x7f27600730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f275ffff700 1 -- 172.21.15.22:0/1146012719 learned_addr learned my addr 172.21.15.22:0/1146012719 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:44.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f27650fd700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2760106140 0x7f2760073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f27648fc700 1 -- 172.21.15.22:0/1146012719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2760101a10 msgr2=0x7f2760073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f27648fc700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2760101a10 0x7f2760073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f27648fc700 1 -- 172.21.15.22:0/1146012719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2760106140 msgr2=0x7f2760073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f27648fc700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2760106140 0x7f2760073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:44.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f27648fc700 1 -- 172.21.15.22:0/1146012719 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27540095f0 con 0x7f2760101060 2023-09-20T21:48:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.874+0000 7f275ffff700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2760101a10 0x7f2760073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.875+0000 7f27648fc700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2760101060 0x7f27600730e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2754000c00 tx=0x7f2754005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.875+0000 7f275dffb700 1 -- 172.21.15.22:0/1146012719 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275401d070 con 0x7f2760101060 2023-09-20T21:48:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.875+0000 7f2766b60700 1 -- 172.21.15.22:0/1146012719 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27601a5980 con 0x7f2760101060 2023-09-20T21:48:44.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.875+0000 7f275dffb700 1 -- 172.21.15.22:0/1146012719 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2754021410 con 0x7f2760101060 2023-09-20T21:48:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.875+0000 7f275dffb700 1 -- 172.21.15.22:0/1146012719 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2754017650 con 0x7f2760101060 2023-09-20T21:48:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.875+0000 7f2766b60700 1 -- 172.21.15.22:0/1146012719 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27601a5df0 con 0x7f2760101060 2023-09-20T21:48:44.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.876+0000 7f2766b60700 1 -- 172.21.15.22:0/1146012719 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276004eae0 con 0x7f2760101060 2023-09-20T21:48:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.878+0000 7f275dffb700 1 -- 172.21.15.22:0/1146012719 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2754021580 con 0x7f2760101060 2023-09-20T21:48:44.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.880+0000 7f275dffb700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27500a1380 0x7f27500a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:44.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.880+0000 7f275dffb700 1 -- 172.21.15.22:0/1146012719 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f27540ce980 con 0x7f2760101060 2023-09-20T21:48:44.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.880+0000 7f275ffff700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27500a1380 0x7f27500a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:44.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.880+0000 7f275ffff700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27500a1380 0x7f27500a3840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f27600748d0 tx=0x7f274c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:44.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:44.882+0000 7f275dffb700 1 -- 172.21.15.22:0/1146012719 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2754097cf0 con 0x7f2760101060 2023-09-20T21:48:45.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.139+0000 7f2766b60700 1 -- 172.21.15.22:0/1146012719 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2760110fa0 con 0x7f2760101060 2023-09-20T21:48:45.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.140+0000 7f275dffb700 1 -- 172.21.15.22:0/1146012719 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7f275409b310 con 0x7f2760101060 2023-09-20T21:48:45.141 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-20T21:48:45.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.142+0000 7f2766b60700 1 -- 172.21.15.22:0/1146012719 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27500a1380 msgr2=0x7f27500a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:45.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.142+0000 7f2766b60700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27500a1380 0x7f27500a3840 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f27600748d0 tx=0x7f274c009260 comp rx=0 tx=0).stop 2023-09-20T21:48:45.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.142+0000 7f2766b60700 1 -- 172.21.15.22:0/1146012719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2760101060 msgr2=0x7f27600730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.142+0000 7f2766b60700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2760101060 0x7f27600730e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2754000c00 tx=0x7f2754005740 comp rx=0 tx=0).stop 2023-09-20T21:48:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.142+0000 7f2766b60700 1 -- 172.21.15.22:0/1146012719 shutdown_connections 2023-09-20T21:48:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.142+0000 7f2766b60700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2760106140 0x7f2760073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:45.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.142+0000 7f2766b60700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f27500a1380 0x7f27500a3840 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:45.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.142+0000 7f2766b60700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2760101060 0x7f27600730e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:45.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.142+0000 7f2766b60700 1 --2- 172.21.15.22:0/1146012719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2760101a10 0x7f2760073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:45.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.142+0000 7f2766b60700 1 -- 172.21.15.22:0/1146012719 >> 172.21.15.22:0/1146012719 conn(0x7f27600fc8d0 msgr2=0x7f27600fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:45.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.143+0000 7f2766b60700 1 -- 172.21.15.22:0/1146012719 shutdown_connections 2023-09-20T21:48:45.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:45.143+0000 7f2766b60700 1 -- 172.21.15.22:0/1146012719 wait complete. 2023-09-20T21:48:45.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:45 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1146012719' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-20T21:48:45.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:45 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1146012719' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-20T21:48:45.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:45 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1146012719' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-09-20T21:48:45.687 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-20T21:48:45.687 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-20T21:48:45.688 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2023-09-20T21:48:45.688 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-20T21:48:45.688 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-09-20T21:48:45.688 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-20T21:48:45.689 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-20T21:48:45.689 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-20T21:48:45.689 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-09-20T21:48:45.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:45.690 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2023-09-20T21:48:45.736 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-20T21:48:45.736 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-20T21:48:45.736 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-09-20T21:48:45.737 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-09-20T21:48:45.737 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-09-20T21:48:45.737 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-09-20T21:48:45.737 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-20T21:48:45.738 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2023-09-20T21:48:45.738 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2023-09-20T21:48:45.738 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-20T21:48:45.738 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2023-09-20T21:48:45.739 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2023-09-20T21:48:45.739 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-20T21:48:45.739 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2023-09-20T21:48:45.739 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2023-09-20T21:48:45.739 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-20T21:48:45.740 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 41617sec preferred_lft 41617sec 2023-09-20T21:48:45.740 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-09-20T21:48:45.740 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-09-20T21:48:45.740 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-20T21:48:45.740 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2023-09-20T21:48:45.741 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2023-09-20T21:48:45.741 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-20T21:48:45.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:45.742 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-09-20T21:48:45.742 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-09-20T21:48:45.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2023-09-20T21:48:45.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2023-09-20T21:48:45.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2023-09-20T21:48:45.744 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-20T21:48:45.745 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-09-20T21:48:45.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:45.987 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-20T21:48:46.058 INFO:teuthology.orchestra.run.smithi022.stdout:1 2023-09-20T21:48:46.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:46.062 DEBUG:teuthology.orchestra.run.smithi022:> route 2023-09-20T21:48:46.110 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2023-09-20T21:48:46.111 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-20T21:48:46.111 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-20T21:48:46.111 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-20T21:48:46.111 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-20T21:48:46.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:46.113 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-09-20T21:48:46.113 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-09-20T21:48:46.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-20T21:48:46.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-20T21:48:46.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-20T21:48:46.114 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-09-20T21:48:46.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:46.330 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-09-20T21:48:46.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:46.376 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2023-09-20T21:48:46.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:46.459 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-09-20T21:48:46.460 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-09-20T21:48:46.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--tmp-cephfs 2023-09-20T21:48:46.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-09-20T21:48:46.460 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-09-20T21:48:46.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:46 smithi022 ceph-mon[97651]: pgmap v342: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:48:46.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:46 smithi123 ceph-mon[134433]: pgmap v342: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:48:46.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:46 smithi089 ceph-mon[134090]: pgmap v342: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T21:48:46.644 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-09-20T21:48:46.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:46.645 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-09-20T21:48:46.645 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-09-20T21:48:46.646 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-09-20T21:48:46.646 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-20T21:48:46.646 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-09-20T21:48:46.646 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-09-20T21:48:46.647 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-09-20T21:48:46.647 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-09-20T21:48:46.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:46.901 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-09-20T21:48:46.901 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-09-20T21:48:46.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2023-09-20T21:48:46.902 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2023-09-20T21:48:46.902 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-09-20T21:48:47.078 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-20T21:48:47.078 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /tmp/cephfs 2023-09-20T21:48:47.122 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/tmp/cephfs' 2023-09-20T21:48:47.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-20T21:48:47.123 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /tmp/cephfs 2023-09-20T21:48:47.205 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-09-20T21:48:47.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:47.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context=system_u:object_r:postgresql_db_t:s0 2023-09-20T21:48:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0" 2023-09-20T21:48:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-09-20T21:48:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2023-09-20T21:48:47.832 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0" 2023-09-20T21:48:47.833 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-09-20T21:48:47.833 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2023-09-20T21:48:47.833 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.89:6789,172.21.15.123:6789" 2023-09-20T21:48:47.833 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.89:6789,172.21.15.123:6789:/ 2023-09-20T21:48:47.833 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da" will pass to kernel 2023-09-20T21:48:47.840 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-09-20T21:48:47.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:47.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /tmp/cephfs 2023-09-20T21:48:47.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:47.913 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-20T21:48:48.000 INFO:teuthology.orchestra.run.smithi022.stdout:client25412 2023-09-20T21:48:48.001 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T21:48:48.001 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-20T21:48:48.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:48.046 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-20T21:48:48.172 INFO:teuthology.orchestra.run.smithi022.stdout:client25412 2023-09-20T21:48:48.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:48.173 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-09-20T21:48:48.260 INFO:teuthology.orchestra.run.smithi022.stdout:client25412 2023-09-20T21:48:48.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T21:48:48.261 DEBUG:teuthology.orchestra.run.smithi022:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da.client25412/status') 2023-09-20T21:48:48.369 INFO:teuthology.orchestra.run.smithi022.stdout:instance: client.25412 (3)172.21.15.22:0/795080241 2023-09-20T21:48:48.370 INFO:teuthology.orchestra.run.smithi022.stdout:blocklisted: false 2023-09-20T21:48:48.370 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records in 2023-09-20T21:48:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records out 2023-09-20T21:48:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:70 bytes copied, 5.6201e-05 s, 1.2 MB/s 2023-09-20T21:48:48.373 INFO:teuthology.run_tasks:Running task check-counter... 2023-09-20T21:48:48.384 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-09-20T21:48:48.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:48:48.464 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:48 smithi022 ceph-mon[97651]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:48.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:48 smithi123 ceph-mon[134433]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:48.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:48 smithi089 ceph-mon[134090]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T21:48:48.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.610+0000 7f95355ff700 1 -- 172.21.15.22:0/1163888786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95300737e0 msgr2=0x7f9530073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:48.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.610+0000 7f95355ff700 1 --2- 172.21.15.22:0/1163888786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95300737e0 0x7f9530073c60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9524009990 tx=0x7f9524009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:48.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.610+0000 7f95355ff700 1 -- 172.21.15.22:0/1163888786 shutdown_connections 2023-09-20T21:48:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.610+0000 7f95355ff700 1 --2- 172.21.15.22:0/1163888786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9530108cb0 0x7f953010b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.610+0000 7f95355ff700 1 --2- 172.21.15.22:0/1163888786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530074dc0 0x7f9530073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.610+0000 7f95355ff700 1 --2- 172.21.15.22:0/1163888786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95300737e0 0x7f9530073c60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.610+0000 7f95355ff700 1 -- 172.21.15.22:0/1163888786 >> 172.21.15.22:0/1163888786 conn(0x7f95300fc310 msgr2=0x7f95300fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.611+0000 7f95355ff700 1 -- 172.21.15.22:0/1163888786 shutdown_connections 2023-09-20T21:48:48.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.611+0000 7f95355ff700 1 -- 172.21.15.22:0/1163888786 wait complete. 2023-09-20T21:48:48.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f95355ff700 1 Processor -- start 2023-09-20T21:48:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f95355ff700 1 -- start start 2023-09-20T21:48:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f95355ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300737e0 0x7f9530198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f95355ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9530074dc0 0x7f95301991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:48.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f95355ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530108cb0 0x7f95301a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f95355ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95301a07e0 con 0x7f9530074dc0 2023-09-20T21:48:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f95355ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95301a0950 con 0x7f9530108cb0 2023-09-20T21:48:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f95355ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95301a0b10 con 0x7f95300737e0 2023-09-20T21:48:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f952e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9530074dc0 0x7f95301991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:48.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f952effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300737e0 0x7f9530198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f952f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530108cb0 0x7f95301a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f952e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9530074dc0 0x7f95301991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59500/0 (socket says 172.21.15.22:59500) 2023-09-20T21:48:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f952effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300737e0 0x7f9530198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:59618/0 (socket says 172.21.15.22:59618) 2023-09-20T21:48:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.612+0000 7f952e7fc700 1 -- 172.21.15.22:0/2376596418 learned_addr learned my addr 172.21.15.22:0/2376596418 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:48.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f952effd700 1 -- 172.21.15.22:0/2376596418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530108cb0 msgr2=0x7f95301a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:48.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f952effd700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530108cb0 0x7f95301a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:48.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f952effd700 1 -- 172.21.15.22:0/2376596418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9530074dc0 msgr2=0x7f95301991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:48.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f952effd700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9530074dc0 0x7f95301991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:48.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f952effd700 1 -- 172.21.15.22:0/2376596418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95240095f0 con 0x7f95300737e0 2023-09-20T21:48:48.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f952e7fc700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9530074dc0 0x7f95301991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f952f7fe700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530108cb0 0x7f95301a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f952effd700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300737e0 0x7f9530198c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f952000e9f0 tx=0x7f952000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f9517fff700 1 -- 172.21.15.22:0/2376596418 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000c9c0 con 0x7f95300737e0 2023-09-20T21:48:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f95355ff700 1 -- 172.21.15.22:0/2376596418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95301a0df0 con 0x7f95300737e0 2023-09-20T21:48:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f9517fff700 1 -- 172.21.15.22:0/2376596418 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9520010430 con 0x7f95300737e0 2023-09-20T21:48:48.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.613+0000 7f9517fff700 1 -- 172.21.15.22:0/2376596418 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520018680 con 0x7f95300737e0 2023-09-20T21:48:48.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.614+0000 7f95355ff700 1 -- 172.21.15.22:0/2376596418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95301a1220 con 0x7f95300737e0 2023-09-20T21:48:48.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.615+0000 7f95355ff700 1 -- 172.21.15.22:0/2376596418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95301073b0 con 0x7f95300737e0 2023-09-20T21:48:48.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.617+0000 7f9517fff700 1 -- 172.21.15.22:0/2376596418 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f9520010860 con 0x7f95300737e0 2023-09-20T21:48:48.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.618+0000 7f9517fff700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95180a1350 0x7f95180a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:48.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.618+0000 7f9517fff700 1 -- 172.21.15.22:0/2376596418 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f9520014070 con 0x7f95300737e0 2023-09-20T21:48:48.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.618+0000 7f952e7fc700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95180a1350 0x7f95180a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:48.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.619+0000 7f952e7fc700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95180a1350 0x7f95180a3810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f953019a170 tx=0x7f9524005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:48.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.620+0000 7f9517fff700 1 -- 172.21.15.22:0/2376596418 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9520096600 con 0x7f95300737e0 2023-09-20T21:48:48.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.838+0000 7f95355ff700 1 -- 172.21.15.22:0/2376596418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f953010e7d0 con 0x7f95300737e0 2023-09-20T21:48:48.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.840+0000 7f9517fff700 1 -- 172.21.15.22:0/2376596418 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7f9520099c20 con 0x7f95300737e0 2023-09-20T21:48:48.841 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:48.841 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:48:48.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.842+0000 7f95355ff700 1 -- 172.21.15.22:0/2376596418 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95180a1350 msgr2=0x7f95180a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:48.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.842+0000 7f95355ff700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95180a1350 0x7f95180a3810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f953019a170 tx=0x7f9524005960 comp rx=0 tx=0).stop 2023-09-20T21:48:48.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.842+0000 7f95355ff700 1 -- 172.21.15.22:0/2376596418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300737e0 msgr2=0x7f9530198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:48.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.842+0000 7f95355ff700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300737e0 0x7f9530198c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f952000e9f0 tx=0x7f952000ed00 comp rx=0 tx=0).stop 2023-09-20T21:48:48.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.842+0000 7f95355ff700 1 -- 172.21.15.22:0/2376596418 shutdown_connections 2023-09-20T21:48:48.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.842+0000 7f95355ff700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f95180a1350 0x7f95180a3810 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:48.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.842+0000 7f95355ff700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9530108cb0 0x7f95301a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:48.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.842+0000 7f95355ff700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95300737e0 0x7f9530198c70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.843+0000 7f95355ff700 1 --2- 172.21.15.22:0/2376596418 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9530074dc0 0x7f95301991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.843+0000 7f95355ff700 1 -- 172.21.15.22:0/2376596418 >> 172.21.15.22:0/2376596418 conn(0x7f95300fc310 msgr2=0x7f95300fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.843+0000 7f95355ff700 1 -- 172.21.15.22:0/2376596418 shutdown_connections 2023-09-20T21:48:48.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:48.843+0000 7f95355ff700 1 -- 172.21.15.22:0/2376596418 wait complete. 2023-09-20T21:48:48.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T21:48:48.863 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:48:49.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.072+0000 7fd4e6bff700 1 -- 172.21.15.22:0/3468844000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4e0101fc0 msgr2=0x7fd4e0102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.072+0000 7fd4e6bff700 1 --2- 172.21.15.22:0/3468844000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4e0101fc0 0x7fd4e0102420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009920 tx=0x7fd4cc009c30 comp rx=0 tx=0).stop 2023-09-20T21:48:49.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.072+0000 7fd4e6bff700 1 -- 172.21.15.22:0/3468844000 shutdown_connections 2023-09-20T21:48:49.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.072+0000 7fd4e6bff700 1 --2- 172.21.15.22:0/3468844000 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e0102960 0x7fd4e0109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.072+0000 7fd4e6bff700 1 --2- 172.21.15.22:0/3468844000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e0100dd0 0x7fd4e01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.072+0000 7fd4e6bff700 1 --2- 172.21.15.22:0/3468844000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4e0101fc0 0x7fd4e0102420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.072+0000 7fd4e6bff700 1 -- 172.21.15.22:0/3468844000 >> 172.21.15.22:0/3468844000 conn(0x7fd4e00fc320 msgr2=0x7fd4e00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:49.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.072+0000 7fd4e6bff700 1 -- 172.21.15.22:0/3468844000 shutdown_connections 2023-09-20T21:48:49.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.072+0000 7fd4e6bff700 1 -- 172.21.15.22:0/3468844000 wait complete. 2023-09-20T21:48:49.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.073+0000 7fd4e6bff700 1 Processor -- start 2023-09-20T21:48:49.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.073+0000 7fd4e6bff700 1 -- start start 2023-09-20T21:48:49.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4e6bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4e0100dd0 0x7fd4e0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4e6bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e0101fc0 0x7fd4e01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4e6bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e0102960 0x7fd4e01a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4e6bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01a08e0 con 0x7fd4e0101fc0 2023-09-20T21:48:49.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4e6bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01a0a20 con 0x7fd4e0100dd0 2023-09-20T21:48:49.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4e6bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01a0bb0 con 0x7fd4e0102960 2023-09-20T21:48:49.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4dffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e0101fc0 0x7fd4e01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4e499b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4e0100dd0 0x7fd4e0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4dffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e0101fc0 0x7fd4e01992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59522/0 (socket says 172.21.15.22:59522) 2023-09-20T21:48:49.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4dffff700 1 -- 172.21.15.22:0/353063634 learned_addr learned my addr 172.21.15.22:0/353063634 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:49.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4e519c700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e0102960 0x7fd4e01a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4dffff700 1 -- 172.21.15.22:0/353063634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e0102960 msgr2=0x7fd4e01a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4dffff700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e0102960 0x7fd4e01a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4dffff700 1 -- 172.21.15.22:0/353063634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4e0100dd0 msgr2=0x7fd4e0198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4dffff700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4e0100dd0 0x7fd4e0198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4dffff700 1 -- 172.21.15.22:0/353063634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d8009d00 con 0x7fd4e0101fc0 2023-09-20T21:48:49.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.074+0000 7fd4e499b700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4e0100dd0 0x7fd4e0198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:49.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.075+0000 7fd4dffff700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e0101fc0 0x7fd4e01992a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009230 tx=0x7fd4cc00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:49.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.075+0000 7fd4ddffb700 1 -- 172.21.15.22:0/353063634 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc01c070 con 0x7fd4e0101fc0 2023-09-20T21:48:49.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.075+0000 7fd4e6bff700 1 -- 172.21.15.22:0/353063634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc009520 con 0x7fd4e0101fc0 2023-09-20T21:48:49.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.075+0000 7fd4ddffb700 1 -- 172.21.15.22:0/353063634 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4cc00bba0 con 0x7fd4e0101fc0 2023-09-20T21:48:49.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.075+0000 7fd4ddffb700 1 -- 172.21.15.22:0/353063634 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc003a40 con 0x7fd4e0101fc0 2023-09-20T21:48:49.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.075+0000 7fd4e6bff700 1 -- 172.21.15.22:0/353063634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e01a1120 con 0x7fd4e0101fc0 2023-09-20T21:48:49.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.076+0000 7fd4e6bff700 1 -- 172.21.15.22:0/353063634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e01933d0 con 0x7fd4e0101fc0 2023-09-20T21:48:49.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.078+0000 7fd4ddffb700 1 -- 172.21.15.22:0/353063634 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fd4cc020030 con 0x7fd4e0101fc0 2023-09-20T21:48:49.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.080+0000 7fd4ddffb700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd4d00a1380 0x7fd4d00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.080+0000 7fd4ddffb700 1 -- 172.21.15.22:0/353063634 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fd4cc0ce360 con 0x7fd4e0101fc0 2023-09-20T21:48:49.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.080+0000 7fd4e499b700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd4d00a1380 0x7fd4d00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.080+0000 7fd4e499b700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd4d00a1380 0x7fd4d00a3840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0101e20 tx=0x7fd4d4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:49.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.082+0000 7fd4ddffb700 1 -- 172.21.15.22:0/353063634 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fd4cc0976d0 con 0x7fd4e0101fc0 2023-09-20T21:48:49.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.292+0000 7fd4e6bff700 1 -- 172.21.15.22:0/353063634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4e010c830 con 0x7fd4e0101fc0 2023-09-20T21:48:49.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.292+0000 7fd4ddffb700 1 -- 172.21.15.22:0/353063634 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4cc005040 con 0x7fd4e0101fc0 2023-09-20T21:48:49.293 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:48:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.294+0000 7fd4e6bff700 1 -- 172.21.15.22:0/353063634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd4d00a1380 msgr2=0x7fd4d00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.294+0000 7fd4e6bff700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd4d00a1380 0x7fd4d00a3840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0101e20 tx=0x7fd4d4009260 comp rx=0 tx=0).stop 2023-09-20T21:48:49.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.294+0000 7fd4e6bff700 1 -- 172.21.15.22:0/353063634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e0101fc0 msgr2=0x7fd4e01992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.294+0000 7fd4e6bff700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e0101fc0 0x7fd4e01992a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009230 tx=0x7fd4cc00bf80 comp rx=0 tx=0).stop 2023-09-20T21:48:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.295+0000 7fd4e6bff700 1 -- 172.21.15.22:0/353063634 shutdown_connections 2023-09-20T21:48:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.295+0000 7fd4e6bff700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4e0102960 0x7fd4e01a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.295+0000 7fd4e6bff700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd4d00a1380 0x7fd4d00a3840 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.295+0000 7fd4e6bff700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd4e0100dd0 0x7fd4e0198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.295+0000 7fd4e6bff700 1 --2- 172.21.15.22:0/353063634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd4e0101fc0 0x7fd4e01992a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.295+0000 7fd4e6bff700 1 -- 172.21.15.22:0/353063634 >> 172.21.15.22:0/353063634 conn(0x7fd4e00fc320 msgr2=0x7fd4e01079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:49.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.295+0000 7fd4e6bff700 1 -- 172.21.15.22:0/353063634 shutdown_connections 2023-09-20T21:48:49.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.295+0000 7fd4e6bff700 1 -- 172.21.15.22:0/353063634 wait complete. 2023-09-20T21:48:49.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T21:48:49.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:49 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2376596418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:49.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.525+0000 7fa2b9bff700 1 -- 172.21.15.22:0/1435173861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b4101fc0 msgr2=0x7fa2b4102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.525+0000 7fa2b9bff700 1 --2- 172.21.15.22:0/1435173861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b4101fc0 0x7fa2b4102420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80099c0 tx=0x7fa2a8009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.526+0000 7fa2b9bff700 1 -- 172.21.15.22:0/1435173861 shutdown_connections 2023-09-20T21:48:49.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.526+0000 7fa2b9bff700 1 --2- 172.21.15.22:0/1435173861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b4102960 0x7fa2b4109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.526+0000 7fa2b9bff700 1 --2- 172.21.15.22:0/1435173861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b4100dd0 0x7fa2b41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.526+0000 7fa2b9bff700 1 --2- 172.21.15.22:0/1435173861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b4101fc0 0x7fa2b4102420 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.526+0000 7fa2b9bff700 1 -- 172.21.15.22:0/1435173861 >> 172.21.15.22:0/1435173861 conn(0x7fa2b40fc320 msgr2=0x7fa2b40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:49.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.526+0000 7fa2b9bff700 1 -- 172.21.15.22:0/1435173861 shutdown_connections 2023-09-20T21:48:49.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.526+0000 7fa2b9bff700 1 -- 172.21.15.22:0/1435173861 wait complete. 2023-09-20T21:48:49.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b9bff700 1 Processor -- start 2023-09-20T21:48:49.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b9bff700 1 -- start start 2023-09-20T21:48:49.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b9bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b4100dd0 0x7fa2b4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b9bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b4101fc0 0x7fa2b4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b9bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b4102960 0x7fa2b41a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b9bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b41a0860 con 0x7fa2b4100dd0 2023-09-20T21:48:49.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b9bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b41a09d0 con 0x7fa2b4102960 2023-09-20T21:48:49.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b9bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b41a0b90 con 0x7fa2b4101fc0 2023-09-20T21:48:49.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b4100dd0 0x7fa2b4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b4100dd0 0x7fa2b4198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59532/0 (socket says 172.21.15.22:59532) 2023-09-20T21:48:49.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.527+0000 7fa2b37fe700 1 -- 172.21.15.22:0/146716777 learned_addr learned my addr 172.21.15.22:0/146716777 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:49.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b3fff700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b4102960 0x7fa2b41a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b2ffd700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b4101fc0 0x7fa2b4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b37fe700 1 -- 172.21.15.22:0/146716777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b4101fc0 msgr2=0x7fa2b4199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b37fe700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b4101fc0 0x7fa2b4199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b37fe700 1 -- 172.21.15.22:0/146716777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b4102960 msgr2=0x7fa2b41a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b37fe700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b4102960 0x7fa2b41a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b37fe700 1 -- 172.21.15.22:0/146716777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a4009520 con 0x7fa2b4100dd0 2023-09-20T21:48:49.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b37fe700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b4100dd0 0x7fa2b4198d20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400ea50 tx=0x7fa2a400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:49.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b0ff9700 1 -- 172.21.15.22:0/146716777 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400cbd0 con 0x7fa2b4100dd0 2023-09-20T21:48:49.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b9bff700 1 -- 172.21.15.22:0/146716777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a80095f0 con 0x7fa2b4100dd0 2023-09-20T21:48:49.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b0ff9700 1 -- 172.21.15.22:0/146716777 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2a400cd30 con 0x7fa2b4100dd0 2023-09-20T21:48:49.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b0ff9700 1 -- 172.21.15.22:0/146716777 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4018680 con 0x7fa2b4100dd0 2023-09-20T21:48:49.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.528+0000 7fa2b9bff700 1 -- 172.21.15.22:0/146716777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b41a11a0 con 0x7fa2b4100dd0 2023-09-20T21:48:49.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.529+0000 7fa2b9bff700 1 -- 172.21.15.22:0/146716777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b41933d0 con 0x7fa2b4100dd0 2023-09-20T21:48:49.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.531+0000 7fa2b0ff9700 1 -- 172.21.15.22:0/146716777 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa2a4010430 con 0x7fa2b4100dd0 2023-09-20T21:48:49.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.533+0000 7fa2b0ff9700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa29c0a1300 0x7fa29c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.533+0000 7fa2b0ff9700 1 -- 172.21.15.22:0/146716777 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa2a4014070 con 0x7fa2b4100dd0 2023-09-20T21:48:49.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.533+0000 7fa2b2ffd700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa29c0a1300 0x7fa29c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.534+0000 7fa2b2ffd700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa29c0a1300 0x7fa29c0a37c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800b580 tx=0x7fa2a8005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:49.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.537+0000 7fa2b0ff9700 1 -- 172.21.15.22:0/146716777 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa2a40967d0 con 0x7fa2b4100dd0 2023-09-20T21:48:49.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:49 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2376596418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:49.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:49 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2376596418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:49.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.753+0000 7fa2b9bff700 1 -- 172.21.15.22:0/146716777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2b402cef0 con 0x7fa2b4100dd0 2023-09-20T21:48:49.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.754+0000 7fa2b0ff9700 1 -- 172.21.15.22:0/146716777 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2a4099df0 con 0x7fa2b4100dd0 2023-09-20T21:48:49.754 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:49.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 -- 172.21.15.22:0/146716777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa29c0a1300 msgr2=0x7fa29c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa29c0a1300 0x7fa29c0a37c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800b580 tx=0x7fa2a8005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 -- 172.21.15.22:0/146716777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b4100dd0 msgr2=0x7fa2b4198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b4100dd0 0x7fa2b4198d20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400ea50 tx=0x7fa2a400ed60 comp rx=0 tx=0).stop 2023-09-20T21:48:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 -- 172.21.15.22:0/146716777 shutdown_connections 2023-09-20T21:48:49.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa29c0a1300 0x7fa29c0a37c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa2b4102960 0x7fa2b41a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa2b4100dd0 0x7fa2b4198d20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 --2- 172.21.15.22:0/146716777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2b4101fc0 0x7fa2b4199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 -- 172.21.15.22:0/146716777 >> 172.21.15.22:0/146716777 conn(0x7fa2b40fc320 msgr2=0x7fa2b41078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:49.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 -- 172.21.15.22:0/146716777 shutdown_connections 2023-09-20T21:48:49.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.756+0000 7fa2b9bff700 1 -- 172.21.15.22:0/146716777 wait complete. 2023-09-20T21:48:49.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T21:48:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.982+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/4118533863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ec4100dd0 msgr2=0x7f5ec41011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.982+0000 7f5ecb5ff700 1 --2- 172.21.15.22:0/4118533863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ec4100dd0 0x7f5ec41011f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00099c0 tx=0x7f5ec0009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.982+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/4118533863 shutdown_connections 2023-09-20T21:48:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.982+0000 7f5ecb5ff700 1 --2- 172.21.15.22:0/4118533863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec4102960 0x7f5ec4109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.982+0000 7f5ecb5ff700 1 --2- 172.21.15.22:0/4118533863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ec4100dd0 0x7f5ec41011f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.982+0000 7f5ecb5ff700 1 --2- 172.21.15.22:0/4118533863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec4101fc0 0x7f5ec4102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.982+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/4118533863 >> 172.21.15.22:0/4118533863 conn(0x7f5ec40fc320 msgr2=0x7f5ec40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:49.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.983+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/4118533863 shutdown_connections 2023-09-20T21:48:49.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.983+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/4118533863 wait complete. 2023-09-20T21:48:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.984+0000 7f5ecb5ff700 1 Processor -- start 2023-09-20T21:48:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.984+0000 7f5ecb5ff700 1 -- start start 2023-09-20T21:48:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ecb5ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec4100dd0 0x7f5ec4196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ecb5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec4101fc0 0x7f5ec41970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ecb5ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ec4102960 0x7f5ec419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ecb5ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec419e6f0 con 0x7f5ec4102960 2023-09-20T21:48:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ecb5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec419e830 con 0x7f5ec4101fc0 2023-09-20T21:48:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ecb5ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec419e9c0 con 0x7f5ec4100dd0 2023-09-20T21:48:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ec9b9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ec4102960 0x7f5ec419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ec8b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec4101fc0 0x7f5ec41970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ec8b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec4101fc0 0x7f5ec41970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37454/0 (socket says 172.21.15.22:37454) 2023-09-20T21:48:49.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ec8b9a700 1 -- 172.21.15.22:0/3223047716 learned_addr learned my addr 172.21.15.22:0/3223047716 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:49.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ec939b700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec4100dd0 0x7f5ec4196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ec8b9a700 1 -- 172.21.15.22:0/3223047716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec4100dd0 msgr2=0x7f5ec4196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ec8b9a700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec4100dd0 0x7f5ec4196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.985+0000 7f5ec8b9a700 1 -- 172.21.15.22:0/3223047716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ec4102960 msgr2=0x7f5ec419e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:49.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.986+0000 7f5ec8b9a700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ec4102960 0x7f5ec419e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:49.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.986+0000 7f5ec8b9a700 1 -- 172.21.15.22:0/3223047716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc009c40 con 0x7f5ec4101fc0 2023-09-20T21:48:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.986+0000 7f5ec8b9a700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec4101fc0 0x7f5ec41970b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400e9a0 tx=0x7f5eb400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.986+0000 7f5eba7fc700 1 -- 172.21.15.22:0/3223047716 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4009790 con 0x7f5ec4101fc0 2023-09-20T21:48:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.986+0000 7f5eba7fc700 1 -- 172.21.15.22:0/3223047716 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb400ce80 con 0x7f5ec4101fc0 2023-09-20T21:48:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.986+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/3223047716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec00095f0 con 0x7f5ec4101fc0 2023-09-20T21:48:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.986+0000 7f5eba7fc700 1 -- 172.21.15.22:0/3223047716 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4018680 con 0x7f5ec4101fc0 2023-09-20T21:48:49.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.986+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/3223047716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec419f070 con 0x7f5ec4101fc0 2023-09-20T21:48:49.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.987+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/3223047716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec410c630 con 0x7f5ec4101fc0 2023-09-20T21:48:49.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.990+0000 7f5eba7fc700 1 -- 172.21.15.22:0/3223047716 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5eb4010430 con 0x7f5ec4101fc0 2023-09-20T21:48:49.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.991+0000 7f5eba7fc700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5eb00a1380 0x7f5eb00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:49.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.991+0000 7f5eba7fc700 1 -- 172.21.15.22:0/3223047716 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f5eb4014070 con 0x7f5ec4101fc0 2023-09-20T21:48:49.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.991+0000 7f5ec939b700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5eb00a1380 0x7f5eb00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:49.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.992+0000 7f5ec939b700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5eb00a1380 0x7f5eb00a3840 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009990 tx=0x7f5ec0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:49.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:49.993+0000 7f5eba7fc700 1 -- 172.21.15.22:0/3223047716 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5eb40967c0 con 0x7f5ec4101fc0 2023-09-20T21:48:50.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.204+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/3223047716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ec4107d50 con 0x7f5ec4101fc0 2023-09-20T21:48:50.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.206+0000 7f5eba7fc700 1 -- 172.21.15.22:0/3223047716 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5eb4026020 con 0x7f5ec4101fc0 2023-09-20T21:48:50.207 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T21:48:50.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/3223047716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5eb00a1380 msgr2=0x7f5eb00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:50.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5eb00a1380 0x7f5eb00a3840 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009990 tx=0x7f5ec0005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/3223047716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec4101fc0 msgr2=0x7f5ec41970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:50.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec4101fc0 0x7f5ec41970b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400e9a0 tx=0x7f5eb400ecb0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/3223047716 shutdown_connections 2023-09-20T21:48:50.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ec4102960 0x7f5ec419e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ec4100dd0 0x7f5ec4196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5eb00a1380 0x7f5eb00a3840 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 --2- 172.21.15.22:0/3223047716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ec4101fc0 0x7f5ec41970b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/3223047716 >> 172.21.15.22:0/3223047716 conn(0x7f5ec40fc320 msgr2=0x7f5ec40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:50.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/3223047716 shutdown_connections 2023-09-20T21:48:50.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.209+0000 7f5ecb5ff700 1 -- 172.21.15.22:0/3223047716 wait complete. 2023-09-20T21:48:50.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T21:48:50.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.435+0000 7f8a975ff700 1 -- 172.21.15.22:0/56011652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90100dd0 msgr2=0x7f8a901011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:50.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.435+0000 7f8a975ff700 1 --2- 172.21.15.22:0/56011652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90100dd0 0x7f8a901011f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0099c0 tx=0x7f8a8c009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.436+0000 7f8a975ff700 1 -- 172.21.15.22:0/56011652 shutdown_connections 2023-09-20T21:48:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.436+0000 7f8a975ff700 1 --2- 172.21.15.22:0/56011652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a90102960 0x7f8a90109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.436+0000 7f8a975ff700 1 --2- 172.21.15.22:0/56011652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90100dd0 0x7f8a901011f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.436+0000 7f8a975ff700 1 --2- 172.21.15.22:0/56011652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a90101fc0 0x7f8a90102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.436+0000 7f8a975ff700 1 -- 172.21.15.22:0/56011652 >> 172.21.15.22:0/56011652 conn(0x7f8a900fc320 msgr2=0x7f8a900fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:50.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.436+0000 7f8a975ff700 1 -- 172.21.15.22:0/56011652 shutdown_connections 2023-09-20T21:48:50.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.436+0000 7f8a975ff700 1 -- 172.21.15.22:0/56011652 wait complete. 2023-09-20T21:48:50.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.437+0000 7f8a975ff700 1 Processor -- start 2023-09-20T21:48:50.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.437+0000 7f8a975ff700 1 -- start start 2023-09-20T21:48:50.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.437+0000 7f8a975ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a90100dd0 0x7f8a90196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:50.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.437+0000 7f8a975ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90101fc0 0x7f8a901970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:50.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.437+0000 7f8a975ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a90102960 0x7f8a9019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:50.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.437+0000 7f8a975ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9019e730 con 0x7f8a90101fc0 2023-09-20T21:48:50.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.437+0000 7f8a975ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9019e870 con 0x7f8a90102960 2023-09-20T21:48:50.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.437+0000 7f8a975ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9019ea00 con 0x7f8a90100dd0 2023-09-20T21:48:50.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a9539b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a90100dd0 0x7f8a90196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:50.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a9539b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a90100dd0 0x7f8a90196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:59712/0 (socket says 172.21.15.22:59712) 2023-09-20T21:48:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a9539b700 1 -- 172.21.15.22:0/1241480632 learned_addr learned my addr 172.21.15.22:0/1241480632 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a94b9a700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90101fc0 0x7f8a901970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a95b9c700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a90102960 0x7f8a9019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a9539b700 1 -- 172.21.15.22:0/1241480632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a90102960 msgr2=0x7f8a9019e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:50.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a9539b700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a90102960 0x7f8a9019e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a9539b700 1 -- 172.21.15.22:0/1241480632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90101fc0 msgr2=0x7f8a901970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a9539b700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90101fc0 0x7f8a901970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a9539b700 1 -- 172.21.15.22:0/1241480632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c0095f0 con 0x7f8a90100dd0 2023-09-20T21:48:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.438+0000 7f8a9539b700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a90100dd0 0x7f8a90196bb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009990 tx=0x7f8a8c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:50.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.439+0000 7f8a867fc700 1 -- 172.21.15.22:0/1241480632 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c01d070 con 0x7f8a90100dd0 2023-09-20T21:48:50.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.439+0000 7f8a867fc700 1 -- 172.21.15.22:0/1241480632 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8c005030 con 0x7f8a90100dd0 2023-09-20T21:48:50.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.439+0000 7f8a975ff700 1 -- 172.21.15.22:0/1241480632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9019ec80 con 0x7f8a90100dd0 2023-09-20T21:48:50.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.439+0000 7f8a867fc700 1 -- 172.21.15.22:0/1241480632 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c004a70 con 0x7f8a90100dd0 2023-09-20T21:48:50.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.439+0000 7f8a975ff700 1 -- 172.21.15.22:0/1241480632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9019f100 con 0x7f8a90100dd0 2023-09-20T21:48:50.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.440+0000 7f8a975ff700 1 -- 172.21.15.22:0/1241480632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a90108460 con 0x7f8a90100dd0 2023-09-20T21:48:50.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.442+0000 7f8a867fc700 1 -- 172.21.15.22:0/1241480632 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f8a8c005410 con 0x7f8a90100dd0 2023-09-20T21:48:50.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.443+0000 7f8a867fc700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a7c0a1380 0x7f8a7c0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:50.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.443+0000 7f8a867fc700 1 -- 172.21.15.22:0/1241480632 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f8a8c0ce0a0 con 0x7f8a90100dd0 2023-09-20T21:48:50.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.443+0000 7f8a94b9a700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a7c0a1380 0x7f8a7c0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:50.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.444+0000 7f8a94b9a700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a7c0a1380 0x7f8a7c0a3840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009c60 tx=0x7f8a80009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:50.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.446+0000 7f8a867fc700 1 -- 172.21.15.22:0/1241480632 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8a8c097410 con 0x7f8a90100dd0 2023-09-20T21:48:50.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:50 smithi022 ceph-mon[97651]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-20T21:48:50.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:50 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/353063634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:50.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:50 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/146716777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:50.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:50 smithi123 ceph-mon[134433]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-20T21:48:50.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:50 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/353063634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:50.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:50 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/146716777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:50.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:50 smithi089 ceph-mon[134090]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-20T21:48:50.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:50 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/353063634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:50.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:50 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/146716777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:50.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.658+0000 7f8a975ff700 1 -- 172.21.15.22:0/1241480632 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8a9010c7c0 con 0x7f8a90100dd0 2023-09-20T21:48:50.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.658+0000 7f8a867fc700 1 -- 172.21.15.22:0/1241480632 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8a8c09aa30 con 0x7f8a90100dd0 2023-09-20T21:48:50.659 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:50.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 -- 172.21.15.22:0/1241480632 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a7c0a1380 msgr2=0x7f8a7c0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:50.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a7c0a1380 0x7f8a7c0a3840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009c60 tx=0x7f8a80009260 comp rx=0 tx=0).stop 2023-09-20T21:48:50.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 -- 172.21.15.22:0/1241480632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a90100dd0 msgr2=0x7f8a90196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:50.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a90100dd0 0x7f8a90196bb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009990 tx=0x7f8a8c00f690 comp rx=0 tx=0).stop 2023-09-20T21:48:50.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 -- 172.21.15.22:0/1241480632 shutdown_connections 2023-09-20T21:48:50.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8a7c0a1380 0x7f8a7c0a3840 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a90102960 0x7f8a9019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a90100dd0 0x7f8a90196bb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 --2- 172.21.15.22:0/1241480632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a90101fc0 0x7f8a901970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 -- 172.21.15.22:0/1241480632 >> 172.21.15.22:0/1241480632 conn(0x7f8a900fc320 msgr2=0x7f8a901055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:50.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 -- 172.21.15.22:0/1241480632 shutdown_connections 2023-09-20T21:48:50.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.661+0000 7f8a975ff700 1 -- 172.21.15.22:0/1241480632 wait complete. 2023-09-20T21:48:50.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:48:50.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.885+0000 7f89ccdff700 1 -- 172.21.15.22:0/4068085214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8101840 msgr2=0x7f89c8103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:50.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.885+0000 7f89ccdff700 1 --2- 172.21.15.22:0/4068085214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8101840 0x7f89c8103c30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009990 tx=0x7f89b8009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.886+0000 7f89ccdff700 1 -- 172.21.15.22:0/4068085214 shutdown_connections 2023-09-20T21:48:50.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.886+0000 7f89ccdff700 1 --2- 172.21.15.22:0/4068085214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c8106aa0 0x7f89c8108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.886+0000 7f89ccdff700 1 --2- 172.21.15.22:0/4068085214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8101840 0x7f89c8103c30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.886+0000 7f89ccdff700 1 --2- 172.21.15.22:0/4068085214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89c8104170 0x7f89c8106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.886+0000 7f89ccdff700 1 -- 172.21.15.22:0/4068085214 >> 172.21.15.22:0/4068085214 conn(0x7f89c80fb210 msgr2=0x7f89c80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:50.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.886+0000 7f89ccdff700 1 -- 172.21.15.22:0/4068085214 shutdown_connections 2023-09-20T21:48:50.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.886+0000 7f89ccdff700 1 -- 172.21.15.22:0/4068085214 wait complete. 2023-09-20T21:48:50.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89ccdff700 1 Processor -- start 2023-09-20T21:48:50.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89ccdff700 1 -- start start 2023-09-20T21:48:50.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89ccdff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8101840 0x7f89c8196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:50.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89ccdff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89c8104170 0x7f89c8196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:50.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89ccdff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c8106aa0 0x7f89c819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:50.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89ccdff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c819e5a0 con 0x7f89c8101840 2023-09-20T21:48:50.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89ccdff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c819e6e0 con 0x7f89c8104170 2023-09-20T21:48:50.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89ccdff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c819e870 con 0x7f89c8106aa0 2023-09-20T21:48:50.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89c659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8101840 0x7f89c8196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89c659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8101840 0x7f89c8196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59584/0 (socket says 172.21.15.22:59584) 2023-09-20T21:48:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.887+0000 7f89c659c700 1 -- 172.21.15.22:0/624279025 learned_addr learned my addr 172.21.15.22:0/624279025 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89c6d9d700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c8106aa0 0x7f89c819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89c5d9b700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89c8104170 0x7f89c8196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89c659c700 1 -- 172.21.15.22:0/624279025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c8106aa0 msgr2=0x7f89c819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89c659c700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c8106aa0 0x7f89c819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89c659c700 1 -- 172.21.15.22:0/624279025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89c8104170 msgr2=0x7f89c8196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89c659c700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89c8104170 0x7f89c8196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89c659c700 1 -- 172.21.15.22:0/624279025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc009520 con 0x7f89c8101840 2023-09-20T21:48:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89c659c700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8101840 0x7f89c8196a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89b8000c00 tx=0x7f89b800f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:50.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89b77fe700 1 -- 172.21.15.22:0/624279025 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b801d070 con 0x7f89c8101840 2023-09-20T21:48:50.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89ccdff700 1 -- 172.21.15.22:0/624279025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b80095f0 con 0x7f89c8101840 2023-09-20T21:48:50.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89b77fe700 1 -- 172.21.15.22:0/624279025 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b800fcb0 con 0x7f89c8101840 2023-09-20T21:48:50.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.888+0000 7f89b77fe700 1 -- 172.21.15.22:0/624279025 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8017650 con 0x7f89c8101840 2023-09-20T21:48:50.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.889+0000 7f89ccdff700 1 -- 172.21.15.22:0/624279025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c819ee10 con 0x7f89c8101840 2023-09-20T21:48:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.889+0000 7f89ccdff700 1 -- 172.21.15.22:0/624279025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c810c5a0 con 0x7f89c8101840 2023-09-20T21:48:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.892+0000 7f89b77fe700 1 -- 172.21.15.22:0/624279025 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f89b8021410 con 0x7f89c8101840 2023-09-20T21:48:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.893+0000 7f89b77fe700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89b00a1350 0x7f89b00a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.893+0000 7f89b77fe700 1 -- 172.21.15.22:0/624279025 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f89b80ce560 con 0x7f89c8101840 2023-09-20T21:48:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.893+0000 7f89c5d9b700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89b00a1350 0x7f89b00a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:50.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.894+0000 7f89c5d9b700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89b00a1350 0x7f89b00a3810 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009d80 tx=0x7f89bc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:50.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:50.897+0000 7f89b77fe700 1 -- 172.21.15.22:0/624279025 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f89b80978d0 con 0x7f89c8101840 2023-09-20T21:48:51.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.110+0000 7f89ccdff700 1 -- 172.21.15.22:0/624279025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89c810c790 con 0x7f89c8101840 2023-09-20T21:48:51.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.111+0000 7f89b77fe700 1 -- 172.21.15.22:0/624279025 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7f89b809aef0 con 0x7f89c8101840 2023-09-20T21:48:51.112 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:51.112 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:48:51.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 -- 172.21.15.22:0/624279025 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89b00a1350 msgr2=0x7f89b00a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:51.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89b00a1350 0x7f89b00a3810 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009d80 tx=0x7f89bc009260 comp rx=0 tx=0).stop 2023-09-20T21:48:51.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 -- 172.21.15.22:0/624279025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8101840 msgr2=0x7f89c8196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:51.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8101840 0x7f89c8196a20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89b8000c00 tx=0x7f89b800f460 comp rx=0 tx=0).stop 2023-09-20T21:48:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 -- 172.21.15.22:0/624279025 shutdown_connections 2023-09-20T21:48:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c8106aa0 0x7f89c819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89c8101840 0x7f89c8196a20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89b00a1350 0x7f89b00a3810 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 --2- 172.21.15.22:0/624279025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f89c8104170 0x7f89c8196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 -- 172.21.15.22:0/624279025 >> 172.21.15.22:0/624279025 conn(0x7f89c80fb210 msgr2=0x7f89c8102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 -- 172.21.15.22:0/624279025 shutdown_connections 2023-09-20T21:48:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.114+0000 7f89ccdff700 1 -- 172.21.15.22:0/624279025 wait complete. 2023-09-20T21:48:51.133 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T21:48:51.134 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:48:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.344+0000 7f6023dff700 1 -- 172.21.15.22:0/2443695575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c100070 msgr2=0x7f601c1004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.344+0000 7f6023dff700 1 --2- 172.21.15.22:0/2443695575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c100070 0x7f601c1004f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f600c0099c0 tx=0x7f600c009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.344+0000 7f6023dff700 1 -- 172.21.15.22:0/2443695575 shutdown_connections 2023-09-20T21:48:51.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.344+0000 7f6023dff700 1 --2- 172.21.15.22:0/2443695575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c100a30 0x7f601c100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.344+0000 7f6023dff700 1 --2- 172.21.15.22:0/2443695575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c0ff710 0x7f601c0ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.344+0000 7f6023dff700 1 --2- 172.21.15.22:0/2443695575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c100070 0x7f601c1004f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.344+0000 7f6023dff700 1 -- 172.21.15.22:0/2443695575 >> 172.21.15.22:0/2443695575 conn(0x7f601c0fb2b0 msgr2=0x7f601c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.344+0000 7f6023dff700 1 -- 172.21.15.22:0/2443695575 shutdown_connections 2023-09-20T21:48:51.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.344+0000 7f6023dff700 1 -- 172.21.15.22:0/2443695575 wait complete. 2023-09-20T21:48:51.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.345+0000 7f6023dff700 1 Processor -- start 2023-09-20T21:48:51.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.345+0000 7f6023dff700 1 -- start start 2023-09-20T21:48:51.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6023dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c0ff710 0x7f601c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:51.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6023dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c100070 0x7f601c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:51.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6023dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c100a30 0x7f601c1a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:51.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6023dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c1a07f0 con 0x7f601c0ff710 2023-09-20T21:48:51.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6023dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c1a0960 con 0x7f601c100070 2023-09-20T21:48:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6023dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c1a0b20 con 0x7f601c100a30 2023-09-20T21:48:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6021b9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c0ff710 0x7f601c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6021b9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c0ff710 0x7f601c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59600/0 (socket says 172.21.15.22:59600) 2023-09-20T21:48:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6021b9b700 1 -- 172.21.15.22:0/2110532847 learned_addr learned my addr 172.21.15.22:0/2110532847 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f602139a700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c100070 0x7f601c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f602239c700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c100a30 0x7f601c1a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6021b9b700 1 -- 172.21.15.22:0/2110532847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c100a30 msgr2=0x7f601c1a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6021b9b700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c100a30 0x7f601c1a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6021b9b700 1 -- 172.21.15.22:0/2110532847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c100070 msgr2=0x7f601c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6021b9b700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c100070 0x7f601c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f6021b9b700 1 -- 172.21.15.22:0/2110532847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0095f0 con 0x7f601c0ff710 2023-09-20T21:48:51.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.346+0000 7f602239c700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c100a30 0x7f601c1a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:51.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.347+0000 7f6021b9b700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c0ff710 0x7f601c198c80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f601800ef10 tx=0x7f601800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:51.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.347+0000 7f6012ffd700 1 -- 172.21.15.22:0/2110532847 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601800ca70 con 0x7f601c0ff710 2023-09-20T21:48:51.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.347+0000 7f6012ffd700 1 -- 172.21.15.22:0/2110532847 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f601800f040 con 0x7f601c0ff710 2023-09-20T21:48:51.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.347+0000 7f6023dff700 1 -- 172.21.15.22:0/2110532847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601c1a0da0 con 0x7f601c0ff710 2023-09-20T21:48:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.347+0000 7f6012ffd700 1 -- 172.21.15.22:0/2110532847 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018014480 con 0x7f601c0ff710 2023-09-20T21:48:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.347+0000 7f6023dff700 1 -- 172.21.15.22:0/2110532847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c1a1210 con 0x7f601c0ff710 2023-09-20T21:48:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.348+0000 7f6023dff700 1 -- 172.21.15.22:0/2110532847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601c10a4f0 con 0x7f601c0ff710 2023-09-20T21:48:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.350+0000 7f6012ffd700 1 -- 172.21.15.22:0/2110532847 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f6018007500 con 0x7f601c0ff710 2023-09-20T21:48:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.351+0000 7f6012ffd700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60080a1380 0x7f60080a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.352+0000 7f6012ffd700 1 -- 172.21.15.22:0/2110532847 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f60180cd2d0 con 0x7f601c0ff710 2023-09-20T21:48:51.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.352+0000 7f602139a700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60080a1380 0x7f60080a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:51.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.352+0000 7f602139a700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60080a1380 0x7f60080a3840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f600c00b580 tx=0x7f600c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:51.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.355+0000 7f6012ffd700 1 -- 172.21.15.22:0/2110532847 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f60180965b0 con 0x7f601c0ff710 2023-09-20T21:48:51.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:51 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3223047716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:51.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:51 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1241480632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:51.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:51 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/624279025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:51.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.566+0000 7f6023dff700 1 -- 172.21.15.22:0/2110532847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f601c02d170 con 0x7f601c0ff710 2023-09-20T21:48:51.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.566+0000 7f6012ffd700 1 -- 172.21.15.22:0/2110532847 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6018099bd0 con 0x7f601c0ff710 2023-09-20T21:48:51.567 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:48:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.568+0000 7f6023dff700 1 -- 172.21.15.22:0/2110532847 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60080a1380 msgr2=0x7f60080a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60080a1380 0x7f60080a3840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f600c00b580 tx=0x7f600c005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 -- 172.21.15.22:0/2110532847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c0ff710 msgr2=0x7f601c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:51.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c0ff710 0x7f601c198c80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f601800ef10 tx=0x7f601800c3a0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 -- 172.21.15.22:0/2110532847 shutdown_connections 2023-09-20T21:48:51.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f601c100a30 0x7f601c1a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f601c0ff710 0x7f601c198c80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60080a1380 0x7f60080a3840 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 --2- 172.21.15.22:0/2110532847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f601c100070 0x7f601c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 -- 172.21.15.22:0/2110532847 >> 172.21.15.22:0/2110532847 conn(0x7f601c0fb2b0 msgr2=0x7f601c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:51.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 -- 172.21.15.22:0/2110532847 shutdown_connections 2023-09-20T21:48:51.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.569+0000 7f6023dff700 1 -- 172.21.15.22:0/2110532847 wait complete. 2023-09-20T21:48:51.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T21:48:51.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:51 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3223047716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:51.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:51 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1241480632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:51.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:51 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/624279025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:51.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:51 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3223047716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:51.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:51 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1241480632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:51.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:51 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/624279025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:51.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.795+0000 7f14f19ff700 1 -- 172.21.15.22:0/41775359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ec102980 msgr2=0x7f14ec109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:51.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.795+0000 7f14f19ff700 1 --2- 172.21.15.22:0/41775359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ec102980 0x7f14ec109230 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f14e400b0d0 tx=0x7f14e400b3e0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.795+0000 7f14f19ff700 1 -- 172.21.15.22:0/41775359 shutdown_connections 2023-09-20T21:48:51.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.795+0000 7f14f19ff700 1 --2- 172.21.15.22:0/41775359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ec102980 0x7f14ec109230 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.795+0000 7f14f19ff700 1 --2- 172.21.15.22:0/41775359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ec100df0 0x7f14ec101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.795+0000 7f14f19ff700 1 --2- 172.21.15.22:0/41775359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ec101fe0 0x7f14ec102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.795+0000 7f14f19ff700 1 -- 172.21.15.22:0/41775359 >> 172.21.15.22:0/41775359 conn(0x7f14ec0fc360 msgr2=0x7f14ec0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:51.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.795+0000 7f14f19ff700 1 -- 172.21.15.22:0/41775359 shutdown_connections 2023-09-20T21:48:51.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.795+0000 7f14f19ff700 1 -- 172.21.15.22:0/41775359 wait complete. 2023-09-20T21:48:51.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.796+0000 7f14f19ff700 1 Processor -- start 2023-09-20T21:48:51.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14f19ff700 1 -- start start 2023-09-20T21:48:51.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14f19ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ec100df0 0x7f14ec198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:51.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14f19ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ec101fe0 0x7f14ec199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:51.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14f19ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ec102980 0x7f14ec1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:51.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14f19ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec1a0990 con 0x7f14ec101fe0 2023-09-20T21:48:51.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14f19ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec1a0ad0 con 0x7f14ec100df0 2023-09-20T21:48:51.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14f19ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec1a0c60 con 0x7f14ec102980 2023-09-20T21:48:51.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14ea7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ec101fe0 0x7f14ec199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:51.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14ea7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ec101fe0 0x7f14ec199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59618/0 (socket says 172.21.15.22:59618) 2023-09-20T21:48:51.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14eaffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ec100df0 0x7f14ec198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:51.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14ea7fc700 1 -- 172.21.15.22:0/1710600188 learned_addr learned my addr 172.21.15.22:0/1710600188 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:51.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.797+0000 7f14eb7fe700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ec102980 0x7f14ec1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:51.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14ea7fc700 1 -- 172.21.15.22:0/1710600188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ec102980 msgr2=0x7f14ec1a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:51.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14ea7fc700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ec102980 0x7f14ec1a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14ea7fc700 1 -- 172.21.15.22:0/1710600188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ec100df0 msgr2=0x7f14ec198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:51.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14ea7fc700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ec100df0 0x7f14ec198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:51.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14ea7fc700 1 -- 172.21.15.22:0/1710600188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e4009d00 con 0x7f14ec101fe0 2023-09-20T21:48:51.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14ea7fc700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ec101fe0 0x7f14ec199350 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f14e000d980 tx=0x7f14e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:51.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14f09fd700 1 -- 172.21.15.22:0/1710600188 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0004220 con 0x7f14ec101fe0 2023-09-20T21:48:51.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14f09fd700 1 -- 172.21.15.22:0/1710600188 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14e0007640 con 0x7f14ec101fe0 2023-09-20T21:48:51.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14f09fd700 1 -- 172.21.15.22:0/1710600188 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0005690 con 0x7f14ec101fe0 2023-09-20T21:48:51.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14f19ff700 1 -- 172.21.15.22:0/1710600188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec1a0f40 con 0x7f14ec101fe0 2023-09-20T21:48:51.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.798+0000 7f14f19ff700 1 -- 172.21.15.22:0/1710600188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec1a1370 con 0x7f14ec101fe0 2023-09-20T21:48:51.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.799+0000 7f14f19ff700 1 -- 172.21.15.22:0/1710600188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec10c660 con 0x7f14ec101fe0 2023-09-20T21:48:51.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.801+0000 7f14f09fd700 1 -- 172.21.15.22:0/1710600188 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f14e0004750 con 0x7f14ec101fe0 2023-09-20T21:48:51.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.803+0000 7f14f09fd700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14d40a1380 0x7f14d40a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:51.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.803+0000 7f14f09fd700 1 -- 172.21.15.22:0/1710600188 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f14e00ccca0 con 0x7f14ec101fe0 2023-09-20T21:48:51.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.803+0000 7f14eaffd700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14d40a1380 0x7f14d40a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:51.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.803+0000 7f14eaffd700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14d40a1380 0x7f14d40a3840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f14ec101e40 tx=0x7f14dc007400 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:51.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:51.806+0000 7f14f09fd700 1 -- 172.21.15.22:0/1710600188 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f14e0095f80 con 0x7f14ec101fe0 2023-09-20T21:48:52.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.016+0000 7f14f19ff700 1 -- 172.21.15.22:0/1710600188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14ec1a17e0 con 0x7f14ec101fe0 2023-09-20T21:48:52.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.016+0000 7f14f09fd700 1 -- 172.21.15.22:0/1710600188 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14e0016020 con 0x7f14ec101fe0 2023-09-20T21:48:52.017 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:52.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 -- 172.21.15.22:0/1710600188 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14d40a1380 msgr2=0x7f14d40a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14d40a1380 0x7f14d40a3840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f14ec101e40 tx=0x7f14dc007400 comp rx=0 tx=0).stop 2023-09-20T21:48:52.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 -- 172.21.15.22:0/1710600188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ec101fe0 msgr2=0x7f14ec199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ec101fe0 0x7f14ec199350 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f14e000d980 tx=0x7f14e000dc90 comp rx=0 tx=0).stop 2023-09-20T21:48:52.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 -- 172.21.15.22:0/1710600188 shutdown_connections 2023-09-20T21:48:52.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ec102980 0x7f14ec1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f14d40a1380 0x7f14d40a3840 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14ec100df0 0x7f14ec198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 --2- 172.21.15.22:0/1710600188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ec101fe0 0x7f14ec199350 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 -- 172.21.15.22:0/1710600188 >> 172.21.15.22:0/1710600188 conn(0x7f14ec0fc360 msgr2=0x7f14ec0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:52.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 -- 172.21.15.22:0/1710600188 shutdown_connections 2023-09-20T21:48:52.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.019+0000 7f14f19ff700 1 -- 172.21.15.22:0/1710600188 wait complete. 2023-09-20T21:48:52.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T21:48:52.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.246+0000 7f3f501ff700 1 -- 172.21.15.22:0/530822839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f48101fc0 msgr2=0x7f3f48102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.246+0000 7f3f501ff700 1 --2- 172.21.15.22:0/530822839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f48101fc0 0x7f3f48102420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f380099c0 tx=0x7f3f38009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.246+0000 7f3f501ff700 1 -- 172.21.15.22:0/530822839 shutdown_connections 2023-09-20T21:48:52.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.246+0000 7f3f501ff700 1 --2- 172.21.15.22:0/530822839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f48102960 0x7f3f48109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.246+0000 7f3f501ff700 1 --2- 172.21.15.22:0/530822839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48100dd0 0x7f3f481011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.246+0000 7f3f501ff700 1 --2- 172.21.15.22:0/530822839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f48101fc0 0x7f3f48102420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.246+0000 7f3f501ff700 1 -- 172.21.15.22:0/530822839 >> 172.21.15.22:0/530822839 conn(0x7f3f480fc320 msgr2=0x7f3f480fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:52.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.246+0000 7f3f501ff700 1 -- 172.21.15.22:0/530822839 shutdown_connections 2023-09-20T21:48:52.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.246+0000 7f3f501ff700 1 -- 172.21.15.22:0/530822839 wait complete. 2023-09-20T21:48:52.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.247+0000 7f3f501ff700 1 Processor -- start 2023-09-20T21:48:52.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.247+0000 7f3f501ff700 1 -- start start 2023-09-20T21:48:52.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f501ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48100dd0 0x7f3f48196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:52.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f501ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f48101fc0 0x7f3f48197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:52.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f501ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f48102960 0x7f3f4819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:52.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f501ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4819e670 con 0x7f3f48102960 2023-09-20T21:48:52.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f501ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4819e7b0 con 0x7f3f48100dd0 2023-09-20T21:48:52.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f501ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4819e940 con 0x7f3f48101fc0 2023-09-20T21:48:52.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f4e79c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f48102960 0x7f3f4819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:52.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f4e79c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f48102960 0x7f3f4819e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59634/0 (socket says 172.21.15.22:59634) 2023-09-20T21:48:52.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f4e79c700 1 -- 172.21.15.22:0/3152952812 learned_addr learned my addr 172.21.15.22:0/3152952812 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:52.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f4df9b700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48100dd0 0x7f3f48196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:52.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.248+0000 7f3f4d79a700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f48101fc0 0x7f3f48197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:52.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f4e79c700 1 -- 172.21.15.22:0/3152952812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f48101fc0 msgr2=0x7f3f48197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f4e79c700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f48101fc0 0x7f3f48197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f4e79c700 1 -- 172.21.15.22:0/3152952812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48100dd0 msgr2=0x7f3f48196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f4e79c700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48100dd0 0x7f3f48196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f4e79c700 1 -- 172.21.15.22:0/3152952812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f380095f0 con 0x7f3f48102960 2023-09-20T21:48:52.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f4d79a700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f48101fc0 0x7f3f48197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:52.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f4e79c700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f48102960 0x7f3f4819e0a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000ef30 tx=0x7f3f4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:52.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f3effd700 1 -- 172.21.15.22:0/3152952812 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40018070 con 0x7f3f48102960 2023-09-20T21:48:52.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f501ff700 1 -- 172.21.15.22:0/3152952812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4819ebc0 con 0x7f3f48102960 2023-09-20T21:48:52.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f3effd700 1 -- 172.21.15.22:0/3152952812 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f4000f040 con 0x7f3f48102960 2023-09-20T21:48:52.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f3effd700 1 -- 172.21.15.22:0/3152952812 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4000cdb0 con 0x7f3f48102960 2023-09-20T21:48:52.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.249+0000 7f3f501ff700 1 -- 172.21.15.22:0/3152952812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4819f0d0 con 0x7f3f48102960 2023-09-20T21:48:52.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.250+0000 7f3f501ff700 1 -- 172.21.15.22:0/3152952812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4810c630 con 0x7f3f48102960 2023-09-20T21:48:52.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.252+0000 7f3f3effd700 1 -- 172.21.15.22:0/3152952812 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f3f40003a00 con 0x7f3f48102960 2023-09-20T21:48:52.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.254+0000 7f3f3effd700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3f340a1300 0x7f3f340a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:52.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.254+0000 7f3f3effd700 1 -- 172.21.15.22:0/3152952812 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f3f400ccbe0 con 0x7f3f48102960 2023-09-20T21:48:52.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.254+0000 7f3f4df9b700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3f340a1300 0x7f3f340a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:52.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.255+0000 7f3f4df9b700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3f340a1300 0x7f3f340a37c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3f48101e20 tx=0x7f3f4400b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.258+0000 7f3f3effd700 1 -- 172.21.15.22:0/3152952812 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3f40095f50 con 0x7f3f48102960 2023-09-20T21:48:52.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.469+0000 7f3f501ff700 1 -- 172.21.15.22:0/3152952812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f48107d00 con 0x7f3f48102960 2023-09-20T21:48:52.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.470+0000 7f3f3effd700 1 -- 172.21.15.22:0/3152952812 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f40099570 con 0x7f3f48102960 2023-09-20T21:48:52.470 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T21:48:52.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 -- 172.21.15.22:0/3152952812 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3f340a1300 msgr2=0x7f3f340a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3f340a1300 0x7f3f340a37c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3f48101e20 tx=0x7f3f4400b410 comp rx=0 tx=0).stop 2023-09-20T21:48:52.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 -- 172.21.15.22:0/3152952812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f48102960 msgr2=0x7f3f4819e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f48102960 0x7f3f4819e0a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000ef30 tx=0x7f3f4000c3a0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 -- 172.21.15.22:0/3152952812 shutdown_connections 2023-09-20T21:48:52.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f48102960 0x7f3f4819e0a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3f340a1300 0x7f3f340a37c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f48100dd0 0x7f3f48196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 --2- 172.21.15.22:0/3152952812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f48101fc0 0x7f3f48197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 -- 172.21.15.22:0/3152952812 >> 172.21.15.22:0/3152952812 conn(0x7f3f480fc320 msgr2=0x7f3f480fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:52.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.472+0000 7f3f501ff700 1 -- 172.21.15.22:0/3152952812 shutdown_connections 2023-09-20T21:48:52.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.473+0000 7f3f501ff700 1 -- 172.21.15.22:0/3152952812 wait complete. 2023-09-20T21:48:52.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:52 smithi022 ceph-mon[97651]: pgmap v345: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-20T21:48:52.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:52 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2110532847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:52.489 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:52 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1710600188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:52.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T21:48:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:52 smithi123 ceph-mon[134433]: pgmap v345: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-20T21:48:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:52 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2110532847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:52 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1710600188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:52.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:52 smithi089 ceph-mon[134090]: pgmap v345: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-20T21:48:52.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:52 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2110532847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:52.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:52 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1710600188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:52.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.699+0000 7f5d07dff700 1 -- 172.21.15.22:0/1858663725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d00100dd0 msgr2=0x7f5d001011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.699+0000 7f5d07dff700 1 --2- 172.21.15.22:0/1858663725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d00100dd0 0x7f5d001011f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0099c0 tx=0x7f5cfc009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.699+0000 7f5d07dff700 1 -- 172.21.15.22:0/1858663725 shutdown_connections 2023-09-20T21:48:52.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.699+0000 7f5d07dff700 1 --2- 172.21.15.22:0/1858663725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00102960 0x7f5d00109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.699+0000 7f5d07dff700 1 --2- 172.21.15.22:0/1858663725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d00100dd0 0x7f5d001011f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.699+0000 7f5d07dff700 1 --2- 172.21.15.22:0/1858663725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d00101fc0 0x7f5d00102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.699+0000 7f5d07dff700 1 -- 172.21.15.22:0/1858663725 >> 172.21.15.22:0/1858663725 conn(0x7f5d000fc320 msgr2=0x7f5d000fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:52.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.700+0000 7f5d07dff700 1 -- 172.21.15.22:0/1858663725 shutdown_connections 2023-09-20T21:48:52.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.700+0000 7f5d07dff700 1 -- 172.21.15.22:0/1858663725 wait complete. 2023-09-20T21:48:52.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d07dff700 1 Processor -- start 2023-09-20T21:48:52.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d07dff700 1 -- start start 2023-09-20T21:48:52.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d07dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d00100dd0 0x7f5d00198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:52.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d07dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101fc0 0x7f5d00199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:52.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d07dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d00102960 0x7f5d001a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:52.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d07dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d001a0860 con 0x7f5d00102960 2023-09-20T21:48:52.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d07dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d001a09d0 con 0x7f5d00100dd0 2023-09-20T21:48:52.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d07dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d001a0b90 con 0x7f5d00101fc0 2023-09-20T21:48:52.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d0639c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d00102960 0x7f5d001a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:52.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d0639c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d00102960 0x7f5d001a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59650/0 (socket says 172.21.15.22:59650) 2023-09-20T21:48:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d0639c700 1 -- 172.21.15.22:0/1055720957 learned_addr learned my addr 172.21.15.22:0/1055720957 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5d05b9b700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d00100dd0 0x7f5d00198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.701+0000 7f5d0539a700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101fc0 0x7f5d00199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5d0639c700 1 -- 172.21.15.22:0/1055720957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101fc0 msgr2=0x7f5d00199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5d0639c700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101fc0 0x7f5d00199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5d0639c700 1 -- 172.21.15.22:0/1055720957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d00100dd0 msgr2=0x7f5d00198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5d0639c700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d00100dd0 0x7f5d00198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5d0639c700 1 -- 172.21.15.22:0/1055720957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0095f0 con 0x7f5d00102960 2023-09-20T21:48:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5d0639c700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d00102960 0x7f5d001a0290 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800ef30 tx=0x7f5cf800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5cf6ffd700 1 -- 172.21.15.22:0/1055720957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8018070 con 0x7f5d00102960 2023-09-20T21:48:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5cf6ffd700 1 -- 172.21.15.22:0/1055720957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cf800f040 con 0x7f5d00102960 2023-09-20T21:48:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5d07dff700 1 -- 172.21.15.22:0/1055720957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d001a0e10 con 0x7f5d00102960 2023-09-20T21:48:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.702+0000 7f5cf6ffd700 1 -- 172.21.15.22:0/1055720957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800cd00 con 0x7f5d00102960 2023-09-20T21:48:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.703+0000 7f5d07dff700 1 -- 172.21.15.22:0/1055720957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d001a11e0 con 0x7f5d00102960 2023-09-20T21:48:52.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.704+0000 7f5d07dff700 1 -- 172.21.15.22:0/1055720957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d00193420 con 0x7f5d00102960 2023-09-20T21:48:52.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.706+0000 7f5cf6ffd700 1 -- 172.21.15.22:0/1055720957 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5cf8003a00 con 0x7f5d00102960 2023-09-20T21:48:52.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.707+0000 7f5cf6ffd700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5cec0a1300 0x7f5cec0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:52.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.707+0000 7f5cf6ffd700 1 -- 172.21.15.22:0/1055720957 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f5cf80ccb10 con 0x7f5d00102960 2023-09-20T21:48:52.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.707+0000 7f5d05b9b700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5cec0a1300 0x7f5cec0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:52.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.708+0000 7f5d05b9b700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5cec0a1300 0x7f5cec0a37c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5d0019a1f0 tx=0x7f5cfc005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:52.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.711+0000 7f5cf6ffd700 1 -- 172.21.15.22:0/1055720957 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5cf8095e80 con 0x7f5d00102960 2023-09-20T21:48:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.924+0000 7f5d07dff700 1 -- 172.21.15.22:0/1055720957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5d0002d140 con 0x7f5d00102960 2023-09-20T21:48:52.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.924+0000 7f5cf6ffd700 1 -- 172.21.15.22:0/1055720957 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5cf80994a0 con 0x7f5d00102960 2023-09-20T21:48:52.925 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:52.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.926+0000 7f5d07dff700 1 -- 172.21.15.22:0/1055720957 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5cec0a1300 msgr2=0x7f5cec0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.926+0000 7f5d07dff700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5cec0a1300 0x7f5cec0a37c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5d0019a1f0 tx=0x7f5cfc005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.926+0000 7f5d07dff700 1 -- 172.21.15.22:0/1055720957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d00102960 msgr2=0x7f5d001a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:52.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.926+0000 7f5d07dff700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d00102960 0x7f5d001a0290 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800ef30 tx=0x7f5cf800c3a0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.926+0000 7f5d07dff700 1 -- 172.21.15.22:0/1055720957 shutdown_connections 2023-09-20T21:48:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.926+0000 7f5d07dff700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d00102960 0x7f5d001a0290 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.926+0000 7f5d07dff700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5cec0a1300 0x7f5cec0a37c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.926+0000 7f5d07dff700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5d00100dd0 0x7f5d00198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.926+0000 7f5d07dff700 1 --2- 172.21.15.22:0/1055720957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101fc0 0x7f5d00199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:52.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.926+0000 7f5d07dff700 1 -- 172.21.15.22:0/1055720957 >> 172.21.15.22:0/1055720957 conn(0x7f5d000fc320 msgr2=0x7f5d001078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:52.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.927+0000 7f5d07dff700 1 -- 172.21.15.22:0/1055720957 shutdown_connections 2023-09-20T21:48:52.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:52.927+0000 7f5d07dff700 1 -- 172.21.15.22:0/1055720957 wait complete. 2023-09-20T21:48:52.945 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-09-20T21:48:52.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:48:53.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.149+0000 7f06d0fff700 1 -- 172.21.15.22:0/2330072764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06cc101fd0 msgr2=0x7f06cc102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:53.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.149+0000 7f06d0fff700 1 --2- 172.21.15.22:0/2330072764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06cc101fd0 0x7f06cc102430 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f06c00099c0 tx=0x7f06c0009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.149+0000 7f06d0fff700 1 -- 172.21.15.22:0/2330072764 shutdown_connections 2023-09-20T21:48:53.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.149+0000 7f06d0fff700 1 --2- 172.21.15.22:0/2330072764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc102970 0x7f06cc109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.149+0000 7f06d0fff700 1 --2- 172.21.15.22:0/2330072764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc100de0 0x7f06cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.149+0000 7f06d0fff700 1 --2- 172.21.15.22:0/2330072764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06cc101fd0 0x7f06cc102430 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.149+0000 7f06d0fff700 1 -- 172.21.15.22:0/2330072764 >> 172.21.15.22:0/2330072764 conn(0x7f06cc0fc330 msgr2=0x7f06cc0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:53.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.149+0000 7f06d0fff700 1 -- 172.21.15.22:0/2330072764 shutdown_connections 2023-09-20T21:48:53.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.150+0000 7f06d0fff700 1 -- 172.21.15.22:0/2330072764 wait complete. 2023-09-20T21:48:53.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.150+0000 7f06d0fff700 1 Processor -- start 2023-09-20T21:48:53.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06d0fff700 1 -- start start 2023-09-20T21:48:53.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06d0fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc100de0 0x7f06cc196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:53.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06d0fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc101fd0 0x7f06cc196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:53.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06d0fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06cc102970 0x7f06cc19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:53.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06d0fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc19e600 con 0x7f06cc102970 2023-09-20T21:48:53.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06d0fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc19e740 con 0x7f06cc101fd0 2023-09-20T21:48:53.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06d0fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc19e8d0 con 0x7f06cc100de0 2023-09-20T21:48:53.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06cad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06cc102970 0x7f06cc19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:53.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06cad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06cc102970 0x7f06cc19e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59682/0 (socket says 172.21.15.22:59682) 2023-09-20T21:48:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06cad9d700 1 -- 172.21.15.22:0/672754346 learned_addr learned my addr 172.21.15.22:0/672754346 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06c9d9b700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc101fd0 0x7f06cc196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.151+0000 7f06ca59c700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc100de0 0x7f06cc196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06cad9d700 1 -- 172.21.15.22:0/672754346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc100de0 msgr2=0x7f06cc196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06cad9d700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc100de0 0x7f06cc196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06cad9d700 1 -- 172.21.15.22:0/672754346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc101fd0 msgr2=0x7f06cc196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06cad9d700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc101fd0 0x7f06cc196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06cad9d700 1 -- 172.21.15.22:0/672754346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00095f0 con 0x7f06cc102970 2023-09-20T21:48:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06cad9d700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06cc102970 0x7f06cc19e030 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f06c400ef30 tx=0x7f06c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06bb7fe700 1 -- 172.21.15.22:0/672754346 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4018070 con 0x7f06cc102970 2023-09-20T21:48:53.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06bb7fe700 1 -- 172.21.15.22:0/672754346 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06c400f040 con 0x7f06cc102970 2023-09-20T21:48:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06bb7fe700 1 -- 172.21.15.22:0/672754346 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c400cd00 con 0x7f06cc102970 2023-09-20T21:48:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06d0fff700 1 -- 172.21.15.22:0/672754346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc19eb50 con 0x7f06cc102970 2023-09-20T21:48:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.152+0000 7f06d0fff700 1 -- 172.21.15.22:0/672754346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc19f060 con 0x7f06cc102970 2023-09-20T21:48:53.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.153+0000 7f06d0fff700 1 -- 172.21.15.22:0/672754346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc108350 con 0x7f06cc102970 2023-09-20T21:48:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.155+0000 7f06bb7fe700 1 -- 172.21.15.22:0/672754346 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f06c4003a00 con 0x7f06cc102970 2023-09-20T21:48:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.158+0000 7f06bb7fe700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06b40a13d0 0x7f06b40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:53.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.158+0000 7f06bb7fe700 1 -- 172.21.15.22:0/672754346 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f06c40cd420 con 0x7f06cc102970 2023-09-20T21:48:53.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.158+0000 7f06ca59c700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06b40a13d0 0x7f06b40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:53.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.158+0000 7f06ca59c700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06b40a13d0 0x7f06b40a3890 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f06cc101e30 tx=0x7f06bc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:53.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.160+0000 7f06bb7fe700 1 -- 172.21.15.22:0/672754346 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f06c40967f0 con 0x7f06cc102970 2023-09-20T21:48:53.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.419+0000 7f06d0fff700 1 -- 172.21.15.22:0/672754346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06cc02cef0 con 0x7f06cc102970 2023-09-20T21:48:53.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.420+0000 7f06bb7fe700 1 -- 172.21.15.22:0/672754346 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11361 (secure 0 0 0) 0x7f06c4025380 con 0x7f06cc102970 2023-09-20T21:48:53.420 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:53.420 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:47:09.607119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T21:48:53.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 -- 172.21.15.22:0/672754346 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06b40a13d0 msgr2=0x7f06b40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:53.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06b40a13d0 0x7f06b40a3890 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f06cc101e30 tx=0x7f06bc009260 comp rx=0 tx=0).stop 2023-09-20T21:48:53.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 -- 172.21.15.22:0/672754346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06cc102970 msgr2=0x7f06cc19e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:53.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06cc102970 0x7f06cc19e030 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f06c400ef30 tx=0x7f06c400c3a0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 -- 172.21.15.22:0/672754346 shutdown_connections 2023-09-20T21:48:53.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06cc102970 0x7f06cc19e030 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc100de0 0x7f06cc196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06b40a13d0 0x7f06b40a3890 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 --2- 172.21.15.22:0/672754346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f06cc101fd0 0x7f06cc196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 -- 172.21.15.22:0/672754346 >> 172.21.15.22:0/672754346 conn(0x7f06cc0fc330 msgr2=0x7f06cc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:53.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.422+0000 7f06d0fff700 1 -- 172.21.15.22:0/672754346 shutdown_connections 2023-09-20T21:48:53.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.423+0000 7f06d0fff700 1 -- 172.21.15.22:0/672754346 wait complete. 2023-09-20T21:48:53.425 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2023-09-20T21:48:53.442 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-09-20T21:48:53.442 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-09-20T21:48:53.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-20T21:48:53.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:53 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3152952812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:53.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:53 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1055720957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:53.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:53 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3152952812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:53.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:53 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1055720957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:53.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:53 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3152952812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:53.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:53 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1055720957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.651+0000 7f2c249ff700 1 -- 172.21.15.22:0/1811850236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20102980 msgr2=0x7f2c20109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.651+0000 7f2c249ff700 1 --2- 172.21.15.22:0/1811850236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20102980 0x7f2c20109230 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800b0d0 tx=0x7f2c1800b3e0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.651+0000 7f2c249ff700 1 -- 172.21.15.22:0/1811850236 shutdown_connections 2023-09-20T21:48:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.651+0000 7f2c249ff700 1 --2- 172.21.15.22:0/1811850236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20102980 0x7f2c20109230 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.651+0000 7f2c249ff700 1 --2- 172.21.15.22:0/1811850236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c20100df0 0x7f2c20101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.651+0000 7f2c249ff700 1 --2- 172.21.15.22:0/1811850236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c20101fe0 0x7f2c20102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.651+0000 7f2c249ff700 1 -- 172.21.15.22:0/1811850236 >> 172.21.15.22:0/1811850236 conn(0x7f2c200fc320 msgr2=0x7f2c200fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:53.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.651+0000 7f2c249ff700 1 -- 172.21.15.22:0/1811850236 shutdown_connections 2023-09-20T21:48:53.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.651+0000 7f2c249ff700 1 -- 172.21.15.22:0/1811850236 wait complete. 2023-09-20T21:48:53.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.652+0000 7f2c249ff700 1 Processor -- start 2023-09-20T21:48:53.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c249ff700 1 -- start start 2023-09-20T21:48:53.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c249ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c20100df0 0x7f2c20198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:53.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c249ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20101fe0 0x7f2c20199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c249ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c20102980 0x7f2c201a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c249ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c201a0900 con 0x7f2c20101fe0 2023-09-20T21:48:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c249ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c201a0a70 con 0x7f2c20100df0 2023-09-20T21:48:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c249ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c201a0c30 con 0x7f2c20102980 2023-09-20T21:48:53.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c1dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20101fe0 0x7f2c20199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:53.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c1dd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20101fe0 0x7f2c20199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59700/0 (socket says 172.21.15.22:59700) 2023-09-20T21:48:53.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c1dd9b700 1 -- 172.21.15.22:0/1378047574 learned_addr learned my addr 172.21.15.22:0/1378047574 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:53.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c1e59c700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c20100df0 0x7f2c20198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:53.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.653+0000 7f2c1ed9d700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c20102980 0x7f2c201a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:53.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c1dd9b700 1 -- 172.21.15.22:0/1378047574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c20102980 msgr2=0x7f2c201a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:53.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c1dd9b700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c20102980 0x7f2c201a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c1dd9b700 1 -- 172.21.15.22:0/1378047574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c20100df0 msgr2=0x7f2c20198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:53.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c1dd9b700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c20100df0 0x7f2c20198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c1dd9b700 1 -- 172.21.15.22:0/1378047574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c18009d00 con 0x7f2c20101fe0 2023-09-20T21:48:53.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c1dd9b700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20101fe0 0x7f2c20199300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400e9a0 tx=0x7f2c1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c0f7fe700 1 -- 172.21.15.22:0/1378047574 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14009790 con 0x7f2c20101fe0 2023-09-20T21:48:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c0f7fe700 1 -- 172.21.15.22:0/1378047574 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c14010430 con 0x7f2c20101fe0 2023-09-20T21:48:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c249ff700 1 -- 172.21.15.22:0/1378047574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c201a0f10 con 0x7f2c20101fe0 2023-09-20T21:48:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c0f7fe700 1 -- 172.21.15.22:0/1378047574 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14018680 con 0x7f2c20101fe0 2023-09-20T21:48:53.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.654+0000 7f2c249ff700 1 -- 172.21.15.22:0/1378047574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c201a1330 con 0x7f2c20101fe0 2023-09-20T21:48:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.655+0000 7f2c249ff700 1 -- 172.21.15.22:0/1378047574 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2010c660 con 0x7f2c20101fe0 2023-09-20T21:48:53.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.657+0000 7f2c0f7fe700 1 -- 172.21.15.22:0/1378047574 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2c140105a0 con 0x7f2c20101fe0 2023-09-20T21:48:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.659+0000 7f2c0f7fe700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2c080a1350 0x7f2c080a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.659+0000 7f2c0f7fe700 1 -- 172.21.15.22:0/1378047574 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f2c14014070 con 0x7f2c20101fe0 2023-09-20T21:48:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.659+0000 7f2c1e59c700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2c080a1350 0x7f2c080a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.659+0000 7f2c1e59c700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2c080a1350 0x7f2c080a3810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2c2019a290 tx=0x7f2c10009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:53.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.662+0000 7f2c0f7fe700 1 -- 172.21.15.22:0/1378047574 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2c14051e10 con 0x7f2c20101fe0 2023-09-20T21:48:53.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.869+0000 7f2c249ff700 1 -- 172.21.15.22:0/1378047574 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c20108ee0 con 0x7f2c080a1350 2023-09-20T21:48:53.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.882+0000 7f2c0f7fe700 1 -- 172.21.15.22:0/1378047574 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426901 (secure 0 0 0) 0x7f2c20108ee0 con 0x7f2c080a1350 2023-09-20T21:48:53.890 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:53.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.887+0000 7f2c249ff700 1 -- 172.21.15.22:0/1378047574 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2c080a1350 msgr2=0x7f2c080a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.887+0000 7f2c249ff700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2c080a1350 0x7f2c080a3810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2c2019a290 tx=0x7f2c10009260 comp rx=0 tx=0).stop 2023-09-20T21:48:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.887+0000 7f2c249ff700 1 -- 172.21.15.22:0/1378047574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20101fe0 msgr2=0x7f2c20199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.887+0000 7f2c249ff700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20101fe0 0x7f2c20199300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400e9a0 tx=0x7f2c1400ecb0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.888+0000 7f2c249ff700 1 -- 172.21.15.22:0/1378047574 shutdown_connections 2023-09-20T21:48:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.888+0000 7f2c249ff700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c20102980 0x7f2c201a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.888+0000 7f2c249ff700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2c080a1350 0x7f2c080a3810 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.888+0000 7f2c249ff700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2c20100df0 0x7f2c20198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.888+0000 7f2c249ff700 1 --2- 172.21.15.22:0/1378047574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c20101fe0 0x7f2c20199300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:53.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.888+0000 7f2c249ff700 1 -- 172.21.15.22:0/1378047574 >> 172.21.15.22:0/1378047574 conn(0x7f2c200fc320 msgr2=0x7f2c201077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:53.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.888+0000 7f2c249ff700 1 -- 172.21.15.22:0/1378047574 shutdown_connections 2023-09-20T21:48:53.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:53.888+0000 7f2c249ff700 1 -- 172.21.15.22:0/1378047574 wait complete. 2023-09-20T21:48:53.898 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-09-20T21:48:53.907 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2023-09-20T21:48:53.132727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600890,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1184,"num_read_kb":656,"num_write":392,"num_write_kb":3007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338640,"kb_used_data":14360,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603184,"statfs":{"total":1151940427776,"available":1151593660416,"internally_reserved":0,"allocated":14704640,"data_stored":7983721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":66,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003652"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.597787+0000","last_change":"2023-09-20T21:46:41.506092+0000","last_active":"2023-09-20T21:48:09.597787+0000","last_peered":"2023-09-20T21:48:09.597787+0000","last_clean":"2023-09-20T21:48:09.597787+0000","last_became_active":"2023-09-20T21:46:41.505779+0000","last_became_peered":"2023-09-20T21:46:41.505779+0000","last_unstale":"2023-09-20T21:48:09.597787+0000","last_undegraded":"2023-09-20T21:48:09.597787+0000","last_fullsized":"2023-09-20T21:48:09.597787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:25.284068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.971254+0000","last_change":"2023-09-20T21:46:41.505000+0000","last_active":"2023-09-20T21:48:09.971254+0000","last_peered":"2023-09-20T21:48:09.971254+0000","last_clean":"2023-09-20T21:48:09.971254+0000","last_became_active":"2023-09-20T21:46:41.504464+0000","last_became_peered":"2023-09-20T21:46:41.504464+0000","last_unstale":"2023-09-20T21:48:09.971254+0000","last_undegraded":"2023-09-20T21:48:09.971254+0000","last_fullsized":"2023-09-20T21:48:09.971254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:42:44.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.970944+0000","last_change":"2023-09-20T21:46:41.513134+0000","last_active":"2023-09-20T21:48:09.970944+0000","last_peered":"2023-09-20T21:48:09.970944+0000","last_clean":"2023-09-20T21:48:09.970944+0000","last_became_active":"2023-09-20T21:46:41.512793+0000","last_became_peered":"2023-09-20T21:46:41.512793+0000","last_unstale":"2023-09-20T21:48:09.970944+0000","last_undegraded":"2023-09-20T21:48:09.970944+0000","last_fullsized":"2023-09-20T21:48:09.970944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:59.978630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.667363+0000","last_change":"2023-09-20T21:46:41.500200+0000","last_active":"2023-09-20T21:48:09.667363+0000","last_peered":"2023-09-20T21:48:09.667363+0000","last_clean":"2023-09-20T21:48:09.667363+0000","last_became_active":"2023-09-20T21:46:41.499866+0000","last_became_peered":"2023-09-20T21:46:41.499866+0000","last_unstale":"2023-09-20T21:48:09.667363+0000","last_undegraded":"2023-09-20T21:48:09.667363+0000","last_fullsized":"2023-09-20T21:48:09.667363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:20:52.989008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584846+0000","last_change":"2023-09-20T21:46:41.507584+0000","last_active":"2023-09-20T21:48:08.584846+0000","last_peered":"2023-09-20T21:48:08.584846+0000","last_clean":"2023-09-20T21:48:08.584846+0000","last_became_active":"2023-09-20T21:46:41.507298+0000","last_became_peered":"2023-09-20T21:46:41.507298+0000","last_unstale":"2023-09-20T21:48:08.584846+0000","last_undegraded":"2023-09-20T21:48:08.584846+0000","last_fullsized":"2023-09-20T21:48:08.584846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:32:14.042414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585516+0000","last_change":"2023-09-20T21:46:41.497002+0000","last_active":"2023-09-20T21:48:08.585516+0000","last_peered":"2023-09-20T21:48:08.585516+0000","last_clean":"2023-09-20T21:48:08.585516+0000","last_became_active":"2023-09-20T21:46:41.496485+0000","last_became_peered":"2023-09-20T21:46:41.496485+0000","last_unstale":"2023-09-20T21:48:08.585516+0000","last_undegraded":"2023-09-20T21:48:08.585516+0000","last_fullsized":"2023-09-20T21:48:08.585516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:48:24.481575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590370+0000","last_change":"2023-09-20T21:46:41.503303+0000","last_active":"2023-09-20T21:48:08.590370+0000","last_peered":"2023-09-20T21:48:08.590370+0000","last_clean":"2023-09-20T21:48:08.590370+0000","last_became_active":"2023-09-20T21:46:41.502982+0000","last_became_peered":"2023-09-20T21:46:41.502982+0000","last_unstale":"2023-09-20T21:48:08.590370+0000","last_undegraded":"2023-09-20T21:48:08.590370+0000","last_fullsized":"2023-09-20T21:48:08.590370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:23:27.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.589897+0000","last_change":"2023-09-20T21:46:41.508243+0000","last_active":"2023-09-20T21:48:08.589897+0000","last_peered":"2023-09-20T21:48:08.589897+0000","last_clean":"2023-09-20T21:48:08.589897+0000","last_became_active":"2023-09-20T21:46:41.506407+0000","last_became_peered":"2023-09-20T21:46:41.506407+0000","last_unstale":"2023-09-20T21:48:08.589897+0000","last_undegraded":"2023-09-20T21:48:08.589897+0000","last_fullsized":"2023-09-20T21:48:08.589897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:39.388716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588390+0000","last_change":"2023-09-20T21:46:41.501513+0000","last_active":"2023-09-20T21:48:08.588390+0000","last_peered":"2023-09-20T21:48:08.588390+0000","last_clean":"2023-09-20T21:48:08.588390+0000","last_became_active":"2023-09-20T21:46:41.500051+0000","last_became_peered":"2023-09-20T21:46:41.500051+0000","last_unstale":"2023-09-20T21:48:08.588390+0000","last_undegraded":"2023-09-20T21:48:08.588390+0000","last_fullsized":"2023-09-20T21:48:08.588390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:40.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585877+0000","last_change":"2023-09-20T21:46:41.501045+0000","last_active":"2023-09-20T21:48:08.585877+0000","last_peered":"2023-09-20T21:48:08.585877+0000","last_clean":"2023-09-20T21:48:08.585877+0000","last_became_active":"2023-09-20T21:46:41.500744+0000","last_became_peered":"2023-09-20T21:46:41.500744+0000","last_unstale":"2023-09-20T21:48:08.585877+0000","last_undegraded":"2023-09-20T21:48:08.585877+0000","last_fullsized":"2023-09-20T21:48:08.585877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:57:29.120472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590634+0000","last_change":"2023-09-20T21:46:41.510235+0000","last_active":"2023-09-20T21:48:08.590634+0000","last_peered":"2023-09-20T21:48:08.590634+0000","last_clean":"2023-09-20T21:48:08.590634+0000","last_became_active":"2023-09-20T21:46:41.509943+0000","last_became_peered":"2023-09-20T21:46:41.509943+0000","last_unstale":"2023-09-20T21:48:08.590634+0000","last_undegraded":"2023-09-20T21:48:08.590634+0000","last_fullsized":"2023-09-20T21:48:08.590634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:09:19.383505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663853+0000","last_change":"2023-09-20T21:46:41.501194+0000","last_active":"2023-09-20T21:48:08.663853+0000","last_peered":"2023-09-20T21:48:08.663853+0000","last_clean":"2023-09-20T21:48:08.663853+0000","last_became_active":"2023-09-20T21:46:41.500095+0000","last_became_peered":"2023-09-20T21:46:41.500095+0000","last_unstale":"2023-09-20T21:48:08.663853+0000","last_undegraded":"2023-09-20T21:48:08.663853+0000","last_fullsized":"2023-09-20T21:48:08.663853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:46:51.343332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587794+0000","last_change":"2023-09-20T21:46:41.507562+0000","last_active":"2023-09-20T21:48:08.587794+0000","last_peered":"2023-09-20T21:48:08.587794+0000","last_clean":"2023-09-20T21:48:08.587794+0000","last_became_active":"2023-09-20T21:46:41.507094+0000","last_became_peered":"2023-09-20T21:46:41.507094+0000","last_unstale":"2023-09-20T21:48:08.587794+0000","last_undegraded":"2023-09-20T21:48:08.587794+0000","last_fullsized":"2023-09-20T21:48:08.587794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:34:32.401415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591639+0000","last_change":"2023-09-20T21:46:41.502864+0000","last_active":"2023-09-20T21:48:08.591639+0000","last_peered":"2023-09-20T21:48:08.591639+0000","last_clean":"2023-09-20T21:48:08.591639+0000","last_became_active":"2023-09-20T21:46:41.502543+0000","last_became_peered":"2023-09-20T21:46:41.502543+0000","last_unstale":"2023-09-20T21:48:08.591639+0000","last_undegraded":"2023-09-20T21:48:08.591639+0000","last_fullsized":"2023-09-20T21:48:08.591639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:02:04.631874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.592056+0000","last_change":"2023-09-20T21:47:08.600445+0000","last_active":"2023-09-20T21:48:08.592056+0000","last_peered":"2023-09-20T21:48:08.592056+0000","last_clean":"2023-09-20T21:48:08.592056+0000","last_became_active":"2023-09-20T21:47:08.600054+0000","last_became_peered":"2023-09-20T21:47:08.600054+0000","last_unstale":"2023-09-20T21:48:08.592056+0000","last_undegraded":"2023-09-20T21:48:08.592056+0000","last_fullsized":"2023-09-20T21:48:08.592056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:43:18.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587667+0000","last_change":"2023-09-20T21:46:41.502036+0000","last_active":"2023-09-20T21:48:08.587667+0000","last_peered":"2023-09-20T21:48:08.587667+0000","last_clean":"2023-09-20T21:48:08.587667+0000","last_became_active":"2023-09-20T21:46:41.501745+0000","last_became_peered":"2023-09-20T21:46:41.501745+0000","last_unstale":"2023-09-20T21:48:08.587667+0000","last_undegraded":"2023-09-20T21:48:08.587667+0000","last_fullsized":"2023-09-20T21:48:08.587667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:08:06.699543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588222+0000","last_change":"2023-09-20T21:46:41.506666+0000","last_active":"2023-09-20T21:48:08.588222+0000","last_peered":"2023-09-20T21:48:08.588222+0000","last_clean":"2023-09-20T21:48:08.588222+0000","last_became_active":"2023-09-20T21:46:41.506266+0000","last_became_peered":"2023-09-20T21:46:41.506266+0000","last_unstale":"2023-09-20T21:48:08.588222+0000","last_undegraded":"2023-09-20T21:48:08.588222+0000","last_fullsized":"2023-09-20T21:48:08.588222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:06:16.809609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585233+0000","last_change":"2023-09-20T21:46:41.497391+0000","last_active":"2023-09-20T21:48:08.585233+0000","last_peered":"2023-09-20T21:48:08.585233+0000","last_clean":"2023-09-20T21:48:08.585233+0000","last_became_active":"2023-09-20T21:46:41.497064+0000","last_became_peered":"2023-09-20T21:46:41.497064+0000","last_unstale":"2023-09-20T21:48:08.585233+0000","last_undegraded":"2023-09-20T21:48:08.585233+0000","last_fullsized":"2023-09-20T21:48:08.585233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:56:11.489585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587770+0000","last_change":"2023-09-20T21:46:41.510602+0000","last_active":"2023-09-20T21:48:08.587770+0000","last_peered":"2023-09-20T21:48:08.587770+0000","last_clean":"2023-09-20T21:48:08.587770+0000","last_became_active":"2023-09-20T21:46:41.510308+0000","last_became_peered":"2023-09-20T21:46:41.510308+0000","last_unstale":"2023-09-20T21:48:08.587770+0000","last_undegraded":"2023-09-20T21:48:08.587770+0000","last_fullsized":"2023-09-20T21:48:08.587770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:43:41.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.972323+0000","last_change":"2023-09-20T21:48:09.972323+0000","last_active":"2023-09-20T21:48:09.972323+0000","last_peered":"2023-09-20T21:48:09.972323+0000","last_clean":"2023-09-20T21:48:09.972323+0000","last_became_active":"2023-09-20T21:48:09.971743+0000","last_became_peered":"2023-09-20T21:48:09.971743+0000","last_unstale":"2023-09-20T21:48:09.972323+0000","last_undegraded":"2023-09-20T21:48:09.972323+0000","last_fullsized":"2023-09-20T21:48:09.972323+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:00:49.679685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":136,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:47.769593+0000","last_change":"2023-09-20T21:46:40.500010+0000","last_active":"2023-09-20T21:48:47.769593+0000","last_peered":"2023-09-20T21:48:47.769593+0000","last_clean":"2023-09-20T21:48:47.769593+0000","last_became_active":"2023-09-20T21:46:40.499632+0000","last_became_peered":"2023-09-20T21:46:40.499632+0000","last_unstale":"2023-09-20T21:48:47.769593+0000","last_undegraded":"2023-09-20T21:48:47.769593+0000","last_fullsized":"2023-09-20T21:48:47.769593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:41:35.590982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594616+0000","last_change":"2023-09-20T21:46:41.516465+0000","last_active":"2023-09-20T21:48:09.594616+0000","last_peered":"2023-09-20T21:48:09.594616+0000","last_clean":"2023-09-20T21:48:09.594616+0000","last_became_active":"2023-09-20T21:46:41.516150+0000","last_became_peered":"2023-09-20T21:46:41.516150+0000","last_unstale":"2023-09-20T21:48:09.594616+0000","last_undegraded":"2023-09-20T21:48:09.594616+0000","last_fullsized":"2023-09-20T21:48:09.594616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:33:37.507049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663434+0000","last_change":"2023-09-20T21:46:45.582916+0000","last_active":"2023-09-20T21:48:08.663434+0000","last_peered":"2023-09-20T21:48:08.663434+0000","last_clean":"2023-09-20T21:48:08.663434+0000","last_became_active":"2023-09-20T21:46:45.582511+0000","last_became_peered":"2023-09-20T21:46:45.582511+0000","last_unstale":"2023-09-20T21:48:08.663434+0000","last_undegraded":"2023-09-20T21:48:08.663434+0000","last_fullsized":"2023-09-20T21:48:08.663434+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:43:47.006768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594814+0000","last_change":"2023-09-20T21:46:40.516867+0000","last_active":"2023-09-20T21:48:09.594814+0000","last_peered":"2023-09-20T21:48:09.594814+0000","last_clean":"2023-09-20T21:48:09.594814+0000","last_became_active":"2023-09-20T21:46:40.516343+0000","last_became_peered":"2023-09-20T21:46:40.516343+0000","last_unstale":"2023-09-20T21:48:09.594814+0000","last_undegraded":"2023-09-20T21:48:09.594814+0000","last_fullsized":"2023-09-20T21:48:09.594814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:17:18.348921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.970829+0000","last_change":"2023-09-20T21:46:41.507093+0000","last_active":"2023-09-20T21:48:09.970829+0000","last_peered":"2023-09-20T21:48:09.970829+0000","last_clean":"2023-09-20T21:48:09.970829+0000","last_became_active":"2023-09-20T21:46:41.506587+0000","last_became_peered":"2023-09-20T21:46:41.506587+0000","last_unstale":"2023-09-20T21:48:09.970829+0000","last_undegraded":"2023-09-20T21:48:09.970829+0000","last_fullsized":"2023-09-20T21:48:09.970829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:26:44.825499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663624+0000","last_change":"2023-09-20T21:46:45.583942+0000","last_active":"2023-09-20T21:48:08.663624+0000","last_peered":"2023-09-20T21:48:08.663624+0000","last_clean":"2023-09-20T21:48:08.663624+0000","last_became_active":"2023-09-20T21:46:45.583484+0000","last_became_peered":"2023-09-20T21:46:45.583484+0000","last_unstale":"2023-09-20T21:48:08.663624+0000","last_undegraded":"2023-09-20T21:48:08.663624+0000","last_fullsized":"2023-09-20T21:48:08.663624+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:48.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585701+0000","last_change":"2023-09-20T21:47:08.615780+0000","last_active":"2023-09-20T21:48:08.585701+0000","last_peered":"2023-09-20T21:48:08.585701+0000","last_clean":"2023-09-20T21:48:08.585701+0000","last_became_active":"2023-09-20T21:47:08.603974+0000","last_became_peered":"2023-09-20T21:47:08.603974+0000","last_unstale":"2023-09-20T21:48:08.585701+0000","last_undegraded":"2023-09-20T21:48:08.585701+0000","last_fullsized":"2023-09-20T21:48:08.585701+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:53:23.172401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587966+0000","last_change":"2023-09-20T21:46:41.504449+0000","last_active":"2023-09-20T21:48:08.587966+0000","last_peered":"2023-09-20T21:48:08.587966+0000","last_clean":"2023-09-20T21:48:08.587966+0000","last_became_active":"2023-09-20T21:46:41.504169+0000","last_became_peered":"2023-09-20T21:46:41.504169+0000","last_unstale":"2023-09-20T21:48:08.587966+0000","last_undegraded":"2023-09-20T21:48:08.587966+0000","last_fullsized":"2023-09-20T21:48:08.587966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:36:43.980902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663507+0000","last_change":"2023-09-20T21:46:45.582269+0000","last_active":"2023-09-20T21:48:08.663507+0000","last_peered":"2023-09-20T21:48:08.663507+0000","last_clean":"2023-09-20T21:48:08.663507+0000","last_became_active":"2023-09-20T21:46:45.581782+0000","last_became_peered":"2023-09-20T21:46:45.581782+0000","last_unstale":"2023-09-20T21:48:08.663507+0000","last_undegraded":"2023-09-20T21:48:08.663507+0000","last_fullsized":"2023-09-20T21:48:08.663507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:32:09.518715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585164+0000","last_change":"2023-09-20T21:46:40.504099+0000","last_active":"2023-09-20T21:48:08.585164+0000","last_peered":"2023-09-20T21:48:08.585164+0000","last_clean":"2023-09-20T21:48:08.585164+0000","last_became_active":"2023-09-20T21:46:40.503774+0000","last_became_peered":"2023-09-20T21:46:40.503774+0000","last_unstale":"2023-09-20T21:48:08.585164+0000","last_undegraded":"2023-09-20T21:48:08.585164+0000","last_fullsized":"2023-09-20T21:48:08.585164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:17.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591062+0000","last_change":"2023-09-20T21:46:41.499802+0000","last_active":"2023-09-20T21:48:08.591062+0000","last_peered":"2023-09-20T21:48:08.591062+0000","last_clean":"2023-09-20T21:48:08.591062+0000","last_became_active":"2023-09-20T21:46:41.499491+0000","last_became_peered":"2023-09-20T21:46:41.499491+0000","last_unstale":"2023-09-20T21:48:08.591062+0000","last_undegraded":"2023-09-20T21:48:08.591062+0000","last_fullsized":"2023-09-20T21:48:08.591062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:44.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587887+0000","last_change":"2023-09-20T21:46:45.578753+0000","last_active":"2023-09-20T21:48:08.587887+0000","last_peered":"2023-09-20T21:48:08.587887+0000","last_clean":"2023-09-20T21:48:08.587887+0000","last_became_active":"2023-09-20T21:46:45.578297+0000","last_became_peered":"2023-09-20T21:46:45.578297+0000","last_unstale":"2023-09-20T21:48:08.587887+0000","last_undegraded":"2023-09-20T21:48:08.587887+0000","last_fullsized":"2023-09-20T21:48:08.587887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:18:22.596550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:48.727867+0000","last_change":"2023-09-20T21:46:40.505675+0000","last_active":"2023-09-20T21:48:48.727867+0000","last_peered":"2023-09-20T21:48:48.727867+0000","last_clean":"2023-09-20T21:48:48.727867+0000","last_became_active":"2023-09-20T21:46:40.505432+0000","last_became_peered":"2023-09-20T21:46:40.505432+0000","last_unstale":"2023-09-20T21:48:48.727867+0000","last_undegraded":"2023-09-20T21:48:48.727867+0000","last_fullsized":"2023-09-20T21:48:48.727867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:14:53.652904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585583+0000","last_change":"2023-09-20T21:46:41.505924+0000","last_active":"2023-09-20T21:48:08.585583+0000","last_peered":"2023-09-20T21:48:08.585583+0000","last_clean":"2023-09-20T21:48:08.585583+0000","last_became_active":"2023-09-20T21:46:41.505609+0000","last_became_peered":"2023-09-20T21:46:41.505609+0000","last_unstale":"2023-09-20T21:48:08.585583+0000","last_undegraded":"2023-09-20T21:48:08.585583+0000","last_fullsized":"2023-09-20T21:48:08.585583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:13:46.506786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588075+0000","last_change":"2023-09-20T21:46:45.587818+0000","last_active":"2023-09-20T21:48:08.588075+0000","last_peered":"2023-09-20T21:48:08.588075+0000","last_clean":"2023-09-20T21:48:08.588075+0000","last_became_active":"2023-09-20T21:46:45.587376+0000","last_became_peered":"2023-09-20T21:46:45.587376+0000","last_unstale":"2023-09-20T21:48:08.588075+0000","last_undegraded":"2023-09-20T21:48:08.588075+0000","last_fullsized":"2023-09-20T21:48:08.588075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:28:53.605158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585426+0000","last_change":"2023-09-20T21:46:40.505610+0000","last_active":"2023-09-20T21:48:08.585426+0000","last_peered":"2023-09-20T21:48:08.585426+0000","last_clean":"2023-09-20T21:48:08.585426+0000","last_became_active":"2023-09-20T21:46:40.505279+0000","last_became_peered":"2023-09-20T21:46:40.505279+0000","last_unstale":"2023-09-20T21:48:08.585426+0000","last_undegraded":"2023-09-20T21:48:08.585426+0000","last_fullsized":"2023-09-20T21:48:08.585426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:41:19.575154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584476+0000","last_change":"2023-09-20T21:47:08.908513+0000","last_active":"2023-09-20T21:48:08.584476+0000","last_peered":"2023-09-20T21:48:08.584476+0000","last_clean":"2023-09-20T21:48:08.584476+0000","last_became_active":"2023-09-20T21:47:08.908142+0000","last_became_peered":"2023-09-20T21:47:08.908142+0000","last_unstale":"2023-09-20T21:48:08.584476+0000","last_undegraded":"2023-09-20T21:48:08.584476+0000","last_fullsized":"2023-09-20T21:48:08.584476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:31:33.992202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588229+0000","last_change":"2023-09-20T21:46:45.581738+0000","last_active":"2023-09-20T21:48:08.588229+0000","last_peered":"2023-09-20T21:48:08.588229+0000","last_clean":"2023-09-20T21:48:08.588229+0000","last_became_active":"2023-09-20T21:46:45.581279+0000","last_became_peered":"2023-09-20T21:46:45.581279+0000","last_unstale":"2023-09-20T21:48:08.588229+0000","last_undegraded":"2023-09-20T21:48:08.588229+0000","last_fullsized":"2023-09-20T21:48:08.588229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:41:41.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587220+0000","last_change":"2023-09-20T21:46:40.510539+0000","last_active":"2023-09-20T21:48:08.587220+0000","last_peered":"2023-09-20T21:48:08.587220+0000","last_clean":"2023-09-20T21:48:08.587220+0000","last_became_active":"2023-09-20T21:46:40.509880+0000","last_became_peered":"2023-09-20T21:46:40.509880+0000","last_unstale":"2023-09-20T21:48:08.587220+0000","last_undegraded":"2023-09-20T21:48:08.587220+0000","last_fullsized":"2023-09-20T21:48:08.587220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:52.877786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.589794+0000","last_change":"2023-09-20T21:46:41.505105+0000","last_active":"2023-09-20T21:48:08.589794+0000","last_peered":"2023-09-20T21:48:08.589794+0000","last_clean":"2023-09-20T21:48:08.589794+0000","last_became_active":"2023-09-20T21:46:41.504785+0000","last_became_peered":"2023-09-20T21:46:41.504785+0000","last_unstale":"2023-09-20T21:48:08.589794+0000","last_undegraded":"2023-09-20T21:48:08.589794+0000","last_fullsized":"2023-09-20T21:48:08.589794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:07:57.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.971145+0000","last_change":"2023-09-20T21:46:45.583160+0000","last_active":"2023-09-20T21:48:09.971145+0000","last_peered":"2023-09-20T21:48:09.971145+0000","last_clean":"2023-09-20T21:48:09.971145+0000","last_became_active":"2023-09-20T21:46:45.582709+0000","last_became_peered":"2023-09-20T21:46:45.582709+0000","last_unstale":"2023-09-20T21:48:09.971145+0000","last_undegraded":"2023-09-20T21:48:09.971145+0000","last_fullsized":"2023-09-20T21:48:09.971145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:24:57.563682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590612+0000","last_change":"2023-09-20T21:46:40.502591+0000","last_active":"2023-09-20T21:48:08.590612+0000","last_peered":"2023-09-20T21:48:08.590612+0000","last_clean":"2023-09-20T21:48:08.590612+0000","last_became_active":"2023-09-20T21:46:40.502288+0000","last_became_peered":"2023-09-20T21:46:40.502288+0000","last_unstale":"2023-09-20T21:48:08.590612+0000","last_undegraded":"2023-09-20T21:48:08.590612+0000","last_fullsized":"2023-09-20T21:48:08.590612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:09:37.983866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584209+0000","last_change":"2023-09-20T21:47:08.599912+0000","last_active":"2023-09-20T21:48:08.584209+0000","last_peered":"2023-09-20T21:48:08.584209+0000","last_clean":"2023-09-20T21:48:08.584209+0000","last_became_active":"2023-09-20T21:47:08.599434+0000","last_became_peered":"2023-09-20T21:47:08.599434+0000","last_unstale":"2023-09-20T21:48:08.584209+0000","last_undegraded":"2023-09-20T21:48:08.584209+0000","last_fullsized":"2023-09-20T21:48:08.584209+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:37:01.125731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590887+0000","last_change":"2023-09-20T21:46:45.580917+0000","last_active":"2023-09-20T21:48:08.590887+0000","last_peered":"2023-09-20T21:48:08.590887+0000","last_clean":"2023-09-20T21:48:08.590887+0000","last_became_active":"2023-09-20T21:46:45.580505+0000","last_became_peered":"2023-09-20T21:46:45.580505+0000","last_unstale":"2023-09-20T21:48:08.590887+0000","last_undegraded":"2023-09-20T21:48:08.590887+0000","last_fullsized":"2023-09-20T21:48:08.590887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:52:19.733919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":230,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:50.730444+0000","last_change":"2023-09-20T21:46:40.507040+0000","last_active":"2023-09-20T21:48:50.730444+0000","last_peered":"2023-09-20T21:48:50.730444+0000","last_clean":"2023-09-20T21:48:50.730444+0000","last_became_active":"2023-09-20T21:46:40.506712+0000","last_became_peered":"2023-09-20T21:46:40.506712+0000","last_unstale":"2023-09-20T21:48:50.730444+0000","last_undegraded":"2023-09-20T21:48:50.730444+0000","last_fullsized":"2023-09-20T21:48:50.730444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:02:56.040911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588376+0000","last_change":"2023-09-20T21:46:41.504794+0000","last_active":"2023-09-20T21:48:08.588376+0000","last_peered":"2023-09-20T21:48:08.588376+0000","last_clean":"2023-09-20T21:48:08.588376+0000","last_became_active":"2023-09-20T21:46:41.504372+0000","last_became_peered":"2023-09-20T21:46:41.504372+0000","last_unstale":"2023-09-20T21:48:08.588376+0000","last_undegraded":"2023-09-20T21:48:08.588376+0000","last_fullsized":"2023-09-20T21:48:08.588376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:42:20.527306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584027+0000","last_change":"2023-09-20T21:46:45.586364+0000","last_active":"2023-09-20T21:48:08.584027+0000","last_peered":"2023-09-20T21:48:08.584027+0000","last_clean":"2023-09-20T21:48:08.584027+0000","last_became_active":"2023-09-20T21:46:45.584620+0000","last_became_peered":"2023-09-20T21:46:45.584620+0000","last_unstale":"2023-09-20T21:48:08.584027+0000","last_undegraded":"2023-09-20T21:48:08.584027+0000","last_fullsized":"2023-09-20T21:48:08.584027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:05:20.158118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590779+0000","last_change":"2023-09-20T21:46:40.500797+0000","last_active":"2023-09-20T21:48:08.590779+0000","last_peered":"2023-09-20T21:48:08.590779+0000","last_clean":"2023-09-20T21:48:08.590779+0000","last_became_active":"2023-09-20T21:46:40.500337+0000","last_became_peered":"2023-09-20T21:46:40.500337+0000","last_unstale":"2023-09-20T21:48:08.590779+0000","last_undegraded":"2023-09-20T21:48:08.590779+0000","last_fullsized":"2023-09-20T21:48:08.590779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:19.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.592706+0000","last_change":"2023-09-20T21:47:08.908615+0000","last_active":"2023-09-20T21:48:08.592706+0000","last_peered":"2023-09-20T21:48:08.592706+0000","last_clean":"2023-09-20T21:48:08.592706+0000","last_became_active":"2023-09-20T21:47:08.908228+0000","last_became_peered":"2023-09-20T21:47:08.908228+0000","last_unstale":"2023-09-20T21:48:08.592706+0000","last_undegraded":"2023-09-20T21:48:08.592706+0000","last_fullsized":"2023-09-20T21:48:08.592706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:15:28.772129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663779+0000","last_change":"2023-09-20T21:46:45.583081+0000","last_active":"2023-09-20T21:48:08.663779+0000","last_peered":"2023-09-20T21:48:08.663779+0000","last_clean":"2023-09-20T21:48:08.663779+0000","last_became_active":"2023-09-20T21:46:45.580496+0000","last_became_peered":"2023-09-20T21:46:45.580496+0000","last_unstale":"2023-09-20T21:48:08.663779+0000","last_undegraded":"2023-09-20T21:48:08.663779+0000","last_fullsized":"2023-09-20T21:48:08.663779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:42.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591031+0000","last_change":"2023-09-20T21:46:40.512739+0000","last_active":"2023-09-20T21:48:08.591031+0000","last_peered":"2023-09-20T21:48:08.591031+0000","last_clean":"2023-09-20T21:48:08.591031+0000","last_became_active":"2023-09-20T21:46:40.512485+0000","last_became_peered":"2023-09-20T21:46:40.512485+0000","last_unstale":"2023-09-20T21:48:08.591031+0000","last_undegraded":"2023-09-20T21:48:08.591031+0000","last_fullsized":"2023-09-20T21:48:08.591031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:02:49.392560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588075+0000","last_change":"2023-09-20T21:47:08.604682+0000","last_active":"2023-09-20T21:48:08.588075+0000","last_peered":"2023-09-20T21:48:08.588075+0000","last_clean":"2023-09-20T21:48:08.588075+0000","last_became_active":"2023-09-20T21:47:08.604271+0000","last_became_peered":"2023-09-20T21:47:08.604271+0000","last_unstale":"2023-09-20T21:48:08.588075+0000","last_undegraded":"2023-09-20T21:48:08.588075+0000","last_fullsized":"2023-09-20T21:48:08.588075+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:40:51.542781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584184+0000","last_change":"2023-09-20T21:46:45.579478+0000","last_active":"2023-09-20T21:48:08.584184+0000","last_peered":"2023-09-20T21:48:08.584184+0000","last_clean":"2023-09-20T21:48:08.584184+0000","last_became_active":"2023-09-20T21:46:45.579055+0000","last_became_peered":"2023-09-20T21:46:45.579055+0000","last_unstale":"2023-09-20T21:48:08.584184+0000","last_undegraded":"2023-09-20T21:48:08.584184+0000","last_fullsized":"2023-09-20T21:48:08.584184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:17:32.013280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'14","reported_seq":244,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:50.730483+0000","last_change":"2023-09-20T21:46:40.507321+0000","last_active":"2023-09-20T21:48:50.730483+0000","last_peered":"2023-09-20T21:48:50.730483+0000","last_clean":"2023-09-20T21:48:50.730483+0000","last_became_active":"2023-09-20T21:46:40.506983+0000","last_became_peered":"2023-09-20T21:46:40.506983+0000","last_unstale":"2023-09-20T21:48:50.730483+0000","last_undegraded":"2023-09-20T21:48:50.730483+0000","last_fullsized":"2023-09-20T21:48:50.730483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:46:40.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":210,"num_read_kb":109,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585464+0000","last_change":"2023-09-20T21:46:41.509646+0000","last_active":"2023-09-20T21:48:08.585464+0000","last_peered":"2023-09-20T21:48:08.585464+0000","last_clean":"2023-09-20T21:48:08.585464+0000","last_became_active":"2023-09-20T21:46:41.509183+0000","last_became_peered":"2023-09-20T21:46:41.509183+0000","last_unstale":"2023-09-20T21:48:08.585464+0000","last_undegraded":"2023-09-20T21:48:08.585464+0000","last_fullsized":"2023-09-20T21:48:08.585464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:17:21.084830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591360+0000","last_change":"2023-09-20T21:46:45.583709+0000","last_active":"2023-09-20T21:48:08.591360+0000","last_peered":"2023-09-20T21:48:08.591360+0000","last_clean":"2023-09-20T21:48:08.591360+0000","last_became_active":"2023-09-20T21:46:45.583302+0000","last_became_peered":"2023-09-20T21:46:45.583302+0000","last_unstale":"2023-09-20T21:48:08.591360+0000","last_undegraded":"2023-09-20T21:48:08.591360+0000","last_fullsized":"2023-09-20T21:48:08.591360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:31:35.472750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587265+0000","last_change":"2023-09-20T21:46:40.506411+0000","last_active":"2023-09-20T21:48:08.587265+0000","last_peered":"2023-09-20T21:48:08.587265+0000","last_clean":"2023-09-20T21:48:08.587265+0000","last_became_active":"2023-09-20T21:46:40.506116+0000","last_became_peered":"2023-09-20T21:46:40.506116+0000","last_unstale":"2023-09-20T21:48:08.587265+0000","last_undegraded":"2023-09-20T21:48:08.587265+0000","last_fullsized":"2023-09-20T21:48:08.587265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:23:19.175007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587561+0000","last_change":"2023-09-20T21:46:41.513793+0000","last_active":"2023-09-20T21:48:08.587561+0000","last_peered":"2023-09-20T21:48:08.587561+0000","last_clean":"2023-09-20T21:48:08.587561+0000","last_became_active":"2023-09-20T21:46:41.513434+0000","last_became_peered":"2023-09-20T21:46:41.513434+0000","last_unstale":"2023-09-20T21:48:08.587561+0000","last_undegraded":"2023-09-20T21:48:08.587561+0000","last_fullsized":"2023-09-20T21:48:08.587561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:00:57.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587768+0000","last_change":"2023-09-20T21:46:45.578188+0000","last_active":"2023-09-20T21:48:08.587768+0000","last_peered":"2023-09-20T21:48:08.587768+0000","last_clean":"2023-09-20T21:48:08.587768+0000","last_became_active":"2023-09-20T21:46:45.577548+0000","last_became_peered":"2023-09-20T21:46:45.577548+0000","last_unstale":"2023-09-20T21:48:08.587768+0000","last_undegraded":"2023-09-20T21:48:08.587768+0000","last_fullsized":"2023-09-20T21:48:08.587768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:54:58.894269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591467+0000","last_change":"2023-09-20T21:46:40.505465+0000","last_active":"2023-09-20T21:48:08.591467+0000","last_peered":"2023-09-20T21:48:08.591467+0000","last_clean":"2023-09-20T21:48:08.591467+0000","last_became_active":"2023-09-20T21:46:40.505174+0000","last_became_peered":"2023-09-20T21:46:40.505174+0000","last_unstale":"2023-09-20T21:48:08.591467+0000","last_undegraded":"2023-09-20T21:48:08.591467+0000","last_fullsized":"2023-09-20T21:48:08.591467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:50:53.191963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590466+0000","last_change":"2023-09-20T21:46:41.507370+0000","last_active":"2023-09-20T21:48:08.590466+0000","last_peered":"2023-09-20T21:48:08.590466+0000","last_clean":"2023-09-20T21:48:08.590466+0000","last_became_active":"2023-09-20T21:46:41.507035+0000","last_became_peered":"2023-09-20T21:46:41.507035+0000","last_unstale":"2023-09-20T21:48:08.590466+0000","last_undegraded":"2023-09-20T21:48:08.590466+0000","last_fullsized":"2023-09-20T21:48:08.590466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:50.298899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584972+0000","last_change":"2023-09-20T21:46:45.584458+0000","last_active":"2023-09-20T21:48:08.584972+0000","last_peered":"2023-09-20T21:48:08.584972+0000","last_clean":"2023-09-20T21:48:08.584972+0000","last_became_active":"2023-09-20T21:46:45.584033+0000","last_became_peered":"2023-09-20T21:46:45.584033+0000","last_unstale":"2023-09-20T21:48:08.584972+0000","last_undegraded":"2023-09-20T21:48:08.584972+0000","last_fullsized":"2023-09-20T21:48:08.584972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:48:44.653880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:48.727034+0000","last_change":"2023-09-20T21:46:40.501329+0000","last_active":"2023-09-20T21:48:48.727034+0000","last_peered":"2023-09-20T21:48:48.727034+0000","last_clean":"2023-09-20T21:48:48.727034+0000","last_became_active":"2023-09-20T21:46:40.500999+0000","last_became_peered":"2023-09-20T21:46:40.500999+0000","last_unstale":"2023-09-20T21:48:48.727034+0000","last_undegraded":"2023-09-20T21:48:48.727034+0000","last_fullsized":"2023-09-20T21:48:48.727034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:38.028398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590301+0000","last_change":"2023-09-20T21:47:08.602123+0000","last_active":"2023-09-20T21:48:08.590301+0000","last_peered":"2023-09-20T21:48:08.590301+0000","last_clean":"2023-09-20T21:48:08.590301+0000","last_became_active":"2023-09-20T21:47:08.601738+0000","last_became_peered":"2023-09-20T21:47:08.601738+0000","last_unstale":"2023-09-20T21:48:08.590301+0000","last_undegraded":"2023-09-20T21:48:08.590301+0000","last_fullsized":"2023-09-20T21:48:08.590301+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:26:36.710230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.599126+0000","last_change":"2023-09-20T21:48:09.599126+0000","last_active":"2023-09-20T21:48:09.599126+0000","last_peered":"2023-09-20T21:48:09.599126+0000","last_clean":"2023-09-20T21:48:09.599126+0000","last_became_active":"2023-09-20T21:48:09.598649+0000","last_became_peered":"2023-09-20T21:48:09.598649+0000","last_unstale":"2023-09-20T21:48:09.599126+0000","last_undegraded":"2023-09-20T21:48:09.599126+0000","last_fullsized":"2023-09-20T21:48:09.599126+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:24:27.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584737+0000","last_change":"2023-09-20T21:46:40.499985+0000","last_active":"2023-09-20T21:48:08.584737+0000","last_peered":"2023-09-20T21:48:08.584737+0000","last_clean":"2023-09-20T21:48:08.584737+0000","last_became_active":"2023-09-20T21:46:40.499646+0000","last_became_peered":"2023-09-20T21:46:40.499646+0000","last_unstale":"2023-09-20T21:48:08.584737+0000","last_undegraded":"2023-09-20T21:48:08.584737+0000","last_fullsized":"2023-09-20T21:48:08.584737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:52:02.758556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590785+0000","last_change":"2023-09-20T21:46:41.501241+0000","last_active":"2023-09-20T21:48:08.590785+0000","last_peered":"2023-09-20T21:48:08.590785+0000","last_clean":"2023-09-20T21:48:08.590785+0000","last_became_active":"2023-09-20T21:46:41.500902+0000","last_became_peered":"2023-09-20T21:46:41.500902+0000","last_unstale":"2023-09-20T21:48:08.590785+0000","last_undegraded":"2023-09-20T21:48:08.590785+0000","last_fullsized":"2023-09-20T21:48:08.590785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:21:55.210065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590708+0000","last_change":"2023-09-20T21:46:45.583405+0000","last_active":"2023-09-20T21:48:08.590708+0000","last_peered":"2023-09-20T21:48:08.590708+0000","last_clean":"2023-09-20T21:48:08.590708+0000","last_became_active":"2023-09-20T21:46:45.582945+0000","last_became_peered":"2023-09-20T21:46:45.582945+0000","last_unstale":"2023-09-20T21:48:08.590708+0000","last_undegraded":"2023-09-20T21:48:08.590708+0000","last_fullsized":"2023-09-20T21:48:08.590708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:27:14.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585473+0000","last_change":"2023-09-20T21:46:40.507744+0000","last_active":"2023-09-20T21:48:08.585473+0000","last_peered":"2023-09-20T21:48:08.585473+0000","last_clean":"2023-09-20T21:48:08.585473+0000","last_became_active":"2023-09-20T21:46:40.507454+0000","last_became_peered":"2023-09-20T21:46:40.507454+0000","last_unstale":"2023-09-20T21:48:08.585473+0000","last_undegraded":"2023-09-20T21:48:08.585473+0000","last_fullsized":"2023-09-20T21:48:08.585473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:00:24.492789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588068+0000","last_change":"2023-09-20T21:46:41.504263+0000","last_active":"2023-09-20T21:48:08.588068+0000","last_peered":"2023-09-20T21:48:08.588068+0000","last_clean":"2023-09-20T21:48:08.588068+0000","last_became_active":"2023-09-20T21:46:41.503961+0000","last_became_peered":"2023-09-20T21:46:41.503961+0000","last_unstale":"2023-09-20T21:48:08.588068+0000","last_undegraded":"2023-09-20T21:48:08.588068+0000","last_fullsized":"2023-09-20T21:48:08.588068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:31:58.035517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.668647+0000","last_change":"2023-09-20T21:48:09.668184+0000","last_active":"2023-09-20T21:48:09.668647+0000","last_peered":"2023-09-20T21:48:09.668647+0000","last_clean":"2023-09-20T21:48:09.668647+0000","last_became_active":"2023-09-20T21:48:09.667732+0000","last_became_peered":"2023-09-20T21:48:09.667732+0000","last_unstale":"2023-09-20T21:48:09.668647+0000","last_undegraded":"2023-09-20T21:48:09.668647+0000","last_fullsized":"2023-09-20T21:48:09.668647+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:06:25.108207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.592116+0000","last_change":"2023-09-20T21:46:40.501850+0000","last_active":"2023-09-20T21:48:09.592116+0000","last_peered":"2023-09-20T21:48:09.592116+0000","last_clean":"2023-09-20T21:48:09.592116+0000","last_became_active":"2023-09-20T21:46:40.501500+0000","last_became_peered":"2023-09-20T21:46:40.501500+0000","last_unstale":"2023-09-20T21:48:09.592116+0000","last_undegraded":"2023-09-20T21:48:09.592116+0000","last_fullsized":"2023-09-20T21:48:09.592116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:14:15.907335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.666100+0000","last_change":"2023-09-20T21:46:41.507432+0000","last_active":"2023-09-20T21:48:09.666100+0000","last_peered":"2023-09-20T21:48:09.666100+0000","last_clean":"2023-09-20T21:48:09.666100+0000","last_became_active":"2023-09-20T21:46:41.507119+0000","last_became_peered":"2023-09-20T21:46:41.507119+0000","last_unstale":"2023-09-20T21:48:09.666100+0000","last_undegraded":"2023-09-20T21:48:09.666100+0000","last_fullsized":"2023-09-20T21:48:09.666100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:14:10.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594895+0000","last_change":"2023-09-20T21:46:45.588455+0000","last_active":"2023-09-20T21:48:09.594895+0000","last_peered":"2023-09-20T21:48:09.594895+0000","last_clean":"2023-09-20T21:48:09.594895+0000","last_became_active":"2023-09-20T21:46:45.587980+0000","last_became_peered":"2023-09-20T21:46:45.587980+0000","last_unstale":"2023-09-20T21:48:09.594895+0000","last_undegraded":"2023-09-20T21:48:09.594895+0000","last_fullsized":"2023-09-20T21:48:09.594895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:07:35.697330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:50.774388+0000","last_change":"2023-09-20T21:46:40.506260+0000","last_active":"2023-09-20T21:48:50.774388+0000","last_peered":"2023-09-20T21:48:50.774388+0000","last_clean":"2023-09-20T21:48:50.774388+0000","last_became_active":"2023-09-20T21:46:40.505999+0000","last_became_peered":"2023-09-20T21:46:40.505999+0000","last_unstale":"2023-09-20T21:48:50.774388+0000","last_undegraded":"2023-09-20T21:48:50.774388+0000","last_fullsized":"2023-09-20T21:48:50.774388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:39:01.748038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.665667+0000","last_change":"2023-09-20T21:47:08.599481+0000","last_active":"2023-09-20T21:48:09.665667+0000","last_peered":"2023-09-20T21:48:09.665667+0000","last_clean":"2023-09-20T21:48:09.665667+0000","last_became_active":"2023-09-20T21:47:08.599104+0000","last_became_peered":"2023-09-20T21:47:08.599104+0000","last_unstale":"2023-09-20T21:48:09.665667+0000","last_undegraded":"2023-09-20T21:48:09.665667+0000","last_fullsized":"2023-09-20T21:48:09.665667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:33:56.845446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588094+0000","last_change":"2023-09-20T21:46:45.587410+0000","last_active":"2023-09-20T21:48:08.588094+0000","last_peered":"2023-09-20T21:48:08.588094+0000","last_clean":"2023-09-20T21:48:08.588094+0000","last_became_active":"2023-09-20T21:46:45.586944+0000","last_became_peered":"2023-09-20T21:46:45.586944+0000","last_unstale":"2023-09-20T21:48:08.588094+0000","last_undegraded":"2023-09-20T21:48:08.588094+0000","last_fullsized":"2023-09-20T21:48:08.588094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:57.053765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.591816+0000","last_change":"2023-09-20T21:46:41.511526+0000","last_active":"2023-09-20T21:48:09.591816+0000","last_peered":"2023-09-20T21:48:09.591816+0000","last_clean":"2023-09-20T21:48:09.591816+0000","last_became_active":"2023-09-20T21:46:41.511213+0000","last_became_peered":"2023-09-20T21:46:41.511213+0000","last_unstale":"2023-09-20T21:48:09.591816+0000","last_undegraded":"2023-09-20T21:48:09.591816+0000","last_fullsized":"2023-09-20T21:48:09.591816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:29:04.199190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:48.726313+0000","last_change":"2023-09-20T21:46:40.514916+0000","last_active":"2023-09-20T21:48:48.726313+0000","last_peered":"2023-09-20T21:48:48.726313+0000","last_clean":"2023-09-20T21:48:48.726313+0000","last_became_active":"2023-09-20T21:46:40.514553+0000","last_became_peered":"2023-09-20T21:46:40.514553+0000","last_unstale":"2023-09-20T21:48:48.726313+0000","last_undegraded":"2023-09-20T21:48:48.726313+0000","last_fullsized":"2023-09-20T21:48:48.726313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:12:32.481852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714807+0000","last_change":"2023-09-20T21:46:45.587489+0000","last_active":"2023-09-20T21:48:09.714807+0000","last_peered":"2023-09-20T21:48:09.714807+0000","last_clean":"2023-09-20T21:48:09.714807+0000","last_became_active":"2023-09-20T21:46:45.587022+0000","last_became_peered":"2023-09-20T21:46:45.587022+0000","last_unstale":"2023-09-20T21:48:09.714807+0000","last_undegraded":"2023-09-20T21:48:09.714807+0000","last_fullsized":"2023-09-20T21:48:09.714807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:09:05.870763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593480+0000","last_change":"2023-09-20T21:46:41.508886+0000","last_active":"2023-09-20T21:48:09.593480+0000","last_peered":"2023-09-20T21:48:09.593480+0000","last_clean":"2023-09-20T21:48:09.593480+0000","last_became_active":"2023-09-20T21:46:41.508470+0000","last_became_peered":"2023-09-20T21:46:41.508470+0000","last_unstale":"2023-09-20T21:48:09.593480+0000","last_undegraded":"2023-09-20T21:48:09.593480+0000","last_fullsized":"2023-09-20T21:48:09.593480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:45:04.888300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.667141+0000","last_change":"2023-09-20T21:46:40.503350+0000","last_active":"2023-09-20T21:48:09.667141+0000","last_peered":"2023-09-20T21:48:09.667141+0000","last_clean":"2023-09-20T21:48:09.667141+0000","last_became_active":"2023-09-20T21:46:40.503045+0000","last_became_peered":"2023-09-20T21:46:40.503045+0000","last_unstale":"2023-09-20T21:48:09.667141+0000","last_undegraded":"2023-09-20T21:48:09.667141+0000","last_fullsized":"2023-09-20T21:48:09.667141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:31:09.004342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":318,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714381+0000","last_change":"2023-09-20T21:45:35.180242+0000","last_active":"2023-09-20T21:48:09.714381+0000","last_peered":"2023-09-20T21:48:09.714381+0000","last_clean":"2023-09-20T21:48:09.714381+0000","last_became_active":"2023-09-20T21:45:35.179705+0000","last_became_peered":"2023-09-20T21:45:35.179705+0000","last_unstale":"2023-09-20T21:48:09.714381+0000","last_undegraded":"2023-09-20T21:48:09.714381+0000","last_fullsized":"2023-09-20T21:48:09.714381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_clean_scrub_stamp":"2023-09-20T21:42:25.826683+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:02:03.448188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591048+0000","last_change":"2023-09-20T21:46:45.581396+0000","last_active":"2023-09-20T21:48:08.591048+0000","last_peered":"2023-09-20T21:48:08.591048+0000","last_clean":"2023-09-20T21:48:08.591048+0000","last_became_active":"2023-09-20T21:46:45.581018+0000","last_became_peered":"2023-09-20T21:46:45.581018+0000","last_unstale":"2023-09-20T21:48:08.591048+0000","last_undegraded":"2023-09-20T21:48:08.591048+0000","last_fullsized":"2023-09-20T21:48:08.591048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:16:35.934460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594684+0000","last_change":"2023-09-20T21:46:41.514392+0000","last_active":"2023-09-20T21:48:09.594684+0000","last_peered":"2023-09-20T21:48:09.594684+0000","last_clean":"2023-09-20T21:48:09.594684+0000","last_became_active":"2023-09-20T21:46:41.514090+0000","last_became_peered":"2023-09-20T21:46:41.514090+0000","last_unstale":"2023-09-20T21:48:09.594684+0000","last_undegraded":"2023-09-20T21:48:09.594684+0000","last_fullsized":"2023-09-20T21:48:09.594684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:05:47.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714387+0000","last_change":"2023-09-20T21:46:40.507346+0000","last_active":"2023-09-20T21:48:09.714387+0000","last_peered":"2023-09-20T21:48:09.714387+0000","last_clean":"2023-09-20T21:48:09.714387+0000","last_became_active":"2023-09-20T21:46:40.506969+0000","last_became_peered":"2023-09-20T21:46:40.506969+0000","last_unstale":"2023-09-20T21:48:09.714387+0000","last_undegraded":"2023-09-20T21:48:09.714387+0000","last_fullsized":"2023-09-20T21:48:09.714387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:01:20.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588355+0000","last_change":"2023-09-20T21:46:45.578033+0000","last_active":"2023-09-20T21:48:08.588355+0000","last_peered":"2023-09-20T21:48:08.588355+0000","last_clean":"2023-09-20T21:48:08.588355+0000","last_became_active":"2023-09-20T21:46:45.577540+0000","last_became_peered":"2023-09-20T21:46:45.577540+0000","last_unstale":"2023-09-20T21:48:08.588355+0000","last_undegraded":"2023-09-20T21:48:08.588355+0000","last_fullsized":"2023-09-20T21:48:08.588355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:24:31.020660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594334+0000","last_change":"2023-09-20T21:46:40.510634+0000","last_active":"2023-09-20T21:48:09.594334+0000","last_peered":"2023-09-20T21:48:09.594334+0000","last_clean":"2023-09-20T21:48:09.594334+0000","last_became_active":"2023-09-20T21:46:40.510390+0000","last_became_peered":"2023-09-20T21:46:40.510390+0000","last_unstale":"2023-09-20T21:48:09.594334+0000","last_undegraded":"2023-09-20T21:48:09.594334+0000","last_fullsized":"2023-09-20T21:48:09.594334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:44:49.272987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595048+0000","last_change":"2023-09-20T21:46:41.502981+0000","last_active":"2023-09-20T21:48:09.595048+0000","last_peered":"2023-09-20T21:48:09.595048+0000","last_clean":"2023-09-20T21:48:09.595048+0000","last_became_active":"2023-09-20T21:46:41.502594+0000","last_became_peered":"2023-09-20T21:46:41.502594+0000","last_unstale":"2023-09-20T21:48:09.595048+0000","last_undegraded":"2023-09-20T21:48:09.595048+0000","last_fullsized":"2023-09-20T21:48:09.595048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:41:15.261478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593135+0000","last_change":"2023-09-20T21:46:45.582903+0000","last_active":"2023-09-20T21:48:09.593135+0000","last_peered":"2023-09-20T21:48:09.593135+0000","last_clean":"2023-09-20T21:48:09.593135+0000","last_became_active":"2023-09-20T21:46:45.582433+0000","last_became_peered":"2023-09-20T21:46:45.582433+0000","last_unstale":"2023-09-20T21:48:09.593135+0000","last_undegraded":"2023-09-20T21:48:09.593135+0000","last_fullsized":"2023-09-20T21:48:09.593135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:38:22.174903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.592545+0000","last_change":"2023-09-20T21:46:40.507671+0000","last_active":"2023-09-20T21:48:09.592545+0000","last_peered":"2023-09-20T21:48:09.592545+0000","last_clean":"2023-09-20T21:48:09.592545+0000","last_became_active":"2023-09-20T21:46:40.507288+0000","last_became_peered":"2023-09-20T21:46:40.507288+0000","last_unstale":"2023-09-20T21:48:09.592545+0000","last_undegraded":"2023-09-20T21:48:09.592545+0000","last_fullsized":"2023-09-20T21:48:09.592545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:27:02.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594926+0000","last_change":"2023-09-20T21:46:41.507654+0000","last_active":"2023-09-20T21:48:09.594926+0000","last_peered":"2023-09-20T21:48:09.594926+0000","last_clean":"2023-09-20T21:48:09.594926+0000","last_became_active":"2023-09-20T21:46:41.507362+0000","last_became_peered":"2023-09-20T21:46:41.507362+0000","last_unstale":"2023-09-20T21:48:09.594926+0000","last_undegraded":"2023-09-20T21:48:09.594926+0000","last_fullsized":"2023-09-20T21:48:09.594926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:32:08.973379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595697+0000","last_change":"2023-09-20T21:46:45.578437+0000","last_active":"2023-09-20T21:48:09.595697+0000","last_peered":"2023-09-20T21:48:09.595697+0000","last_clean":"2023-09-20T21:48:09.595697+0000","last_became_active":"2023-09-20T21:46:45.578010+0000","last_became_peered":"2023-09-20T21:46:45.578010+0000","last_unstale":"2023-09-20T21:48:09.595697+0000","last_undegraded":"2023-09-20T21:48:09.595697+0000","last_fullsized":"2023-09-20T21:48:09.595697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:59:05.494861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.664112+0000","last_change":"2023-09-20T21:46:40.508937+0000","last_active":"2023-09-20T21:48:08.664112+0000","last_peered":"2023-09-20T21:48:08.664112+0000","last_clean":"2023-09-20T21:48:08.664112+0000","last_became_active":"2023-09-20T21:46:40.508639+0000","last_became_peered":"2023-09-20T21:46:40.508639+0000","last_unstale":"2023-09-20T21:48:08.664112+0000","last_undegraded":"2023-09-20T21:48:08.664112+0000","last_fullsized":"2023-09-20T21:48:08.664112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:50:48.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.971221+0000","last_change":"2023-09-20T21:46:41.510702+0000","last_active":"2023-09-20T21:48:09.971221+0000","last_peered":"2023-09-20T21:48:09.971221+0000","last_clean":"2023-09-20T21:48:09.971221+0000","last_became_active":"2023-09-20T21:46:41.510357+0000","last_became_peered":"2023-09-20T21:46:41.510357+0000","last_unstale":"2023-09-20T21:48:09.971221+0000","last_undegraded":"2023-09-20T21:48:09.971221+0000","last_fullsized":"2023-09-20T21:48:09.971221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:35:44.455981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663990+0000","last_change":"2023-09-20T21:46:45.581110+0000","last_active":"2023-09-20T21:48:08.663990+0000","last_peered":"2023-09-20T21:48:08.663990+0000","last_clean":"2023-09-20T21:48:08.663990+0000","last_became_active":"2023-09-20T21:46:45.580498+0000","last_became_peered":"2023-09-20T21:46:45.580498+0000","last_unstale":"2023-09-20T21:48:08.663990+0000","last_undegraded":"2023-09-20T21:48:08.663990+0000","last_fullsized":"2023-09-20T21:48:08.663990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:19:22.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587563+0000","last_change":"2023-09-20T21:46:40.497017+0000","last_active":"2023-09-20T21:48:08.587563+0000","last_peered":"2023-09-20T21:48:08.587563+0000","last_clean":"2023-09-20T21:48:08.587563+0000","last_became_active":"2023-09-20T21:46:40.496722+0000","last_became_peered":"2023-09-20T21:46:40.496722+0000","last_unstale":"2023-09-20T21:48:08.587563+0000","last_undegraded":"2023-09-20T21:48:08.587563+0000","last_fullsized":"2023-09-20T21:48:08.587563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:12:07.017318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593744+0000","last_change":"2023-09-20T21:46:41.508793+0000","last_active":"2023-09-20T21:48:09.593744+0000","last_peered":"2023-09-20T21:48:09.593744+0000","last_clean":"2023-09-20T21:48:09.593744+0000","last_became_active":"2023-09-20T21:46:41.508453+0000","last_became_peered":"2023-09-20T21:46:41.508453+0000","last_unstale":"2023-09-20T21:48:09.593744+0000","last_undegraded":"2023-09-20T21:48:09.593744+0000","last_fullsized":"2023-09-20T21:48:09.593744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:35:12.711570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585166+0000","last_change":"2023-09-20T21:46:45.583519+0000","last_active":"2023-09-20T21:48:08.585166+0000","last_peered":"2023-09-20T21:48:08.585166+0000","last_clean":"2023-09-20T21:48:08.585166+0000","last_became_active":"2023-09-20T21:46:45.582933+0000","last_became_peered":"2023-09-20T21:46:45.582933+0000","last_unstale":"2023-09-20T21:48:08.585166+0000","last_undegraded":"2023-09-20T21:48:08.585166+0000","last_fullsized":"2023-09-20T21:48:08.585166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:13:20.237865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593456+0000","last_change":"2023-09-20T21:46:40.507384+0000","last_active":"2023-09-20T21:48:09.593456+0000","last_peered":"2023-09-20T21:48:09.593456+0000","last_clean":"2023-09-20T21:48:09.593456+0000","last_became_active":"2023-09-20T21:46:40.507021+0000","last_became_peered":"2023-09-20T21:46:40.507021+0000","last_unstale":"2023-09-20T21:48:09.593456+0000","last_undegraded":"2023-09-20T21:48:09.593456+0000","last_fullsized":"2023-09-20T21:48:09.593456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:48:04.360664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588413+0000","last_change":"2023-09-20T21:46:41.511537+0000","last_active":"2023-09-20T21:48:08.588413+0000","last_peered":"2023-09-20T21:48:08.588413+0000","last_clean":"2023-09-20T21:48:08.588413+0000","last_became_active":"2023-09-20T21:46:41.511238+0000","last_became_peered":"2023-09-20T21:46:41.511238+0000","last_unstale":"2023-09-20T21:48:08.588413+0000","last_undegraded":"2023-09-20T21:48:08.588413+0000","last_fullsized":"2023-09-20T21:48:08.588413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:01:41.492824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663746+0000","last_change":"2023-09-20T21:46:45.585750+0000","last_active":"2023-09-20T21:48:08.663746+0000","last_peered":"2023-09-20T21:48:08.663746+0000","last_clean":"2023-09-20T21:48:08.663746+0000","last_became_active":"2023-09-20T21:46:45.585213+0000","last_became_peered":"2023-09-20T21:46:45.585213+0000","last_unstale":"2023-09-20T21:48:08.663746+0000","last_undegraded":"2023-09-20T21:48:08.663746+0000","last_fullsized":"2023-09-20T21:48:08.663746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:31:15.650766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.602996+0000","last_change":"2023-09-20T21:48:09.602996+0000","last_active":"2023-09-20T21:48:09.602996+0000","last_peered":"2023-09-20T21:48:09.602996+0000","last_clean":"2023-09-20T21:48:09.602996+0000","last_became_active":"2023-09-20T21:48:09.598415+0000","last_became_peered":"2023-09-20T21:48:09.598415+0000","last_unstale":"2023-09-20T21:48:09.602996+0000","last_undegraded":"2023-09-20T21:48:09.602996+0000","last_fullsized":"2023-09-20T21:48:09.602996+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:39:45.484653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590150+0000","last_change":"2023-09-20T21:46:41.509491+0000","last_active":"2023-09-20T21:48:08.590150+0000","last_peered":"2023-09-20T21:48:08.590150+0000","last_clean":"2023-09-20T21:48:08.590150+0000","last_became_active":"2023-09-20T21:46:41.509039+0000","last_became_peered":"2023-09-20T21:46:41.509039+0000","last_unstale":"2023-09-20T21:48:08.590150+0000","last_undegraded":"2023-09-20T21:48:08.590150+0000","last_fullsized":"2023-09-20T21:48:08.590150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:57:39.003154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663975+0000","last_change":"2023-09-20T21:46:45.581651+0000","last_active":"2023-09-20T21:48:08.663975+0000","last_peered":"2023-09-20T21:48:08.663975+0000","last_clean":"2023-09-20T21:48:08.663975+0000","last_became_active":"2023-09-20T21:46:45.581224+0000","last_became_peered":"2023-09-20T21:46:45.581224+0000","last_unstale":"2023-09-20T21:48:08.663975+0000","last_undegraded":"2023-09-20T21:48:08.663975+0000","last_fullsized":"2023-09-20T21:48:08.663975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:32:17.778301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588268+0000","last_change":"2023-09-20T21:47:08.610413+0000","last_active":"2023-09-20T21:48:08.588268+0000","last_peered":"2023-09-20T21:48:08.588268+0000","last_clean":"2023-09-20T21:48:08.588268+0000","last_became_active":"2023-09-20T21:47:08.603734+0000","last_became_peered":"2023-09-20T21:47:08.603734+0000","last_unstale":"2023-09-20T21:48:08.588268+0000","last_undegraded":"2023-09-20T21:48:08.588268+0000","last_fullsized":"2023-09-20T21:48:08.588268+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:43:42.569371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714928+0000","last_change":"2023-09-20T21:46:41.507218+0000","last_active":"2023-09-20T21:48:09.714928+0000","last_peered":"2023-09-20T21:48:09.714928+0000","last_clean":"2023-09-20T21:48:09.714928+0000","last_became_active":"2023-09-20T21:46:41.506889+0000","last_became_peered":"2023-09-20T21:46:41.506889+0000","last_unstale":"2023-09-20T21:48:09.714928+0000","last_undegraded":"2023-09-20T21:48:09.714928+0000","last_fullsized":"2023-09-20T21:48:09.714928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:08:40.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.716076+0000","last_change":"2023-09-20T21:48:09.716076+0000","last_active":"2023-09-20T21:48:09.716076+0000","last_peered":"2023-09-20T21:48:09.716076+0000","last_clean":"2023-09-20T21:48:09.716076+0000","last_became_active":"2023-09-20T21:48:09.715521+0000","last_became_peered":"2023-09-20T21:48:09.715521+0000","last_unstale":"2023-09-20T21:48:09.716076+0000","last_undegraded":"2023-09-20T21:48:09.716076+0000","last_fullsized":"2023-09-20T21:48:09.716076+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:59:27.762388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":130,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:49.823237+0000","last_change":"2023-09-20T21:46:40.508774+0000","last_active":"2023-09-20T21:48:49.823237+0000","last_peered":"2023-09-20T21:48:49.823237+0000","last_clean":"2023-09-20T21:48:49.823237+0000","last_became_active":"2023-09-20T21:46:40.508480+0000","last_became_peered":"2023-09-20T21:46:40.508480+0000","last_unstale":"2023-09-20T21:48:49.823237+0000","last_undegraded":"2023-09-20T21:48:49.823237+0000","last_fullsized":"2023-09-20T21:48:49.823237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:22.643724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590718+0000","last_change":"2023-09-20T21:46:41.503763+0000","last_active":"2023-09-20T21:48:08.590718+0000","last_peered":"2023-09-20T21:48:08.590718+0000","last_clean":"2023-09-20T21:48:08.590718+0000","last_became_active":"2023-09-20T21:46:41.503360+0000","last_became_peered":"2023-09-20T21:46:41.503360+0000","last_unstale":"2023-09-20T21:48:08.590718+0000","last_undegraded":"2023-09-20T21:48:08.590718+0000","last_fullsized":"2023-09-20T21:48:08.590718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:03:00.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591725+0000","last_change":"2023-09-20T21:46:45.586702+0000","last_active":"2023-09-20T21:48:08.591725+0000","last_peered":"2023-09-20T21:48:08.591725+0000","last_clean":"2023-09-20T21:48:08.591725+0000","last_became_active":"2023-09-20T21:46:45.586177+0000","last_became_peered":"2023-09-20T21:46:45.586177+0000","last_unstale":"2023-09-20T21:48:08.591725+0000","last_undegraded":"2023-09-20T21:48:08.591725+0000","last_fullsized":"2023-09-20T21:48:08.591725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:19:03.883440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591133+0000","last_change":"2023-09-20T21:46:40.507270+0000","last_active":"2023-09-20T21:48:08.591133+0000","last_peered":"2023-09-20T21:48:08.591133+0000","last_clean":"2023-09-20T21:48:08.591133+0000","last_became_active":"2023-09-20T21:46:40.506933+0000","last_became_peered":"2023-09-20T21:46:40.506933+0000","last_unstale":"2023-09-20T21:48:08.591133+0000","last_undegraded":"2023-09-20T21:48:08.591133+0000","last_fullsized":"2023-09-20T21:48:08.591133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:59.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590595+0000","last_change":"2023-09-20T21:46:41.507868+0000","last_active":"2023-09-20T21:48:08.590595+0000","last_peered":"2023-09-20T21:48:08.590595+0000","last_clean":"2023-09-20T21:48:08.590595+0000","last_became_active":"2023-09-20T21:46:41.507568+0000","last_became_peered":"2023-09-20T21:46:41.507568+0000","last_unstale":"2023-09-20T21:48:08.590595+0000","last_undegraded":"2023-09-20T21:48:08.590595+0000","last_fullsized":"2023-09-20T21:48:08.590595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:44.615620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590490+0000","last_change":"2023-09-20T21:46:45.578726+0000","last_active":"2023-09-20T21:48:08.590490+0000","last_peered":"2023-09-20T21:48:08.590490+0000","last_clean":"2023-09-20T21:48:08.590490+0000","last_became_active":"2023-09-20T21:46:45.578326+0000","last_became_peered":"2023-09-20T21:46:45.578326+0000","last_unstale":"2023-09-20T21:48:08.590490+0000","last_undegraded":"2023-09-20T21:48:08.590490+0000","last_fullsized":"2023-09-20T21:48:08.590490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:21:34.564840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587838+0000","last_change":"2023-09-20T21:46:40.509024+0000","last_active":"2023-09-20T21:48:08.587838+0000","last_peered":"2023-09-20T21:48:08.587838+0000","last_clean":"2023-09-20T21:48:08.587838+0000","last_became_active":"2023-09-20T21:46:40.508659+0000","last_became_peered":"2023-09-20T21:46:40.508659+0000","last_unstale":"2023-09-20T21:48:08.587838+0000","last_undegraded":"2023-09-20T21:48:08.587838+0000","last_fullsized":"2023-09-20T21:48:08.587838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:12:08.302068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590780+0000","last_change":"2023-09-20T21:46:41.510307+0000","last_active":"2023-09-20T21:48:08.590780+0000","last_peered":"2023-09-20T21:48:08.590780+0000","last_clean":"2023-09-20T21:48:08.590780+0000","last_became_active":"2023-09-20T21:46:41.510014+0000","last_became_peered":"2023-09-20T21:46:41.510014+0000","last_unstale":"2023-09-20T21:48:08.590780+0000","last_undegraded":"2023-09-20T21:48:08.590780+0000","last_fullsized":"2023-09-20T21:48:08.590780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:05:55.062515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.596849+0000","last_change":"2023-09-20T21:46:41.511525+0000","last_active":"2023-09-20T21:48:09.596849+0000","last_peered":"2023-09-20T21:48:09.596849+0000","last_clean":"2023-09-20T21:48:09.596849+0000","last_became_active":"2023-09-20T21:46:41.511203+0000","last_became_peered":"2023-09-20T21:46:41.511203+0000","last_unstale":"2023-09-20T21:48:09.596849+0000","last_undegraded":"2023-09-20T21:48:09.596849+0000","last_fullsized":"2023-09-20T21:48:09.596849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:42:15.757422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.596012+0000","last_change":"2023-09-20T21:46:41.502936+0000","last_active":"2023-09-20T21:48:09.596012+0000","last_peered":"2023-09-20T21:48:09.596012+0000","last_clean":"2023-09-20T21:48:09.596012+0000","last_became_active":"2023-09-20T21:46:41.502617+0000","last_became_peered":"2023-09-20T21:46:41.502617+0000","last_unstale":"2023-09-20T21:48:09.596012+0000","last_undegraded":"2023-09-20T21:48:09.596012+0000","last_fullsized":"2023-09-20T21:48:09.596012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:45:21.197652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593522+0000","last_change":"2023-09-20T21:46:41.510766+0000","last_active":"2023-09-20T21:48:09.593522+0000","last_peered":"2023-09-20T21:48:09.593522+0000","last_clean":"2023-09-20T21:48:09.593522+0000","last_became_active":"2023-09-20T21:46:41.510433+0000","last_became_peered":"2023-09-20T21:46:41.510433+0000","last_unstale":"2023-09-20T21:48:09.593522+0000","last_undegraded":"2023-09-20T21:48:09.593522+0000","last_fullsized":"2023-09-20T21:48:09.593522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:56:12.085926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.970963+0000","last_change":"2023-09-20T21:46:41.508846+0000","last_active":"2023-09-20T21:48:09.970963+0000","last_peered":"2023-09-20T21:48:09.970963+0000","last_clean":"2023-09-20T21:48:09.970963+0000","last_became_active":"2023-09-20T21:46:41.508518+0000","last_became_peered":"2023-09-20T21:46:41.508518+0000","last_unstale":"2023-09-20T21:48:09.970963+0000","last_undegraded":"2023-09-20T21:48:09.970963+0000","last_fullsized":"2023-09-20T21:48:09.970963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:01:58.195327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588468+0000","last_change":"2023-09-20T21:46:41.505929+0000","last_active":"2023-09-20T21:48:08.588468+0000","last_peered":"2023-09-20T21:48:08.588468+0000","last_clean":"2023-09-20T21:48:08.588468+0000","last_became_active":"2023-09-20T21:46:41.505542+0000","last_became_peered":"2023-09-20T21:46:41.505542+0000","last_unstale":"2023-09-20T21:48:08.588468+0000","last_undegraded":"2023-09-20T21:48:08.588468+0000","last_fullsized":"2023-09-20T21:48:08.588468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:24:47.520344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587952+0000","last_change":"2023-09-20T21:46:41.507603+0000","last_active":"2023-09-20T21:48:08.587952+0000","last_peered":"2023-09-20T21:48:08.587952+0000","last_clean":"2023-09-20T21:48:08.587952+0000","last_became_active":"2023-09-20T21:46:41.507340+0000","last_became_peered":"2023-09-20T21:46:41.507340+0000","last_unstale":"2023-09-20T21:48:08.587952+0000","last_undegraded":"2023-09-20T21:48:08.587952+0000","last_fullsized":"2023-09-20T21:48:08.587952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:50:19.756491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595430+0000","last_change":"2023-09-20T21:46:41.509625+0000","last_active":"2023-09-20T21:48:09.595430+0000","last_peered":"2023-09-20T21:48:09.595430+0000","last_clean":"2023-09-20T21:48:09.595430+0000","last_became_active":"2023-09-20T21:46:41.509301+0000","last_became_peered":"2023-09-20T21:46:41.509301+0000","last_unstale":"2023-09-20T21:48:09.595430+0000","last_undegraded":"2023-09-20T21:48:09.595430+0000","last_fullsized":"2023-09-20T21:48:09.595430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:15:03.108060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595097+0000","last_change":"2023-09-20T21:46:41.516781+0000","last_active":"2023-09-20T21:48:09.595097+0000","last_peered":"2023-09-20T21:48:09.595097+0000","last_clean":"2023-09-20T21:48:09.595097+0000","last_became_active":"2023-09-20T21:46:41.516496+0000","last_became_peered":"2023-09-20T21:46:41.516496+0000","last_unstale":"2023-09-20T21:48:09.595097+0000","last_undegraded":"2023-09-20T21:48:09.595097+0000","last_fullsized":"2023-09-20T21:48:09.595097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:17:26.492851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590654+0000","last_change":"2023-09-20T21:46:41.499927+0000","last_active":"2023-09-20T21:48:08.590654+0000","last_peered":"2023-09-20T21:48:08.590654+0000","last_clean":"2023-09-20T21:48:08.590654+0000","last_became_active":"2023-09-20T21:46:41.499605+0000","last_became_peered":"2023-09-20T21:46:41.499605+0000","last_unstale":"2023-09-20T21:48:08.590654+0000","last_undegraded":"2023-09-20T21:48:08.590654+0000","last_fullsized":"2023-09-20T21:48:08.590654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:18:16.306349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590204+0000","last_change":"2023-09-20T21:46:41.515842+0000","last_active":"2023-09-20T21:48:08.590204+0000","last_peered":"2023-09-20T21:48:08.590204+0000","last_clean":"2023-09-20T21:48:08.590204+0000","last_became_active":"2023-09-20T21:46:41.515600+0000","last_became_peered":"2023-09-20T21:46:41.515600+0000","last_unstale":"2023-09-20T21:48:08.590204+0000","last_undegraded":"2023-09-20T21:48:08.590204+0000","last_fullsized":"2023-09-20T21:48:08.590204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:50:58.088103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588120+0000","last_change":"2023-09-20T21:46:41.510695+0000","last_active":"2023-09-20T21:48:08.588120+0000","last_peered":"2023-09-20T21:48:08.588120+0000","last_clean":"2023-09-20T21:48:08.588120+0000","last_became_active":"2023-09-20T21:46:41.510345+0000","last_became_peered":"2023-09-20T21:46:41.510345+0000","last_unstale":"2023-09-20T21:48:08.588120+0000","last_undegraded":"2023-09-20T21:48:08.588120+0000","last_fullsized":"2023-09-20T21:48:08.588120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:38:24.201382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587592+0000","last_change":"2023-09-20T21:46:41.507809+0000","last_active":"2023-09-20T21:48:08.587592+0000","last_peered":"2023-09-20T21:48:08.587592+0000","last_clean":"2023-09-20T21:48:08.587592+0000","last_became_active":"2023-09-20T21:46:41.507513+0000","last_became_peered":"2023-09-20T21:46:41.507513+0000","last_unstale":"2023-09-20T21:48:08.587592+0000","last_undegraded":"2023-09-20T21:48:08.587592+0000","last_fullsized":"2023-09-20T21:48:08.587592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:30:43.132728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587980+0000","last_change":"2023-09-20T21:46:41.503590+0000","last_active":"2023-09-20T21:48:08.587980+0000","last_peered":"2023-09-20T21:48:08.587980+0000","last_clean":"2023-09-20T21:48:08.587980+0000","last_became_active":"2023-09-20T21:46:41.503295+0000","last_became_peered":"2023-09-20T21:46:41.503295+0000","last_unstale":"2023-09-20T21:48:08.587980+0000","last_undegraded":"2023-09-20T21:48:08.587980+0000","last_fullsized":"2023-09-20T21:48:08.587980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:28:33.088873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10522,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1058,"num_read_kb":547,"num_write":211,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":31566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383719,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":516303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.38500000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83099999999999996,"5min":2.923,"15min":2.923},"last":0.53000000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.43099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80400000000000005,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.628},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73399999999999999,"5min":0.872,"15min":0.872},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.49299999999999999,"15min":0.498},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.68700000000000006,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.29299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79800000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.55800000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83099999999999996,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42799999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.81799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72399999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.60399999999999998,"15min":0.61799999999999999},"min":{"1min":0.32700000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.88500000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.64200000000000002,"15min":0.65500000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77700000000000002,"5min":1.351,"15min":1.351},"last":0.73599999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64900000000000002,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.40799999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.76400000000000001}]},{"osd":7,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.53800000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88100000000000001,"5min":1.246,"15min":1.246},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.54800000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89800000000000002,"5min":1.117,"15min":1.117},"last":0.80000000000000004}]},{"osd":8,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.52400000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.69799999999999995}]},{"osd":9,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.82999999999999996,"15min":0.84699999999999998},"min":{"1min":0.52800000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95099999999999996,"5min":1.992,"15min":1.992},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82499999999999996,"15min":0.82999999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.99299999999999999,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83399999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.61699999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92300000000000004,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.751},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.60499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.91600000000000004,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.71199999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1111315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":1.0169999999999999,"15min":1.093},"min":{"1min":0.433,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76100000000000001,"5min":7.9690000000000003,"15min":7.9690000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.85499999999999998,"15min":0.90900000000000003},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81399999999999995,"5min":5.6280000000000001,"15min":5.6280000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46899999999999997,"15min":0.47899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.60299999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80000000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79900000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67000000000000004,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.748,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.82399999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60799999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78100000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.72799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.68000000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.79000000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68200000000000005,"15min":0.69699999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.62}]},{"osd":6,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73499999999999999,"15min":0.76100000000000001},"min":{"1min":0.47099999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.873,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.89700000000000002,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":8.0440000000000005,"15min":8.0440000000000005},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.75800000000000001,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89700000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72099999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.89800000000000002,"15min":0.85999999999999999},"min":{"1min":0.67300000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.93200000000000005,"5min":5.6449999999999996,"15min":5.6449999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.89500000000000002,"15min":0.86799999999999999},"min":{"1min":0.61199999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.88400000000000001,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":0.74199999999999999}]},{"osd":11,"last update":"Wed Sep 20 21:47:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76600000000000001,"15min":0.76500000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77100000000000002}]}]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":516233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.622,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":2.7269999999999999,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.46400000000000002}]},{"osd":1,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5289999999999999,"5min":4.6799999999999997,"15min":4.6799999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78700000000000003,"15min":0.81599999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.996,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.56200000000000006,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.417,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.70699999999999996}]},{"osd":3,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71899999999999997,"15min":0.69499999999999995},"min":{"1min":0.41499999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.8170000000000002,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.82499999999999996,"15min":0.84499999999999997},"min":{"1min":0.47699999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.7690000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.627}]},{"osd":4,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.94899999999999995,"15min":0.93500000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.798,"5min":6.3449999999999998,"15min":6.3449999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.86099999999999999,"15min":0.86599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5920000000000001,"5min":4.7279999999999998,"15min":4.7279999999999998},"last":0.58399999999999996}]},{"osd":5,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.98899999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4970000000000001,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.92100000000000004,"15min":0.93000000000000005},"min":{"1min":0.54800000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.671,"5min":4.7679999999999998,"15min":4.7679999999999998},"last":0.76600000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.95799999999999996,"15min":0.96599999999999997},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6279999999999999,"5min":4.806,"15min":4.806},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.96599999999999997,"15min":1.0189999999999999},"min":{"1min":0.67400000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1000000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0629999999999999,"15min":1.119},"min":{"1min":0.67300000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.744,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0389999999999999,"15min":1.0600000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.702,"5min":4.843,"15min":4.843},"last":0.85299999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":1.0069999999999999,"15min":1.075},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.2150000000000001,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.749},{"interface":"front","average":{"1min":0.86299999999999999,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.2470000000000001,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.64100000000000001}]},{"osd":11,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.79400000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":516611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":0.52500000000000002}]},{"osd":1,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.497,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.75800000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.625},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.48099999999999998,"15min":0.46300000000000002},"min":{"1min":0.38300000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.53900000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.67500000000000004,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.93400000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.76600000000000001,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.82499999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.57699999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0109999999999999,"5min":1.548,"15min":1.548},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.57499999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.90400000000000003}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.79400000000000004,"15min":0.76800000000000002},"min":{"1min":0.69099999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.81000000000000005,"15min":0.78300000000000003},"min":{"1min":0.73599999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89300000000000002}]},{"osd":10,"last update":"Wed Sep 20 21:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.78000000000000003,"15min":0.745},"min":{"1min":0.69299999999999995,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.80500000000000005,"15min":0.77100000000000002},"min":{"1min":0.72399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.077,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.91700000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81899999999999995,"15min":0.81399999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84499999999999997,"15min":0.84199999999999997},"min":{"1min":0.70699999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":0.97999999999999998,"5min":1.038,"15min":1.038},"last":0.93700000000000006}]}]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1108879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.43099999999999999,"15min":0.42199999999999999},"min":{"1min":0.36899999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45300000000000001,"15min":0.438},"min":{"1min":0.34100000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.442}]},{"osd":2,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76900000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.51700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81799999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.89400000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63100000000000001,"15min":0.60099999999999998},"min":{"1min":0.496,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51900000000000002,"15min":0.48899999999999999},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71399999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.35599999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.498}]},{"osd":5,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63,"15min":0.62},"min":{"1min":0.495,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81000000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.63700000000000001},"min":{"1min":0.54100000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96099999999999997,"5min":1.006,"15min":1.006},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0069999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.66000000000000003},"min":{"1min":0.53700000000000003,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63600000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:48:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85699999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.63500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90000000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78300000000000003}]},{"osd":9,"last update":"Wed Sep 20 21:48:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.629,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77700000000000002,"15min":0.77600000000000002},"min":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.90500000000000003,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.76200000000000001}]},{"osd":10,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.52800000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91500000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.48599999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83299999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69499999999999995}]},{"osd":11,"last update":"Wed Sep 20 21:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83999999999999997,"15min":0.84499999999999997},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.93000000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":0.97099999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.84299999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":515833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81999999999999995,"5min":1.018,"15min":1.018},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66500000000000004,"15min":0.629},"min":{"1min":0.438,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.04,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79600000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.41699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":1.2,"15min":1.2},"last":0.92000000000000004}]},{"osd":2,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.873,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72399999999999998,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.99399999999999999,"5min":1.466,"15min":1.466},"last":0.873}]},{"osd":3,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69899999999999995,"15min":0.72299999999999998},"min":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.048,"5min":1.093,"15min":1.093},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0720000000000001,"5min":1.407,"15min":1.407},"last":0.98899999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.82799999999999996,"15min":0.80800000000000005},"min":{"1min":0.59999999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.749,"15min":0.73899999999999999},"min":{"1min":0.51800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Sep 20 21:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84999999999999998,"15min":0.83799999999999997},"min":{"1min":0.5,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2989999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.85099999999999998},"min":{"1min":0.64800000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.214,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0109999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.82599999999999996,"15min":0.85299999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.55900000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.206,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0329999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.1899999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1619999999999999,"5min":1.333,"15min":1.333},"last":1.0489999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.93899999999999995,"15min":0.94699999999999995},"min":{"1min":0.63600000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2370000000000001,"5min":1.302,"15min":1.302},"last":1.077},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.96799999999999997,"15min":0.97899999999999998},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2589999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.071}]},{"osd":11,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.0920000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":520705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.59399999999999997,"15min":0.54300000000000004},"min":{"1min":0.316,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.9539999999999997,"5min":4.9539999999999997,"15min":4.9539999999999997},"last":0.747},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.53700000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69599999999999995,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56000000000000005,"15min":0.51400000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.82299999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54400000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58699999999999997,"15min":0.53700000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.626}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.41899999999999998,"15min":0.39300000000000002},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.58599999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.84299999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.499},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64700000000000002,"15min":0.60499999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.878,"5min":1.01,"15min":1.01},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.70099999999999996,"15min":0.68200000000000005},"min":{"1min":0.44600000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":1.27,"15min":1.27},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:48:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77300000000000002,"5min":1.218,"15min":1.218},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.41299999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.321,"15min":1.321},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67500000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73799999999999999,"15min":0.751},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85299999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96999999999999997,"5min":1.054,"15min":1.054},"last":0.73399999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.72799999999999998,"15min":0.74199999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91200000000000003,"5min":1,"15min":1},"last":0.443},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.435,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93999999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.63800000000000001}]},{"osd":11,"last update":"Wed Sep 20 21:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78700000000000003,"15min":0.79100000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95199999999999996,"5min":1.018,"15min":1.018},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92700000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.70299999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":517289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 20 21:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":0.755,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":7.0289999999999999,"5min":7.0289999999999999,"15min":7.0289999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.76700000000000002,"15min":0.67400000000000004},"min":{"1min":0.53800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.4390000000000001,"5min":6.4390000000000001,"15min":6.4390000000000001},"last":0.76000000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72999999999999998,"15min":0.65400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.7589999999999999,"5min":8.1639999999999997,"15min":8.1639999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.67500000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97199999999999998,"5min":6.5270000000000001,"15min":6.5270000000000001},"last":0.71899999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.97399999999999998,"15min":1.038},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88500000000000001,"5min":8.9499999999999993,"15min":8.9499999999999993},"last":0.623},{"interface":"front","average":{"1min":0.63,"5min":0.88600000000000001,"15min":0.92800000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2110000000000001,"5min":8.6620000000000008,"15min":8.6620000000000008},"last":0.83699999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":0.80100000000000005,"15min":0.68700000000000006},"min":{"1min":0.45900000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":8.2560000000000002,"5min":8.2560000000000002,"15min":8.2560000000000002},"last":0.441},{"interface":"front","average":{"1min":1.123,"5min":0.80800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47999999999999998,"5min":0.443,"15min":0.443},"max":{"1min":6.5650000000000004,"5min":6.5650000000000004,"15min":6.5650000000000004},"last":0.81299999999999994}]},{"osd":5,"last update":"Wed Sep 20 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84199999999999997,"15min":0.73299999999999998},"min":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.792,"5min":8.2919999999999998,"15min":8.2919999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80400000000000005,"15min":0.70799999999999996},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.095,"5min":6.601,"15min":6.601},"last":0.78700000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":0.85099999999999998,"15min":0.72199999999999998},"min":{"1min":0.505,"5min":0.375,"15min":0.375},"max":{"1min":8.9049999999999994,"5min":8.9049999999999994,"15min":8.9049999999999994},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.79700000000000004,"15min":0.74299999999999999},"min":{"1min":0.49199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":8.3659999999999997,"5min":8.3659999999999997,"15min":8.3659999999999997},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":6.6440000000000001,"5min":6.6440000000000001,"15min":6.6440000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":8.2159999999999993,"5min":8.2159999999999993,"15min":8.2159999999999993},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":0.93999999999999995,"15min":0.86799999999999999},"min":{"1min":0.628,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":8.3290000000000006,"5min":8.3290000000000006,"15min":8.3290000000000006},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.1990000000000001,"5min":0.96099999999999997,"15min":0.93500000000000005},"min":{"1min":0.79600000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":6.6799999999999997,"5min":6.6799999999999997,"15min":6.6799999999999997},"last":0.89700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.0129999999999999,"15min":0.96199999999999997},"min":{"1min":0.66400000000000003,"5min":0.502,"15min":0.502},"max":{"1min":8.9879999999999995,"5min":8.9879999999999995,"15min":8.9879999999999995},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.2869999999999999,"5min":0.88300000000000001,"15min":0.83199999999999996},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":8.702,"5min":8.702,"15min":8.702},"last":0.879}]},{"osd":10,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0249999999999999,"15min":0.96899999999999997},"min":{"1min":0.73099999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.5,"5min":6.7149999999999999,"15min":6.7149999999999999},"last":1.04},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.90100000000000002,"15min":0.83699999999999997},"min":{"1min":0.61099999999999999,"5min":0.498,"15min":0.498},"max":{"1min":1.931,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":0.92500000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":0.877,"15min":0.82699999999999996},"min":{"1min":0.58999999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.0669999999999999,"15min":0.98599999999999999},"min":{"1min":0.85399999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":1.0629999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":520925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.55600000000000005},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.872,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64700000000000002,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94499999999999995,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64800000000000002,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83899999999999997,"5min":7.0330000000000004,"15min":7.0330000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.57699999999999996,"15min":0.52600000000000002},"min":{"1min":0.41799999999999998,"5min":0.33000000000000002,"15min":0.307},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.66800000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:47:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84599999999999997,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.505},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66800000000000004,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90900000000000003,"5min":3.645,"15min":3.645},"last":0.40100000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.82099999999999995,"15min":0.84299999999999997},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88800000000000001,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.749,"5min":1.181,"15min":1.2529999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0469999999999999,"5min":9.8659999999999997,"15min":9.8659999999999997},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.42099999999999999},"min":{"1min":0.34499999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73799999999999999,"5min":3.798,"15min":3.798},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.51000000000000001},"min":{"1min":0.40300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80000000000000004,"5min":7.1500000000000004,"15min":7.1500000000000004},"last":0.44700000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71199999999999997,"15min":0.64200000000000002},"min":{"1min":0.55700000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86299999999999999,"5min":7.367,"15min":7.367},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.754,"15min":0.67400000000000004},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.018,"5min":6.4649999999999999,"15min":6.4649999999999999},"last":0.69899999999999995}]},{"osd":7,"last update":"Wed Sep 20 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.58499999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.054,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.873},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.54900000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60299999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.63300000000000001},"min":{"1min":0.44900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95699999999999996,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.65200000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:48:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78800000000000003,"15min":0.72999999999999998},"min":{"1min":0.59599999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.99399999999999999,"5min":7.1200000000000001,"15min":7.1200000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.82699999999999996,"15min":0.78500000000000003},"min":{"1min":0.59299999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.137,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.81699999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80500000000000005,"15min":0.72299999999999998},"min":{"1min":0.626,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0600000000000001,"5min":7.1799999999999997,"15min":7.1799999999999997},"last":0.747},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.745,"15min":0.71299999999999997},"min":{"1min":0.50600000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0309999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.54300000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.143,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.0940000000000001,"15min":1.139},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.099,"5min":7.2110000000000003,"15min":7.2110000000000003},"last":0.78100000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":515541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.47999999999999998,"15min":0.40600000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.50800000000000001,"15min":0.42999999999999999},"min":{"1min":0.34300000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.504}]},{"osd":1,"last update":"Wed Sep 20 21:48:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63800000000000001,"15min":0.57799999999999996},"min":{"1min":0.432,"5min":0.376,"15min":0.376},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.73699999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:48:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65800000000000003,"15min":0.60999999999999999},"min":{"1min":0.42899999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0329999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed Sep 20 21:48:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67800000000000005,"15min":0.63400000000000001},"min":{"1min":0.504,"5min":0.436,"15min":0.436},"max":{"1min":0.877,"5min":1.601,"15min":1.601},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.754,"15min":0.66700000000000004},"min":{"1min":0.5,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93100000000000005,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.93999999999999995}]},{"osd":5,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.502,"5min":0.434,"15min":0.434},"max":{"1min":0.83199999999999996,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77400000000000002,"15min":0.752},"min":{"1min":0.52700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92900000000000005,"5min":1.167,"15min":1.167},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.69599999999999995,"15min":0.65000000000000002},"min":{"1min":0.50900000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.73499999999999999,"15min":0.71199999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Wed Sep 20 21:48:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.77600000000000002},"min":{"1min":0.56699999999999995,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96699999999999997,"5min":1.248,"15min":1.248},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82799999999999996,"15min":0.81899999999999995},"min":{"1min":0.64700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98499999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.97699999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91300000000000003,"15min":0.94299999999999995},"min":{"1min":0.65000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.1819999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.92200000000000004,"15min":0.93899999999999995},"min":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.74299999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90000000000000002,"15min":0.91300000000000003},"min":{"1min":0.73399999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0389999999999999,"5min":1.228,"15min":1.228},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.071,"5min":0.96099999999999997,"15min":0.93700000000000006},"min":{"1min":0.64000000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"last":0.90400000000000003}]},{"osd":11,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.109,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.879},{"interface":"front","average":{"1min":0.874,"5min":0.99199999999999999,"15min":1.012},"min":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.0109999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.81599999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":516069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.41399999999999998,"15min":0.37},"min":{"1min":0.314,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59999999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.38200000000000001},"min":{"1min":0.308,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.57599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47899999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.59999999999999998,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.53800000000000003},"min":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69099999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56000000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.57899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed Sep 20 21:48:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.621,"15min":0.57999999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.626},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.59799999999999998},"min":{"1min":0.503,"5min":0.379,"15min":0.379},"max":{"1min":0.81000000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:48:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":0.71899999999999997,"15min":0.61299999999999999},"min":{"1min":0.47399999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.5129999999999999,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.69299999999999995,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:48:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84299999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79500000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53700000000000003}]},{"osd":8,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.54200000000000004,"5min":0.371,"15min":0.371},"max":{"1min":0.89500000000000002,"5min":1,"15min":1},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84999999999999998}]},{"osd":9,"last update":"Wed Sep 20 21:48:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.83099999999999996,"15min":0.80400000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.109,"5min":4.109,"15min":4.109},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.78200000000000003,"15min":0.76300000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":0.81899999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:48:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.749},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.78600000000000003,"15min":0.79800000000000004},"min":{"1min":0.48899999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97399999999999998,"5min":1.048,"15min":1.048},"last":0.67000000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:48:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84599999999999997,"15min":0.85299999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.006,"5min":1.115,"15min":1.115},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.94599999999999995,"5min":1.008,"15min":1.008},"last":0.92400000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1108018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.087,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.1240000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.56299999999999994}]},{"osd":1,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56799999999999995,"15min":0.54100000000000004},"min":{"1min":0.41899999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.70999999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.53400000000000003},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55200000000000005}]},{"osd":2,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.378},"min":{"1min":0.30599999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58099999999999996,"5min":0.755,"15min":0.755},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.433,"15min":0.39900000000000002},"min":{"1min":0.311,"5min":0.255,"15min":0.255},"max":{"1min":0.66600000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.40699999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.436},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73399999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.57299999999999995},"min":{"1min":0.44700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80000000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.59699999999999998},"min":{"1min":0.45900000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77600000000000002,"5min":1.008,"15min":1.008},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed Sep 20 21:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.65500000000000003},"min":{"1min":0.54500000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84799999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.64900000000000002,"15min":0.626},"min":{"1min":0.48799999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.627}]},{"osd":7,"last update":"Wed Sep 20 21:48:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.87,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.66500000000000004},"min":{"1min":0.47499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81599999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.57999999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":0.879,"15min":0.879},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.51800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53800000000000003}]},{"osd":9,"last update":"Wed Sep 20 21:48:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.79800000000000004},"min":{"1min":0.626,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.68400000000000005}]},{"osd":10,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.751,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93400000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.89200000000000002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-20T21:48:53.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-20T21:48:54.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.150+0000 7fb3851ff700 1 -- 172.21.15.22:0/3820180835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb380102970 msgr2=0x7fb380109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:54.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.150+0000 7fb3851ff700 1 --2- 172.21.15.22:0/3820180835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb380102970 0x7fb380109220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb378009fc0 tx=0x7fb37800b330 comp rx=0 tx=0).stop 2023-09-20T21:48:54.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.150+0000 7fb3851ff700 1 -- 172.21.15.22:0/3820180835 shutdown_connections 2023-09-20T21:48:54.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.150+0000 7fb3851ff700 1 --2- 172.21.15.22:0/3820180835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb380102970 0x7fb380109220 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.150+0000 7fb3851ff700 1 --2- 172.21.15.22:0/3820180835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb380100de0 0x7fb380101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.150+0000 7fb3851ff700 1 --2- 172.21.15.22:0/3820180835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb380101fd0 0x7fb380102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.150+0000 7fb3851ff700 1 -- 172.21.15.22:0/3820180835 >> 172.21.15.22:0/3820180835 conn(0x7fb3800fc330 msgr2=0x7fb3800fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:54.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.150+0000 7fb3851ff700 1 -- 172.21.15.22:0/3820180835 shutdown_connections 2023-09-20T21:48:54.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.150+0000 7fb3851ff700 1 -- 172.21.15.22:0/3820180835 wait complete. 2023-09-20T21:48:54.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.151+0000 7fb3851ff700 1 Processor -- start 2023-09-20T21:48:54.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.151+0000 7fb3851ff700 1 -- start start 2023-09-20T21:48:54.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb3851ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb380100de0 0x7fb380198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:54.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb3851ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb380101fd0 0x7fb380199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:54.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb3851ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb380102970 0x7fb3801a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:54.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb3851ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3801a08c0 con 0x7fb380102970 2023-09-20T21:48:54.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb3851ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3801a0a00 con 0x7fb380100de0 2023-09-20T21:48:54.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb3851ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3801a0b90 con 0x7fb380101fd0 2023-09-20T21:48:54.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb37ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb380100de0 0x7fb380198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:54.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb37ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb380100de0 0x7fb380198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37626/0 (socket says 172.21.15.22:37626) 2023-09-20T21:48:54.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb37ed9d700 1 -- 172.21.15.22:0/2972608747 learned_addr learned my addr 172.21.15.22:0/2972608747 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:54.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb37f59e700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb380102970 0x7fb3801a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:54.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb37e59c700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb380101fd0 0x7fb380199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:54.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.152+0000 7fb37ed9d700 1 -- 172.21.15.22:0/2972608747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb380101fd0 msgr2=0x7fb380199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:54.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.153+0000 7fb37ed9d700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb380101fd0 0x7fb380199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.153+0000 7fb37ed9d700 1 -- 172.21.15.22:0/2972608747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb380102970 msgr2=0x7fb3801a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:54.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.153+0000 7fb37ed9d700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb380102970 0x7fb3801a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.153+0000 7fb37ed9d700 1 -- 172.21.15.22:0/2972608747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3740095f0 con 0x7fb380100de0 2023-09-20T21:48:54.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.153+0000 7fb37ed9d700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb380100de0 0x7fb380198d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e970 tx=0x7fb37000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:54.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.153+0000 7fb367fff700 1 -- 172.21.15.22:0/2972608747 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370009790 con 0x7fb380100de0 2023-09-20T21:48:54.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.153+0000 7fb3851ff700 1 -- 172.21.15.22:0/2972608747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb378009c40 con 0x7fb380100de0 2023-09-20T21:48:54.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.153+0000 7fb367fff700 1 -- 172.21.15.22:0/2972608747 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb37000c7e0 con 0x7fb380100de0 2023-09-20T21:48:54.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.153+0000 7fb367fff700 1 -- 172.21.15.22:0/2972608747 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370003c70 con 0x7fb380100de0 2023-09-20T21:48:54.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.153+0000 7fb3851ff700 1 -- 172.21.15.22:0/2972608747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3801a1190 con 0x7fb380100de0 2023-09-20T21:48:54.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.154+0000 7fb3851ff700 1 -- 172.21.15.22:0/2972608747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38010c670 con 0x7fb380100de0 2023-09-20T21:48:54.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.156+0000 7fb367fff700 1 -- 172.21.15.22:0/2972608747 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fb37000cc50 con 0x7fb380100de0 2023-09-20T21:48:54.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.158+0000 7fb367fff700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3680a13d0 0x7fb3680a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:54.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.158+0000 7fb367fff700 1 -- 172.21.15.22:0/2972608747 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fb370014070 con 0x7fb380100de0 2023-09-20T21:48:54.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.158+0000 7fb37e59c700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3680a13d0 0x7fb3680a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:54.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.159+0000 7fb37e59c700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3680a13d0 0x7fb3680a3890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb374005fd0 tx=0x7fb374009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:54.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.160+0000 7fb367fff700 1 -- 172.21.15.22:0/2972608747 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb3700965a0 con 0x7fb380100de0 2023-09-20T21:48:54.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.370+0000 7fb3851ff700 1 -- 172.21.15.22:0/2972608747 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb380108f40 con 0x7fb3680a13d0 2023-09-20T21:48:54.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.383+0000 7fb367fff700 1 -- 172.21.15.22:0/2972608747 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426901 (secure 0 0 0) 0x7fb380108f40 con 0x7fb3680a13d0 2023-09-20T21:48:54.389 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:54.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 -- 172.21.15.22:0/2972608747 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3680a13d0 msgr2=0x7fb3680a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:54.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3680a13d0 0x7fb3680a3890 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb374005fd0 tx=0x7fb374009310 comp rx=0 tx=0).stop 2023-09-20T21:48:54.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 -- 172.21.15.22:0/2972608747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb380100de0 msgr2=0x7fb380198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:54.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb380100de0 0x7fb380198d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e970 tx=0x7fb37000ec80 comp rx=0 tx=0).stop 2023-09-20T21:48:54.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 -- 172.21.15.22:0/2972608747 shutdown_connections 2023-09-20T21:48:54.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb380102970 0x7fb3801a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb3680a13d0 0x7fb3680a3890 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb380100de0 0x7fb380198d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 --2- 172.21.15.22:0/2972608747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb380101fd0 0x7fb380199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 -- 172.21.15.22:0/2972608747 >> 172.21.15.22:0/2972608747 conn(0x7fb3800fc330 msgr2=0x7fb380107820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:54.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 -- 172.21.15.22:0/2972608747 shutdown_connections 2023-09-20T21:48:54.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.388+0000 7fb3851ff700 1 -- 172.21.15.22:0/2972608747 wait complete. 2023-09-20T21:48:54.395 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-09-20T21:48:54.407 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2023-09-20T21:48:53.132727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600890,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1184,"num_read_kb":656,"num_write":392,"num_write_kb":3007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338640,"kb_used_data":14360,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603184,"statfs":{"total":1151940427776,"available":1151593660416,"internally_reserved":0,"allocated":14704640,"data_stored":7983721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":66,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003652"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.597787+0000","last_change":"2023-09-20T21:46:41.506092+0000","last_active":"2023-09-20T21:48:09.597787+0000","last_peered":"2023-09-20T21:48:09.597787+0000","last_clean":"2023-09-20T21:48:09.597787+0000","last_became_active":"2023-09-20T21:46:41.505779+0000","last_became_peered":"2023-09-20T21:46:41.505779+0000","last_unstale":"2023-09-20T21:48:09.597787+0000","last_undegraded":"2023-09-20T21:48:09.597787+0000","last_fullsized":"2023-09-20T21:48:09.597787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:25.284068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.971254+0000","last_change":"2023-09-20T21:46:41.505000+0000","last_active":"2023-09-20T21:48:09.971254+0000","last_peered":"2023-09-20T21:48:09.971254+0000","last_clean":"2023-09-20T21:48:09.971254+0000","last_became_active":"2023-09-20T21:46:41.504464+0000","last_became_peered":"2023-09-20T21:46:41.504464+0000","last_unstale":"2023-09-20T21:48:09.971254+0000","last_undegraded":"2023-09-20T21:48:09.971254+0000","last_fullsized":"2023-09-20T21:48:09.971254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:42:44.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.970944+0000","last_change":"2023-09-20T21:46:41.513134+0000","last_active":"2023-09-20T21:48:09.970944+0000","last_peered":"2023-09-20T21:48:09.970944+0000","last_clean":"2023-09-20T21:48:09.970944+0000","last_became_active":"2023-09-20T21:46:41.512793+0000","last_became_peered":"2023-09-20T21:46:41.512793+0000","last_unstale":"2023-09-20T21:48:09.970944+0000","last_undegraded":"2023-09-20T21:48:09.970944+0000","last_fullsized":"2023-09-20T21:48:09.970944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:59.978630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.667363+0000","last_change":"2023-09-20T21:46:41.500200+0000","last_active":"2023-09-20T21:48:09.667363+0000","last_peered":"2023-09-20T21:48:09.667363+0000","last_clean":"2023-09-20T21:48:09.667363+0000","last_became_active":"2023-09-20T21:46:41.499866+0000","last_became_peered":"2023-09-20T21:46:41.499866+0000","last_unstale":"2023-09-20T21:48:09.667363+0000","last_undegraded":"2023-09-20T21:48:09.667363+0000","last_fullsized":"2023-09-20T21:48:09.667363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:20:52.989008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584846+0000","last_change":"2023-09-20T21:46:41.507584+0000","last_active":"2023-09-20T21:48:08.584846+0000","last_peered":"2023-09-20T21:48:08.584846+0000","last_clean":"2023-09-20T21:48:08.584846+0000","last_became_active":"2023-09-20T21:46:41.507298+0000","last_became_peered":"2023-09-20T21:46:41.507298+0000","last_unstale":"2023-09-20T21:48:08.584846+0000","last_undegraded":"2023-09-20T21:48:08.584846+0000","last_fullsized":"2023-09-20T21:48:08.584846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:32:14.042414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585516+0000","last_change":"2023-09-20T21:46:41.497002+0000","last_active":"2023-09-20T21:48:08.585516+0000","last_peered":"2023-09-20T21:48:08.585516+0000","last_clean":"2023-09-20T21:48:08.585516+0000","last_became_active":"2023-09-20T21:46:41.496485+0000","last_became_peered":"2023-09-20T21:46:41.496485+0000","last_unstale":"2023-09-20T21:48:08.585516+0000","last_undegraded":"2023-09-20T21:48:08.585516+0000","last_fullsized":"2023-09-20T21:48:08.585516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:48:24.481575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590370+0000","last_change":"2023-09-20T21:46:41.503303+0000","last_active":"2023-09-20T21:48:08.590370+0000","last_peered":"2023-09-20T21:48:08.590370+0000","last_clean":"2023-09-20T21:48:08.590370+0000","last_became_active":"2023-09-20T21:46:41.502982+0000","last_became_peered":"2023-09-20T21:46:41.502982+0000","last_unstale":"2023-09-20T21:48:08.590370+0000","last_undegraded":"2023-09-20T21:48:08.590370+0000","last_fullsized":"2023-09-20T21:48:08.590370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:23:27.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.589897+0000","last_change":"2023-09-20T21:46:41.508243+0000","last_active":"2023-09-20T21:48:08.589897+0000","last_peered":"2023-09-20T21:48:08.589897+0000","last_clean":"2023-09-20T21:48:08.589897+0000","last_became_active":"2023-09-20T21:46:41.506407+0000","last_became_peered":"2023-09-20T21:46:41.506407+0000","last_unstale":"2023-09-20T21:48:08.589897+0000","last_undegraded":"2023-09-20T21:48:08.589897+0000","last_fullsized":"2023-09-20T21:48:08.589897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:39.388716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588390+0000","last_change":"2023-09-20T21:46:41.501513+0000","last_active":"2023-09-20T21:48:08.588390+0000","last_peered":"2023-09-20T21:48:08.588390+0000","last_clean":"2023-09-20T21:48:08.588390+0000","last_became_active":"2023-09-20T21:46:41.500051+0000","last_became_peered":"2023-09-20T21:46:41.500051+0000","last_unstale":"2023-09-20T21:48:08.588390+0000","last_undegraded":"2023-09-20T21:48:08.588390+0000","last_fullsized":"2023-09-20T21:48:08.588390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:40.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585877+0000","last_change":"2023-09-20T21:46:41.501045+0000","last_active":"2023-09-20T21:48:08.585877+0000","last_peered":"2023-09-20T21:48:08.585877+0000","last_clean":"2023-09-20T21:48:08.585877+0000","last_became_active":"2023-09-20T21:46:41.500744+0000","last_became_peered":"2023-09-20T21:46:41.500744+0000","last_unstale":"2023-09-20T21:48:08.585877+0000","last_undegraded":"2023-09-20T21:48:08.585877+0000","last_fullsized":"2023-09-20T21:48:08.585877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:57:29.120472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590634+0000","last_change":"2023-09-20T21:46:41.510235+0000","last_active":"2023-09-20T21:48:08.590634+0000","last_peered":"2023-09-20T21:48:08.590634+0000","last_clean":"2023-09-20T21:48:08.590634+0000","last_became_active":"2023-09-20T21:46:41.509943+0000","last_became_peered":"2023-09-20T21:46:41.509943+0000","last_unstale":"2023-09-20T21:48:08.590634+0000","last_undegraded":"2023-09-20T21:48:08.590634+0000","last_fullsized":"2023-09-20T21:48:08.590634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:09:19.383505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663853+0000","last_change":"2023-09-20T21:46:41.501194+0000","last_active":"2023-09-20T21:48:08.663853+0000","last_peered":"2023-09-20T21:48:08.663853+0000","last_clean":"2023-09-20T21:48:08.663853+0000","last_became_active":"2023-09-20T21:46:41.500095+0000","last_became_peered":"2023-09-20T21:46:41.500095+0000","last_unstale":"2023-09-20T21:48:08.663853+0000","last_undegraded":"2023-09-20T21:48:08.663853+0000","last_fullsized":"2023-09-20T21:48:08.663853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:46:51.343332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587794+0000","last_change":"2023-09-20T21:46:41.507562+0000","last_active":"2023-09-20T21:48:08.587794+0000","last_peered":"2023-09-20T21:48:08.587794+0000","last_clean":"2023-09-20T21:48:08.587794+0000","last_became_active":"2023-09-20T21:46:41.507094+0000","last_became_peered":"2023-09-20T21:46:41.507094+0000","last_unstale":"2023-09-20T21:48:08.587794+0000","last_undegraded":"2023-09-20T21:48:08.587794+0000","last_fullsized":"2023-09-20T21:48:08.587794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:34:32.401415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591639+0000","last_change":"2023-09-20T21:46:41.502864+0000","last_active":"2023-09-20T21:48:08.591639+0000","last_peered":"2023-09-20T21:48:08.591639+0000","last_clean":"2023-09-20T21:48:08.591639+0000","last_became_active":"2023-09-20T21:46:41.502543+0000","last_became_peered":"2023-09-20T21:46:41.502543+0000","last_unstale":"2023-09-20T21:48:08.591639+0000","last_undegraded":"2023-09-20T21:48:08.591639+0000","last_fullsized":"2023-09-20T21:48:08.591639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:02:04.631874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.592056+0000","last_change":"2023-09-20T21:47:08.600445+0000","last_active":"2023-09-20T21:48:08.592056+0000","last_peered":"2023-09-20T21:48:08.592056+0000","last_clean":"2023-09-20T21:48:08.592056+0000","last_became_active":"2023-09-20T21:47:08.600054+0000","last_became_peered":"2023-09-20T21:47:08.600054+0000","last_unstale":"2023-09-20T21:48:08.592056+0000","last_undegraded":"2023-09-20T21:48:08.592056+0000","last_fullsized":"2023-09-20T21:48:08.592056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:43:18.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587667+0000","last_change":"2023-09-20T21:46:41.502036+0000","last_active":"2023-09-20T21:48:08.587667+0000","last_peered":"2023-09-20T21:48:08.587667+0000","last_clean":"2023-09-20T21:48:08.587667+0000","last_became_active":"2023-09-20T21:46:41.501745+0000","last_became_peered":"2023-09-20T21:46:41.501745+0000","last_unstale":"2023-09-20T21:48:08.587667+0000","last_undegraded":"2023-09-20T21:48:08.587667+0000","last_fullsized":"2023-09-20T21:48:08.587667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:08:06.699543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588222+0000","last_change":"2023-09-20T21:46:41.506666+0000","last_active":"2023-09-20T21:48:08.588222+0000","last_peered":"2023-09-20T21:48:08.588222+0000","last_clean":"2023-09-20T21:48:08.588222+0000","last_became_active":"2023-09-20T21:46:41.506266+0000","last_became_peered":"2023-09-20T21:46:41.506266+0000","last_unstale":"2023-09-20T21:48:08.588222+0000","last_undegraded":"2023-09-20T21:48:08.588222+0000","last_fullsized":"2023-09-20T21:48:08.588222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:06:16.809609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585233+0000","last_change":"2023-09-20T21:46:41.497391+0000","last_active":"2023-09-20T21:48:08.585233+0000","last_peered":"2023-09-20T21:48:08.585233+0000","last_clean":"2023-09-20T21:48:08.585233+0000","last_became_active":"2023-09-20T21:46:41.497064+0000","last_became_peered":"2023-09-20T21:46:41.497064+0000","last_unstale":"2023-09-20T21:48:08.585233+0000","last_undegraded":"2023-09-20T21:48:08.585233+0000","last_fullsized":"2023-09-20T21:48:08.585233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:56:11.489585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587770+0000","last_change":"2023-09-20T21:46:41.510602+0000","last_active":"2023-09-20T21:48:08.587770+0000","last_peered":"2023-09-20T21:48:08.587770+0000","last_clean":"2023-09-20T21:48:08.587770+0000","last_became_active":"2023-09-20T21:46:41.510308+0000","last_became_peered":"2023-09-20T21:46:41.510308+0000","last_unstale":"2023-09-20T21:48:08.587770+0000","last_undegraded":"2023-09-20T21:48:08.587770+0000","last_fullsized":"2023-09-20T21:48:08.587770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:43:41.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.972323+0000","last_change":"2023-09-20T21:48:09.972323+0000","last_active":"2023-09-20T21:48:09.972323+0000","last_peered":"2023-09-20T21:48:09.972323+0000","last_clean":"2023-09-20T21:48:09.972323+0000","last_became_active":"2023-09-20T21:48:09.971743+0000","last_became_peered":"2023-09-20T21:48:09.971743+0000","last_unstale":"2023-09-20T21:48:09.972323+0000","last_undegraded":"2023-09-20T21:48:09.972323+0000","last_fullsized":"2023-09-20T21:48:09.972323+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:00:49.679685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":136,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:47.769593+0000","last_change":"2023-09-20T21:46:40.500010+0000","last_active":"2023-09-20T21:48:47.769593+0000","last_peered":"2023-09-20T21:48:47.769593+0000","last_clean":"2023-09-20T21:48:47.769593+0000","last_became_active":"2023-09-20T21:46:40.499632+0000","last_became_peered":"2023-09-20T21:46:40.499632+0000","last_unstale":"2023-09-20T21:48:47.769593+0000","last_undegraded":"2023-09-20T21:48:47.769593+0000","last_fullsized":"2023-09-20T21:48:47.769593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:41:35.590982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594616+0000","last_change":"2023-09-20T21:46:41.516465+0000","last_active":"2023-09-20T21:48:09.594616+0000","last_peered":"2023-09-20T21:48:09.594616+0000","last_clean":"2023-09-20T21:48:09.594616+0000","last_became_active":"2023-09-20T21:46:41.516150+0000","last_became_peered":"2023-09-20T21:46:41.516150+0000","last_unstale":"2023-09-20T21:48:09.594616+0000","last_undegraded":"2023-09-20T21:48:09.594616+0000","last_fullsized":"2023-09-20T21:48:09.594616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:33:37.507049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663434+0000","last_change":"2023-09-20T21:46:45.582916+0000","last_active":"2023-09-20T21:48:08.663434+0000","last_peered":"2023-09-20T21:48:08.663434+0000","last_clean":"2023-09-20T21:48:08.663434+0000","last_became_active":"2023-09-20T21:46:45.582511+0000","last_became_peered":"2023-09-20T21:46:45.582511+0000","last_unstale":"2023-09-20T21:48:08.663434+0000","last_undegraded":"2023-09-20T21:48:08.663434+0000","last_fullsized":"2023-09-20T21:48:08.663434+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:43:47.006768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594814+0000","last_change":"2023-09-20T21:46:40.516867+0000","last_active":"2023-09-20T21:48:09.594814+0000","last_peered":"2023-09-20T21:48:09.594814+0000","last_clean":"2023-09-20T21:48:09.594814+0000","last_became_active":"2023-09-20T21:46:40.516343+0000","last_became_peered":"2023-09-20T21:46:40.516343+0000","last_unstale":"2023-09-20T21:48:09.594814+0000","last_undegraded":"2023-09-20T21:48:09.594814+0000","last_fullsized":"2023-09-20T21:48:09.594814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:17:18.348921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.970829+0000","last_change":"2023-09-20T21:46:41.507093+0000","last_active":"2023-09-20T21:48:09.970829+0000","last_peered":"2023-09-20T21:48:09.970829+0000","last_clean":"2023-09-20T21:48:09.970829+0000","last_became_active":"2023-09-20T21:46:41.506587+0000","last_became_peered":"2023-09-20T21:46:41.506587+0000","last_unstale":"2023-09-20T21:48:09.970829+0000","last_undegraded":"2023-09-20T21:48:09.970829+0000","last_fullsized":"2023-09-20T21:48:09.970829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:26:44.825499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663624+0000","last_change":"2023-09-20T21:46:45.583942+0000","last_active":"2023-09-20T21:48:08.663624+0000","last_peered":"2023-09-20T21:48:08.663624+0000","last_clean":"2023-09-20T21:48:08.663624+0000","last_became_active":"2023-09-20T21:46:45.583484+0000","last_became_peered":"2023-09-20T21:46:45.583484+0000","last_unstale":"2023-09-20T21:48:08.663624+0000","last_undegraded":"2023-09-20T21:48:08.663624+0000","last_fullsized":"2023-09-20T21:48:08.663624+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:48.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585701+0000","last_change":"2023-09-20T21:47:08.615780+0000","last_active":"2023-09-20T21:48:08.585701+0000","last_peered":"2023-09-20T21:48:08.585701+0000","last_clean":"2023-09-20T21:48:08.585701+0000","last_became_active":"2023-09-20T21:47:08.603974+0000","last_became_peered":"2023-09-20T21:47:08.603974+0000","last_unstale":"2023-09-20T21:48:08.585701+0000","last_undegraded":"2023-09-20T21:48:08.585701+0000","last_fullsized":"2023-09-20T21:48:08.585701+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:53:23.172401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587966+0000","last_change":"2023-09-20T21:46:41.504449+0000","last_active":"2023-09-20T21:48:08.587966+0000","last_peered":"2023-09-20T21:48:08.587966+0000","last_clean":"2023-09-20T21:48:08.587966+0000","last_became_active":"2023-09-20T21:46:41.504169+0000","last_became_peered":"2023-09-20T21:46:41.504169+0000","last_unstale":"2023-09-20T21:48:08.587966+0000","last_undegraded":"2023-09-20T21:48:08.587966+0000","last_fullsized":"2023-09-20T21:48:08.587966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:36:43.980902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663507+0000","last_change":"2023-09-20T21:46:45.582269+0000","last_active":"2023-09-20T21:48:08.663507+0000","last_peered":"2023-09-20T21:48:08.663507+0000","last_clean":"2023-09-20T21:48:08.663507+0000","last_became_active":"2023-09-20T21:46:45.581782+0000","last_became_peered":"2023-09-20T21:46:45.581782+0000","last_unstale":"2023-09-20T21:48:08.663507+0000","last_undegraded":"2023-09-20T21:48:08.663507+0000","last_fullsized":"2023-09-20T21:48:08.663507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:32:09.518715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585164+0000","last_change":"2023-09-20T21:46:40.504099+0000","last_active":"2023-09-20T21:48:08.585164+0000","last_peered":"2023-09-20T21:48:08.585164+0000","last_clean":"2023-09-20T21:48:08.585164+0000","last_became_active":"2023-09-20T21:46:40.503774+0000","last_became_peered":"2023-09-20T21:46:40.503774+0000","last_unstale":"2023-09-20T21:48:08.585164+0000","last_undegraded":"2023-09-20T21:48:08.585164+0000","last_fullsized":"2023-09-20T21:48:08.585164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:17.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591062+0000","last_change":"2023-09-20T21:46:41.499802+0000","last_active":"2023-09-20T21:48:08.591062+0000","last_peered":"2023-09-20T21:48:08.591062+0000","last_clean":"2023-09-20T21:48:08.591062+0000","last_became_active":"2023-09-20T21:46:41.499491+0000","last_became_peered":"2023-09-20T21:46:41.499491+0000","last_unstale":"2023-09-20T21:48:08.591062+0000","last_undegraded":"2023-09-20T21:48:08.591062+0000","last_fullsized":"2023-09-20T21:48:08.591062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:44.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587887+0000","last_change":"2023-09-20T21:46:45.578753+0000","last_active":"2023-09-20T21:48:08.587887+0000","last_peered":"2023-09-20T21:48:08.587887+0000","last_clean":"2023-09-20T21:48:08.587887+0000","last_became_active":"2023-09-20T21:46:45.578297+0000","last_became_peered":"2023-09-20T21:46:45.578297+0000","last_unstale":"2023-09-20T21:48:08.587887+0000","last_undegraded":"2023-09-20T21:48:08.587887+0000","last_fullsized":"2023-09-20T21:48:08.587887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:18:22.596550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:48.727867+0000","last_change":"2023-09-20T21:46:40.505675+0000","last_active":"2023-09-20T21:48:48.727867+0000","last_peered":"2023-09-20T21:48:48.727867+0000","last_clean":"2023-09-20T21:48:48.727867+0000","last_became_active":"2023-09-20T21:46:40.505432+0000","last_became_peered":"2023-09-20T21:46:40.505432+0000","last_unstale":"2023-09-20T21:48:48.727867+0000","last_undegraded":"2023-09-20T21:48:48.727867+0000","last_fullsized":"2023-09-20T21:48:48.727867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:14:53.652904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585583+0000","last_change":"2023-09-20T21:46:41.505924+0000","last_active":"2023-09-20T21:48:08.585583+0000","last_peered":"2023-09-20T21:48:08.585583+0000","last_clean":"2023-09-20T21:48:08.585583+0000","last_became_active":"2023-09-20T21:46:41.505609+0000","last_became_peered":"2023-09-20T21:46:41.505609+0000","last_unstale":"2023-09-20T21:48:08.585583+0000","last_undegraded":"2023-09-20T21:48:08.585583+0000","last_fullsized":"2023-09-20T21:48:08.585583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:13:46.506786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588075+0000","last_change":"2023-09-20T21:46:45.587818+0000","last_active":"2023-09-20T21:48:08.588075+0000","last_peered":"2023-09-20T21:48:08.588075+0000","last_clean":"2023-09-20T21:48:08.588075+0000","last_became_active":"2023-09-20T21:46:45.587376+0000","last_became_peered":"2023-09-20T21:46:45.587376+0000","last_unstale":"2023-09-20T21:48:08.588075+0000","last_undegraded":"2023-09-20T21:48:08.588075+0000","last_fullsized":"2023-09-20T21:48:08.588075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:28:53.605158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585426+0000","last_change":"2023-09-20T21:46:40.505610+0000","last_active":"2023-09-20T21:48:08.585426+0000","last_peered":"2023-09-20T21:48:08.585426+0000","last_clean":"2023-09-20T21:48:08.585426+0000","last_became_active":"2023-09-20T21:46:40.505279+0000","last_became_peered":"2023-09-20T21:46:40.505279+0000","last_unstale":"2023-09-20T21:48:08.585426+0000","last_undegraded":"2023-09-20T21:48:08.585426+0000","last_fullsized":"2023-09-20T21:48:08.585426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:41:19.575154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584476+0000","last_change":"2023-09-20T21:47:08.908513+0000","last_active":"2023-09-20T21:48:08.584476+0000","last_peered":"2023-09-20T21:48:08.584476+0000","last_clean":"2023-09-20T21:48:08.584476+0000","last_became_active":"2023-09-20T21:47:08.908142+0000","last_became_peered":"2023-09-20T21:47:08.908142+0000","last_unstale":"2023-09-20T21:48:08.584476+0000","last_undegraded":"2023-09-20T21:48:08.584476+0000","last_fullsized":"2023-09-20T21:48:08.584476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:31:33.992202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588229+0000","last_change":"2023-09-20T21:46:45.581738+0000","last_active":"2023-09-20T21:48:08.588229+0000","last_peered":"2023-09-20T21:48:08.588229+0000","last_clean":"2023-09-20T21:48:08.588229+0000","last_became_active":"2023-09-20T21:46:45.581279+0000","last_became_peered":"2023-09-20T21:46:45.581279+0000","last_unstale":"2023-09-20T21:48:08.588229+0000","last_undegraded":"2023-09-20T21:48:08.588229+0000","last_fullsized":"2023-09-20T21:48:08.588229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:41:41.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587220+0000","last_change":"2023-09-20T21:46:40.510539+0000","last_active":"2023-09-20T21:48:08.587220+0000","last_peered":"2023-09-20T21:48:08.587220+0000","last_clean":"2023-09-20T21:48:08.587220+0000","last_became_active":"2023-09-20T21:46:40.509880+0000","last_became_peered":"2023-09-20T21:46:40.509880+0000","last_unstale":"2023-09-20T21:48:08.587220+0000","last_undegraded":"2023-09-20T21:48:08.587220+0000","last_fullsized":"2023-09-20T21:48:08.587220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:52.877786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.589794+0000","last_change":"2023-09-20T21:46:41.505105+0000","last_active":"2023-09-20T21:48:08.589794+0000","last_peered":"2023-09-20T21:48:08.589794+0000","last_clean":"2023-09-20T21:48:08.589794+0000","last_became_active":"2023-09-20T21:46:41.504785+0000","last_became_peered":"2023-09-20T21:46:41.504785+0000","last_unstale":"2023-09-20T21:48:08.589794+0000","last_undegraded":"2023-09-20T21:48:08.589794+0000","last_fullsized":"2023-09-20T21:48:08.589794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:07:57.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.971145+0000","last_change":"2023-09-20T21:46:45.583160+0000","last_active":"2023-09-20T21:48:09.971145+0000","last_peered":"2023-09-20T21:48:09.971145+0000","last_clean":"2023-09-20T21:48:09.971145+0000","last_became_active":"2023-09-20T21:46:45.582709+0000","last_became_peered":"2023-09-20T21:46:45.582709+0000","last_unstale":"2023-09-20T21:48:09.971145+0000","last_undegraded":"2023-09-20T21:48:09.971145+0000","last_fullsized":"2023-09-20T21:48:09.971145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:24:57.563682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590612+0000","last_change":"2023-09-20T21:46:40.502591+0000","last_active":"2023-09-20T21:48:08.590612+0000","last_peered":"2023-09-20T21:48:08.590612+0000","last_clean":"2023-09-20T21:48:08.590612+0000","last_became_active":"2023-09-20T21:46:40.502288+0000","last_became_peered":"2023-09-20T21:46:40.502288+0000","last_unstale":"2023-09-20T21:48:08.590612+0000","last_undegraded":"2023-09-20T21:48:08.590612+0000","last_fullsized":"2023-09-20T21:48:08.590612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:09:37.983866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584209+0000","last_change":"2023-09-20T21:47:08.599912+0000","last_active":"2023-09-20T21:48:08.584209+0000","last_peered":"2023-09-20T21:48:08.584209+0000","last_clean":"2023-09-20T21:48:08.584209+0000","last_became_active":"2023-09-20T21:47:08.599434+0000","last_became_peered":"2023-09-20T21:47:08.599434+0000","last_unstale":"2023-09-20T21:48:08.584209+0000","last_undegraded":"2023-09-20T21:48:08.584209+0000","last_fullsized":"2023-09-20T21:48:08.584209+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:37:01.125731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590887+0000","last_change":"2023-09-20T21:46:45.580917+0000","last_active":"2023-09-20T21:48:08.590887+0000","last_peered":"2023-09-20T21:48:08.590887+0000","last_clean":"2023-09-20T21:48:08.590887+0000","last_became_active":"2023-09-20T21:46:45.580505+0000","last_became_peered":"2023-09-20T21:46:45.580505+0000","last_unstale":"2023-09-20T21:48:08.590887+0000","last_undegraded":"2023-09-20T21:48:08.590887+0000","last_fullsized":"2023-09-20T21:48:08.590887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:52:19.733919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":230,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:50.730444+0000","last_change":"2023-09-20T21:46:40.507040+0000","last_active":"2023-09-20T21:48:50.730444+0000","last_peered":"2023-09-20T21:48:50.730444+0000","last_clean":"2023-09-20T21:48:50.730444+0000","last_became_active":"2023-09-20T21:46:40.506712+0000","last_became_peered":"2023-09-20T21:46:40.506712+0000","last_unstale":"2023-09-20T21:48:50.730444+0000","last_undegraded":"2023-09-20T21:48:50.730444+0000","last_fullsized":"2023-09-20T21:48:50.730444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:02:56.040911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588376+0000","last_change":"2023-09-20T21:46:41.504794+0000","last_active":"2023-09-20T21:48:08.588376+0000","last_peered":"2023-09-20T21:48:08.588376+0000","last_clean":"2023-09-20T21:48:08.588376+0000","last_became_active":"2023-09-20T21:46:41.504372+0000","last_became_peered":"2023-09-20T21:46:41.504372+0000","last_unstale":"2023-09-20T21:48:08.588376+0000","last_undegraded":"2023-09-20T21:48:08.588376+0000","last_fullsized":"2023-09-20T21:48:08.588376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:42:20.527306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584027+0000","last_change":"2023-09-20T21:46:45.586364+0000","last_active":"2023-09-20T21:48:08.584027+0000","last_peered":"2023-09-20T21:48:08.584027+0000","last_clean":"2023-09-20T21:48:08.584027+0000","last_became_active":"2023-09-20T21:46:45.584620+0000","last_became_peered":"2023-09-20T21:46:45.584620+0000","last_unstale":"2023-09-20T21:48:08.584027+0000","last_undegraded":"2023-09-20T21:48:08.584027+0000","last_fullsized":"2023-09-20T21:48:08.584027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:05:20.158118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590779+0000","last_change":"2023-09-20T21:46:40.500797+0000","last_active":"2023-09-20T21:48:08.590779+0000","last_peered":"2023-09-20T21:48:08.590779+0000","last_clean":"2023-09-20T21:48:08.590779+0000","last_became_active":"2023-09-20T21:46:40.500337+0000","last_became_peered":"2023-09-20T21:46:40.500337+0000","last_unstale":"2023-09-20T21:48:08.590779+0000","last_undegraded":"2023-09-20T21:48:08.590779+0000","last_fullsized":"2023-09-20T21:48:08.590779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:19.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.592706+0000","last_change":"2023-09-20T21:47:08.908615+0000","last_active":"2023-09-20T21:48:08.592706+0000","last_peered":"2023-09-20T21:48:08.592706+0000","last_clean":"2023-09-20T21:48:08.592706+0000","last_became_active":"2023-09-20T21:47:08.908228+0000","last_became_peered":"2023-09-20T21:47:08.908228+0000","last_unstale":"2023-09-20T21:48:08.592706+0000","last_undegraded":"2023-09-20T21:48:08.592706+0000","last_fullsized":"2023-09-20T21:48:08.592706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:15:28.772129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663779+0000","last_change":"2023-09-20T21:46:45.583081+0000","last_active":"2023-09-20T21:48:08.663779+0000","last_peered":"2023-09-20T21:48:08.663779+0000","last_clean":"2023-09-20T21:48:08.663779+0000","last_became_active":"2023-09-20T21:46:45.580496+0000","last_became_peered":"2023-09-20T21:46:45.580496+0000","last_unstale":"2023-09-20T21:48:08.663779+0000","last_undegraded":"2023-09-20T21:48:08.663779+0000","last_fullsized":"2023-09-20T21:48:08.663779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:42.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591031+0000","last_change":"2023-09-20T21:46:40.512739+0000","last_active":"2023-09-20T21:48:08.591031+0000","last_peered":"2023-09-20T21:48:08.591031+0000","last_clean":"2023-09-20T21:48:08.591031+0000","last_became_active":"2023-09-20T21:46:40.512485+0000","last_became_peered":"2023-09-20T21:46:40.512485+0000","last_unstale":"2023-09-20T21:48:08.591031+0000","last_undegraded":"2023-09-20T21:48:08.591031+0000","last_fullsized":"2023-09-20T21:48:08.591031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:02:49.392560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588075+0000","last_change":"2023-09-20T21:47:08.604682+0000","last_active":"2023-09-20T21:48:08.588075+0000","last_peered":"2023-09-20T21:48:08.588075+0000","last_clean":"2023-09-20T21:48:08.588075+0000","last_became_active":"2023-09-20T21:47:08.604271+0000","last_became_peered":"2023-09-20T21:47:08.604271+0000","last_unstale":"2023-09-20T21:48:08.588075+0000","last_undegraded":"2023-09-20T21:48:08.588075+0000","last_fullsized":"2023-09-20T21:48:08.588075+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:40:51.542781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584184+0000","last_change":"2023-09-20T21:46:45.579478+0000","last_active":"2023-09-20T21:48:08.584184+0000","last_peered":"2023-09-20T21:48:08.584184+0000","last_clean":"2023-09-20T21:48:08.584184+0000","last_became_active":"2023-09-20T21:46:45.579055+0000","last_became_peered":"2023-09-20T21:46:45.579055+0000","last_unstale":"2023-09-20T21:48:08.584184+0000","last_undegraded":"2023-09-20T21:48:08.584184+0000","last_fullsized":"2023-09-20T21:48:08.584184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:17:32.013280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'14","reported_seq":244,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:50.730483+0000","last_change":"2023-09-20T21:46:40.507321+0000","last_active":"2023-09-20T21:48:50.730483+0000","last_peered":"2023-09-20T21:48:50.730483+0000","last_clean":"2023-09-20T21:48:50.730483+0000","last_became_active":"2023-09-20T21:46:40.506983+0000","last_became_peered":"2023-09-20T21:46:40.506983+0000","last_unstale":"2023-09-20T21:48:50.730483+0000","last_undegraded":"2023-09-20T21:48:50.730483+0000","last_fullsized":"2023-09-20T21:48:50.730483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:46:40.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":210,"num_read_kb":109,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585464+0000","last_change":"2023-09-20T21:46:41.509646+0000","last_active":"2023-09-20T21:48:08.585464+0000","last_peered":"2023-09-20T21:48:08.585464+0000","last_clean":"2023-09-20T21:48:08.585464+0000","last_became_active":"2023-09-20T21:46:41.509183+0000","last_became_peered":"2023-09-20T21:46:41.509183+0000","last_unstale":"2023-09-20T21:48:08.585464+0000","last_undegraded":"2023-09-20T21:48:08.585464+0000","last_fullsized":"2023-09-20T21:48:08.585464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:17:21.084830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591360+0000","last_change":"2023-09-20T21:46:45.583709+0000","last_active":"2023-09-20T21:48:08.591360+0000","last_peered":"2023-09-20T21:48:08.591360+0000","last_clean":"2023-09-20T21:48:08.591360+0000","last_became_active":"2023-09-20T21:46:45.583302+0000","last_became_peered":"2023-09-20T21:46:45.583302+0000","last_unstale":"2023-09-20T21:48:08.591360+0000","last_undegraded":"2023-09-20T21:48:08.591360+0000","last_fullsized":"2023-09-20T21:48:08.591360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:31:35.472750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587265+0000","last_change":"2023-09-20T21:46:40.506411+0000","last_active":"2023-09-20T21:48:08.587265+0000","last_peered":"2023-09-20T21:48:08.587265+0000","last_clean":"2023-09-20T21:48:08.587265+0000","last_became_active":"2023-09-20T21:46:40.506116+0000","last_became_peered":"2023-09-20T21:46:40.506116+0000","last_unstale":"2023-09-20T21:48:08.587265+0000","last_undegraded":"2023-09-20T21:48:08.587265+0000","last_fullsized":"2023-09-20T21:48:08.587265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:23:19.175007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587561+0000","last_change":"2023-09-20T21:46:41.513793+0000","last_active":"2023-09-20T21:48:08.587561+0000","last_peered":"2023-09-20T21:48:08.587561+0000","last_clean":"2023-09-20T21:48:08.587561+0000","last_became_active":"2023-09-20T21:46:41.513434+0000","last_became_peered":"2023-09-20T21:46:41.513434+0000","last_unstale":"2023-09-20T21:48:08.587561+0000","last_undegraded":"2023-09-20T21:48:08.587561+0000","last_fullsized":"2023-09-20T21:48:08.587561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:00:57.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587768+0000","last_change":"2023-09-20T21:46:45.578188+0000","last_active":"2023-09-20T21:48:08.587768+0000","last_peered":"2023-09-20T21:48:08.587768+0000","last_clean":"2023-09-20T21:48:08.587768+0000","last_became_active":"2023-09-20T21:46:45.577548+0000","last_became_peered":"2023-09-20T21:46:45.577548+0000","last_unstale":"2023-09-20T21:48:08.587768+0000","last_undegraded":"2023-09-20T21:48:08.587768+0000","last_fullsized":"2023-09-20T21:48:08.587768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:54:58.894269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591467+0000","last_change":"2023-09-20T21:46:40.505465+0000","last_active":"2023-09-20T21:48:08.591467+0000","last_peered":"2023-09-20T21:48:08.591467+0000","last_clean":"2023-09-20T21:48:08.591467+0000","last_became_active":"2023-09-20T21:46:40.505174+0000","last_became_peered":"2023-09-20T21:46:40.505174+0000","last_unstale":"2023-09-20T21:48:08.591467+0000","last_undegraded":"2023-09-20T21:48:08.591467+0000","last_fullsized":"2023-09-20T21:48:08.591467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:50:53.191963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590466+0000","last_change":"2023-09-20T21:46:41.507370+0000","last_active":"2023-09-20T21:48:08.590466+0000","last_peered":"2023-09-20T21:48:08.590466+0000","last_clean":"2023-09-20T21:48:08.590466+0000","last_became_active":"2023-09-20T21:46:41.507035+0000","last_became_peered":"2023-09-20T21:46:41.507035+0000","last_unstale":"2023-09-20T21:48:08.590466+0000","last_undegraded":"2023-09-20T21:48:08.590466+0000","last_fullsized":"2023-09-20T21:48:08.590466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:50.298899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584972+0000","last_change":"2023-09-20T21:46:45.584458+0000","last_active":"2023-09-20T21:48:08.584972+0000","last_peered":"2023-09-20T21:48:08.584972+0000","last_clean":"2023-09-20T21:48:08.584972+0000","last_became_active":"2023-09-20T21:46:45.584033+0000","last_became_peered":"2023-09-20T21:46:45.584033+0000","last_unstale":"2023-09-20T21:48:08.584972+0000","last_undegraded":"2023-09-20T21:48:08.584972+0000","last_fullsized":"2023-09-20T21:48:08.584972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:48:44.653880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:48.727034+0000","last_change":"2023-09-20T21:46:40.501329+0000","last_active":"2023-09-20T21:48:48.727034+0000","last_peered":"2023-09-20T21:48:48.727034+0000","last_clean":"2023-09-20T21:48:48.727034+0000","last_became_active":"2023-09-20T21:46:40.500999+0000","last_became_peered":"2023-09-20T21:46:40.500999+0000","last_unstale":"2023-09-20T21:48:48.727034+0000","last_undegraded":"2023-09-20T21:48:48.727034+0000","last_fullsized":"2023-09-20T21:48:48.727034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:38.028398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590301+0000","last_change":"2023-09-20T21:47:08.602123+0000","last_active":"2023-09-20T21:48:08.590301+0000","last_peered":"2023-09-20T21:48:08.590301+0000","last_clean":"2023-09-20T21:48:08.590301+0000","last_became_active":"2023-09-20T21:47:08.601738+0000","last_became_peered":"2023-09-20T21:47:08.601738+0000","last_unstale":"2023-09-20T21:48:08.590301+0000","last_undegraded":"2023-09-20T21:48:08.590301+0000","last_fullsized":"2023-09-20T21:48:08.590301+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:26:36.710230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.599126+0000","last_change":"2023-09-20T21:48:09.599126+0000","last_active":"2023-09-20T21:48:09.599126+0000","last_peered":"2023-09-20T21:48:09.599126+0000","last_clean":"2023-09-20T21:48:09.599126+0000","last_became_active":"2023-09-20T21:48:09.598649+0000","last_became_peered":"2023-09-20T21:48:09.598649+0000","last_unstale":"2023-09-20T21:48:09.599126+0000","last_undegraded":"2023-09-20T21:48:09.599126+0000","last_fullsized":"2023-09-20T21:48:09.599126+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:24:27.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584737+0000","last_change":"2023-09-20T21:46:40.499985+0000","last_active":"2023-09-20T21:48:08.584737+0000","last_peered":"2023-09-20T21:48:08.584737+0000","last_clean":"2023-09-20T21:48:08.584737+0000","last_became_active":"2023-09-20T21:46:40.499646+0000","last_became_peered":"2023-09-20T21:46:40.499646+0000","last_unstale":"2023-09-20T21:48:08.584737+0000","last_undegraded":"2023-09-20T21:48:08.584737+0000","last_fullsized":"2023-09-20T21:48:08.584737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:52:02.758556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590785+0000","last_change":"2023-09-20T21:46:41.501241+0000","last_active":"2023-09-20T21:48:08.590785+0000","last_peered":"2023-09-20T21:48:08.590785+0000","last_clean":"2023-09-20T21:48:08.590785+0000","last_became_active":"2023-09-20T21:46:41.500902+0000","last_became_peered":"2023-09-20T21:46:41.500902+0000","last_unstale":"2023-09-20T21:48:08.590785+0000","last_undegraded":"2023-09-20T21:48:08.590785+0000","last_fullsized":"2023-09-20T21:48:08.590785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:21:55.210065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590708+0000","last_change":"2023-09-20T21:46:45.583405+0000","last_active":"2023-09-20T21:48:08.590708+0000","last_peered":"2023-09-20T21:48:08.590708+0000","last_clean":"2023-09-20T21:48:08.590708+0000","last_became_active":"2023-09-20T21:46:45.582945+0000","last_became_peered":"2023-09-20T21:46:45.582945+0000","last_unstale":"2023-09-20T21:48:08.590708+0000","last_undegraded":"2023-09-20T21:48:08.590708+0000","last_fullsized":"2023-09-20T21:48:08.590708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:27:14.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585473+0000","last_change":"2023-09-20T21:46:40.507744+0000","last_active":"2023-09-20T21:48:08.585473+0000","last_peered":"2023-09-20T21:48:08.585473+0000","last_clean":"2023-09-20T21:48:08.585473+0000","last_became_active":"2023-09-20T21:46:40.507454+0000","last_became_peered":"2023-09-20T21:46:40.507454+0000","last_unstale":"2023-09-20T21:48:08.585473+0000","last_undegraded":"2023-09-20T21:48:08.585473+0000","last_fullsized":"2023-09-20T21:48:08.585473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:00:24.492789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588068+0000","last_change":"2023-09-20T21:46:41.504263+0000","last_active":"2023-09-20T21:48:08.588068+0000","last_peered":"2023-09-20T21:48:08.588068+0000","last_clean":"2023-09-20T21:48:08.588068+0000","last_became_active":"2023-09-20T21:46:41.503961+0000","last_became_peered":"2023-09-20T21:46:41.503961+0000","last_unstale":"2023-09-20T21:48:08.588068+0000","last_undegraded":"2023-09-20T21:48:08.588068+0000","last_fullsized":"2023-09-20T21:48:08.588068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:31:58.035517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.668647+0000","last_change":"2023-09-20T21:48:09.668184+0000","last_active":"2023-09-20T21:48:09.668647+0000","last_peered":"2023-09-20T21:48:09.668647+0000","last_clean":"2023-09-20T21:48:09.668647+0000","last_became_active":"2023-09-20T21:48:09.667732+0000","last_became_peered":"2023-09-20T21:48:09.667732+0000","last_unstale":"2023-09-20T21:48:09.668647+0000","last_undegraded":"2023-09-20T21:48:09.668647+0000","last_fullsized":"2023-09-20T21:48:09.668647+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:06:25.108207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.592116+0000","last_change":"2023-09-20T21:46:40.501850+0000","last_active":"2023-09-20T21:48:09.592116+0000","last_peered":"2023-09-20T21:48:09.592116+0000","last_clean":"2023-09-20T21:48:09.592116+0000","last_became_active":"2023-09-20T21:46:40.501500+0000","last_became_peered":"2023-09-20T21:46:40.501500+0000","last_unstale":"2023-09-20T21:48:09.592116+0000","last_undegraded":"2023-09-20T21:48:09.592116+0000","last_fullsized":"2023-09-20T21:48:09.592116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:14:15.907335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.666100+0000","last_change":"2023-09-20T21:46:41.507432+0000","last_active":"2023-09-20T21:48:09.666100+0000","last_peered":"2023-09-20T21:48:09.666100+0000","last_clean":"2023-09-20T21:48:09.666100+0000","last_became_active":"2023-09-20T21:46:41.507119+0000","last_became_peered":"2023-09-20T21:46:41.507119+0000","last_unstale":"2023-09-20T21:48:09.666100+0000","last_undegraded":"2023-09-20T21:48:09.666100+0000","last_fullsized":"2023-09-20T21:48:09.666100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:14:10.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594895+0000","last_change":"2023-09-20T21:46:45.588455+0000","last_active":"2023-09-20T21:48:09.594895+0000","last_peered":"2023-09-20T21:48:09.594895+0000","last_clean":"2023-09-20T21:48:09.594895+0000","last_became_active":"2023-09-20T21:46:45.587980+0000","last_became_peered":"2023-09-20T21:46:45.587980+0000","last_unstale":"2023-09-20T21:48:09.594895+0000","last_undegraded":"2023-09-20T21:48:09.594895+0000","last_fullsized":"2023-09-20T21:48:09.594895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:07:35.697330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:50.774388+0000","last_change":"2023-09-20T21:46:40.506260+0000","last_active":"2023-09-20T21:48:50.774388+0000","last_peered":"2023-09-20T21:48:50.774388+0000","last_clean":"2023-09-20T21:48:50.774388+0000","last_became_active":"2023-09-20T21:46:40.505999+0000","last_became_peered":"2023-09-20T21:46:40.505999+0000","last_unstale":"2023-09-20T21:48:50.774388+0000","last_undegraded":"2023-09-20T21:48:50.774388+0000","last_fullsized":"2023-09-20T21:48:50.774388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:39:01.748038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.665667+0000","last_change":"2023-09-20T21:47:08.599481+0000","last_active":"2023-09-20T21:48:09.665667+0000","last_peered":"2023-09-20T21:48:09.665667+0000","last_clean":"2023-09-20T21:48:09.665667+0000","last_became_active":"2023-09-20T21:47:08.599104+0000","last_became_peered":"2023-09-20T21:47:08.599104+0000","last_unstale":"2023-09-20T21:48:09.665667+0000","last_undegraded":"2023-09-20T21:48:09.665667+0000","last_fullsized":"2023-09-20T21:48:09.665667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:33:56.845446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588094+0000","last_change":"2023-09-20T21:46:45.587410+0000","last_active":"2023-09-20T21:48:08.588094+0000","last_peered":"2023-09-20T21:48:08.588094+0000","last_clean":"2023-09-20T21:48:08.588094+0000","last_became_active":"2023-09-20T21:46:45.586944+0000","last_became_peered":"2023-09-20T21:46:45.586944+0000","last_unstale":"2023-09-20T21:48:08.588094+0000","last_undegraded":"2023-09-20T21:48:08.588094+0000","last_fullsized":"2023-09-20T21:48:08.588094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:57.053765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.591816+0000","last_change":"2023-09-20T21:46:41.511526+0000","last_active":"2023-09-20T21:48:09.591816+0000","last_peered":"2023-09-20T21:48:09.591816+0000","last_clean":"2023-09-20T21:48:09.591816+0000","last_became_active":"2023-09-20T21:46:41.511213+0000","last_became_peered":"2023-09-20T21:46:41.511213+0000","last_unstale":"2023-09-20T21:48:09.591816+0000","last_undegraded":"2023-09-20T21:48:09.591816+0000","last_fullsized":"2023-09-20T21:48:09.591816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:29:04.199190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:48.726313+0000","last_change":"2023-09-20T21:46:40.514916+0000","last_active":"2023-09-20T21:48:48.726313+0000","last_peered":"2023-09-20T21:48:48.726313+0000","last_clean":"2023-09-20T21:48:48.726313+0000","last_became_active":"2023-09-20T21:46:40.514553+0000","last_became_peered":"2023-09-20T21:46:40.514553+0000","last_unstale":"2023-09-20T21:48:48.726313+0000","last_undegraded":"2023-09-20T21:48:48.726313+0000","last_fullsized":"2023-09-20T21:48:48.726313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:12:32.481852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714807+0000","last_change":"2023-09-20T21:46:45.587489+0000","last_active":"2023-09-20T21:48:09.714807+0000","last_peered":"2023-09-20T21:48:09.714807+0000","last_clean":"2023-09-20T21:48:09.714807+0000","last_became_active":"2023-09-20T21:46:45.587022+0000","last_became_peered":"2023-09-20T21:46:45.587022+0000","last_unstale":"2023-09-20T21:48:09.714807+0000","last_undegraded":"2023-09-20T21:48:09.714807+0000","last_fullsized":"2023-09-20T21:48:09.714807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:09:05.870763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593480+0000","last_change":"2023-09-20T21:46:41.508886+0000","last_active":"2023-09-20T21:48:09.593480+0000","last_peered":"2023-09-20T21:48:09.593480+0000","last_clean":"2023-09-20T21:48:09.593480+0000","last_became_active":"2023-09-20T21:46:41.508470+0000","last_became_peered":"2023-09-20T21:46:41.508470+0000","last_unstale":"2023-09-20T21:48:09.593480+0000","last_undegraded":"2023-09-20T21:48:09.593480+0000","last_fullsized":"2023-09-20T21:48:09.593480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:45:04.888300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.667141+0000","last_change":"2023-09-20T21:46:40.503350+0000","last_active":"2023-09-20T21:48:09.667141+0000","last_peered":"2023-09-20T21:48:09.667141+0000","last_clean":"2023-09-20T21:48:09.667141+0000","last_became_active":"2023-09-20T21:46:40.503045+0000","last_became_peered":"2023-09-20T21:46:40.503045+0000","last_unstale":"2023-09-20T21:48:09.667141+0000","last_undegraded":"2023-09-20T21:48:09.667141+0000","last_fullsized":"2023-09-20T21:48:09.667141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:31:09.004342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":318,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714381+0000","last_change":"2023-09-20T21:45:35.180242+0000","last_active":"2023-09-20T21:48:09.714381+0000","last_peered":"2023-09-20T21:48:09.714381+0000","last_clean":"2023-09-20T21:48:09.714381+0000","last_became_active":"2023-09-20T21:45:35.179705+0000","last_became_peered":"2023-09-20T21:45:35.179705+0000","last_unstale":"2023-09-20T21:48:09.714381+0000","last_undegraded":"2023-09-20T21:48:09.714381+0000","last_fullsized":"2023-09-20T21:48:09.714381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_clean_scrub_stamp":"2023-09-20T21:42:25.826683+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:02:03.448188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591048+0000","last_change":"2023-09-20T21:46:45.581396+0000","last_active":"2023-09-20T21:48:08.591048+0000","last_peered":"2023-09-20T21:48:08.591048+0000","last_clean":"2023-09-20T21:48:08.591048+0000","last_became_active":"2023-09-20T21:46:45.581018+0000","last_became_peered":"2023-09-20T21:46:45.581018+0000","last_unstale":"2023-09-20T21:48:08.591048+0000","last_undegraded":"2023-09-20T21:48:08.591048+0000","last_fullsized":"2023-09-20T21:48:08.591048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:16:35.934460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594684+0000","last_change":"2023-09-20T21:46:41.514392+0000","last_active":"2023-09-20T21:48:09.594684+0000","last_peered":"2023-09-20T21:48:09.594684+0000","last_clean":"2023-09-20T21:48:09.594684+0000","last_became_active":"2023-09-20T21:46:41.514090+0000","last_became_peered":"2023-09-20T21:46:41.514090+0000","last_unstale":"2023-09-20T21:48:09.594684+0000","last_undegraded":"2023-09-20T21:48:09.594684+0000","last_fullsized":"2023-09-20T21:48:09.594684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:05:47.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714387+0000","last_change":"2023-09-20T21:46:40.507346+0000","last_active":"2023-09-20T21:48:09.714387+0000","last_peered":"2023-09-20T21:48:09.714387+0000","last_clean":"2023-09-20T21:48:09.714387+0000","last_became_active":"2023-09-20T21:46:40.506969+0000","last_became_peered":"2023-09-20T21:46:40.506969+0000","last_unstale":"2023-09-20T21:48:09.714387+0000","last_undegraded":"2023-09-20T21:48:09.714387+0000","last_fullsized":"2023-09-20T21:48:09.714387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:01:20.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588355+0000","last_change":"2023-09-20T21:46:45.578033+0000","last_active":"2023-09-20T21:48:08.588355+0000","last_peered":"2023-09-20T21:48:08.588355+0000","last_clean":"2023-09-20T21:48:08.588355+0000","last_became_active":"2023-09-20T21:46:45.577540+0000","last_became_peered":"2023-09-20T21:46:45.577540+0000","last_unstale":"2023-09-20T21:48:08.588355+0000","last_undegraded":"2023-09-20T21:48:08.588355+0000","last_fullsized":"2023-09-20T21:48:08.588355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:24:31.020660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594334+0000","last_change":"2023-09-20T21:46:40.510634+0000","last_active":"2023-09-20T21:48:09.594334+0000","last_peered":"2023-09-20T21:48:09.594334+0000","last_clean":"2023-09-20T21:48:09.594334+0000","last_became_active":"2023-09-20T21:46:40.510390+0000","last_became_peered":"2023-09-20T21:46:40.510390+0000","last_unstale":"2023-09-20T21:48:09.594334+0000","last_undegraded":"2023-09-20T21:48:09.594334+0000","last_fullsized":"2023-09-20T21:48:09.594334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:44:49.272987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595048+0000","last_change":"2023-09-20T21:46:41.502981+0000","last_active":"2023-09-20T21:48:09.595048+0000","last_peered":"2023-09-20T21:48:09.595048+0000","last_clean":"2023-09-20T21:48:09.595048+0000","last_became_active":"2023-09-20T21:46:41.502594+0000","last_became_peered":"2023-09-20T21:46:41.502594+0000","last_unstale":"2023-09-20T21:48:09.595048+0000","last_undegraded":"2023-09-20T21:48:09.595048+0000","last_fullsized":"2023-09-20T21:48:09.595048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:41:15.261478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593135+0000","last_change":"2023-09-20T21:46:45.582903+0000","last_active":"2023-09-20T21:48:09.593135+0000","last_peered":"2023-09-20T21:48:09.593135+0000","last_clean":"2023-09-20T21:48:09.593135+0000","last_became_active":"2023-09-20T21:46:45.582433+0000","last_became_peered":"2023-09-20T21:46:45.582433+0000","last_unstale":"2023-09-20T21:48:09.593135+0000","last_undegraded":"2023-09-20T21:48:09.593135+0000","last_fullsized":"2023-09-20T21:48:09.593135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:38:22.174903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.592545+0000","last_change":"2023-09-20T21:46:40.507671+0000","last_active":"2023-09-20T21:48:09.592545+0000","last_peered":"2023-09-20T21:48:09.592545+0000","last_clean":"2023-09-20T21:48:09.592545+0000","last_became_active":"2023-09-20T21:46:40.507288+0000","last_became_peered":"2023-09-20T21:46:40.507288+0000","last_unstale":"2023-09-20T21:48:09.592545+0000","last_undegraded":"2023-09-20T21:48:09.592545+0000","last_fullsized":"2023-09-20T21:48:09.592545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:27:02.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594926+0000","last_change":"2023-09-20T21:46:41.507654+0000","last_active":"2023-09-20T21:48:09.594926+0000","last_peered":"2023-09-20T21:48:09.594926+0000","last_clean":"2023-09-20T21:48:09.594926+0000","last_became_active":"2023-09-20T21:46:41.507362+0000","last_became_peered":"2023-09-20T21:46:41.507362+0000","last_unstale":"2023-09-20T21:48:09.594926+0000","last_undegraded":"2023-09-20T21:48:09.594926+0000","last_fullsized":"2023-09-20T21:48:09.594926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:32:08.973379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595697+0000","last_change":"2023-09-20T21:46:45.578437+0000","last_active":"2023-09-20T21:48:09.595697+0000","last_peered":"2023-09-20T21:48:09.595697+0000","last_clean":"2023-09-20T21:48:09.595697+0000","last_became_active":"2023-09-20T21:46:45.578010+0000","last_became_peered":"2023-09-20T21:46:45.578010+0000","last_unstale":"2023-09-20T21:48:09.595697+0000","last_undegraded":"2023-09-20T21:48:09.595697+0000","last_fullsized":"2023-09-20T21:48:09.595697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:59:05.494861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.664112+0000","last_change":"2023-09-20T21:46:40.508937+0000","last_active":"2023-09-20T21:48:08.664112+0000","last_peered":"2023-09-20T21:48:08.664112+0000","last_clean":"2023-09-20T21:48:08.664112+0000","last_became_active":"2023-09-20T21:46:40.508639+0000","last_became_peered":"2023-09-20T21:46:40.508639+0000","last_unstale":"2023-09-20T21:48:08.664112+0000","last_undegraded":"2023-09-20T21:48:08.664112+0000","last_fullsized":"2023-09-20T21:48:08.664112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:50:48.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.971221+0000","last_change":"2023-09-20T21:46:41.510702+0000","last_active":"2023-09-20T21:48:09.971221+0000","last_peered":"2023-09-20T21:48:09.971221+0000","last_clean":"2023-09-20T21:48:09.971221+0000","last_became_active":"2023-09-20T21:46:41.510357+0000","last_became_peered":"2023-09-20T21:46:41.510357+0000","last_unstale":"2023-09-20T21:48:09.971221+0000","last_undegraded":"2023-09-20T21:48:09.971221+0000","last_fullsized":"2023-09-20T21:48:09.971221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:35:44.455981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663990+0000","last_change":"2023-09-20T21:46:45.581110+0000","last_active":"2023-09-20T21:48:08.663990+0000","last_peered":"2023-09-20T21:48:08.663990+0000","last_clean":"2023-09-20T21:48:08.663990+0000","last_became_active":"2023-09-20T21:46:45.580498+0000","last_became_peered":"2023-09-20T21:46:45.580498+0000","last_unstale":"2023-09-20T21:48:08.663990+0000","last_undegraded":"2023-09-20T21:48:08.663990+0000","last_fullsized":"2023-09-20T21:48:08.663990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:19:22.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587563+0000","last_change":"2023-09-20T21:46:40.497017+0000","last_active":"2023-09-20T21:48:08.587563+0000","last_peered":"2023-09-20T21:48:08.587563+0000","last_clean":"2023-09-20T21:48:08.587563+0000","last_became_active":"2023-09-20T21:46:40.496722+0000","last_became_peered":"2023-09-20T21:46:40.496722+0000","last_unstale":"2023-09-20T21:48:08.587563+0000","last_undegraded":"2023-09-20T21:48:08.587563+0000","last_fullsized":"2023-09-20T21:48:08.587563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:12:07.017318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593744+0000","last_change":"2023-09-20T21:46:41.508793+0000","last_active":"2023-09-20T21:48:09.593744+0000","last_peered":"2023-09-20T21:48:09.593744+0000","last_clean":"2023-09-20T21:48:09.593744+0000","last_became_active":"2023-09-20T21:46:41.508453+0000","last_became_peered":"2023-09-20T21:46:41.508453+0000","last_unstale":"2023-09-20T21:48:09.593744+0000","last_undegraded":"2023-09-20T21:48:09.593744+0000","last_fullsized":"2023-09-20T21:48:09.593744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:35:12.711570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585166+0000","last_change":"2023-09-20T21:46:45.583519+0000","last_active":"2023-09-20T21:48:08.585166+0000","last_peered":"2023-09-20T21:48:08.585166+0000","last_clean":"2023-09-20T21:48:08.585166+0000","last_became_active":"2023-09-20T21:46:45.582933+0000","last_became_peered":"2023-09-20T21:46:45.582933+0000","last_unstale":"2023-09-20T21:48:08.585166+0000","last_undegraded":"2023-09-20T21:48:08.585166+0000","last_fullsized":"2023-09-20T21:48:08.585166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:13:20.237865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593456+0000","last_change":"2023-09-20T21:46:40.507384+0000","last_active":"2023-09-20T21:48:09.593456+0000","last_peered":"2023-09-20T21:48:09.593456+0000","last_clean":"2023-09-20T21:48:09.593456+0000","last_became_active":"2023-09-20T21:46:40.507021+0000","last_became_peered":"2023-09-20T21:46:40.507021+0000","last_unstale":"2023-09-20T21:48:09.593456+0000","last_undegraded":"2023-09-20T21:48:09.593456+0000","last_fullsized":"2023-09-20T21:48:09.593456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:48:04.360664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588413+0000","last_change":"2023-09-20T21:46:41.511537+0000","last_active":"2023-09-20T21:48:08.588413+0000","last_peered":"2023-09-20T21:48:08.588413+0000","last_clean":"2023-09-20T21:48:08.588413+0000","last_became_active":"2023-09-20T21:46:41.511238+0000","last_became_peered":"2023-09-20T21:46:41.511238+0000","last_unstale":"2023-09-20T21:48:08.588413+0000","last_undegraded":"2023-09-20T21:48:08.588413+0000","last_fullsized":"2023-09-20T21:48:08.588413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:01:41.492824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663746+0000","last_change":"2023-09-20T21:46:45.585750+0000","last_active":"2023-09-20T21:48:08.663746+0000","last_peered":"2023-09-20T21:48:08.663746+0000","last_clean":"2023-09-20T21:48:08.663746+0000","last_became_active":"2023-09-20T21:46:45.585213+0000","last_became_peered":"2023-09-20T21:46:45.585213+0000","last_unstale":"2023-09-20T21:48:08.663746+0000","last_undegraded":"2023-09-20T21:48:08.663746+0000","last_fullsized":"2023-09-20T21:48:08.663746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:31:15.650766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.602996+0000","last_change":"2023-09-20T21:48:09.602996+0000","last_active":"2023-09-20T21:48:09.602996+0000","last_peered":"2023-09-20T21:48:09.602996+0000","last_clean":"2023-09-20T21:48:09.602996+0000","last_became_active":"2023-09-20T21:48:09.598415+0000","last_became_peered":"2023-09-20T21:48:09.598415+0000","last_unstale":"2023-09-20T21:48:09.602996+0000","last_undegraded":"2023-09-20T21:48:09.602996+0000","last_fullsized":"2023-09-20T21:48:09.602996+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:39:45.484653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590150+0000","last_change":"2023-09-20T21:46:41.509491+0000","last_active":"2023-09-20T21:48:08.590150+0000","last_peered":"2023-09-20T21:48:08.590150+0000","last_clean":"2023-09-20T21:48:08.590150+0000","last_became_active":"2023-09-20T21:46:41.509039+0000","last_became_peered":"2023-09-20T21:46:41.509039+0000","last_unstale":"2023-09-20T21:48:08.590150+0000","last_undegraded":"2023-09-20T21:48:08.590150+0000","last_fullsized":"2023-09-20T21:48:08.590150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:57:39.003154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663975+0000","last_change":"2023-09-20T21:46:45.581651+0000","last_active":"2023-09-20T21:48:08.663975+0000","last_peered":"2023-09-20T21:48:08.663975+0000","last_clean":"2023-09-20T21:48:08.663975+0000","last_became_active":"2023-09-20T21:46:45.581224+0000","last_became_peered":"2023-09-20T21:46:45.581224+0000","last_unstale":"2023-09-20T21:48:08.663975+0000","last_undegraded":"2023-09-20T21:48:08.663975+0000","last_fullsized":"2023-09-20T21:48:08.663975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:32:17.778301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588268+0000","last_change":"2023-09-20T21:47:08.610413+0000","last_active":"2023-09-20T21:48:08.588268+0000","last_peered":"2023-09-20T21:48:08.588268+0000","last_clean":"2023-09-20T21:48:08.588268+0000","last_became_active":"2023-09-20T21:47:08.603734+0000","last_became_peered":"2023-09-20T21:47:08.603734+0000","last_unstale":"2023-09-20T21:48:08.588268+0000","last_undegraded":"2023-09-20T21:48:08.588268+0000","last_fullsized":"2023-09-20T21:48:08.588268+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:43:42.569371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714928+0000","last_change":"2023-09-20T21:46:41.507218+0000","last_active":"2023-09-20T21:48:09.714928+0000","last_peered":"2023-09-20T21:48:09.714928+0000","last_clean":"2023-09-20T21:48:09.714928+0000","last_became_active":"2023-09-20T21:46:41.506889+0000","last_became_peered":"2023-09-20T21:46:41.506889+0000","last_unstale":"2023-09-20T21:48:09.714928+0000","last_undegraded":"2023-09-20T21:48:09.714928+0000","last_fullsized":"2023-09-20T21:48:09.714928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:08:40.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.716076+0000","last_change":"2023-09-20T21:48:09.716076+0000","last_active":"2023-09-20T21:48:09.716076+0000","last_peered":"2023-09-20T21:48:09.716076+0000","last_clean":"2023-09-20T21:48:09.716076+0000","last_became_active":"2023-09-20T21:48:09.715521+0000","last_became_peered":"2023-09-20T21:48:09.715521+0000","last_unstale":"2023-09-20T21:48:09.716076+0000","last_undegraded":"2023-09-20T21:48:09.716076+0000","last_fullsized":"2023-09-20T21:48:09.716076+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:59:27.762388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":130,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:49.823237+0000","last_change":"2023-09-20T21:46:40.508774+0000","last_active":"2023-09-20T21:48:49.823237+0000","last_peered":"2023-09-20T21:48:49.823237+0000","last_clean":"2023-09-20T21:48:49.823237+0000","last_became_active":"2023-09-20T21:46:40.508480+0000","last_became_peered":"2023-09-20T21:46:40.508480+0000","last_unstale":"2023-09-20T21:48:49.823237+0000","last_undegraded":"2023-09-20T21:48:49.823237+0000","last_fullsized":"2023-09-20T21:48:49.823237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:22.643724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590718+0000","last_change":"2023-09-20T21:46:41.503763+0000","last_active":"2023-09-20T21:48:08.590718+0000","last_peered":"2023-09-20T21:48:08.590718+0000","last_clean":"2023-09-20T21:48:08.590718+0000","last_became_active":"2023-09-20T21:46:41.503360+0000","last_became_peered":"2023-09-20T21:46:41.503360+0000","last_unstale":"2023-09-20T21:48:08.590718+0000","last_undegraded":"2023-09-20T21:48:08.590718+0000","last_fullsized":"2023-09-20T21:48:08.590718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:03:00.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591725+0000","last_change":"2023-09-20T21:46:45.586702+0000","last_active":"2023-09-20T21:48:08.591725+0000","last_peered":"2023-09-20T21:48:08.591725+0000","last_clean":"2023-09-20T21:48:08.591725+0000","last_became_active":"2023-09-20T21:46:45.586177+0000","last_became_peered":"2023-09-20T21:46:45.586177+0000","last_unstale":"2023-09-20T21:48:08.591725+0000","last_undegraded":"2023-09-20T21:48:08.591725+0000","last_fullsized":"2023-09-20T21:48:08.591725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:19:03.883440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591133+0000","last_change":"2023-09-20T21:46:40.507270+0000","last_active":"2023-09-20T21:48:08.591133+0000","last_peered":"2023-09-20T21:48:08.591133+0000","last_clean":"2023-09-20T21:48:08.591133+0000","last_became_active":"2023-09-20T21:46:40.506933+0000","last_became_peered":"2023-09-20T21:46:40.506933+0000","last_unstale":"2023-09-20T21:48:08.591133+0000","last_undegraded":"2023-09-20T21:48:08.591133+0000","last_fullsized":"2023-09-20T21:48:08.591133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:59.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590595+0000","last_change":"2023-09-20T21:46:41.507868+0000","last_active":"2023-09-20T21:48:08.590595+0000","last_peered":"2023-09-20T21:48:08.590595+0000","last_clean":"2023-09-20T21:48:08.590595+0000","last_became_active":"2023-09-20T21:46:41.507568+0000","last_became_peered":"2023-09-20T21:46:41.507568+0000","last_unstale":"2023-09-20T21:48:08.590595+0000","last_undegraded":"2023-09-20T21:48:08.590595+0000","last_fullsized":"2023-09-20T21:48:08.590595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:44.615620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590490+0000","last_change":"2023-09-20T21:46:45.578726+0000","last_active":"2023-09-20T21:48:08.590490+0000","last_peered":"2023-09-20T21:48:08.590490+0000","last_clean":"2023-09-20T21:48:08.590490+0000","last_became_active":"2023-09-20T21:46:45.578326+0000","last_became_peered":"2023-09-20T21:46:45.578326+0000","last_unstale":"2023-09-20T21:48:08.590490+0000","last_undegraded":"2023-09-20T21:48:08.590490+0000","last_fullsized":"2023-09-20T21:48:08.590490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:21:34.564840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587838+0000","last_change":"2023-09-20T21:46:40.509024+0000","last_active":"2023-09-20T21:48:08.587838+0000","last_peered":"2023-09-20T21:48:08.587838+0000","last_clean":"2023-09-20T21:48:08.587838+0000","last_became_active":"2023-09-20T21:46:40.508659+0000","last_became_peered":"2023-09-20T21:46:40.508659+0000","last_unstale":"2023-09-20T21:48:08.587838+0000","last_undegraded":"2023-09-20T21:48:08.587838+0000","last_fullsized":"2023-09-20T21:48:08.587838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:12:08.302068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590780+0000","last_change":"2023-09-20T21:46:41.510307+0000","last_active":"2023-09-20T21:48:08.590780+0000","last_peered":"2023-09-20T21:48:08.590780+0000","last_clean":"2023-09-20T21:48:08.590780+0000","last_became_active":"2023-09-20T21:46:41.510014+0000","last_became_peered":"2023-09-20T21:46:41.510014+0000","last_unstale":"2023-09-20T21:48:08.590780+0000","last_undegraded":"2023-09-20T21:48:08.590780+0000","last_fullsized":"2023-09-20T21:48:08.590780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:05:55.062515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.596849+0000","last_change":"2023-09-20T21:46:41.511525+0000","last_active":"2023-09-20T21:48:09.596849+0000","last_peered":"2023-09-20T21:48:09.596849+0000","last_clean":"2023-09-20T21:48:09.596849+0000","last_became_active":"2023-09-20T21:46:41.511203+0000","last_became_peered":"2023-09-20T21:46:41.511203+0000","last_unstale":"2023-09-20T21:48:09.596849+0000","last_undegraded":"2023-09-20T21:48:09.596849+0000","last_fullsized":"2023-09-20T21:48:09.596849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:42:15.757422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.596012+0000","last_change":"2023-09-20T21:46:41.502936+0000","last_active":"2023-09-20T21:48:09.596012+0000","last_peered":"2023-09-20T21:48:09.596012+0000","last_clean":"2023-09-20T21:48:09.596012+0000","last_became_active":"2023-09-20T21:46:41.502617+0000","last_became_peered":"2023-09-20T21:46:41.502617+0000","last_unstale":"2023-09-20T21:48:09.596012+0000","last_undegraded":"2023-09-20T21:48:09.596012+0000","last_fullsized":"2023-09-20T21:48:09.596012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:45:21.197652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593522+0000","last_change":"2023-09-20T21:46:41.510766+0000","last_active":"2023-09-20T21:48:09.593522+0000","last_peered":"2023-09-20T21:48:09.593522+0000","last_clean":"2023-09-20T21:48:09.593522+0000","last_became_active":"2023-09-20T21:46:41.510433+0000","last_became_peered":"2023-09-20T21:46:41.510433+0000","last_unstale":"2023-09-20T21:48:09.593522+0000","last_undegraded":"2023-09-20T21:48:09.593522+0000","last_fullsized":"2023-09-20T21:48:09.593522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:56:12.085926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.970963+0000","last_change":"2023-09-20T21:46:41.508846+0000","last_active":"2023-09-20T21:48:09.970963+0000","last_peered":"2023-09-20T21:48:09.970963+0000","last_clean":"2023-09-20T21:48:09.970963+0000","last_became_active":"2023-09-20T21:46:41.508518+0000","last_became_peered":"2023-09-20T21:46:41.508518+0000","last_unstale":"2023-09-20T21:48:09.970963+0000","last_undegraded":"2023-09-20T21:48:09.970963+0000","last_fullsized":"2023-09-20T21:48:09.970963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:01:58.195327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588468+0000","last_change":"2023-09-20T21:46:41.505929+0000","last_active":"2023-09-20T21:48:08.588468+0000","last_peered":"2023-09-20T21:48:08.588468+0000","last_clean":"2023-09-20T21:48:08.588468+0000","last_became_active":"2023-09-20T21:46:41.505542+0000","last_became_peered":"2023-09-20T21:46:41.505542+0000","last_unstale":"2023-09-20T21:48:08.588468+0000","last_undegraded":"2023-09-20T21:48:08.588468+0000","last_fullsized":"2023-09-20T21:48:08.588468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:24:47.520344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587952+0000","last_change":"2023-09-20T21:46:41.507603+0000","last_active":"2023-09-20T21:48:08.587952+0000","last_peered":"2023-09-20T21:48:08.587952+0000","last_clean":"2023-09-20T21:48:08.587952+0000","last_became_active":"2023-09-20T21:46:41.507340+0000","last_became_peered":"2023-09-20T21:46:41.507340+0000","last_unstale":"2023-09-20T21:48:08.587952+0000","last_undegraded":"2023-09-20T21:48:08.587952+0000","last_fullsized":"2023-09-20T21:48:08.587952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:50:19.756491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595430+0000","last_change":"2023-09-20T21:46:41.509625+0000","last_active":"2023-09-20T21:48:09.595430+0000","last_peered":"2023-09-20T21:48:09.595430+0000","last_clean":"2023-09-20T21:48:09.595430+0000","last_became_active":"2023-09-20T21:46:41.509301+0000","last_became_peered":"2023-09-20T21:46:41.509301+0000","last_unstale":"2023-09-20T21:48:09.595430+0000","last_undegraded":"2023-09-20T21:48:09.595430+0000","last_fullsized":"2023-09-20T21:48:09.595430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:15:03.108060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595097+0000","last_change":"2023-09-20T21:46:41.516781+0000","last_active":"2023-09-20T21:48:09.595097+0000","last_peered":"2023-09-20T21:48:09.595097+0000","last_clean":"2023-09-20T21:48:09.595097+0000","last_became_active":"2023-09-20T21:46:41.516496+0000","last_became_peered":"2023-09-20T21:46:41.516496+0000","last_unstale":"2023-09-20T21:48:09.595097+0000","last_undegraded":"2023-09-20T21:48:09.595097+0000","last_fullsized":"2023-09-20T21:48:09.595097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:17:26.492851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590654+0000","last_change":"2023-09-20T21:46:41.499927+0000","last_active":"2023-09-20T21:48:08.590654+0000","last_peered":"2023-09-20T21:48:08.590654+0000","last_clean":"2023-09-20T21:48:08.590654+0000","last_became_active":"2023-09-20T21:46:41.499605+0000","last_became_peered":"2023-09-20T21:46:41.499605+0000","last_unstale":"2023-09-20T21:48:08.590654+0000","last_undegraded":"2023-09-20T21:48:08.590654+0000","last_fullsized":"2023-09-20T21:48:08.590654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:18:16.306349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590204+0000","last_change":"2023-09-20T21:46:41.515842+0000","last_active":"2023-09-20T21:48:08.590204+0000","last_peered":"2023-09-20T21:48:08.590204+0000","last_clean":"2023-09-20T21:48:08.590204+0000","last_became_active":"2023-09-20T21:46:41.515600+0000","last_became_peered":"2023-09-20T21:46:41.515600+0000","last_unstale":"2023-09-20T21:48:08.590204+0000","last_undegraded":"2023-09-20T21:48:08.590204+0000","last_fullsized":"2023-09-20T21:48:08.590204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:50:58.088103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588120+0000","last_change":"2023-09-20T21:46:41.510695+0000","last_active":"2023-09-20T21:48:08.588120+0000","last_peered":"2023-09-20T21:48:08.588120+0000","last_clean":"2023-09-20T21:48:08.588120+0000","last_became_active":"2023-09-20T21:46:41.510345+0000","last_became_peered":"2023-09-20T21:46:41.510345+0000","last_unstale":"2023-09-20T21:48:08.588120+0000","last_undegraded":"2023-09-20T21:48:08.588120+0000","last_fullsized":"2023-09-20T21:48:08.588120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:38:24.201382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587592+0000","last_change":"2023-09-20T21:46:41.507809+0000","last_active":"2023-09-20T21:48:08.587592+0000","last_peered":"2023-09-20T21:48:08.587592+0000","last_clean":"2023-09-20T21:48:08.587592+0000","last_became_active":"2023-09-20T21:46:41.507513+0000","last_became_peered":"2023-09-20T21:46:41.507513+0000","last_unstale":"2023-09-20T21:48:08.587592+0000","last_undegraded":"2023-09-20T21:48:08.587592+0000","last_fullsized":"2023-09-20T21:48:08.587592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:30:43.132728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587980+0000","last_change":"2023-09-20T21:46:41.503590+0000","last_active":"2023-09-20T21:48:08.587980+0000","last_peered":"2023-09-20T21:48:08.587980+0000","last_clean":"2023-09-20T21:48:08.587980+0000","last_became_active":"2023-09-20T21:46:41.503295+0000","last_became_peered":"2023-09-20T21:46:41.503295+0000","last_unstale":"2023-09-20T21:48:08.587980+0000","last_undegraded":"2023-09-20T21:48:08.587980+0000","last_fullsized":"2023-09-20T21:48:08.587980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:28:33.088873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10522,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1058,"num_read_kb":547,"num_write":211,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":31566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383719,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":516303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.38500000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83099999999999996,"5min":2.923,"15min":2.923},"last":0.53000000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.43099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80400000000000005,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.628},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73399999999999999,"5min":0.872,"15min":0.872},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.49299999999999999,"15min":0.498},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.68700000000000006,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.29299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79800000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.55800000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83099999999999996,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42799999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.81799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72399999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.60399999999999998,"15min":0.61799999999999999},"min":{"1min":0.32700000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.88500000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.64200000000000002,"15min":0.65500000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77700000000000002,"5min":1.351,"15min":1.351},"last":0.73599999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64900000000000002,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.40799999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.76400000000000001}]},{"osd":7,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.53800000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88100000000000001,"5min":1.246,"15min":1.246},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.54800000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89800000000000002,"5min":1.117,"15min":1.117},"last":0.80000000000000004}]},{"osd":8,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.52400000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.69799999999999995}]},{"osd":9,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.82999999999999996,"15min":0.84699999999999998},"min":{"1min":0.52800000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95099999999999996,"5min":1.992,"15min":1.992},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82499999999999996,"15min":0.82999999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.99299999999999999,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83399999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.61699999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92300000000000004,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.751},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.60499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.91600000000000004,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.71199999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1111315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":1.0169999999999999,"15min":1.093},"min":{"1min":0.433,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76100000000000001,"5min":7.9690000000000003,"15min":7.9690000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.85499999999999998,"15min":0.90900000000000003},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81399999999999995,"5min":5.6280000000000001,"15min":5.6280000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46899999999999997,"15min":0.47899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.60299999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80000000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79900000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67000000000000004,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.748,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.82399999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60799999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78100000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.72799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.68000000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.79000000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68200000000000005,"15min":0.69699999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.62}]},{"osd":6,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73499999999999999,"15min":0.76100000000000001},"min":{"1min":0.47099999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.873,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.89700000000000002,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":8.0440000000000005,"15min":8.0440000000000005},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.75800000000000001,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89700000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72099999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.89800000000000002,"15min":0.85999999999999999},"min":{"1min":0.67300000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.93200000000000005,"5min":5.6449999999999996,"15min":5.6449999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.89500000000000002,"15min":0.86799999999999999},"min":{"1min":0.61199999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.88400000000000001,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":0.74199999999999999}]},{"osd":11,"last update":"Wed Sep 20 21:47:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76600000000000001,"15min":0.76500000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77100000000000002}]}]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":516233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.622,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":2.7269999999999999,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.46400000000000002}]},{"osd":1,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5289999999999999,"5min":4.6799999999999997,"15min":4.6799999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78700000000000003,"15min":0.81599999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.996,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.56200000000000006,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.417,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.70699999999999996}]},{"osd":3,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71899999999999997,"15min":0.69499999999999995},"min":{"1min":0.41499999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.8170000000000002,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.82499999999999996,"15min":0.84499999999999997},"min":{"1min":0.47699999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.7690000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.627}]},{"osd":4,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.94899999999999995,"15min":0.93500000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.798,"5min":6.3449999999999998,"15min":6.3449999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.86099999999999999,"15min":0.86599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5920000000000001,"5min":4.7279999999999998,"15min":4.7279999999999998},"last":0.58399999999999996}]},{"osd":5,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.98899999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4970000000000001,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.92100000000000004,"15min":0.93000000000000005},"min":{"1min":0.54800000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.671,"5min":4.7679999999999998,"15min":4.7679999999999998},"last":0.76600000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.95799999999999996,"15min":0.96599999999999997},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6279999999999999,"5min":4.806,"15min":4.806},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.96599999999999997,"15min":1.0189999999999999},"min":{"1min":0.67400000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1000000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0629999999999999,"15min":1.119},"min":{"1min":0.67300000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.744,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0389999999999999,"15min":1.0600000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.702,"5min":4.843,"15min":4.843},"last":0.85299999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":1.0069999999999999,"15min":1.075},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.2150000000000001,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.749},{"interface":"front","average":{"1min":0.86299999999999999,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.2470000000000001,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.64100000000000001}]},{"osd":11,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.79400000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":516611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":0.52500000000000002}]},{"osd":1,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.497,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.75800000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.625},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.48099999999999998,"15min":0.46300000000000002},"min":{"1min":0.38300000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.53900000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.67500000000000004,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.93400000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.76600000000000001,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.82499999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.57699999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0109999999999999,"5min":1.548,"15min":1.548},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.57499999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.90400000000000003}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.79400000000000004,"15min":0.76800000000000002},"min":{"1min":0.69099999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.81000000000000005,"15min":0.78300000000000003},"min":{"1min":0.73599999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89300000000000002}]},{"osd":10,"last update":"Wed Sep 20 21:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.78000000000000003,"15min":0.745},"min":{"1min":0.69299999999999995,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.80500000000000005,"15min":0.77100000000000002},"min":{"1min":0.72399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.077,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.91700000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81899999999999995,"15min":0.81399999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84499999999999997,"15min":0.84199999999999997},"min":{"1min":0.70699999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":0.97999999999999998,"5min":1.038,"15min":1.038},"last":0.93700000000000006}]}]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1108879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.43099999999999999,"15min":0.42199999999999999},"min":{"1min":0.36899999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45300000000000001,"15min":0.438},"min":{"1min":0.34100000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.442}]},{"osd":2,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76900000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.51700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81799999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.89400000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63100000000000001,"15min":0.60099999999999998},"min":{"1min":0.496,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51900000000000002,"15min":0.48899999999999999},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71399999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.35599999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.498}]},{"osd":5,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63,"15min":0.62},"min":{"1min":0.495,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81000000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.63700000000000001},"min":{"1min":0.54100000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96099999999999997,"5min":1.006,"15min":1.006},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0069999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.66000000000000003},"min":{"1min":0.53700000000000003,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63600000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:48:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85699999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.63500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90000000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78300000000000003}]},{"osd":9,"last update":"Wed Sep 20 21:48:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.629,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77700000000000002,"15min":0.77600000000000002},"min":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.90500000000000003,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.76200000000000001}]},{"osd":10,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.52800000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91500000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.48599999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83299999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69499999999999995}]},{"osd":11,"last update":"Wed Sep 20 21:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83999999999999997,"15min":0.84499999999999997},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.93000000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":0.97099999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.84299999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":515833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81999999999999995,"5min":1.018,"15min":1.018},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66500000000000004,"15min":0.629},"min":{"1min":0.438,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.04,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79600000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.41699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":1.2,"15min":1.2},"last":0.92000000000000004}]},{"osd":2,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.873,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72399999999999998,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.99399999999999999,"5min":1.466,"15min":1.466},"last":0.873}]},{"osd":3,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69899999999999995,"15min":0.72299999999999998},"min":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.048,"5min":1.093,"15min":1.093},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0720000000000001,"5min":1.407,"15min":1.407},"last":0.98899999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.82799999999999996,"15min":0.80800000000000005},"min":{"1min":0.59999999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.749,"15min":0.73899999999999999},"min":{"1min":0.51800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Sep 20 21:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84999999999999998,"15min":0.83799999999999997},"min":{"1min":0.5,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2989999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.85099999999999998},"min":{"1min":0.64800000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.214,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0109999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.82599999999999996,"15min":0.85299999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.55900000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.206,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0329999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.1899999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1619999999999999,"5min":1.333,"15min":1.333},"last":1.0489999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.93899999999999995,"15min":0.94699999999999995},"min":{"1min":0.63600000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2370000000000001,"5min":1.302,"15min":1.302},"last":1.077},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.96799999999999997,"15min":0.97899999999999998},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2589999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.071}]},{"osd":11,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.0920000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":520705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.59399999999999997,"15min":0.54300000000000004},"min":{"1min":0.316,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.9539999999999997,"5min":4.9539999999999997,"15min":4.9539999999999997},"last":0.747},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.53700000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69599999999999995,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56000000000000005,"15min":0.51400000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.82299999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54400000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58699999999999997,"15min":0.53700000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.626}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.41899999999999998,"15min":0.39300000000000002},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.58599999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.84299999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.499},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64700000000000002,"15min":0.60499999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.878,"5min":1.01,"15min":1.01},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.70099999999999996,"15min":0.68200000000000005},"min":{"1min":0.44600000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":1.27,"15min":1.27},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:48:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77300000000000002,"5min":1.218,"15min":1.218},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.41299999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.321,"15min":1.321},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67500000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73799999999999999,"15min":0.751},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85299999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96999999999999997,"5min":1.054,"15min":1.054},"last":0.73399999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.72799999999999998,"15min":0.74199999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91200000000000003,"5min":1,"15min":1},"last":0.443},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.435,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93999999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.63800000000000001}]},{"osd":11,"last update":"Wed Sep 20 21:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78700000000000003,"15min":0.79100000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95199999999999996,"5min":1.018,"15min":1.018},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92700000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.70299999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":517289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 20 21:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":0.755,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":7.0289999999999999,"5min":7.0289999999999999,"15min":7.0289999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.76700000000000002,"15min":0.67400000000000004},"min":{"1min":0.53800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.4390000000000001,"5min":6.4390000000000001,"15min":6.4390000000000001},"last":0.76000000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72999999999999998,"15min":0.65400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.7589999999999999,"5min":8.1639999999999997,"15min":8.1639999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.67500000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97199999999999998,"5min":6.5270000000000001,"15min":6.5270000000000001},"last":0.71899999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.97399999999999998,"15min":1.038},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88500000000000001,"5min":8.9499999999999993,"15min":8.9499999999999993},"last":0.623},{"interface":"front","average":{"1min":0.63,"5min":0.88600000000000001,"15min":0.92800000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2110000000000001,"5min":8.6620000000000008,"15min":8.6620000000000008},"last":0.83699999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":0.80100000000000005,"15min":0.68700000000000006},"min":{"1min":0.45900000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":8.2560000000000002,"5min":8.2560000000000002,"15min":8.2560000000000002},"last":0.441},{"interface":"front","average":{"1min":1.123,"5min":0.80800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47999999999999998,"5min":0.443,"15min":0.443},"max":{"1min":6.5650000000000004,"5min":6.5650000000000004,"15min":6.5650000000000004},"last":0.81299999999999994}]},{"osd":5,"last update":"Wed Sep 20 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84199999999999997,"15min":0.73299999999999998},"min":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.792,"5min":8.2919999999999998,"15min":8.2919999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80400000000000005,"15min":0.70799999999999996},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.095,"5min":6.601,"15min":6.601},"last":0.78700000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":0.85099999999999998,"15min":0.72199999999999998},"min":{"1min":0.505,"5min":0.375,"15min":0.375},"max":{"1min":8.9049999999999994,"5min":8.9049999999999994,"15min":8.9049999999999994},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.79700000000000004,"15min":0.74299999999999999},"min":{"1min":0.49199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":8.3659999999999997,"5min":8.3659999999999997,"15min":8.3659999999999997},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":6.6440000000000001,"5min":6.6440000000000001,"15min":6.6440000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":8.2159999999999993,"5min":8.2159999999999993,"15min":8.2159999999999993},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":0.93999999999999995,"15min":0.86799999999999999},"min":{"1min":0.628,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":8.3290000000000006,"5min":8.3290000000000006,"15min":8.3290000000000006},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.1990000000000001,"5min":0.96099999999999997,"15min":0.93500000000000005},"min":{"1min":0.79600000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":6.6799999999999997,"5min":6.6799999999999997,"15min":6.6799999999999997},"last":0.89700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.0129999999999999,"15min":0.96199999999999997},"min":{"1min":0.66400000000000003,"5min":0.502,"15min":0.502},"max":{"1min":8.9879999999999995,"5min":8.9879999999999995,"15min":8.9879999999999995},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.2869999999999999,"5min":0.88300000000000001,"15min":0.83199999999999996},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":8.702,"5min":8.702,"15min":8.702},"last":0.879}]},{"osd":10,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0249999999999999,"15min":0.96899999999999997},"min":{"1min":0.73099999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.5,"5min":6.7149999999999999,"15min":6.7149999999999999},"last":1.04},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.90100000000000002,"15min":0.83699999999999997},"min":{"1min":0.61099999999999999,"5min":0.498,"15min":0.498},"max":{"1min":1.931,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":0.92500000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":0.877,"15min":0.82699999999999996},"min":{"1min":0.58999999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.0669999999999999,"15min":0.98599999999999999},"min":{"1min":0.85399999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":1.0629999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":520925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.55600000000000005},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.872,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64700000000000002,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94499999999999995,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64800000000000002,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83899999999999997,"5min":7.0330000000000004,"15min":7.0330000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.57699999999999996,"15min":0.52600000000000002},"min":{"1min":0.41799999999999998,"5min":0.33000000000000002,"15min":0.307},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.66800000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:47:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84599999999999997,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.505},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66800000000000004,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90900000000000003,"5min":3.645,"15min":3.645},"last":0.40100000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.82099999999999995,"15min":0.84299999999999997},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88800000000000001,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.749,"5min":1.181,"15min":1.2529999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0469999999999999,"5min":9.8659999999999997,"15min":9.8659999999999997},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.42099999999999999},"min":{"1min":0.34499999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73799999999999999,"5min":3.798,"15min":3.798},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.51000000000000001},"min":{"1min":0.40300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80000000000000004,"5min":7.1500000000000004,"15min":7.1500000000000004},"last":0.44700000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71199999999999997,"15min":0.64200000000000002},"min":{"1min":0.55700000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86299999999999999,"5min":7.367,"15min":7.367},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.754,"15min":0.67400000000000004},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.018,"5min":6.4649999999999999,"15min":6.4649999999999999},"last":0.69899999999999995}]},{"osd":7,"last update":"Wed Sep 20 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.58499999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.054,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.873},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.54900000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60299999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.63300000000000001},"min":{"1min":0.44900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95699999999999996,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.65200000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:48:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78800000000000003,"15min":0.72999999999999998},"min":{"1min":0.59599999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.99399999999999999,"5min":7.1200000000000001,"15min":7.1200000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.82699999999999996,"15min":0.78500000000000003},"min":{"1min":0.59299999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.137,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.81699999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80500000000000005,"15min":0.72299999999999998},"min":{"1min":0.626,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0600000000000001,"5min":7.1799999999999997,"15min":7.1799999999999997},"last":0.747},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.745,"15min":0.71299999999999997},"min":{"1min":0.50600000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0309999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.54300000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.143,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.0940000000000001,"15min":1.139},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.099,"5min":7.2110000000000003,"15min":7.2110000000000003},"last":0.78100000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":515541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.47999999999999998,"15min":0.40600000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.50800000000000001,"15min":0.42999999999999999},"min":{"1min":0.34300000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.504}]},{"osd":1,"last update":"Wed Sep 20 21:48:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63800000000000001,"15min":0.57799999999999996},"min":{"1min":0.432,"5min":0.376,"15min":0.376},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.73699999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:48:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65800000000000003,"15min":0.60999999999999999},"min":{"1min":0.42899999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0329999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed Sep 20 21:48:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67800000000000005,"15min":0.63400000000000001},"min":{"1min":0.504,"5min":0.436,"15min":0.436},"max":{"1min":0.877,"5min":1.601,"15min":1.601},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.754,"15min":0.66700000000000004},"min":{"1min":0.5,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93100000000000005,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.93999999999999995}]},{"osd":5,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.502,"5min":0.434,"15min":0.434},"max":{"1min":0.83199999999999996,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77400000000000002,"15min":0.752},"min":{"1min":0.52700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92900000000000005,"5min":1.167,"15min":1.167},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.69599999999999995,"15min":0.65000000000000002},"min":{"1min":0.50900000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.73499999999999999,"15min":0.71199999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Wed Sep 20 21:48:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.77600000000000002},"min":{"1min":0.56699999999999995,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96699999999999997,"5min":1.248,"15min":1.248},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82799999999999996,"15min":0.81899999999999995},"min":{"1min":0.64700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98499999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.97699999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91300000000000003,"15min":0.94299999999999995},"min":{"1min":0.65000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.1819999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.92200000000000004,"15min":0.93899999999999995},"min":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.74299999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90000000000000002,"15min":0.91300000000000003},"min":{"1min":0.73399999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0389999999999999,"5min":1.228,"15min":1.228},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.071,"5min":0.96099999999999997,"15min":0.93700000000000006},"min":{"1min":0.64000000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"last":0.90400000000000003}]},{"osd":11,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.109,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.879},{"interface":"front","average":{"1min":0.874,"5min":0.99199999999999999,"15min":1.012},"min":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.0109999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.81599999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":516069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.41399999999999998,"15min":0.37},"min":{"1min":0.314,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59999999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.38200000000000001},"min":{"1min":0.308,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.57599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47899999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.59999999999999998,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.53800000000000003},"min":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69099999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56000000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.57899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed Sep 20 21:48:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.621,"15min":0.57999999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.626},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.59799999999999998},"min":{"1min":0.503,"5min":0.379,"15min":0.379},"max":{"1min":0.81000000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:48:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":0.71899999999999997,"15min":0.61299999999999999},"min":{"1min":0.47399999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.5129999999999999,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.69299999999999995,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:48:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84299999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79500000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53700000000000003}]},{"osd":8,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.54200000000000004,"5min":0.371,"15min":0.371},"max":{"1min":0.89500000000000002,"5min":1,"15min":1},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84999999999999998}]},{"osd":9,"last update":"Wed Sep 20 21:48:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.83099999999999996,"15min":0.80400000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.109,"5min":4.109,"15min":4.109},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.78200000000000003,"15min":0.76300000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":0.81899999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:48:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.749},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.78600000000000003,"15min":0.79800000000000004},"min":{"1min":0.48899999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97399999999999998,"5min":1.048,"15min":1.048},"last":0.67000000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:48:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84599999999999997,"15min":0.85299999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.006,"5min":1.115,"15min":1.115},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.94599999999999995,"5min":1.008,"15min":1.008},"last":0.92400000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1108018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.087,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.1240000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.56299999999999994}]},{"osd":1,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56799999999999995,"15min":0.54100000000000004},"min":{"1min":0.41899999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.70999999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.53400000000000003},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55200000000000005}]},{"osd":2,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.378},"min":{"1min":0.30599999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58099999999999996,"5min":0.755,"15min":0.755},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.433,"15min":0.39900000000000002},"min":{"1min":0.311,"5min":0.255,"15min":0.255},"max":{"1min":0.66600000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.40699999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.436},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73399999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.57299999999999995},"min":{"1min":0.44700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80000000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.59699999999999998},"min":{"1min":0.45900000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77600000000000002,"5min":1.008,"15min":1.008},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed Sep 20 21:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.65500000000000003},"min":{"1min":0.54500000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84799999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.64900000000000002,"15min":0.626},"min":{"1min":0.48799999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.627}]},{"osd":7,"last update":"Wed Sep 20 21:48:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.87,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.66500000000000004},"min":{"1min":0.47499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81599999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.57999999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":0.879,"15min":0.879},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.51800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53800000000000003}]},{"osd":9,"last update":"Wed Sep 20 21:48:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.79800000000000004},"min":{"1min":0.626,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.68400000000000005}]},{"osd":10,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.751,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93400000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.89200000000000002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-20T21:48:54.416 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-09-20T21:48:54.417 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-20T21:48:54.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:54 smithi022 ceph-mon[97651]: pgmap v346: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-20T21:48:54.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:54 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/672754346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:54.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:54 smithi123 ceph-mon[134433]: pgmap v346: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-20T21:48:54.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:54 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/672754346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:54.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:54 smithi089 ceph-mon[134090]: pgmap v346: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2023-09-20T21:48:54.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:54 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/672754346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.652+0000 7fbb267ff700 1 -- 172.21.15.22:0/3970711925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb20102970 msgr2=0x7fbb20109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.652+0000 7fbb267ff700 1 --2- 172.21.15.22:0/3970711925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb20102970 0x7fbb20109220 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800b0d0 tx=0x7fbb1800b3e0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.652+0000 7fbb267ff700 1 -- 172.21.15.22:0/3970711925 shutdown_connections 2023-09-20T21:48:54.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.652+0000 7fbb267ff700 1 --2- 172.21.15.22:0/3970711925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb20102970 0x7fbb20109220 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.652+0000 7fbb267ff700 1 --2- 172.21.15.22:0/3970711925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb20100de0 0x7fbb20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.652+0000 7fbb267ff700 1 --2- 172.21.15.22:0/3970711925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20101fd0 0x7fbb20102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.652+0000 7fbb267ff700 1 -- 172.21.15.22:0/3970711925 >> 172.21.15.22:0/3970711925 conn(0x7fbb200fc310 msgr2=0x7fbb200fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.652+0000 7fbb267ff700 1 -- 172.21.15.22:0/3970711925 shutdown_connections 2023-09-20T21:48:54.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.652+0000 7fbb267ff700 1 -- 172.21.15.22:0/3970711925 wait complete. 2023-09-20T21:48:54.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.653+0000 7fbb267ff700 1 Processor -- start 2023-09-20T21:48:54.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.653+0000 7fbb267ff700 1 -- start start 2023-09-20T21:48:54.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb267ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb20100de0 0x7fbb20074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:54.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb267ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20101fd0 0x7fbb200731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:54.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb267ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb20102970 0x7fbb20073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:54.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb267ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20073eb0 con 0x7fbb20102970 2023-09-20T21:48:54.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb267ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20074020 con 0x7fbb20101fd0 2023-09-20T21:48:54.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb267ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb201a0c00 con 0x7fbb20100de0 2023-09-20T21:48:54.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb1f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20101fd0 0x7fbb200731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:54.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb1f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20101fd0 0x7fbb200731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37636/0 (socket says 172.21.15.22:37636) 2023-09-20T21:48:54.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb1f7fe700 1 -- 172.21.15.22:0/2885360014 learned_addr learned my addr 172.21.15.22:0/2885360014 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:54.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb24d9c700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb20102970 0x7fbb20073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:54.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb1ffff700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb20100de0 0x7fbb20074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:54.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb24d9c700 1 -- 172.21.15.22:0/2885360014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb20100de0 msgr2=0x7fbb20074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:54.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb24d9c700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb20100de0 0x7fbb20074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb24d9c700 1 -- 172.21.15.22:0/2885360014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20101fd0 msgr2=0x7fbb200731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:54.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb24d9c700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20101fd0 0x7fbb200731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb24d9c700 1 -- 172.21.15.22:0/2885360014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18009d00 con 0x7fbb20102970 2023-09-20T21:48:54.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.654+0000 7fbb1f7fe700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20101fd0 0x7fbb200731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:54.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.655+0000 7fbb24d9c700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb20102970 0x7fbb20073710 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbb18007a30 tx=0x7fbb18008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:54.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.655+0000 7fbb1d7fa700 1 -- 172.21.15.22:0/2885360014 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800e040 con 0x7fbb20102970 2023-09-20T21:48:54.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.655+0000 7fbb1d7fa700 1 -- 172.21.15.22:0/2885360014 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb1800bd30 con 0x7fbb20102970 2023-09-20T21:48:54.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.655+0000 7fbb267ff700 1 -- 172.21.15.22:0/2885360014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb201a0e80 con 0x7fbb20102970 2023-09-20T21:48:54.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.655+0000 7fbb1d7fa700 1 -- 172.21.15.22:0/2885360014 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1801d470 con 0x7fbb20102970 2023-09-20T21:48:54.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.655+0000 7fbb267ff700 1 -- 172.21.15.22:0/2885360014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb201a1250 con 0x7fbb20102970 2023-09-20T21:48:54.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.656+0000 7fbb267ff700 1 -- 172.21.15.22:0/2885360014 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2010c650 con 0x7fbb20102970 2023-09-20T21:48:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.658+0000 7fbb1d7fa700 1 -- 172.21.15.22:0/2885360014 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fbb18005000 con 0x7fbb20102970 2023-09-20T21:48:54.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.660+0000 7fbb1d7fa700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb080a1350 0x7fbb080a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.660+0000 7fbb1d7fa700 1 -- 172.21.15.22:0/2885360014 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fbb180ce990 con 0x7fbb20102970 2023-09-20T21:48:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.660+0000 7fbb1ffff700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb080a1350 0x7fbb080a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.660+0000 7fbb1ffff700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb080a1350 0x7fbb080a3810 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbb20101e30 tx=0x7fbb10009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:54.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.662+0000 7fbb1d7fa700 1 -- 172.21.15.22:0/2885360014 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbb18097d00 con 0x7fbb20102970 2023-09-20T21:48:54.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.874+0000 7fbb267ff700 1 -- 172.21.15.22:0/2885360014 --> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb20061220 con 0x7fbb080a1350 2023-09-20T21:48:54.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.888+0000 7fbb1d7fa700 1 -- 172.21.15.22:0/2885360014 <== mgr.14154 v2:172.21.15.22:6800/718010003 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426901 (secure 0 0 0) 0x7fbb20061220 con 0x7fbb080a1350 2023-09-20T21:48:54.893 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:54.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 -- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb080a1350 msgr2=0x7fbb080a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:54.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb080a1350 0x7fbb080a3810 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbb20101e30 tx=0x7fbb10009260 comp rx=0 tx=0).stop 2023-09-20T21:48:54.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 -- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb20102970 msgr2=0x7fbb20073710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:54.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb20102970 0x7fbb20073710 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbb18007a30 tx=0x7fbb18008eb0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 -- 172.21.15.22:0/2885360014 shutdown_connections 2023-09-20T21:48:54.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb20102970 0x7fbb20073710 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb20100de0 0x7fbb20074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb080a1350 0x7fbb080a3810 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 --2- 172.21.15.22:0/2885360014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb20101fd0 0x7fbb200731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:54.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 -- 172.21.15.22:0/2885360014 >> 172.21.15.22:0/2885360014 conn(0x7fbb200fc310 msgr2=0x7fbb200fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:54.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 -- 172.21.15.22:0/2885360014 shutdown_connections 2023-09-20T21:48:54.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:54.893+0000 7fbb267ff700 1 -- 172.21.15.22:0/2885360014 wait complete. 2023-09-20T21:48:54.899 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-09-20T21:48:54.913 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2023-09-20T21:48:53.132727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600890,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1184,"num_read_kb":656,"num_write":392,"num_write_kb":3007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338640,"kb_used_data":14360,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603184,"statfs":{"total":1151940427776,"available":1151593660416,"internally_reserved":0,"allocated":14704640,"data_stored":7983721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19129,"internal_metadata":331855175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":66,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003652"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.597787+0000","last_change":"2023-09-20T21:46:41.506092+0000","last_active":"2023-09-20T21:48:09.597787+0000","last_peered":"2023-09-20T21:48:09.597787+0000","last_clean":"2023-09-20T21:48:09.597787+0000","last_became_active":"2023-09-20T21:46:41.505779+0000","last_became_peered":"2023-09-20T21:46:41.505779+0000","last_unstale":"2023-09-20T21:48:09.597787+0000","last_undegraded":"2023-09-20T21:48:09.597787+0000","last_fullsized":"2023-09-20T21:48:09.597787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:25.284068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.971254+0000","last_change":"2023-09-20T21:46:41.505000+0000","last_active":"2023-09-20T21:48:09.971254+0000","last_peered":"2023-09-20T21:48:09.971254+0000","last_clean":"2023-09-20T21:48:09.971254+0000","last_became_active":"2023-09-20T21:46:41.504464+0000","last_became_peered":"2023-09-20T21:46:41.504464+0000","last_unstale":"2023-09-20T21:48:09.971254+0000","last_undegraded":"2023-09-20T21:48:09.971254+0000","last_fullsized":"2023-09-20T21:48:09.971254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:42:44.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.970944+0000","last_change":"2023-09-20T21:46:41.513134+0000","last_active":"2023-09-20T21:48:09.970944+0000","last_peered":"2023-09-20T21:48:09.970944+0000","last_clean":"2023-09-20T21:48:09.970944+0000","last_became_active":"2023-09-20T21:46:41.512793+0000","last_became_peered":"2023-09-20T21:46:41.512793+0000","last_unstale":"2023-09-20T21:48:09.970944+0000","last_undegraded":"2023-09-20T21:48:09.970944+0000","last_fullsized":"2023-09-20T21:48:09.970944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:06:59.978630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.667363+0000","last_change":"2023-09-20T21:46:41.500200+0000","last_active":"2023-09-20T21:48:09.667363+0000","last_peered":"2023-09-20T21:48:09.667363+0000","last_clean":"2023-09-20T21:48:09.667363+0000","last_became_active":"2023-09-20T21:46:41.499866+0000","last_became_peered":"2023-09-20T21:46:41.499866+0000","last_unstale":"2023-09-20T21:48:09.667363+0000","last_undegraded":"2023-09-20T21:48:09.667363+0000","last_fullsized":"2023-09-20T21:48:09.667363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:20:52.989008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584846+0000","last_change":"2023-09-20T21:46:41.507584+0000","last_active":"2023-09-20T21:48:08.584846+0000","last_peered":"2023-09-20T21:48:08.584846+0000","last_clean":"2023-09-20T21:48:08.584846+0000","last_became_active":"2023-09-20T21:46:41.507298+0000","last_became_peered":"2023-09-20T21:46:41.507298+0000","last_unstale":"2023-09-20T21:48:08.584846+0000","last_undegraded":"2023-09-20T21:48:08.584846+0000","last_fullsized":"2023-09-20T21:48:08.584846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:32:14.042414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585516+0000","last_change":"2023-09-20T21:46:41.497002+0000","last_active":"2023-09-20T21:48:08.585516+0000","last_peered":"2023-09-20T21:48:08.585516+0000","last_clean":"2023-09-20T21:48:08.585516+0000","last_became_active":"2023-09-20T21:46:41.496485+0000","last_became_peered":"2023-09-20T21:46:41.496485+0000","last_unstale":"2023-09-20T21:48:08.585516+0000","last_undegraded":"2023-09-20T21:48:08.585516+0000","last_fullsized":"2023-09-20T21:48:08.585516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:48:24.481575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590370+0000","last_change":"2023-09-20T21:46:41.503303+0000","last_active":"2023-09-20T21:48:08.590370+0000","last_peered":"2023-09-20T21:48:08.590370+0000","last_clean":"2023-09-20T21:48:08.590370+0000","last_became_active":"2023-09-20T21:46:41.502982+0000","last_became_peered":"2023-09-20T21:46:41.502982+0000","last_unstale":"2023-09-20T21:48:08.590370+0000","last_undegraded":"2023-09-20T21:48:08.590370+0000","last_fullsized":"2023-09-20T21:48:08.590370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:23:27.119491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.589897+0000","last_change":"2023-09-20T21:46:41.508243+0000","last_active":"2023-09-20T21:48:08.589897+0000","last_peered":"2023-09-20T21:48:08.589897+0000","last_clean":"2023-09-20T21:48:08.589897+0000","last_became_active":"2023-09-20T21:46:41.506407+0000","last_became_peered":"2023-09-20T21:46:41.506407+0000","last_unstale":"2023-09-20T21:48:08.589897+0000","last_undegraded":"2023-09-20T21:48:08.589897+0000","last_fullsized":"2023-09-20T21:48:08.589897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:39.388716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588390+0000","last_change":"2023-09-20T21:46:41.501513+0000","last_active":"2023-09-20T21:48:08.588390+0000","last_peered":"2023-09-20T21:48:08.588390+0000","last_clean":"2023-09-20T21:48:08.588390+0000","last_became_active":"2023-09-20T21:46:41.500051+0000","last_became_peered":"2023-09-20T21:46:41.500051+0000","last_unstale":"2023-09-20T21:48:08.588390+0000","last_undegraded":"2023-09-20T21:48:08.588390+0000","last_fullsized":"2023-09-20T21:48:08.588390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:04:40.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585877+0000","last_change":"2023-09-20T21:46:41.501045+0000","last_active":"2023-09-20T21:48:08.585877+0000","last_peered":"2023-09-20T21:48:08.585877+0000","last_clean":"2023-09-20T21:48:08.585877+0000","last_became_active":"2023-09-20T21:46:41.500744+0000","last_became_peered":"2023-09-20T21:46:41.500744+0000","last_unstale":"2023-09-20T21:48:08.585877+0000","last_undegraded":"2023-09-20T21:48:08.585877+0000","last_fullsized":"2023-09-20T21:48:08.585877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:57:29.120472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590634+0000","last_change":"2023-09-20T21:46:41.510235+0000","last_active":"2023-09-20T21:48:08.590634+0000","last_peered":"2023-09-20T21:48:08.590634+0000","last_clean":"2023-09-20T21:48:08.590634+0000","last_became_active":"2023-09-20T21:46:41.509943+0000","last_became_peered":"2023-09-20T21:46:41.509943+0000","last_unstale":"2023-09-20T21:48:08.590634+0000","last_undegraded":"2023-09-20T21:48:08.590634+0000","last_fullsized":"2023-09-20T21:48:08.590634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:09:19.383505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663853+0000","last_change":"2023-09-20T21:46:41.501194+0000","last_active":"2023-09-20T21:48:08.663853+0000","last_peered":"2023-09-20T21:48:08.663853+0000","last_clean":"2023-09-20T21:48:08.663853+0000","last_became_active":"2023-09-20T21:46:41.500095+0000","last_became_peered":"2023-09-20T21:46:41.500095+0000","last_unstale":"2023-09-20T21:48:08.663853+0000","last_undegraded":"2023-09-20T21:48:08.663853+0000","last_fullsized":"2023-09-20T21:48:08.663853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:46:51.343332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587794+0000","last_change":"2023-09-20T21:46:41.507562+0000","last_active":"2023-09-20T21:48:08.587794+0000","last_peered":"2023-09-20T21:48:08.587794+0000","last_clean":"2023-09-20T21:48:08.587794+0000","last_became_active":"2023-09-20T21:46:41.507094+0000","last_became_peered":"2023-09-20T21:46:41.507094+0000","last_unstale":"2023-09-20T21:48:08.587794+0000","last_undegraded":"2023-09-20T21:48:08.587794+0000","last_fullsized":"2023-09-20T21:48:08.587794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:34:32.401415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591639+0000","last_change":"2023-09-20T21:46:41.502864+0000","last_active":"2023-09-20T21:48:08.591639+0000","last_peered":"2023-09-20T21:48:08.591639+0000","last_clean":"2023-09-20T21:48:08.591639+0000","last_became_active":"2023-09-20T21:46:41.502543+0000","last_became_peered":"2023-09-20T21:46:41.502543+0000","last_unstale":"2023-09-20T21:48:08.591639+0000","last_undegraded":"2023-09-20T21:48:08.591639+0000","last_fullsized":"2023-09-20T21:48:08.591639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:02:04.631874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.592056+0000","last_change":"2023-09-20T21:47:08.600445+0000","last_active":"2023-09-20T21:48:08.592056+0000","last_peered":"2023-09-20T21:48:08.592056+0000","last_clean":"2023-09-20T21:48:08.592056+0000","last_became_active":"2023-09-20T21:47:08.600054+0000","last_became_peered":"2023-09-20T21:47:08.600054+0000","last_unstale":"2023-09-20T21:48:08.592056+0000","last_undegraded":"2023-09-20T21:48:08.592056+0000","last_fullsized":"2023-09-20T21:48:08.592056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:43:18.055278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587667+0000","last_change":"2023-09-20T21:46:41.502036+0000","last_active":"2023-09-20T21:48:08.587667+0000","last_peered":"2023-09-20T21:48:08.587667+0000","last_clean":"2023-09-20T21:48:08.587667+0000","last_became_active":"2023-09-20T21:46:41.501745+0000","last_became_peered":"2023-09-20T21:46:41.501745+0000","last_unstale":"2023-09-20T21:48:08.587667+0000","last_undegraded":"2023-09-20T21:48:08.587667+0000","last_fullsized":"2023-09-20T21:48:08.587667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:08:06.699543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588222+0000","last_change":"2023-09-20T21:46:41.506666+0000","last_active":"2023-09-20T21:48:08.588222+0000","last_peered":"2023-09-20T21:48:08.588222+0000","last_clean":"2023-09-20T21:48:08.588222+0000","last_became_active":"2023-09-20T21:46:41.506266+0000","last_became_peered":"2023-09-20T21:46:41.506266+0000","last_unstale":"2023-09-20T21:48:08.588222+0000","last_undegraded":"2023-09-20T21:48:08.588222+0000","last_fullsized":"2023-09-20T21:48:08.588222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:06:16.809609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585233+0000","last_change":"2023-09-20T21:46:41.497391+0000","last_active":"2023-09-20T21:48:08.585233+0000","last_peered":"2023-09-20T21:48:08.585233+0000","last_clean":"2023-09-20T21:48:08.585233+0000","last_became_active":"2023-09-20T21:46:41.497064+0000","last_became_peered":"2023-09-20T21:46:41.497064+0000","last_unstale":"2023-09-20T21:48:08.585233+0000","last_undegraded":"2023-09-20T21:48:08.585233+0000","last_fullsized":"2023-09-20T21:48:08.585233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:56:11.489585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587770+0000","last_change":"2023-09-20T21:46:41.510602+0000","last_active":"2023-09-20T21:48:08.587770+0000","last_peered":"2023-09-20T21:48:08.587770+0000","last_clean":"2023-09-20T21:48:08.587770+0000","last_became_active":"2023-09-20T21:46:41.510308+0000","last_became_peered":"2023-09-20T21:46:41.510308+0000","last_unstale":"2023-09-20T21:48:08.587770+0000","last_undegraded":"2023-09-20T21:48:08.587770+0000","last_fullsized":"2023-09-20T21:48:08.587770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:43:41.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.972323+0000","last_change":"2023-09-20T21:48:09.972323+0000","last_active":"2023-09-20T21:48:09.972323+0000","last_peered":"2023-09-20T21:48:09.972323+0000","last_clean":"2023-09-20T21:48:09.972323+0000","last_became_active":"2023-09-20T21:48:09.971743+0000","last_became_peered":"2023-09-20T21:48:09.971743+0000","last_unstale":"2023-09-20T21:48:09.972323+0000","last_undegraded":"2023-09-20T21:48:09.972323+0000","last_fullsized":"2023-09-20T21:48:09.972323+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:00:49.679685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":136,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:47.769593+0000","last_change":"2023-09-20T21:46:40.500010+0000","last_active":"2023-09-20T21:48:47.769593+0000","last_peered":"2023-09-20T21:48:47.769593+0000","last_clean":"2023-09-20T21:48:47.769593+0000","last_became_active":"2023-09-20T21:46:40.499632+0000","last_became_peered":"2023-09-20T21:46:40.499632+0000","last_unstale":"2023-09-20T21:48:47.769593+0000","last_undegraded":"2023-09-20T21:48:47.769593+0000","last_fullsized":"2023-09-20T21:48:47.769593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:41:35.590982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594616+0000","last_change":"2023-09-20T21:46:41.516465+0000","last_active":"2023-09-20T21:48:09.594616+0000","last_peered":"2023-09-20T21:48:09.594616+0000","last_clean":"2023-09-20T21:48:09.594616+0000","last_became_active":"2023-09-20T21:46:41.516150+0000","last_became_peered":"2023-09-20T21:46:41.516150+0000","last_unstale":"2023-09-20T21:48:09.594616+0000","last_undegraded":"2023-09-20T21:48:09.594616+0000","last_fullsized":"2023-09-20T21:48:09.594616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:33:37.507049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663434+0000","last_change":"2023-09-20T21:46:45.582916+0000","last_active":"2023-09-20T21:48:08.663434+0000","last_peered":"2023-09-20T21:48:08.663434+0000","last_clean":"2023-09-20T21:48:08.663434+0000","last_became_active":"2023-09-20T21:46:45.582511+0000","last_became_peered":"2023-09-20T21:46:45.582511+0000","last_unstale":"2023-09-20T21:48:08.663434+0000","last_undegraded":"2023-09-20T21:48:08.663434+0000","last_fullsized":"2023-09-20T21:48:08.663434+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:43:47.006768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594814+0000","last_change":"2023-09-20T21:46:40.516867+0000","last_active":"2023-09-20T21:48:09.594814+0000","last_peered":"2023-09-20T21:48:09.594814+0000","last_clean":"2023-09-20T21:48:09.594814+0000","last_became_active":"2023-09-20T21:46:40.516343+0000","last_became_peered":"2023-09-20T21:46:40.516343+0000","last_unstale":"2023-09-20T21:48:09.594814+0000","last_undegraded":"2023-09-20T21:48:09.594814+0000","last_fullsized":"2023-09-20T21:48:09.594814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:17:18.348921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.970829+0000","last_change":"2023-09-20T21:46:41.507093+0000","last_active":"2023-09-20T21:48:09.970829+0000","last_peered":"2023-09-20T21:48:09.970829+0000","last_clean":"2023-09-20T21:48:09.970829+0000","last_became_active":"2023-09-20T21:46:41.506587+0000","last_became_peered":"2023-09-20T21:46:41.506587+0000","last_unstale":"2023-09-20T21:48:09.970829+0000","last_undegraded":"2023-09-20T21:48:09.970829+0000","last_fullsized":"2023-09-20T21:48:09.970829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:26:44.825499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663624+0000","last_change":"2023-09-20T21:46:45.583942+0000","last_active":"2023-09-20T21:48:08.663624+0000","last_peered":"2023-09-20T21:48:08.663624+0000","last_clean":"2023-09-20T21:48:08.663624+0000","last_became_active":"2023-09-20T21:46:45.583484+0000","last_became_peered":"2023-09-20T21:46:45.583484+0000","last_unstale":"2023-09-20T21:48:08.663624+0000","last_undegraded":"2023-09-20T21:48:08.663624+0000","last_fullsized":"2023-09-20T21:48:08.663624+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:49:48.982309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"91'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585701+0000","last_change":"2023-09-20T21:47:08.615780+0000","last_active":"2023-09-20T21:48:08.585701+0000","last_peered":"2023-09-20T21:48:08.585701+0000","last_clean":"2023-09-20T21:48:08.585701+0000","last_became_active":"2023-09-20T21:47:08.603974+0000","last_became_peered":"2023-09-20T21:47:08.603974+0000","last_unstale":"2023-09-20T21:48:08.585701+0000","last_undegraded":"2023-09-20T21:48:08.585701+0000","last_fullsized":"2023-09-20T21:48:08.585701+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:53:23.172401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587966+0000","last_change":"2023-09-20T21:46:41.504449+0000","last_active":"2023-09-20T21:48:08.587966+0000","last_peered":"2023-09-20T21:48:08.587966+0000","last_clean":"2023-09-20T21:48:08.587966+0000","last_became_active":"2023-09-20T21:46:41.504169+0000","last_became_peered":"2023-09-20T21:46:41.504169+0000","last_unstale":"2023-09-20T21:48:08.587966+0000","last_undegraded":"2023-09-20T21:48:08.587966+0000","last_fullsized":"2023-09-20T21:48:08.587966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:36:43.980902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663507+0000","last_change":"2023-09-20T21:46:45.582269+0000","last_active":"2023-09-20T21:48:08.663507+0000","last_peered":"2023-09-20T21:48:08.663507+0000","last_clean":"2023-09-20T21:48:08.663507+0000","last_became_active":"2023-09-20T21:46:45.581782+0000","last_became_peered":"2023-09-20T21:46:45.581782+0000","last_unstale":"2023-09-20T21:48:08.663507+0000","last_undegraded":"2023-09-20T21:48:08.663507+0000","last_fullsized":"2023-09-20T21:48:08.663507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:32:09.518715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585164+0000","last_change":"2023-09-20T21:46:40.504099+0000","last_active":"2023-09-20T21:48:08.585164+0000","last_peered":"2023-09-20T21:48:08.585164+0000","last_clean":"2023-09-20T21:48:08.585164+0000","last_became_active":"2023-09-20T21:46:40.503774+0000","last_became_peered":"2023-09-20T21:46:40.503774+0000","last_unstale":"2023-09-20T21:48:08.585164+0000","last_undegraded":"2023-09-20T21:48:08.585164+0000","last_fullsized":"2023-09-20T21:48:08.585164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:07:17.008746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591062+0000","last_change":"2023-09-20T21:46:41.499802+0000","last_active":"2023-09-20T21:48:08.591062+0000","last_peered":"2023-09-20T21:48:08.591062+0000","last_clean":"2023-09-20T21:48:08.591062+0000","last_became_active":"2023-09-20T21:46:41.499491+0000","last_became_peered":"2023-09-20T21:46:41.499491+0000","last_unstale":"2023-09-20T21:48:08.591062+0000","last_undegraded":"2023-09-20T21:48:08.591062+0000","last_fullsized":"2023-09-20T21:48:08.591062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:44.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587887+0000","last_change":"2023-09-20T21:46:45.578753+0000","last_active":"2023-09-20T21:48:08.587887+0000","last_peered":"2023-09-20T21:48:08.587887+0000","last_clean":"2023-09-20T21:48:08.587887+0000","last_became_active":"2023-09-20T21:46:45.578297+0000","last_became_peered":"2023-09-20T21:46:45.578297+0000","last_unstale":"2023-09-20T21:48:08.587887+0000","last_undegraded":"2023-09-20T21:48:08.587887+0000","last_fullsized":"2023-09-20T21:48:08.587887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:18:22.596550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:48.727867+0000","last_change":"2023-09-20T21:46:40.505675+0000","last_active":"2023-09-20T21:48:48.727867+0000","last_peered":"2023-09-20T21:48:48.727867+0000","last_clean":"2023-09-20T21:48:48.727867+0000","last_became_active":"2023-09-20T21:46:40.505432+0000","last_became_peered":"2023-09-20T21:46:40.505432+0000","last_unstale":"2023-09-20T21:48:48.727867+0000","last_undegraded":"2023-09-20T21:48:48.727867+0000","last_fullsized":"2023-09-20T21:48:48.727867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:14:53.652904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585583+0000","last_change":"2023-09-20T21:46:41.505924+0000","last_active":"2023-09-20T21:48:08.585583+0000","last_peered":"2023-09-20T21:48:08.585583+0000","last_clean":"2023-09-20T21:48:08.585583+0000","last_became_active":"2023-09-20T21:46:41.505609+0000","last_became_peered":"2023-09-20T21:46:41.505609+0000","last_unstale":"2023-09-20T21:48:08.585583+0000","last_undegraded":"2023-09-20T21:48:08.585583+0000","last_fullsized":"2023-09-20T21:48:08.585583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:13:46.506786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588075+0000","last_change":"2023-09-20T21:46:45.587818+0000","last_active":"2023-09-20T21:48:08.588075+0000","last_peered":"2023-09-20T21:48:08.588075+0000","last_clean":"2023-09-20T21:48:08.588075+0000","last_became_active":"2023-09-20T21:46:45.587376+0000","last_became_peered":"2023-09-20T21:46:45.587376+0000","last_unstale":"2023-09-20T21:48:08.588075+0000","last_undegraded":"2023-09-20T21:48:08.588075+0000","last_fullsized":"2023-09-20T21:48:08.588075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:28:53.605158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585426+0000","last_change":"2023-09-20T21:46:40.505610+0000","last_active":"2023-09-20T21:48:08.585426+0000","last_peered":"2023-09-20T21:48:08.585426+0000","last_clean":"2023-09-20T21:48:08.585426+0000","last_became_active":"2023-09-20T21:46:40.505279+0000","last_became_peered":"2023-09-20T21:46:40.505279+0000","last_unstale":"2023-09-20T21:48:08.585426+0000","last_undegraded":"2023-09-20T21:48:08.585426+0000","last_fullsized":"2023-09-20T21:48:08.585426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:41:19.575154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584476+0000","last_change":"2023-09-20T21:47:08.908513+0000","last_active":"2023-09-20T21:48:08.584476+0000","last_peered":"2023-09-20T21:48:08.584476+0000","last_clean":"2023-09-20T21:48:08.584476+0000","last_became_active":"2023-09-20T21:47:08.908142+0000","last_became_peered":"2023-09-20T21:47:08.908142+0000","last_unstale":"2023-09-20T21:48:08.584476+0000","last_undegraded":"2023-09-20T21:48:08.584476+0000","last_fullsized":"2023-09-20T21:48:08.584476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:31:33.992202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588229+0000","last_change":"2023-09-20T21:46:45.581738+0000","last_active":"2023-09-20T21:48:08.588229+0000","last_peered":"2023-09-20T21:48:08.588229+0000","last_clean":"2023-09-20T21:48:08.588229+0000","last_became_active":"2023-09-20T21:46:45.581279+0000","last_became_peered":"2023-09-20T21:46:45.581279+0000","last_unstale":"2023-09-20T21:48:08.588229+0000","last_undegraded":"2023-09-20T21:48:08.588229+0000","last_fullsized":"2023-09-20T21:48:08.588229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:41:41.862345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587220+0000","last_change":"2023-09-20T21:46:40.510539+0000","last_active":"2023-09-20T21:48:08.587220+0000","last_peered":"2023-09-20T21:48:08.587220+0000","last_clean":"2023-09-20T21:48:08.587220+0000","last_became_active":"2023-09-20T21:46:40.509880+0000","last_became_peered":"2023-09-20T21:46:40.509880+0000","last_unstale":"2023-09-20T21:48:08.587220+0000","last_undegraded":"2023-09-20T21:48:08.587220+0000","last_fullsized":"2023-09-20T21:48:08.587220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:52.877786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.589794+0000","last_change":"2023-09-20T21:46:41.505105+0000","last_active":"2023-09-20T21:48:08.589794+0000","last_peered":"2023-09-20T21:48:08.589794+0000","last_clean":"2023-09-20T21:48:08.589794+0000","last_became_active":"2023-09-20T21:46:41.504785+0000","last_became_peered":"2023-09-20T21:46:41.504785+0000","last_unstale":"2023-09-20T21:48:08.589794+0000","last_undegraded":"2023-09-20T21:48:08.589794+0000","last_fullsized":"2023-09-20T21:48:08.589794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:07:57.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.971145+0000","last_change":"2023-09-20T21:46:45.583160+0000","last_active":"2023-09-20T21:48:09.971145+0000","last_peered":"2023-09-20T21:48:09.971145+0000","last_clean":"2023-09-20T21:48:09.971145+0000","last_became_active":"2023-09-20T21:46:45.582709+0000","last_became_peered":"2023-09-20T21:46:45.582709+0000","last_unstale":"2023-09-20T21:48:09.971145+0000","last_undegraded":"2023-09-20T21:48:09.971145+0000","last_fullsized":"2023-09-20T21:48:09.971145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:24:57.563682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590612+0000","last_change":"2023-09-20T21:46:40.502591+0000","last_active":"2023-09-20T21:48:08.590612+0000","last_peered":"2023-09-20T21:48:08.590612+0000","last_clean":"2023-09-20T21:48:08.590612+0000","last_became_active":"2023-09-20T21:46:40.502288+0000","last_became_peered":"2023-09-20T21:46:40.502288+0000","last_unstale":"2023-09-20T21:48:08.590612+0000","last_undegraded":"2023-09-20T21:48:08.590612+0000","last_fullsized":"2023-09-20T21:48:08.590612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:09:37.983866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584209+0000","last_change":"2023-09-20T21:47:08.599912+0000","last_active":"2023-09-20T21:48:08.584209+0000","last_peered":"2023-09-20T21:48:08.584209+0000","last_clean":"2023-09-20T21:48:08.584209+0000","last_became_active":"2023-09-20T21:47:08.599434+0000","last_became_peered":"2023-09-20T21:47:08.599434+0000","last_unstale":"2023-09-20T21:48:08.584209+0000","last_undegraded":"2023-09-20T21:48:08.584209+0000","last_fullsized":"2023-09-20T21:48:08.584209+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:37:01.125731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590887+0000","last_change":"2023-09-20T21:46:45.580917+0000","last_active":"2023-09-20T21:48:08.590887+0000","last_peered":"2023-09-20T21:48:08.590887+0000","last_clean":"2023-09-20T21:48:08.590887+0000","last_became_active":"2023-09-20T21:46:45.580505+0000","last_became_peered":"2023-09-20T21:46:45.580505+0000","last_unstale":"2023-09-20T21:48:08.590887+0000","last_undegraded":"2023-09-20T21:48:08.590887+0000","last_fullsized":"2023-09-20T21:48:08.590887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:52:19.733919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":230,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:50.730444+0000","last_change":"2023-09-20T21:46:40.507040+0000","last_active":"2023-09-20T21:48:50.730444+0000","last_peered":"2023-09-20T21:48:50.730444+0000","last_clean":"2023-09-20T21:48:50.730444+0000","last_became_active":"2023-09-20T21:46:40.506712+0000","last_became_peered":"2023-09-20T21:46:40.506712+0000","last_unstale":"2023-09-20T21:48:50.730444+0000","last_undegraded":"2023-09-20T21:48:50.730444+0000","last_fullsized":"2023-09-20T21:48:50.730444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:02:56.040911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588376+0000","last_change":"2023-09-20T21:46:41.504794+0000","last_active":"2023-09-20T21:48:08.588376+0000","last_peered":"2023-09-20T21:48:08.588376+0000","last_clean":"2023-09-20T21:48:08.588376+0000","last_became_active":"2023-09-20T21:46:41.504372+0000","last_became_peered":"2023-09-20T21:46:41.504372+0000","last_unstale":"2023-09-20T21:48:08.588376+0000","last_undegraded":"2023-09-20T21:48:08.588376+0000","last_fullsized":"2023-09-20T21:48:08.588376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:42:20.527306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584027+0000","last_change":"2023-09-20T21:46:45.586364+0000","last_active":"2023-09-20T21:48:08.584027+0000","last_peered":"2023-09-20T21:48:08.584027+0000","last_clean":"2023-09-20T21:48:08.584027+0000","last_became_active":"2023-09-20T21:46:45.584620+0000","last_became_peered":"2023-09-20T21:46:45.584620+0000","last_unstale":"2023-09-20T21:48:08.584027+0000","last_undegraded":"2023-09-20T21:48:08.584027+0000","last_fullsized":"2023-09-20T21:48:08.584027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:05:20.158118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590779+0000","last_change":"2023-09-20T21:46:40.500797+0000","last_active":"2023-09-20T21:48:08.590779+0000","last_peered":"2023-09-20T21:48:08.590779+0000","last_clean":"2023-09-20T21:48:08.590779+0000","last_became_active":"2023-09-20T21:46:40.500337+0000","last_became_peered":"2023-09-20T21:46:40.500337+0000","last_unstale":"2023-09-20T21:48:08.590779+0000","last_undegraded":"2023-09-20T21:48:08.590779+0000","last_fullsized":"2023-09-20T21:48:08.590779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:19.704483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.592706+0000","last_change":"2023-09-20T21:47:08.908615+0000","last_active":"2023-09-20T21:48:08.592706+0000","last_peered":"2023-09-20T21:48:08.592706+0000","last_clean":"2023-09-20T21:48:08.592706+0000","last_became_active":"2023-09-20T21:47:08.908228+0000","last_became_peered":"2023-09-20T21:47:08.908228+0000","last_unstale":"2023-09-20T21:48:08.592706+0000","last_undegraded":"2023-09-20T21:48:08.592706+0000","last_fullsized":"2023-09-20T21:48:08.592706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:15:28.772129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663779+0000","last_change":"2023-09-20T21:46:45.583081+0000","last_active":"2023-09-20T21:48:08.663779+0000","last_peered":"2023-09-20T21:48:08.663779+0000","last_clean":"2023-09-20T21:48:08.663779+0000","last_became_active":"2023-09-20T21:46:45.580496+0000","last_became_peered":"2023-09-20T21:46:45.580496+0000","last_unstale":"2023-09-20T21:48:08.663779+0000","last_undegraded":"2023-09-20T21:48:08.663779+0000","last_fullsized":"2023-09-20T21:48:08.663779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:42.708165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591031+0000","last_change":"2023-09-20T21:46:40.512739+0000","last_active":"2023-09-20T21:48:08.591031+0000","last_peered":"2023-09-20T21:48:08.591031+0000","last_clean":"2023-09-20T21:48:08.591031+0000","last_became_active":"2023-09-20T21:46:40.512485+0000","last_became_peered":"2023-09-20T21:46:40.512485+0000","last_unstale":"2023-09-20T21:48:08.591031+0000","last_undegraded":"2023-09-20T21:48:08.591031+0000","last_fullsized":"2023-09-20T21:48:08.591031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:02:49.392560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588075+0000","last_change":"2023-09-20T21:47:08.604682+0000","last_active":"2023-09-20T21:48:08.588075+0000","last_peered":"2023-09-20T21:48:08.588075+0000","last_clean":"2023-09-20T21:48:08.588075+0000","last_became_active":"2023-09-20T21:47:08.604271+0000","last_became_peered":"2023-09-20T21:47:08.604271+0000","last_unstale":"2023-09-20T21:48:08.588075+0000","last_undegraded":"2023-09-20T21:48:08.588075+0000","last_fullsized":"2023-09-20T21:48:08.588075+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:40:51.542781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584184+0000","last_change":"2023-09-20T21:46:45.579478+0000","last_active":"2023-09-20T21:48:08.584184+0000","last_peered":"2023-09-20T21:48:08.584184+0000","last_clean":"2023-09-20T21:48:08.584184+0000","last_became_active":"2023-09-20T21:46:45.579055+0000","last_became_peered":"2023-09-20T21:46:45.579055+0000","last_unstale":"2023-09-20T21:48:08.584184+0000","last_undegraded":"2023-09-20T21:48:08.584184+0000","last_fullsized":"2023-09-20T21:48:08.584184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:17:32.013280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'14","reported_seq":244,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:50.730483+0000","last_change":"2023-09-20T21:46:40.507321+0000","last_active":"2023-09-20T21:48:50.730483+0000","last_peered":"2023-09-20T21:48:50.730483+0000","last_clean":"2023-09-20T21:48:50.730483+0000","last_became_active":"2023-09-20T21:46:40.506983+0000","last_became_peered":"2023-09-20T21:46:40.506983+0000","last_unstale":"2023-09-20T21:48:50.730483+0000","last_undegraded":"2023-09-20T21:48:50.730483+0000","last_fullsized":"2023-09-20T21:48:50.730483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:46:40.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":210,"num_read_kb":109,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585464+0000","last_change":"2023-09-20T21:46:41.509646+0000","last_active":"2023-09-20T21:48:08.585464+0000","last_peered":"2023-09-20T21:48:08.585464+0000","last_clean":"2023-09-20T21:48:08.585464+0000","last_became_active":"2023-09-20T21:46:41.509183+0000","last_became_peered":"2023-09-20T21:46:41.509183+0000","last_unstale":"2023-09-20T21:48:08.585464+0000","last_undegraded":"2023-09-20T21:48:08.585464+0000","last_fullsized":"2023-09-20T21:48:08.585464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:17:21.084830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591360+0000","last_change":"2023-09-20T21:46:45.583709+0000","last_active":"2023-09-20T21:48:08.591360+0000","last_peered":"2023-09-20T21:48:08.591360+0000","last_clean":"2023-09-20T21:48:08.591360+0000","last_became_active":"2023-09-20T21:46:45.583302+0000","last_became_peered":"2023-09-20T21:46:45.583302+0000","last_unstale":"2023-09-20T21:48:08.591360+0000","last_undegraded":"2023-09-20T21:48:08.591360+0000","last_fullsized":"2023-09-20T21:48:08.591360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:31:35.472750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587265+0000","last_change":"2023-09-20T21:46:40.506411+0000","last_active":"2023-09-20T21:48:08.587265+0000","last_peered":"2023-09-20T21:48:08.587265+0000","last_clean":"2023-09-20T21:48:08.587265+0000","last_became_active":"2023-09-20T21:46:40.506116+0000","last_became_peered":"2023-09-20T21:46:40.506116+0000","last_unstale":"2023-09-20T21:48:08.587265+0000","last_undegraded":"2023-09-20T21:48:08.587265+0000","last_fullsized":"2023-09-20T21:48:08.587265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:23:19.175007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587561+0000","last_change":"2023-09-20T21:46:41.513793+0000","last_active":"2023-09-20T21:48:08.587561+0000","last_peered":"2023-09-20T21:48:08.587561+0000","last_clean":"2023-09-20T21:48:08.587561+0000","last_became_active":"2023-09-20T21:46:41.513434+0000","last_became_peered":"2023-09-20T21:46:41.513434+0000","last_unstale":"2023-09-20T21:48:08.587561+0000","last_undegraded":"2023-09-20T21:48:08.587561+0000","last_fullsized":"2023-09-20T21:48:08.587561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:00:57.666310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587768+0000","last_change":"2023-09-20T21:46:45.578188+0000","last_active":"2023-09-20T21:48:08.587768+0000","last_peered":"2023-09-20T21:48:08.587768+0000","last_clean":"2023-09-20T21:48:08.587768+0000","last_became_active":"2023-09-20T21:46:45.577548+0000","last_became_peered":"2023-09-20T21:46:45.577548+0000","last_unstale":"2023-09-20T21:48:08.587768+0000","last_undegraded":"2023-09-20T21:48:08.587768+0000","last_fullsized":"2023-09-20T21:48:08.587768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:54:58.894269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591467+0000","last_change":"2023-09-20T21:46:40.505465+0000","last_active":"2023-09-20T21:48:08.591467+0000","last_peered":"2023-09-20T21:48:08.591467+0000","last_clean":"2023-09-20T21:48:08.591467+0000","last_became_active":"2023-09-20T21:46:40.505174+0000","last_became_peered":"2023-09-20T21:46:40.505174+0000","last_unstale":"2023-09-20T21:48:08.591467+0000","last_undegraded":"2023-09-20T21:48:08.591467+0000","last_fullsized":"2023-09-20T21:48:08.591467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:50:53.191963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590466+0000","last_change":"2023-09-20T21:46:41.507370+0000","last_active":"2023-09-20T21:48:08.590466+0000","last_peered":"2023-09-20T21:48:08.590466+0000","last_clean":"2023-09-20T21:48:08.590466+0000","last_became_active":"2023-09-20T21:46:41.507035+0000","last_became_peered":"2023-09-20T21:46:41.507035+0000","last_unstale":"2023-09-20T21:48:08.590466+0000","last_undegraded":"2023-09-20T21:48:08.590466+0000","last_fullsized":"2023-09-20T21:48:08.590466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:11:50.298899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584972+0000","last_change":"2023-09-20T21:46:45.584458+0000","last_active":"2023-09-20T21:48:08.584972+0000","last_peered":"2023-09-20T21:48:08.584972+0000","last_clean":"2023-09-20T21:48:08.584972+0000","last_became_active":"2023-09-20T21:46:45.584033+0000","last_became_peered":"2023-09-20T21:46:45.584033+0000","last_unstale":"2023-09-20T21:48:08.584972+0000","last_undegraded":"2023-09-20T21:48:08.584972+0000","last_fullsized":"2023-09-20T21:48:08.584972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:48:44.653880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:48.727034+0000","last_change":"2023-09-20T21:46:40.501329+0000","last_active":"2023-09-20T21:48:48.727034+0000","last_peered":"2023-09-20T21:48:48.727034+0000","last_clean":"2023-09-20T21:48:48.727034+0000","last_became_active":"2023-09-20T21:46:40.500999+0000","last_became_peered":"2023-09-20T21:46:40.500999+0000","last_unstale":"2023-09-20T21:48:48.727034+0000","last_undegraded":"2023-09-20T21:48:48.727034+0000","last_fullsized":"2023-09-20T21:48:48.727034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:41:38.028398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590301+0000","last_change":"2023-09-20T21:47:08.602123+0000","last_active":"2023-09-20T21:48:08.590301+0000","last_peered":"2023-09-20T21:48:08.590301+0000","last_clean":"2023-09-20T21:48:08.590301+0000","last_became_active":"2023-09-20T21:47:08.601738+0000","last_became_peered":"2023-09-20T21:47:08.601738+0000","last_unstale":"2023-09-20T21:48:08.590301+0000","last_undegraded":"2023-09-20T21:48:08.590301+0000","last_fullsized":"2023-09-20T21:48:08.590301+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:26:36.710230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.599126+0000","last_change":"2023-09-20T21:48:09.599126+0000","last_active":"2023-09-20T21:48:09.599126+0000","last_peered":"2023-09-20T21:48:09.599126+0000","last_clean":"2023-09-20T21:48:09.599126+0000","last_became_active":"2023-09-20T21:48:09.598649+0000","last_became_peered":"2023-09-20T21:48:09.598649+0000","last_unstale":"2023-09-20T21:48:09.599126+0000","last_undegraded":"2023-09-20T21:48:09.599126+0000","last_fullsized":"2023-09-20T21:48:09.599126+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:24:27.566874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.584737+0000","last_change":"2023-09-20T21:46:40.499985+0000","last_active":"2023-09-20T21:48:08.584737+0000","last_peered":"2023-09-20T21:48:08.584737+0000","last_clean":"2023-09-20T21:48:08.584737+0000","last_became_active":"2023-09-20T21:46:40.499646+0000","last_became_peered":"2023-09-20T21:46:40.499646+0000","last_unstale":"2023-09-20T21:48:08.584737+0000","last_undegraded":"2023-09-20T21:48:08.584737+0000","last_fullsized":"2023-09-20T21:48:08.584737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:52:02.758556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590785+0000","last_change":"2023-09-20T21:46:41.501241+0000","last_active":"2023-09-20T21:48:08.590785+0000","last_peered":"2023-09-20T21:48:08.590785+0000","last_clean":"2023-09-20T21:48:08.590785+0000","last_became_active":"2023-09-20T21:46:41.500902+0000","last_became_peered":"2023-09-20T21:46:41.500902+0000","last_unstale":"2023-09-20T21:48:08.590785+0000","last_undegraded":"2023-09-20T21:48:08.590785+0000","last_fullsized":"2023-09-20T21:48:08.590785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:21:55.210065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590708+0000","last_change":"2023-09-20T21:46:45.583405+0000","last_active":"2023-09-20T21:48:08.590708+0000","last_peered":"2023-09-20T21:48:08.590708+0000","last_clean":"2023-09-20T21:48:08.590708+0000","last_became_active":"2023-09-20T21:46:45.582945+0000","last_became_peered":"2023-09-20T21:46:45.582945+0000","last_unstale":"2023-09-20T21:48:08.590708+0000","last_undegraded":"2023-09-20T21:48:08.590708+0000","last_fullsized":"2023-09-20T21:48:08.590708+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:27:14.973402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585473+0000","last_change":"2023-09-20T21:46:40.507744+0000","last_active":"2023-09-20T21:48:08.585473+0000","last_peered":"2023-09-20T21:48:08.585473+0000","last_clean":"2023-09-20T21:48:08.585473+0000","last_became_active":"2023-09-20T21:46:40.507454+0000","last_became_peered":"2023-09-20T21:46:40.507454+0000","last_unstale":"2023-09-20T21:48:08.585473+0000","last_undegraded":"2023-09-20T21:48:08.585473+0000","last_fullsized":"2023-09-20T21:48:08.585473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:00:24.492789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588068+0000","last_change":"2023-09-20T21:46:41.504263+0000","last_active":"2023-09-20T21:48:08.588068+0000","last_peered":"2023-09-20T21:48:08.588068+0000","last_clean":"2023-09-20T21:48:08.588068+0000","last_became_active":"2023-09-20T21:46:41.503961+0000","last_became_peered":"2023-09-20T21:46:41.503961+0000","last_unstale":"2023-09-20T21:48:08.588068+0000","last_undegraded":"2023-09-20T21:48:08.588068+0000","last_fullsized":"2023-09-20T21:48:08.588068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:31:58.035517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.668647+0000","last_change":"2023-09-20T21:48:09.668184+0000","last_active":"2023-09-20T21:48:09.668647+0000","last_peered":"2023-09-20T21:48:09.668647+0000","last_clean":"2023-09-20T21:48:09.668647+0000","last_became_active":"2023-09-20T21:48:09.667732+0000","last_became_peered":"2023-09-20T21:48:09.667732+0000","last_unstale":"2023-09-20T21:48:09.668647+0000","last_undegraded":"2023-09-20T21:48:09.668647+0000","last_fullsized":"2023-09-20T21:48:09.668647+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:06:25.108207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.592116+0000","last_change":"2023-09-20T21:46:40.501850+0000","last_active":"2023-09-20T21:48:09.592116+0000","last_peered":"2023-09-20T21:48:09.592116+0000","last_clean":"2023-09-20T21:48:09.592116+0000","last_became_active":"2023-09-20T21:46:40.501500+0000","last_became_peered":"2023-09-20T21:46:40.501500+0000","last_unstale":"2023-09-20T21:48:09.592116+0000","last_undegraded":"2023-09-20T21:48:09.592116+0000","last_fullsized":"2023-09-20T21:48:09.592116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:14:15.907335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.666100+0000","last_change":"2023-09-20T21:46:41.507432+0000","last_active":"2023-09-20T21:48:09.666100+0000","last_peered":"2023-09-20T21:48:09.666100+0000","last_clean":"2023-09-20T21:48:09.666100+0000","last_became_active":"2023-09-20T21:46:41.507119+0000","last_became_peered":"2023-09-20T21:46:41.507119+0000","last_unstale":"2023-09-20T21:48:09.666100+0000","last_undegraded":"2023-09-20T21:48:09.666100+0000","last_fullsized":"2023-09-20T21:48:09.666100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:14:10.298810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594895+0000","last_change":"2023-09-20T21:46:45.588455+0000","last_active":"2023-09-20T21:48:09.594895+0000","last_peered":"2023-09-20T21:48:09.594895+0000","last_clean":"2023-09-20T21:48:09.594895+0000","last_became_active":"2023-09-20T21:46:45.587980+0000","last_became_peered":"2023-09-20T21:46:45.587980+0000","last_unstale":"2023-09-20T21:48:09.594895+0000","last_undegraded":"2023-09-20T21:48:09.594895+0000","last_fullsized":"2023-09-20T21:48:09.594895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:07:35.697330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:50.774388+0000","last_change":"2023-09-20T21:46:40.506260+0000","last_active":"2023-09-20T21:48:50.774388+0000","last_peered":"2023-09-20T21:48:50.774388+0000","last_clean":"2023-09-20T21:48:50.774388+0000","last_became_active":"2023-09-20T21:46:40.505999+0000","last_became_peered":"2023-09-20T21:46:40.505999+0000","last_unstale":"2023-09-20T21:48:50.774388+0000","last_undegraded":"2023-09-20T21:48:50.774388+0000","last_fullsized":"2023-09-20T21:48:50.774388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:39:01.748038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.665667+0000","last_change":"2023-09-20T21:47:08.599481+0000","last_active":"2023-09-20T21:48:09.665667+0000","last_peered":"2023-09-20T21:48:09.665667+0000","last_clean":"2023-09-20T21:48:09.665667+0000","last_became_active":"2023-09-20T21:47:08.599104+0000","last_became_peered":"2023-09-20T21:47:08.599104+0000","last_unstale":"2023-09-20T21:48:09.665667+0000","last_undegraded":"2023-09-20T21:48:09.665667+0000","last_fullsized":"2023-09-20T21:48:09.665667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:33:56.845446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588094+0000","last_change":"2023-09-20T21:46:45.587410+0000","last_active":"2023-09-20T21:48:08.588094+0000","last_peered":"2023-09-20T21:48:08.588094+0000","last_clean":"2023-09-20T21:48:08.588094+0000","last_became_active":"2023-09-20T21:46:45.586944+0000","last_became_peered":"2023-09-20T21:46:45.586944+0000","last_unstale":"2023-09-20T21:48:08.588094+0000","last_undegraded":"2023-09-20T21:48:08.588094+0000","last_fullsized":"2023-09-20T21:48:08.588094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:43:57.053765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.591816+0000","last_change":"2023-09-20T21:46:41.511526+0000","last_active":"2023-09-20T21:48:09.591816+0000","last_peered":"2023-09-20T21:48:09.591816+0000","last_clean":"2023-09-20T21:48:09.591816+0000","last_became_active":"2023-09-20T21:46:41.511213+0000","last_became_peered":"2023-09-20T21:46:41.511213+0000","last_unstale":"2023-09-20T21:48:09.591816+0000","last_undegraded":"2023-09-20T21:48:09.591816+0000","last_fullsized":"2023-09-20T21:48:09.591816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:29:04.199190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:48.726313+0000","last_change":"2023-09-20T21:46:40.514916+0000","last_active":"2023-09-20T21:48:48.726313+0000","last_peered":"2023-09-20T21:48:48.726313+0000","last_clean":"2023-09-20T21:48:48.726313+0000","last_became_active":"2023-09-20T21:46:40.514553+0000","last_became_peered":"2023-09-20T21:46:40.514553+0000","last_unstale":"2023-09-20T21:48:48.726313+0000","last_undegraded":"2023-09-20T21:48:48.726313+0000","last_fullsized":"2023-09-20T21:48:48.726313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:12:32.481852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714807+0000","last_change":"2023-09-20T21:46:45.587489+0000","last_active":"2023-09-20T21:48:09.714807+0000","last_peered":"2023-09-20T21:48:09.714807+0000","last_clean":"2023-09-20T21:48:09.714807+0000","last_became_active":"2023-09-20T21:46:45.587022+0000","last_became_peered":"2023-09-20T21:46:45.587022+0000","last_unstale":"2023-09-20T21:48:09.714807+0000","last_undegraded":"2023-09-20T21:48:09.714807+0000","last_fullsized":"2023-09-20T21:48:09.714807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:09:05.870763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593480+0000","last_change":"2023-09-20T21:46:41.508886+0000","last_active":"2023-09-20T21:48:09.593480+0000","last_peered":"2023-09-20T21:48:09.593480+0000","last_clean":"2023-09-20T21:48:09.593480+0000","last_became_active":"2023-09-20T21:46:41.508470+0000","last_became_peered":"2023-09-20T21:46:41.508470+0000","last_unstale":"2023-09-20T21:48:09.593480+0000","last_undegraded":"2023-09-20T21:48:09.593480+0000","last_fullsized":"2023-09-20T21:48:09.593480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:45:04.888300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.667141+0000","last_change":"2023-09-20T21:46:40.503350+0000","last_active":"2023-09-20T21:48:09.667141+0000","last_peered":"2023-09-20T21:48:09.667141+0000","last_clean":"2023-09-20T21:48:09.667141+0000","last_became_active":"2023-09-20T21:46:40.503045+0000","last_became_peered":"2023-09-20T21:46:40.503045+0000","last_unstale":"2023-09-20T21:48:09.667141+0000","last_undegraded":"2023-09-20T21:48:09.667141+0000","last_fullsized":"2023-09-20T21:48:09.667141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:31:09.004342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":318,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714381+0000","last_change":"2023-09-20T21:45:35.180242+0000","last_active":"2023-09-20T21:48:09.714381+0000","last_peered":"2023-09-20T21:48:09.714381+0000","last_clean":"2023-09-20T21:48:09.714381+0000","last_became_active":"2023-09-20T21:45:35.179705+0000","last_became_peered":"2023-09-20T21:45:35.179705+0000","last_unstale":"2023-09-20T21:48:09.714381+0000","last_undegraded":"2023-09-20T21:48:09.714381+0000","last_fullsized":"2023-09-20T21:48:09.714381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:42:25.826683+0000","last_clean_scrub_stamp":"2023-09-20T21:42:25.826683+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:02:03.448188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591048+0000","last_change":"2023-09-20T21:46:45.581396+0000","last_active":"2023-09-20T21:48:08.591048+0000","last_peered":"2023-09-20T21:48:08.591048+0000","last_clean":"2023-09-20T21:48:08.591048+0000","last_became_active":"2023-09-20T21:46:45.581018+0000","last_became_peered":"2023-09-20T21:46:45.581018+0000","last_unstale":"2023-09-20T21:48:08.591048+0000","last_undegraded":"2023-09-20T21:48:08.591048+0000","last_fullsized":"2023-09-20T21:48:08.591048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:16:35.934460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594684+0000","last_change":"2023-09-20T21:46:41.514392+0000","last_active":"2023-09-20T21:48:09.594684+0000","last_peered":"2023-09-20T21:48:09.594684+0000","last_clean":"2023-09-20T21:48:09.594684+0000","last_became_active":"2023-09-20T21:46:41.514090+0000","last_became_peered":"2023-09-20T21:46:41.514090+0000","last_unstale":"2023-09-20T21:48:09.594684+0000","last_undegraded":"2023-09-20T21:48:09.594684+0000","last_fullsized":"2023-09-20T21:48:09.594684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:05:47.145753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714387+0000","last_change":"2023-09-20T21:46:40.507346+0000","last_active":"2023-09-20T21:48:09.714387+0000","last_peered":"2023-09-20T21:48:09.714387+0000","last_clean":"2023-09-20T21:48:09.714387+0000","last_became_active":"2023-09-20T21:46:40.506969+0000","last_became_peered":"2023-09-20T21:46:40.506969+0000","last_unstale":"2023-09-20T21:48:09.714387+0000","last_undegraded":"2023-09-20T21:48:09.714387+0000","last_fullsized":"2023-09-20T21:48:09.714387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:01:20.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588355+0000","last_change":"2023-09-20T21:46:45.578033+0000","last_active":"2023-09-20T21:48:08.588355+0000","last_peered":"2023-09-20T21:48:08.588355+0000","last_clean":"2023-09-20T21:48:08.588355+0000","last_became_active":"2023-09-20T21:46:45.577540+0000","last_became_peered":"2023-09-20T21:46:45.577540+0000","last_unstale":"2023-09-20T21:48:08.588355+0000","last_undegraded":"2023-09-20T21:48:08.588355+0000","last_fullsized":"2023-09-20T21:48:08.588355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:24:31.020660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594334+0000","last_change":"2023-09-20T21:46:40.510634+0000","last_active":"2023-09-20T21:48:09.594334+0000","last_peered":"2023-09-20T21:48:09.594334+0000","last_clean":"2023-09-20T21:48:09.594334+0000","last_became_active":"2023-09-20T21:46:40.510390+0000","last_became_peered":"2023-09-20T21:46:40.510390+0000","last_unstale":"2023-09-20T21:48:09.594334+0000","last_undegraded":"2023-09-20T21:48:09.594334+0000","last_fullsized":"2023-09-20T21:48:09.594334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:44:49.272987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595048+0000","last_change":"2023-09-20T21:46:41.502981+0000","last_active":"2023-09-20T21:48:09.595048+0000","last_peered":"2023-09-20T21:48:09.595048+0000","last_clean":"2023-09-20T21:48:09.595048+0000","last_became_active":"2023-09-20T21:46:41.502594+0000","last_became_peered":"2023-09-20T21:46:41.502594+0000","last_unstale":"2023-09-20T21:48:09.595048+0000","last_undegraded":"2023-09-20T21:48:09.595048+0000","last_fullsized":"2023-09-20T21:48:09.595048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T04:41:15.261478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593135+0000","last_change":"2023-09-20T21:46:45.582903+0000","last_active":"2023-09-20T21:48:09.593135+0000","last_peered":"2023-09-20T21:48:09.593135+0000","last_clean":"2023-09-20T21:48:09.593135+0000","last_became_active":"2023-09-20T21:46:45.582433+0000","last_became_peered":"2023-09-20T21:46:45.582433+0000","last_unstale":"2023-09-20T21:48:09.593135+0000","last_undegraded":"2023-09-20T21:48:09.593135+0000","last_fullsized":"2023-09-20T21:48:09.593135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:38:22.174903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.592545+0000","last_change":"2023-09-20T21:46:40.507671+0000","last_active":"2023-09-20T21:48:09.592545+0000","last_peered":"2023-09-20T21:48:09.592545+0000","last_clean":"2023-09-20T21:48:09.592545+0000","last_became_active":"2023-09-20T21:46:40.507288+0000","last_became_peered":"2023-09-20T21:46:40.507288+0000","last_unstale":"2023-09-20T21:48:09.592545+0000","last_undegraded":"2023-09-20T21:48:09.592545+0000","last_fullsized":"2023-09-20T21:48:09.592545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:27:02.432900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.594926+0000","last_change":"2023-09-20T21:46:41.507654+0000","last_active":"2023-09-20T21:48:09.594926+0000","last_peered":"2023-09-20T21:48:09.594926+0000","last_clean":"2023-09-20T21:48:09.594926+0000","last_became_active":"2023-09-20T21:46:41.507362+0000","last_became_peered":"2023-09-20T21:46:41.507362+0000","last_unstale":"2023-09-20T21:48:09.594926+0000","last_undegraded":"2023-09-20T21:48:09.594926+0000","last_fullsized":"2023-09-20T21:48:09.594926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:32:08.973379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595697+0000","last_change":"2023-09-20T21:46:45.578437+0000","last_active":"2023-09-20T21:48:09.595697+0000","last_peered":"2023-09-20T21:48:09.595697+0000","last_clean":"2023-09-20T21:48:09.595697+0000","last_became_active":"2023-09-20T21:46:45.578010+0000","last_became_peered":"2023-09-20T21:46:45.578010+0000","last_unstale":"2023-09-20T21:48:09.595697+0000","last_undegraded":"2023-09-20T21:48:09.595697+0000","last_fullsized":"2023-09-20T21:48:09.595697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:59:05.494861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.664112+0000","last_change":"2023-09-20T21:46:40.508937+0000","last_active":"2023-09-20T21:48:08.664112+0000","last_peered":"2023-09-20T21:48:08.664112+0000","last_clean":"2023-09-20T21:48:08.664112+0000","last_became_active":"2023-09-20T21:46:40.508639+0000","last_became_peered":"2023-09-20T21:46:40.508639+0000","last_unstale":"2023-09-20T21:48:08.664112+0000","last_undegraded":"2023-09-20T21:48:08.664112+0000","last_fullsized":"2023-09-20T21:48:08.664112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:50:48.629958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.971221+0000","last_change":"2023-09-20T21:46:41.510702+0000","last_active":"2023-09-20T21:48:09.971221+0000","last_peered":"2023-09-20T21:48:09.971221+0000","last_clean":"2023-09-20T21:48:09.971221+0000","last_became_active":"2023-09-20T21:46:41.510357+0000","last_became_peered":"2023-09-20T21:46:41.510357+0000","last_unstale":"2023-09-20T21:48:09.971221+0000","last_undegraded":"2023-09-20T21:48:09.971221+0000","last_fullsized":"2023-09-20T21:48:09.971221+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T03:35:44.455981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663990+0000","last_change":"2023-09-20T21:46:45.581110+0000","last_active":"2023-09-20T21:48:08.663990+0000","last_peered":"2023-09-20T21:48:08.663990+0000","last_clean":"2023-09-20T21:48:08.663990+0000","last_became_active":"2023-09-20T21:46:45.580498+0000","last_became_peered":"2023-09-20T21:46:45.580498+0000","last_unstale":"2023-09-20T21:48:08.663990+0000","last_undegraded":"2023-09-20T21:48:08.663990+0000","last_fullsized":"2023-09-20T21:48:08.663990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:19:22.576271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587563+0000","last_change":"2023-09-20T21:46:40.497017+0000","last_active":"2023-09-20T21:48:08.587563+0000","last_peered":"2023-09-20T21:48:08.587563+0000","last_clean":"2023-09-20T21:48:08.587563+0000","last_became_active":"2023-09-20T21:46:40.496722+0000","last_became_peered":"2023-09-20T21:46:40.496722+0000","last_unstale":"2023-09-20T21:48:08.587563+0000","last_undegraded":"2023-09-20T21:48:08.587563+0000","last_fullsized":"2023-09-20T21:48:08.587563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:12:07.017318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593744+0000","last_change":"2023-09-20T21:46:41.508793+0000","last_active":"2023-09-20T21:48:09.593744+0000","last_peered":"2023-09-20T21:48:09.593744+0000","last_clean":"2023-09-20T21:48:09.593744+0000","last_became_active":"2023-09-20T21:46:41.508453+0000","last_became_peered":"2023-09-20T21:46:41.508453+0000","last_unstale":"2023-09-20T21:48:09.593744+0000","last_undegraded":"2023-09-20T21:48:09.593744+0000","last_fullsized":"2023-09-20T21:48:09.593744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:35:12.711570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.585166+0000","last_change":"2023-09-20T21:46:45.583519+0000","last_active":"2023-09-20T21:48:08.585166+0000","last_peered":"2023-09-20T21:48:08.585166+0000","last_clean":"2023-09-20T21:48:08.585166+0000","last_became_active":"2023-09-20T21:46:45.582933+0000","last_became_peered":"2023-09-20T21:46:45.582933+0000","last_unstale":"2023-09-20T21:48:08.585166+0000","last_undegraded":"2023-09-20T21:48:08.585166+0000","last_fullsized":"2023-09-20T21:48:08.585166+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:13:20.237865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593456+0000","last_change":"2023-09-20T21:46:40.507384+0000","last_active":"2023-09-20T21:48:09.593456+0000","last_peered":"2023-09-20T21:48:09.593456+0000","last_clean":"2023-09-20T21:48:09.593456+0000","last_became_active":"2023-09-20T21:46:40.507021+0000","last_became_peered":"2023-09-20T21:46:40.507021+0000","last_unstale":"2023-09-20T21:48:09.593456+0000","last_undegraded":"2023-09-20T21:48:09.593456+0000","last_fullsized":"2023-09-20T21:48:09.593456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:48:04.360664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588413+0000","last_change":"2023-09-20T21:46:41.511537+0000","last_active":"2023-09-20T21:48:08.588413+0000","last_peered":"2023-09-20T21:48:08.588413+0000","last_clean":"2023-09-20T21:48:08.588413+0000","last_became_active":"2023-09-20T21:46:41.511238+0000","last_became_peered":"2023-09-20T21:46:41.511238+0000","last_unstale":"2023-09-20T21:48:08.588413+0000","last_undegraded":"2023-09-20T21:48:08.588413+0000","last_fullsized":"2023-09-20T21:48:08.588413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:01:41.492824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663746+0000","last_change":"2023-09-20T21:46:45.585750+0000","last_active":"2023-09-20T21:48:08.663746+0000","last_peered":"2023-09-20T21:48:08.663746+0000","last_clean":"2023-09-20T21:48:08.663746+0000","last_became_active":"2023-09-20T21:46:45.585213+0000","last_became_peered":"2023-09-20T21:46:45.585213+0000","last_unstale":"2023-09-20T21:48:08.663746+0000","last_undegraded":"2023-09-20T21:48:08.663746+0000","last_fullsized":"2023-09-20T21:48:08.663746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:31:15.650766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.602996+0000","last_change":"2023-09-20T21:48:09.602996+0000","last_active":"2023-09-20T21:48:09.602996+0000","last_peered":"2023-09-20T21:48:09.602996+0000","last_clean":"2023-09-20T21:48:09.602996+0000","last_became_active":"2023-09-20T21:48:09.598415+0000","last_became_peered":"2023-09-20T21:48:09.598415+0000","last_unstale":"2023-09-20T21:48:09.602996+0000","last_undegraded":"2023-09-20T21:48:09.602996+0000","last_fullsized":"2023-09-20T21:48:09.602996+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:39:45.484653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590150+0000","last_change":"2023-09-20T21:46:41.509491+0000","last_active":"2023-09-20T21:48:08.590150+0000","last_peered":"2023-09-20T21:48:08.590150+0000","last_clean":"2023-09-20T21:48:08.590150+0000","last_became_active":"2023-09-20T21:46:41.509039+0000","last_became_peered":"2023-09-20T21:46:41.509039+0000","last_unstale":"2023-09-20T21:48:08.590150+0000","last_undegraded":"2023-09-20T21:48:08.590150+0000","last_fullsized":"2023-09-20T21:48:08.590150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:57:39.003154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.663975+0000","last_change":"2023-09-20T21:46:45.581651+0000","last_active":"2023-09-20T21:48:08.663975+0000","last_peered":"2023-09-20T21:48:08.663975+0000","last_clean":"2023-09-20T21:48:08.663975+0000","last_became_active":"2023-09-20T21:46:45.581224+0000","last_became_peered":"2023-09-20T21:46:45.581224+0000","last_unstale":"2023-09-20T21:48:08.663975+0000","last_undegraded":"2023-09-20T21:48:08.663975+0000","last_fullsized":"2023-09-20T21:48:08.663975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T07:32:17.778301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588268+0000","last_change":"2023-09-20T21:47:08.610413+0000","last_active":"2023-09-20T21:48:08.588268+0000","last_peered":"2023-09-20T21:48:08.588268+0000","last_clean":"2023-09-20T21:48:08.588268+0000","last_became_active":"2023-09-20T21:47:08.603734+0000","last_became_peered":"2023-09-20T21:47:08.603734+0000","last_unstale":"2023-09-20T21:48:08.588268+0000","last_undegraded":"2023-09-20T21:48:08.588268+0000","last_fullsized":"2023-09-20T21:48:08.588268+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:43:42.569371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.714928+0000","last_change":"2023-09-20T21:46:41.507218+0000","last_active":"2023-09-20T21:48:09.714928+0000","last_peered":"2023-09-20T21:48:09.714928+0000","last_clean":"2023-09-20T21:48:09.714928+0000","last_became_active":"2023-09-20T21:46:41.506889+0000","last_became_peered":"2023-09-20T21:46:41.506889+0000","last_unstale":"2023-09-20T21:48:09.714928+0000","last_undegraded":"2023-09-20T21:48:09.714928+0000","last_fullsized":"2023-09-20T21:48:09.714928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T06:08:40.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.716076+0000","last_change":"2023-09-20T21:48:09.716076+0000","last_active":"2023-09-20T21:48:09.716076+0000","last_peered":"2023-09-20T21:48:09.716076+0000","last_clean":"2023-09-20T21:48:09.716076+0000","last_became_active":"2023-09-20T21:48:09.715521+0000","last_became_peered":"2023-09-20T21:48:09.715521+0000","last_unstale":"2023-09-20T21:48:09.716076+0000","last_undegraded":"2023-09-20T21:48:09.716076+0000","last_fullsized":"2023-09-20T21:48:09.716076+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:59:27.762388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":130,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:49.823237+0000","last_change":"2023-09-20T21:46:40.508774+0000","last_active":"2023-09-20T21:48:49.823237+0000","last_peered":"2023-09-20T21:48:49.823237+0000","last_clean":"2023-09-20T21:48:49.823237+0000","last_became_active":"2023-09-20T21:46:40.508480+0000","last_became_peered":"2023-09-20T21:46:40.508480+0000","last_unstale":"2023-09-20T21:48:49.823237+0000","last_undegraded":"2023-09-20T21:48:49.823237+0000","last_fullsized":"2023-09-20T21:48:49.823237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:11:22.643724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590718+0000","last_change":"2023-09-20T21:46:41.503763+0000","last_active":"2023-09-20T21:48:08.590718+0000","last_peered":"2023-09-20T21:48:08.590718+0000","last_clean":"2023-09-20T21:48:08.590718+0000","last_became_active":"2023-09-20T21:46:41.503360+0000","last_became_peered":"2023-09-20T21:46:41.503360+0000","last_unstale":"2023-09-20T21:48:08.590718+0000","last_undegraded":"2023-09-20T21:48:08.590718+0000","last_fullsized":"2023-09-20T21:48:08.590718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:03:00.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591725+0000","last_change":"2023-09-20T21:46:45.586702+0000","last_active":"2023-09-20T21:48:08.591725+0000","last_peered":"2023-09-20T21:48:08.591725+0000","last_clean":"2023-09-20T21:48:08.591725+0000","last_became_active":"2023-09-20T21:46:45.586177+0000","last_became_peered":"2023-09-20T21:46:45.586177+0000","last_unstale":"2023-09-20T21:48:08.591725+0000","last_undegraded":"2023-09-20T21:48:08.591725+0000","last_fullsized":"2023-09-20T21:48:08.591725+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:19:03.883440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.591133+0000","last_change":"2023-09-20T21:46:40.507270+0000","last_active":"2023-09-20T21:48:08.591133+0000","last_peered":"2023-09-20T21:48:08.591133+0000","last_clean":"2023-09-20T21:48:08.591133+0000","last_became_active":"2023-09-20T21:46:40.506933+0000","last_became_peered":"2023-09-20T21:46:40.506933+0000","last_unstale":"2023-09-20T21:48:08.591133+0000","last_undegraded":"2023-09-20T21:48:08.591133+0000","last_fullsized":"2023-09-20T21:48:08.591133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:34:59.854513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590595+0000","last_change":"2023-09-20T21:46:41.507868+0000","last_active":"2023-09-20T21:48:08.590595+0000","last_peered":"2023-09-20T21:48:08.590595+0000","last_clean":"2023-09-20T21:48:08.590595+0000","last_became_active":"2023-09-20T21:46:41.507568+0000","last_became_peered":"2023-09-20T21:46:41.507568+0000","last_unstale":"2023-09-20T21:48:08.590595+0000","last_undegraded":"2023-09-20T21:48:08.590595+0000","last_fullsized":"2023-09-20T21:48:08.590595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:42:44.615620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590490+0000","last_change":"2023-09-20T21:46:45.578726+0000","last_active":"2023-09-20T21:48:08.590490+0000","last_peered":"2023-09-20T21:48:08.590490+0000","last_clean":"2023-09-20T21:48:08.590490+0000","last_became_active":"2023-09-20T21:46:45.578326+0000","last_became_peered":"2023-09-20T21:46:45.578326+0000","last_unstale":"2023-09-20T21:48:08.590490+0000","last_undegraded":"2023-09-20T21:48:08.590490+0000","last_fullsized":"2023-09-20T21:48:08.590490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:44.552474+0000","last_clean_scrub_stamp":"2023-09-20T21:46:44.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:21:34.564840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587838+0000","last_change":"2023-09-20T21:46:40.509024+0000","last_active":"2023-09-20T21:48:08.587838+0000","last_peered":"2023-09-20T21:48:08.587838+0000","last_clean":"2023-09-20T21:48:08.587838+0000","last_became_active":"2023-09-20T21:46:40.508659+0000","last_became_peered":"2023-09-20T21:46:40.508659+0000","last_unstale":"2023-09-20T21:48:08.587838+0000","last_undegraded":"2023-09-20T21:48:08.587838+0000","last_fullsized":"2023-09-20T21:48:08.587838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:39.413748+0000","last_clean_scrub_stamp":"2023-09-20T21:46:39.413748+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T09:12:08.302068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590780+0000","last_change":"2023-09-20T21:46:41.510307+0000","last_active":"2023-09-20T21:48:08.590780+0000","last_peered":"2023-09-20T21:48:08.590780+0000","last_clean":"2023-09-20T21:48:08.590780+0000","last_became_active":"2023-09-20T21:46:41.510014+0000","last_became_peered":"2023-09-20T21:46:41.510014+0000","last_unstale":"2023-09-20T21:48:08.590780+0000","last_undegraded":"2023-09-20T21:48:08.590780+0000","last_fullsized":"2023-09-20T21:48:08.590780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:05:55.062515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.596849+0000","last_change":"2023-09-20T21:46:41.511525+0000","last_active":"2023-09-20T21:48:09.596849+0000","last_peered":"2023-09-20T21:48:09.596849+0000","last_clean":"2023-09-20T21:48:09.596849+0000","last_became_active":"2023-09-20T21:46:41.511203+0000","last_became_peered":"2023-09-20T21:46:41.511203+0000","last_unstale":"2023-09-20T21:48:09.596849+0000","last_undegraded":"2023-09-20T21:48:09.596849+0000","last_fullsized":"2023-09-20T21:48:09.596849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:42:15.757422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.596012+0000","last_change":"2023-09-20T21:46:41.502936+0000","last_active":"2023-09-20T21:48:09.596012+0000","last_peered":"2023-09-20T21:48:09.596012+0000","last_clean":"2023-09-20T21:48:09.596012+0000","last_became_active":"2023-09-20T21:46:41.502617+0000","last_became_peered":"2023-09-20T21:46:41.502617+0000","last_unstale":"2023-09-20T21:48:09.596012+0000","last_undegraded":"2023-09-20T21:48:09.596012+0000","last_fullsized":"2023-09-20T21:48:09.596012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:45:21.197652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.593522+0000","last_change":"2023-09-20T21:46:41.510766+0000","last_active":"2023-09-20T21:48:09.593522+0000","last_peered":"2023-09-20T21:48:09.593522+0000","last_clean":"2023-09-20T21:48:09.593522+0000","last_became_active":"2023-09-20T21:46:41.510433+0000","last_became_peered":"2023-09-20T21:46:41.510433+0000","last_unstale":"2023-09-20T21:48:09.593522+0000","last_undegraded":"2023-09-20T21:48:09.593522+0000","last_fullsized":"2023-09-20T21:48:09.593522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T21:56:12.085926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.970963+0000","last_change":"2023-09-20T21:46:41.508846+0000","last_active":"2023-09-20T21:48:09.970963+0000","last_peered":"2023-09-20T21:48:09.970963+0000","last_clean":"2023-09-20T21:48:09.970963+0000","last_became_active":"2023-09-20T21:46:41.508518+0000","last_became_peered":"2023-09-20T21:46:41.508518+0000","last_unstale":"2023-09-20T21:48:09.970963+0000","last_undegraded":"2023-09-20T21:48:09.970963+0000","last_fullsized":"2023-09-20T21:48:09.970963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T02:01:58.195327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588468+0000","last_change":"2023-09-20T21:46:41.505929+0000","last_active":"2023-09-20T21:48:08.588468+0000","last_peered":"2023-09-20T21:48:08.588468+0000","last_clean":"2023-09-20T21:48:08.588468+0000","last_became_active":"2023-09-20T21:46:41.505542+0000","last_became_peered":"2023-09-20T21:46:41.505542+0000","last_unstale":"2023-09-20T21:48:08.588468+0000","last_undegraded":"2023-09-20T21:48:08.588468+0000","last_fullsized":"2023-09-20T21:48:08.588468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:24:47.520344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587952+0000","last_change":"2023-09-20T21:46:41.507603+0000","last_active":"2023-09-20T21:48:08.587952+0000","last_peered":"2023-09-20T21:48:08.587952+0000","last_clean":"2023-09-20T21:48:08.587952+0000","last_became_active":"2023-09-20T21:46:41.507340+0000","last_became_peered":"2023-09-20T21:46:41.507340+0000","last_unstale":"2023-09-20T21:48:08.587952+0000","last_undegraded":"2023-09-20T21:48:08.587952+0000","last_fullsized":"2023-09-20T21:48:08.587952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T23:50:19.756491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595430+0000","last_change":"2023-09-20T21:46:41.509625+0000","last_active":"2023-09-20T21:48:09.595430+0000","last_peered":"2023-09-20T21:48:09.595430+0000","last_clean":"2023-09-20T21:48:09.595430+0000","last_became_active":"2023-09-20T21:46:41.509301+0000","last_became_peered":"2023-09-20T21:46:41.509301+0000","last_unstale":"2023-09-20T21:48:09.595430+0000","last_undegraded":"2023-09-20T21:48:09.595430+0000","last_fullsized":"2023-09-20T21:48:09.595430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:15:03.108060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-09-20T21:48:09.595097+0000","last_change":"2023-09-20T21:46:41.516781+0000","last_active":"2023-09-20T21:48:09.595097+0000","last_peered":"2023-09-20T21:48:09.595097+0000","last_clean":"2023-09-20T21:48:09.595097+0000","last_became_active":"2023-09-20T21:46:41.516496+0000","last_became_peered":"2023-09-20T21:46:41.516496+0000","last_unstale":"2023-09-20T21:48:09.595097+0000","last_undegraded":"2023-09-20T21:48:09.595097+0000","last_fullsized":"2023-09-20T21:48:09.595097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:17:26.492851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590654+0000","last_change":"2023-09-20T21:46:41.499927+0000","last_active":"2023-09-20T21:48:08.590654+0000","last_peered":"2023-09-20T21:48:08.590654+0000","last_clean":"2023-09-20T21:48:08.590654+0000","last_became_active":"2023-09-20T21:46:41.499605+0000","last_became_peered":"2023-09-20T21:46:41.499605+0000","last_unstale":"2023-09-20T21:48:08.590654+0000","last_undegraded":"2023-09-20T21:48:08.590654+0000","last_fullsized":"2023-09-20T21:48:08.590654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T01:18:16.306349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.590204+0000","last_change":"2023-09-20T21:46:41.515842+0000","last_active":"2023-09-20T21:48:08.590204+0000","last_peered":"2023-09-20T21:48:08.590204+0000","last_clean":"2023-09-20T21:48:08.590204+0000","last_became_active":"2023-09-20T21:46:41.515600+0000","last_became_peered":"2023-09-20T21:46:41.515600+0000","last_unstale":"2023-09-20T21:48:08.590204+0000","last_undegraded":"2023-09-20T21:48:08.590204+0000","last_fullsized":"2023-09-20T21:48:08.590204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T08:50:58.088103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.588120+0000","last_change":"2023-09-20T21:46:41.510695+0000","last_active":"2023-09-20T21:48:08.588120+0000","last_peered":"2023-09-20T21:48:08.588120+0000","last_clean":"2023-09-20T21:48:08.588120+0000","last_became_active":"2023-09-20T21:46:41.510345+0000","last_became_peered":"2023-09-20T21:46:41.510345+0000","last_unstale":"2023-09-20T21:48:08.588120+0000","last_undegraded":"2023-09-20T21:48:08.588120+0000","last_fullsized":"2023-09-20T21:48:08.588120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-21T22:38:24.201382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587592+0000","last_change":"2023-09-20T21:46:41.507809+0000","last_active":"2023-09-20T21:48:08.587592+0000","last_peered":"2023-09-20T21:48:08.587592+0000","last_clean":"2023-09-20T21:48:08.587592+0000","last_became_active":"2023-09-20T21:46:41.507513+0000","last_became_peered":"2023-09-20T21:46:41.507513+0000","last_unstale":"2023-09-20T21:48:08.587592+0000","last_undegraded":"2023-09-20T21:48:08.587592+0000","last_fullsized":"2023-09-20T21:48:08.587592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T05:30:43.132728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-09-20T21:48:08.587980+0000","last_change":"2023-09-20T21:46:41.503590+0000","last_active":"2023-09-20T21:48:08.587980+0000","last_peered":"2023-09-20T21:48:08.587980+0000","last_clean":"2023-09-20T21:48:08.587980+0000","last_became_active":"2023-09-20T21:46:41.503295+0000","last_became_peered":"2023-09-20T21:46:41.503295+0000","last_unstale":"2023-09-20T21:48:08.587980+0000","last_undegraded":"2023-09-20T21:48:08.587980+0000","last_fullsized":"2023-09-20T21:48:08.587980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-20T21:46:40.479611+0000","last_clean_scrub_stamp":"2023-09-20T21:46:40.479611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-22T00:28:33.088873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10522,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1058,"num_read_kb":547,"num_write":211,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":31566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":185,"ondisk_log_size":185,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383719,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":516303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.38500000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83099999999999996,"5min":2.923,"15min":2.923},"last":0.53000000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.43099999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.80400000000000005,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.628},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73399999999999999,"5min":0.872,"15min":0.872},"last":0.64500000000000002}]},{"osd":2,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.49299999999999999,"15min":0.498},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.68700000000000006,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.29299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.79800000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.55800000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83099999999999996,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42799999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.81799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72399999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.60399999999999998,"15min":0.61799999999999999},"min":{"1min":0.32700000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.88500000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.64200000000000002,"15min":0.65500000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77700000000000002,"5min":1.351,"15min":1.351},"last":0.73599999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64900000000000002,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.40799999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.76400000000000001}]},{"osd":7,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.53800000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88100000000000001,"5min":1.246,"15min":1.246},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.54800000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89800000000000002,"5min":1.117,"15min":1.117},"last":0.80000000000000004}]},{"osd":8,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76500000000000001,"15min":0.75700000000000001},"min":{"1min":0.52400000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.69799999999999995}]},{"osd":9,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.82999999999999996,"15min":0.84699999999999998},"min":{"1min":0.52800000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95099999999999996,"5min":1.992,"15min":1.992},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82499999999999996,"15min":0.82999999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.99299999999999999,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83399999999999996}]},{"osd":10,"last update":"Wed Sep 20 21:47:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.61699999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92300000000000004,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.751},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76500000000000001,"15min":0.77100000000000002},"min":{"1min":0.60499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.91600000000000004,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.71199999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1111315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":1.0169999999999999,"15min":1.093},"min":{"1min":0.433,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76100000000000001,"5min":7.9690000000000003,"15min":7.9690000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.85499999999999998,"15min":0.90900000000000003},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81399999999999995,"5min":5.6280000000000001,"15min":5.6280000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46899999999999997,"15min":0.47899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.60299999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.34699999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80000000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46999999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79900000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67000000000000004,"15min":0.68100000000000005},"min":{"1min":0.49299999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.748,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.48399999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.82399999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60799999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78100000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.72799999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.68000000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.79000000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68200000000000005,"15min":0.69699999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.62}]},{"osd":6,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73499999999999999,"15min":0.76100000000000001},"min":{"1min":0.47099999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.873,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.89700000000000002,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90100000000000002,"5min":8.0440000000000005,"15min":8.0440000000000005},"last":0.78300000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:46:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94799999999999995,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.75800000000000001,"15min":0.77700000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89700000000000002,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72099999999999997}]},{"osd":9,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.89800000000000002,"15min":0.85999999999999999},"min":{"1min":0.67300000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.93200000000000005,"5min":5.6449999999999996,"15min":5.6449999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.89500000000000002,"15min":0.86799999999999999},"min":{"1min":0.61199999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.88400000000000001,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":0.74199999999999999}]},{"osd":11,"last update":"Wed Sep 20 21:47:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76600000000000001,"15min":0.76500000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.77100000000000002}]}]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":516233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1598,"internal_metadata":27654594},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.622,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":2.7269999999999999,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.69699999999999995,"15min":0.68200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.1130000000000004,"5min":4.1130000000000004,"15min":4.1130000000000004},"last":0.46400000000000002}]},{"osd":1,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5289999999999999,"5min":4.6799999999999997,"15min":4.6799999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.78700000000000003,"15min":0.81599999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.55400000000000005}]},{"osd":2,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.996,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.56200000000000006,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.417,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.70699999999999996}]},{"osd":3,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71899999999999997,"15min":0.69499999999999995},"min":{"1min":0.41499999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.8170000000000002,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.82499999999999996,"15min":0.84499999999999997},"min":{"1min":0.47699999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.7690000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.627}]},{"osd":4,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.94899999999999995,"15min":0.93500000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.798,"5min":6.3449999999999998,"15min":6.3449999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.86099999999999999,"15min":0.86599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.5920000000000001,"5min":4.7279999999999998,"15min":4.7279999999999998},"last":0.58399999999999996}]},{"osd":5,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.94499999999999995,"15min":0.98899999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4970000000000001,"5min":2.6469999999999998,"15min":2.6469999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.92100000000000004,"15min":0.93000000000000005},"min":{"1min":0.54800000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.671,"5min":4.7679999999999998,"15min":4.7679999999999998},"last":0.76600000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:46:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.95799999999999996,"15min":0.96599999999999997},"min":{"1min":0.63400000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6279999999999999,"5min":4.806,"15min":4.806},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.96599999999999997,"15min":1.0189999999999999},"min":{"1min":0.67400000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1000000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0629999999999999,"15min":1.119},"min":{"1min":0.67300000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.744,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0389999999999999,"15min":1.0600000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.702,"5min":4.843,"15min":4.843},"last":0.85299999999999998}]},{"osd":10,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":1.0069999999999999,"15min":1.075},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.2150000000000001,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":0.749},{"interface":"front","average":{"1min":0.86299999999999999,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.2470000000000001,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.64100000000000001}]},{"osd":11,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.79400000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":516611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.441,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":0.52500000000000002}]},{"osd":1,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.497,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.75800000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.46200000000000002,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.625},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.48099999999999998,"15min":0.46300000000000002},"min":{"1min":0.38300000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.53900000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.67500000000000004,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.40600000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.46300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.93400000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.76600000000000001,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.82499999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.57699999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0109999999999999,"5min":1.548,"15min":1.548},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.57499999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.90400000000000003}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.79400000000000004,"15min":0.76800000000000002},"min":{"1min":0.69099999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.81000000000000005,"15min":0.78300000000000003},"min":{"1min":0.73599999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89300000000000002}]},{"osd":10,"last update":"Wed Sep 20 21:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.78000000000000003,"15min":0.745},"min":{"1min":0.69299999999999995,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.80500000000000005,"15min":0.77100000000000002},"min":{"1min":0.72399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.077,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.91700000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81899999999999995,"15min":0.81399999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84499999999999997,"15min":0.84199999999999997},"min":{"1min":0.70699999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":0.97999999999999998,"5min":1.038,"15min":1.038},"last":0.93700000000000006}]}]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1108879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:25 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.39600000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.51600000000000001}]},{"osd":1,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.43099999999999999,"15min":0.42199999999999999},"min":{"1min":0.36899999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45300000000000001,"15min":0.438},"min":{"1min":0.34100000000000003,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.442}]},{"osd":2,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76900000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60599999999999998,"15min":0.57999999999999996},"min":{"1min":0.51700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81799999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.61799999999999999,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.89400000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63100000000000001,"15min":0.60099999999999998},"min":{"1min":0.496,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51900000000000002,"15min":0.48899999999999999},"min":{"1min":0.35499999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71399999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.501,"15min":0.49099999999999999},"min":{"1min":0.35599999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70399999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.498}]},{"osd":5,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63,"15min":0.62},"min":{"1min":0.495,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81000000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67100000000000004,"15min":0.63700000000000001},"min":{"1min":0.54100000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96099999999999997,"5min":1.006,"15min":1.006},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.69999999999999996,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0069999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.66000000000000003},"min":{"1min":0.53700000000000003,"5min":0.435,"15min":0.435},"max":{"1min":0.85399999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63600000000000001}]},{"osd":8,"last update":"Wed Sep 20 21:48:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85699999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.63500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90000000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78300000000000003}]},{"osd":9,"last update":"Wed Sep 20 21:48:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.629,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77700000000000002,"15min":0.77600000000000002},"min":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.90500000000000003,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.76200000000000001}]},{"osd":10,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.52800000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91500000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.48599999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83299999999999996,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69499999999999995}]},{"osd":11,"last update":"Wed Sep 20 21:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83999999999999997,"15min":0.84499999999999997},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.93000000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.86499999999999999,"15min":0.871},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":0.97099999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.84299999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":515833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.5,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81999999999999995,"5min":1.018,"15min":1.018},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66500000000000004,"15min":0.629},"min":{"1min":0.438,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.04,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.82699999999999996}]},{"osd":1,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79600000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65200000000000002,"15min":0.63},"min":{"1min":0.41699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":1.2,"15min":1.2},"last":0.92000000000000004}]},{"osd":2,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.873,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72399999999999998,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.99399999999999999,"5min":1.466,"15min":1.466},"last":0.873}]},{"osd":3,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.89500000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69899999999999995,"15min":0.72299999999999998},"min":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.048,"5min":1.093,"15min":1.093},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0720000000000001,"5min":1.407,"15min":1.407},"last":0.98899999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:48:04 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.82799999999999996,"15min":0.80800000000000005},"min":{"1min":0.59999999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.749,"15min":0.73899999999999999},"min":{"1min":0.51800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Wed Sep 20 21:48:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84999999999999998,"15min":0.83799999999999997},"min":{"1min":0.5,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2989999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.85099999999999998},"min":{"1min":0.64800000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.214,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0109999999999999}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.82599999999999996,"15min":0.85299999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.85899999999999999,"15min":0.86499999999999999},"min":{"1min":0.55900000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.206,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0329999999999999}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.92400000000000004,"15min":0.94599999999999995},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.1899999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.95599999999999996,"15min":0.98099999999999998},"min":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1619999999999999,"5min":1.333,"15min":1.333},"last":1.0489999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.93899999999999995,"15min":0.94699999999999995},"min":{"1min":0.63600000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2370000000000001,"5min":1.302,"15min":1.302},"last":1.077},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.96799999999999997,"15min":0.97899999999999998},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.2589999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.071}]},{"osd":11,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.0920000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":520705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.59399999999999997,"15min":0.54300000000000004},"min":{"1min":0.316,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.9539999999999997,"5min":4.9539999999999997,"15min":4.9539999999999997},"last":0.747},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.53700000000000003},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69599999999999995,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56000000000000005,"15min":0.51400000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.82299999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54400000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58699999999999997,"15min":0.53700000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.748,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.626}]},{"osd":4,"last update":"Wed Sep 20 21:46:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.41799999999999998,"15min":0.39400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.41899999999999998,"15min":0.39300000000000002},"min":{"1min":0.33300000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Wed Sep 20 21:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.58599999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.84299999999999997,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.499},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64700000000000002,"15min":0.60499999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.878,"5min":1.01,"15min":1.01},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Sep 20 21:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.70099999999999996,"15min":0.68200000000000005},"min":{"1min":0.44600000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":1.27,"15min":1.27},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed Sep 20 21:48:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77300000000000002,"5min":1.218,"15min":1.218},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.41299999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:47:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":1.321,"15min":1.321},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70799999999999996,"15min":0.69199999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67500000000000004}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73799999999999999,"15min":0.751},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85299999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96999999999999997,"5min":1.054,"15min":1.054},"last":0.73399999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.72799999999999998,"15min":0.74199999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91200000000000003,"5min":1,"15min":1},"last":0.443},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.435,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93999999999999995,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.63800000000000001}]},{"osd":11,"last update":"Wed Sep 20 21:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.78700000000000003,"15min":0.79100000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95199999999999996,"5min":1.018,"15min":1.018},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.80600000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92700000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.70299999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":517289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 20 21:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":0.755,"15min":0.66700000000000004},"min":{"1min":0.45400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":7.0289999999999999,"5min":7.0289999999999999,"15min":7.0289999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.76700000000000002,"15min":0.67400000000000004},"min":{"1min":0.53800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.4390000000000001,"5min":6.4390000000000001,"15min":6.4390000000000001},"last":0.76000000000000001}]},{"osd":2,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72999999999999998,"15min":0.65400000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.7589999999999999,"5min":8.1639999999999997,"15min":8.1639999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73899999999999999,"15min":0.67500000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97199999999999998,"5min":6.5270000000000001,"15min":6.5270000000000001},"last":0.71899999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.97399999999999998,"15min":1.038},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88500000000000001,"5min":8.9499999999999993,"15min":8.9499999999999993},"last":0.623},{"interface":"front","average":{"1min":0.63,"5min":0.88600000000000001,"15min":0.92800000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2110000000000001,"5min":8.6620000000000008,"15min":8.6620000000000008},"last":0.83699999999999997}]},{"osd":4,"last update":"Wed Sep 20 21:48:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":0.80100000000000005,"15min":0.68700000000000006},"min":{"1min":0.45900000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":8.2560000000000002,"5min":8.2560000000000002,"15min":8.2560000000000002},"last":0.441},{"interface":"front","average":{"1min":1.123,"5min":0.80800000000000005,"15min":0.67200000000000004},"min":{"1min":0.47999999999999998,"5min":0.443,"15min":0.443},"max":{"1min":6.5650000000000004,"5min":6.5650000000000004,"15min":6.5650000000000004},"last":0.81299999999999994}]},{"osd":5,"last update":"Wed Sep 20 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84199999999999997,"15min":0.73299999999999998},"min":{"1min":0.47299999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.792,"5min":8.2919999999999998,"15min":8.2919999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80400000000000005,"15min":0.70799999999999996},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.095,"5min":6.601,"15min":6.601},"last":0.78700000000000003}]},{"osd":6,"last update":"Wed Sep 20 21:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":0.85099999999999998,"15min":0.72199999999999998},"min":{"1min":0.505,"5min":0.375,"15min":0.375},"max":{"1min":8.9049999999999994,"5min":8.9049999999999994,"15min":8.9049999999999994},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":0.79700000000000004,"15min":0.74299999999999999},"min":{"1min":0.49199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":8.3659999999999997,"5min":8.3659999999999997,"15min":8.3659999999999997},"last":0.67000000000000004}]},{"osd":7,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":6.6440000000000001,"5min":6.6440000000000001,"15min":6.6440000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":8.2159999999999993,"5min":8.2159999999999993,"15min":8.2159999999999993},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":0.93999999999999995,"15min":0.86799999999999999},"min":{"1min":0.628,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":8.3290000000000006,"5min":8.3290000000000006,"15min":8.3290000000000006},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.1990000000000001,"5min":0.96099999999999997,"15min":0.93500000000000005},"min":{"1min":0.79600000000000004,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":6.6799999999999997,"5min":6.6799999999999997,"15min":6.6799999999999997},"last":0.89700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:48:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.0129999999999999,"15min":0.96199999999999997},"min":{"1min":0.66400000000000003,"5min":0.502,"15min":0.502},"max":{"1min":8.9879999999999995,"5min":8.9879999999999995,"15min":8.9879999999999995},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.2869999999999999,"5min":0.88300000000000001,"15min":0.83199999999999996},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":8.702,"5min":8.702,"15min":8.702},"last":0.879}]},{"osd":10,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.0249999999999999,"15min":0.96899999999999997},"min":{"1min":0.73099999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.5,"5min":6.7149999999999999,"15min":6.7149999999999999},"last":1.04},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.90100000000000002,"15min":0.83699999999999997},"min":{"1min":0.61099999999999999,"5min":0.498,"15min":0.498},"max":{"1min":1.931,"5min":6.8520000000000003,"15min":6.8520000000000003},"last":0.92500000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":0.877,"15min":0.82699999999999996},"min":{"1min":0.58999999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.0669999999999999,"15min":0.98599999999999999},"min":{"1min":0.85399999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":1.0629999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":520925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.55600000000000005},"min":{"1min":0.373,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.872,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64700000000000002,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94499999999999995,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.53100000000000003}]},{"osd":1,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64800000000000002,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83899999999999997,"5min":7.0330000000000004,"15min":7.0330000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.57699999999999996,"15min":0.52600000000000002},"min":{"1min":0.41799999999999998,"5min":0.33000000000000002,"15min":0.307},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.66800000000000004}]},{"osd":3,"last update":"Wed Sep 20 21:47:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84599999999999997,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.505},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66800000000000004,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90900000000000003,"5min":3.645,"15min":3.645},"last":0.40100000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.82099999999999995,"15min":0.84299999999999997},"min":{"1min":0.44700000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88800000000000001,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.749,"5min":1.181,"15min":1.2529999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0469999999999999,"5min":9.8659999999999997,"15min":9.8659999999999997},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.42099999999999999},"min":{"1min":0.34499999999999997,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73799999999999999,"5min":3.798,"15min":3.798},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60199999999999998,"15min":0.51000000000000001},"min":{"1min":0.40300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.80000000000000004,"5min":7.1500000000000004,"15min":7.1500000000000004},"last":0.44700000000000001}]},{"osd":6,"last update":"Wed Sep 20 21:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71199999999999997,"15min":0.64200000000000002},"min":{"1min":0.55700000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86299999999999999,"5min":7.367,"15min":7.367},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.754,"15min":0.67400000000000004},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.018,"5min":6.4649999999999999,"15min":6.4649999999999999},"last":0.69899999999999995}]},{"osd":7,"last update":"Wed Sep 20 21:48:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.58499999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.054,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.873},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.54900000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60299999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.40100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.63300000000000001},"min":{"1min":0.44900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95699999999999996,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.65200000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:48:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78800000000000003,"15min":0.72999999999999998},"min":{"1min":0.59599999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.99399999999999999,"5min":7.1200000000000001,"15min":7.1200000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.82699999999999996,"15min":0.78500000000000003},"min":{"1min":0.59299999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.137,"5min":3.7690000000000001,"15min":3.7690000000000001},"last":0.81699999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:48:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80500000000000005,"15min":0.72299999999999998},"min":{"1min":0.626,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0600000000000001,"5min":7.1799999999999997,"15min":7.1799999999999997},"last":0.747},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.745,"15min":0.71299999999999997},"min":{"1min":0.50600000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0309999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.54300000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.143,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.0940000000000001,"15min":1.139},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.099,"5min":7.2110000000000003,"15min":7.2110000000000003},"last":0.78100000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":515541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.47999999999999998,"15min":0.40600000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.50800000000000001,"15min":0.42999999999999999},"min":{"1min":0.34300000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":0.504}]},{"osd":1,"last update":"Wed Sep 20 21:48:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.63800000000000001,"15min":0.57799999999999996},"min":{"1min":0.432,"5min":0.376,"15min":0.376},"max":{"1min":2.9910000000000001,"5min":2.9910000000000001,"15min":2.9910000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61299999999999999,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.73699999999999999}]},{"osd":2,"last update":"Wed Sep 20 21:48:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65800000000000003,"15min":0.60999999999999999},"min":{"1min":0.42899999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0329999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed Sep 20 21:48:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67800000000000005,"15min":0.63400000000000001},"min":{"1min":0.504,"5min":0.436,"15min":0.436},"max":{"1min":0.877,"5min":1.601,"15min":1.601},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.754,"15min":0.66700000000000004},"min":{"1min":0.5,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93100000000000005,"5min":2.9009999999999998,"15min":2.9009999999999998},"last":0.93999999999999995}]},{"osd":5,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71099999999999997,"15min":0.69599999999999995},"min":{"1min":0.502,"5min":0.434,"15min":0.434},"max":{"1min":0.83199999999999996,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77400000000000002,"15min":0.752},"min":{"1min":0.52700000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92900000000000005,"5min":1.167,"15min":1.167},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed Sep 20 21:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.69599999999999995,"15min":0.65000000000000002},"min":{"1min":0.50900000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.73499999999999999,"15min":0.71199999999999997},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Wed Sep 20 21:48:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78000000000000003,"15min":0.77600000000000002},"min":{"1min":0.56699999999999995,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96699999999999997,"5min":1.248,"15min":1.248},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.82799999999999996,"15min":0.81899999999999995},"min":{"1min":0.64700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98499999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.97699999999999998}]},{"osd":8,"last update":"Wed Sep 20 21:45:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Sep 20 21:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91300000000000003,"15min":0.94299999999999995},"min":{"1min":0.65000000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.1819999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.92200000000000004,"15min":0.93899999999999995},"min":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":0.74299999999999999}]},{"osd":10,"last update":"Wed Sep 20 21:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90000000000000002,"15min":0.91300000000000003},"min":{"1min":0.73399999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0389999999999999,"5min":1.228,"15min":1.228},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.071,"5min":0.96099999999999997,"15min":0.93700000000000006},"min":{"1min":0.64000000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"last":0.90400000000000003}]},{"osd":11,"last update":"Wed Sep 20 21:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.109,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.879},{"interface":"front","average":{"1min":0.874,"5min":0.99199999999999999,"15min":1.012},"min":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.0109999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.81599999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":516069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:46:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.40400000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.45300000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 20 21:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.41399999999999998,"15min":0.37},"min":{"1min":0.314,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59999999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.38200000000000001},"min":{"1min":0.308,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.57599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47899999999999998}]},{"osd":2,"last update":"Wed Sep 20 21:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.59999999999999998,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.53800000000000003},"min":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69099999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56000000000000005}]},{"osd":3,"last update":"Wed Sep 20 21:48:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59099999999999997,"15min":0.56499999999999995},"min":{"1min":0.42999999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.57899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91300000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed Sep 20 21:48:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.621,"15min":0.57999999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.626},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.59799999999999998},"min":{"1min":0.503,"5min":0.379,"15min":0.379},"max":{"1min":0.81000000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Wed Sep 20 21:48:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":0.71899999999999997,"15min":0.61299999999999999},"min":{"1min":0.47399999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.5129999999999999,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.69299999999999995,"15min":0.65500000000000003},"min":{"1min":0.48999999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Sep 20 21:48:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.54100000000000004},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84299999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59799999999999998,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79500000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.53700000000000003}]},{"osd":8,"last update":"Wed Sep 20 21:47:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.54200000000000004,"5min":0.371,"15min":0.371},"max":{"1min":0.89500000000000002,"5min":1,"15min":1},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.49299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.84999999999999998}]},{"osd":9,"last update":"Wed Sep 20 21:48:17 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.83099999999999996,"15min":0.80400000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.109,"5min":4.109,"15min":4.109},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.78200000000000003,"15min":0.76300000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":0.81899999999999995}]},{"osd":10,"last update":"Wed Sep 20 21:48:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.749},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.78600000000000003,"15min":0.79800000000000004},"min":{"1min":0.48899999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97399999999999998,"5min":1.048,"15min":1.048},"last":0.67000000000000004}]},{"osd":11,"last update":"Wed Sep 20 21:48:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84599999999999997,"15min":0.85299999999999998},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.006,"5min":1.115,"15min":1.115},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.63300000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.94599999999999995,"5min":1.008,"15min":1.008},"last":0.92400000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1108018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.087,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.1240000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.56299999999999994}]},{"osd":1,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56799999999999995,"15min":0.54100000000000004},"min":{"1min":0.41899999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.70999999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58599999999999997,"15min":0.53400000000000003},"min":{"1min":0.36499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55200000000000005}]},{"osd":2,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.378},"min":{"1min":0.30599999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58099999999999996,"5min":0.755,"15min":0.755},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.433,"15min":0.39900000000000002},"min":{"1min":0.311,"5min":0.255,"15min":0.255},"max":{"1min":0.66600000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.40699999999999997}]},{"osd":3,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.436},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73399999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.57299999999999995},"min":{"1min":0.44700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80000000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.59699999999999998},"min":{"1min":0.45900000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77600000000000002,"5min":1.008,"15min":1.008},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed Sep 20 21:48:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.65500000000000003},"min":{"1min":0.54500000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84799999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.64900000000000002,"15min":0.626},"min":{"1min":0.48799999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.627}]},{"osd":7,"last update":"Wed Sep 20 21:48:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.87,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67500000000000004,"15min":0.66500000000000004},"min":{"1min":0.47499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81599999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.57999999999999996}]},{"osd":8,"last update":"Wed Sep 20 21:47:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":0.879,"15min":0.879},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.51800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53800000000000003}]},{"osd":9,"last update":"Wed Sep 20 21:48:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.79700000000000004,"15min":0.79800000000000004},"min":{"1min":0.626,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.68400000000000005}]},{"osd":10,"last update":"Wed Sep 20 21:48:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.751,"15min":0.73699999999999999},"min":{"1min":0.52100000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93400000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.89200000000000002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":5836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-20T21:48:54.922 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:48:55.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.159+0000 7f9c01dff700 1 -- 172.21.15.22:0/154567574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bfc100df0 msgr2=0x7f9bfc101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:55.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.159+0000 7f9c01dff700 1 --2- 172.21.15.22:0/154567574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bfc100df0 0x7f9bfc101210 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0099c0 tx=0x7f9bec009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.159+0000 7f9c01dff700 1 -- 172.21.15.22:0/154567574 shutdown_connections 2023-09-20T21:48:55.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.159+0000 7f9c01dff700 1 --2- 172.21.15.22:0/154567574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bfc102980 0x7f9bfc109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.159+0000 7f9c01dff700 1 --2- 172.21.15.22:0/154567574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bfc100df0 0x7f9bfc101210 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.159+0000 7f9c01dff700 1 --2- 172.21.15.22:0/154567574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc101fe0 0x7f9bfc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.159+0000 7f9c01dff700 1 -- 172.21.15.22:0/154567574 >> 172.21.15.22:0/154567574 conn(0x7f9bfc0fc320 msgr2=0x7f9bfc0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:55.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.159+0000 7f9c01dff700 1 -- 172.21.15.22:0/154567574 shutdown_connections 2023-09-20T21:48:55.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.159+0000 7f9c01dff700 1 -- 172.21.15.22:0/154567574 wait complete. 2023-09-20T21:48:55.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.160+0000 7f9c01dff700 1 Processor -- start 2023-09-20T21:48:55.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.160+0000 7f9c01dff700 1 -- start start 2023-09-20T21:48:55.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.160+0000 7f9c01dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bfc100df0 0x7f9bfc196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:55.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9c01dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bfc101fe0 0x7f9bfc196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:55.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9c01dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc102980 0x7f9bfc19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:55.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9c01dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc19e630 con 0x7f9bfc100df0 2023-09-20T21:48:55.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9c01dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc19e770 con 0x7f9bfc102980 2023-09-20T21:48:55.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9c01dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc19e900 con 0x7f9bfc101fe0 2023-09-20T21:48:55.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bfc100df0 0x7f9bfc196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:55.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bfc100df0 0x7f9bfc196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59756/0 (socket says 172.21.15.22:59756) 2023-09-20T21:48:55.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2093152591 learned_addr learned my addr 172.21.15.22:0/2093152591 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:55.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9bfbfff700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc102980 0x7f9bfc19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:55.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9bfaffd700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bfc101fe0 0x7f9bfc196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:55.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2093152591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bfc101fe0 msgr2=0x7f9bfc196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:55.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9bfb7fe700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bfc101fe0 0x7f9bfc196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2093152591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc102980 msgr2=0x7f9bfc19e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:55.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9bfb7fe700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc102980 0x7f9bfc19e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.161+0000 7f9bfb7fe700 1 -- 172.21.15.22:0/2093152591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf4009c40 con 0x7f9bfc100df0 2023-09-20T21:48:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.162+0000 7f9bfb7fe700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bfc100df0 0x7f9bfc196ab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0058e0 tx=0x7f9bec00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.162+0000 7f9bf8ff9700 1 -- 172.21.15.22:0/2093152591 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec01d070 con 0x7f9bfc100df0 2023-09-20T21:48:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.162+0000 7f9c01dff700 1 -- 172.21.15.22:0/2093152591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bec0095f0 con 0x7f9bfc100df0 2023-09-20T21:48:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.162+0000 7f9bf8ff9700 1 -- 172.21.15.22:0/2093152591 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bec005510 con 0x7f9bfc100df0 2023-09-20T21:48:55.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.162+0000 7f9bf8ff9700 1 -- 172.21.15.22:0/2093152591 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec004a90 con 0x7f9bfc100df0 2023-09-20T21:48:55.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.162+0000 7f9c01dff700 1 -- 172.21.15.22:0/2093152591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc19ee10 con 0x7f9bfc100df0 2023-09-20T21:48:55.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.163+0000 7f9c01dff700 1 -- 172.21.15.22:0/2093152591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc107d20 con 0x7f9bfc100df0 2023-09-20T21:48:55.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.165+0000 7f9bf8ff9700 1 -- 172.21.15.22:0/2093152591 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f9bec004bf0 con 0x7f9bfc100df0 2023-09-20T21:48:55.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.166+0000 7f9bf8ff9700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9be40a1350 0x7f9be40a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:55.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.166+0000 7f9bf8ff9700 1 -- 172.21.15.22:0/2093152591 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f9bec0ce1d0 con 0x7f9bfc100df0 2023-09-20T21:48:55.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.166+0000 7f9bfaffd700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9be40a1350 0x7f9be40a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:55.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.167+0000 7f9bfaffd700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9be40a1350 0x7f9be40a3810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009c60 tx=0x7f9bf0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:55.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.168+0000 7f9bf8ff9700 1 -- 172.21.15.22:0/2093152591 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f9bec097660 con 0x7f9bfc100df0 2023-09-20T21:48:55.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.380+0000 7f9c01dff700 1 -- 172.21.15.22:0/2093152591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bfc10c7d0 con 0x7f9bfc100df0 2023-09-20T21:48:55.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.382+0000 7f9bf8ff9700 1 -- 172.21.15.22:0/2093152591 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7f9bec09ac80 con 0x7f9bfc100df0 2023-09-20T21:48:55.383 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:55.383 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:48:55.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.384+0000 7f9c01dff700 1 -- 172.21.15.22:0/2093152591 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9be40a1350 msgr2=0x7f9be40a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:55.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.384+0000 7f9c01dff700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9be40a1350 0x7f9be40a3810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009c60 tx=0x7f9bf0009260 comp rx=0 tx=0).stop 2023-09-20T21:48:55.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.384+0000 7f9c01dff700 1 -- 172.21.15.22:0/2093152591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bfc100df0 msgr2=0x7f9bfc196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.384+0000 7f9c01dff700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bfc100df0 0x7f9bfc196ab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0058e0 tx=0x7f9bec00f690 comp rx=0 tx=0).stop 2023-09-20T21:48:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.385+0000 7f9c01dff700 1 -- 172.21.15.22:0/2093152591 shutdown_connections 2023-09-20T21:48:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.385+0000 7f9c01dff700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9be40a1350 0x7f9be40a3810 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.385+0000 7f9c01dff700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9bfc102980 0x7f9bfc19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.385+0000 7f9c01dff700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bfc100df0 0x7f9bfc196ab0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.385+0000 7f9c01dff700 1 --2- 172.21.15.22:0/2093152591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bfc101fe0 0x7f9bfc196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.385+0000 7f9c01dff700 1 -- 172.21.15.22:0/2093152591 >> 172.21.15.22:0/2093152591 conn(0x7f9bfc0fc320 msgr2=0x7f9bfc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.385+0000 7f9c01dff700 1 -- 172.21.15.22:0/2093152591 shutdown_connections 2023-09-20T21:48:55.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.385+0000 7f9c01dff700 1 -- 172.21.15.22:0/2093152591 wait complete. 2023-09-20T21:48:55.404 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T21:48:55.405 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T21:48:55.466 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:55 smithi022 ceph-mon[97651]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:55.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:55 smithi123 ceph-mon[134433]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:55.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:55 smithi089 ceph-mon[134090]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:55.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.612+0000 7f830ebff700 1 -- 172.21.15.22:0/1298448273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8308100df0 msgr2=0x7f8308101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:55.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.612+0000 7f830ebff700 1 --2- 172.21.15.22:0/1298448273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8308100df0 0x7f8308101210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0099c0 tx=0x7f82fc009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.612+0000 7f830ebff700 1 -- 172.21.15.22:0/1298448273 shutdown_connections 2023-09-20T21:48:55.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.612+0000 7f830ebff700 1 --2- 172.21.15.22:0/1298448273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8308102980 0x7f8308109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.612+0000 7f830ebff700 1 --2- 172.21.15.22:0/1298448273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8308100df0 0x7f8308101210 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.612+0000 7f830ebff700 1 --2- 172.21.15.22:0/1298448273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8308101fe0 0x7f8308102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.612+0000 7f830ebff700 1 -- 172.21.15.22:0/1298448273 >> 172.21.15.22:0/1298448273 conn(0x7f83080fc360 msgr2=0x7f83080fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:55.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.612+0000 7f830ebff700 1 -- 172.21.15.22:0/1298448273 shutdown_connections 2023-09-20T21:48:55.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.612+0000 7f830ebff700 1 -- 172.21.15.22:0/1298448273 wait complete. 2023-09-20T21:48:55.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.613+0000 7f830ebff700 1 Processor -- start 2023-09-20T21:48:55.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.614+0000 7f830ebff700 1 -- start start 2023-09-20T21:48:55.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.614+0000 7f830ebff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8308100df0 0x7f8308074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.614+0000 7f830ebff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8308101fe0 0x7f83080731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.614+0000 7f830ebff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8308102980 0x7f8308073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.614+0000 7f830ebff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8308073eb0 con 0x7f8308100df0 2023-09-20T21:48:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.614+0000 7f830ebff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8308074020 con 0x7f8308102980 2023-09-20T21:48:55.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.614+0000 7f830ebff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a0ba0 con 0x7f8308101fe0 2023-09-20T21:48:55.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.614+0000 7f830c99b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8308100df0 0x7f8308074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:55.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.614+0000 7f830c99b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8308100df0 0x7f8308074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59776/0 (socket says 172.21.15.22:59776) 2023-09-20T21:48:55.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.614+0000 7f830c99b700 1 -- 172.21.15.22:0/873883039 learned_addr learned my addr 172.21.15.22:0/873883039 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:55.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f8307fff700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8308101fe0 0x7f83080731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:55.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f830d19c700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8308102980 0x7f8308073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:55.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f830c99b700 1 -- 172.21.15.22:0/873883039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8308101fe0 msgr2=0x7f83080731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:55.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f830c99b700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8308101fe0 0x7f83080731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f830c99b700 1 -- 172.21.15.22:0/873883039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8308102980 msgr2=0x7f8308073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:55.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f830c99b700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8308102980 0x7f8308073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f830c99b700 1 -- 172.21.15.22:0/873883039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4009520 con 0x7f8308100df0 2023-09-20T21:48:55.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f830c99b700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8308100df0 0x7f8308074b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0098e0 tx=0x7f82fc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:55.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f8305ffb700 1 -- 172.21.15.22:0/873883039 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc01d070 con 0x7f8308100df0 2023-09-20T21:48:55.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f8305ffb700 1 -- 172.21.15.22:0/873883039 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc005050 con 0x7f8308100df0 2023-09-20T21:48:55.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f8305ffb700 1 -- 172.21.15.22:0/873883039 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc004b40 con 0x7f8308100df0 2023-09-20T21:48:55.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.615+0000 7f830ebff700 1 -- 172.21.15.22:0/873883039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc0095f0 con 0x7f8308100df0 2023-09-20T21:48:55.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.616+0000 7f830ebff700 1 -- 172.21.15.22:0/873883039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081a1060 con 0x7f8308100df0 2023-09-20T21:48:55.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.617+0000 7f830ebff700 1 -- 172.21.15.22:0/873883039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830810c6c0 con 0x7f8308100df0 2023-09-20T21:48:55.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.619+0000 7f8305ffb700 1 -- 172.21.15.22:0/873883039 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f82fc004ce0 con 0x7f8308100df0 2023-09-20T21:48:55.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.620+0000 7f8305ffb700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f82f80a1300 0x7f82f80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:55.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.620+0000 7f8305ffb700 1 -- 172.21.15.22:0/873883039 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f82fc0cdde0 con 0x7f8308100df0 2023-09-20T21:48:55.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.620+0000 7f8307fff700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f82f80a1300 0x7f82f80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:55.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.621+0000 7f8307fff700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f82f80a1300 0x7f82f80a37c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009df0 tx=0x7f82f4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:55.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.624+0000 7f8305ffb700 1 -- 172.21.15.22:0/873883039 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f82fc097150 con 0x7f8308100df0 2023-09-20T21:48:55.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.834+0000 7f830ebff700 1 -- 172.21.15.22:0/873883039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8308107d70 con 0x7f8308100df0 2023-09-20T21:48:55.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.834+0000 7f8305ffb700 1 -- 172.21.15.22:0/873883039 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82fc09a770 con 0x7f8308100df0 2023-09-20T21:48:55.835 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T21:48:55.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 -- 172.21.15.22:0/873883039 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f82f80a1300 msgr2=0x7f82f80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f82f80a1300 0x7f82f80a37c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009df0 tx=0x7f82f4009260 comp rx=0 tx=0).stop 2023-09-20T21:48:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 -- 172.21.15.22:0/873883039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8308100df0 msgr2=0x7f8308074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8308100df0 0x7f8308074b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0098e0 tx=0x7f82fc00f690 comp rx=0 tx=0).stop 2023-09-20T21:48:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 -- 172.21.15.22:0/873883039 shutdown_connections 2023-09-20T21:48:55.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f82f80a1300 0x7f82f80a37c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8308102980 0x7f8308073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8308100df0 0x7f8308074b90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 --2- 172.21.15.22:0/873883039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8308101fe0 0x7f83080731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:55.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 -- 172.21.15.22:0/873883039 >> 172.21.15.22:0/873883039 conn(0x7f83080fc360 msgr2=0x7f83080fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:55.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 -- 172.21.15.22:0/873883039 shutdown_connections 2023-09-20T21:48:55.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:55.837+0000 7f830ebff700 1 -- 172.21.15.22:0/873883039 wait complete. 2023-09-20T21:48:55.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T21:48:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.066+0000 7f10db5ff700 1 -- 172.21.15.22:0/3090730424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10d4101fc0 msgr2=0x7f10d4102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.066+0000 7f10db5ff700 1 --2- 172.21.15.22:0/3090730424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10d4101fc0 0x7f10d4102420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f10c40099c0 tx=0x7f10c4009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.066+0000 7f10db5ff700 1 -- 172.21.15.22:0/3090730424 shutdown_connections 2023-09-20T21:48:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.066+0000 7f10db5ff700 1 --2- 172.21.15.22:0/3090730424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d4102960 0x7f10d4109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.067+0000 7f10db5ff700 1 --2- 172.21.15.22:0/3090730424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10d4100dd0 0x7f10d41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.067+0000 7f10db5ff700 1 --2- 172.21.15.22:0/3090730424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10d4101fc0 0x7f10d4102420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.067+0000 7f10db5ff700 1 -- 172.21.15.22:0/3090730424 >> 172.21.15.22:0/3090730424 conn(0x7f10d40fc320 msgr2=0x7f10d40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.067+0000 7f10db5ff700 1 -- 172.21.15.22:0/3090730424 shutdown_connections 2023-09-20T21:48:56.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.067+0000 7f10db5ff700 1 -- 172.21.15.22:0/3090730424 wait complete. 2023-09-20T21:48:56.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.068+0000 7f10db5ff700 1 Processor -- start 2023-09-20T21:48:56.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.068+0000 7f10db5ff700 1 -- start start 2023-09-20T21:48:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.068+0000 7f10db5ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d4100dd0 0x7f10d4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.068+0000 7f10db5ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10d4101fc0 0x7f10d4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.068+0000 7f10db5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10d4102960 0x7f10d41a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.068+0000 7f10db5ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d41a0880 con 0x7f10d4101fc0 2023-09-20T21:48:56.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.068+0000 7f10db5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d41a09f0 con 0x7f10d4102960 2023-09-20T21:48:56.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.068+0000 7f10db5ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d41a0bb0 con 0x7f10d4100dd0 2023-09-20T21:48:56.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.068+0000 7f10d8b9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10d4101fc0 0x7f10d4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10d8b9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10d4101fc0 0x7f10d4199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45216/0 (socket says 172.21.15.22:45216) 2023-09-20T21:48:56.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10d8b9a700 1 -- 172.21.15.22:0/2670658643 learned_addr learned my addr 172.21.15.22:0/2670658643 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:56.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10d939b700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d4100dd0 0x7f10d4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10d9b9c700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10d4102960 0x7f10d41a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10d8b9a700 1 -- 172.21.15.22:0/2670658643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d4100dd0 msgr2=0x7f10d4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10d8b9a700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d4100dd0 0x7f10d4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10d8b9a700 1 -- 172.21.15.22:0/2670658643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10d4102960 msgr2=0x7f10d41a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10d8b9a700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10d4102960 0x7f10d41a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10d8b9a700 1 -- 172.21.15.22:0/2670658643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d0009520 con 0x7f10d4101fc0 2023-09-20T21:48:56.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10d8b9a700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10d4101fc0 0x7f10d4199280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009990 tx=0x7f10c400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:56.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10ca7fc700 1 -- 172.21.15.22:0/2670658643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c401d070 con 0x7f10d4101fc0 2023-09-20T21:48:56.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.069+0000 7f10ca7fc700 1 -- 172.21.15.22:0/2670658643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c4005030 con 0x7f10d4101fc0 2023-09-20T21:48:56.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.070+0000 7f10db5ff700 1 -- 172.21.15.22:0/2670658643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c40095f0 con 0x7f10d4101fc0 2023-09-20T21:48:56.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.070+0000 7f10ca7fc700 1 -- 172.21.15.22:0/2670658643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4004a70 con 0x7f10d4101fc0 2023-09-20T21:48:56.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.070+0000 7f10db5ff700 1 -- 172.21.15.22:0/2670658643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d41a1120 con 0x7f10d4101fc0 2023-09-20T21:48:56.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.071+0000 7f10db5ff700 1 -- 172.21.15.22:0/2670658643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d410c630 con 0x7f10d4101fc0 2023-09-20T21:48:56.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.073+0000 7f10ca7fc700 1 -- 172.21.15.22:0/2670658643 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f10c4005410 con 0x7f10d4101fc0 2023-09-20T21:48:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.074+0000 7f10ca7fc700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f10c00a1380 0x7f10c00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.074+0000 7f10ca7fc700 1 -- 172.21.15.22:0/2670658643 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f10c40cdd20 con 0x7f10d4101fc0 2023-09-20T21:48:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.074+0000 7f10d939b700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f10c00a1380 0x7f10c00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.075+0000 7f10d939b700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f10c00a1380 0x7f10c00a3840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f10d419a210 tx=0x7f10d0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:56.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.078+0000 7f10ca7fc700 1 -- 172.21.15.22:0/2670658643 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f10c4097090 con 0x7f10d4101fc0 2023-09-20T21:48:56.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.291+0000 7f10db5ff700 1 -- 172.21.15.22:0/2670658643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10d41a1370 con 0x7f10d4101fc0 2023-09-20T21:48:56.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.292+0000 7f10ca7fc700 1 -- 172.21.15.22:0/2670658643 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f10c409a6b0 con 0x7f10d4101fc0 2023-09-20T21:48:56.293 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:56.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.294+0000 7f10db5ff700 1 -- 172.21.15.22:0/2670658643 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f10c00a1380 msgr2=0x7f10c00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.294+0000 7f10db5ff700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f10c00a1380 0x7f10c00a3840 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f10d419a210 tx=0x7f10d0009260 comp rx=0 tx=0).stop 2023-09-20T21:48:56.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.294+0000 7f10db5ff700 1 -- 172.21.15.22:0/2670658643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10d4101fc0 msgr2=0x7f10d4199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.294+0000 7f10db5ff700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10d4101fc0 0x7f10d4199280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009990 tx=0x7f10c400f690 comp rx=0 tx=0).stop 2023-09-20T21:48:56.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.295+0000 7f10db5ff700 1 -- 172.21.15.22:0/2670658643 shutdown_connections 2023-09-20T21:48:56.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.295+0000 7f10db5ff700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f10c00a1380 0x7f10c00a3840 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.295+0000 7f10db5ff700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10d4102960 0x7f10d41a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.295+0000 7f10db5ff700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10d4100dd0 0x7f10d4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.295+0000 7f10db5ff700 1 --2- 172.21.15.22:0/2670658643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10d4101fc0 0x7f10d4199280 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.295+0000 7f10db5ff700 1 -- 172.21.15.22:0/2670658643 >> 172.21.15.22:0/2670658643 conn(0x7f10d40fc320 msgr2=0x7f10d40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:56.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.295+0000 7f10db5ff700 1 -- 172.21.15.22:0/2670658643 shutdown_connections 2023-09-20T21:48:56.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.295+0000 7f10db5ff700 1 -- 172.21.15.22:0/2670658643 wait complete. 2023-09-20T21:48:56.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T21:48:56.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:56 smithi022 ceph-mon[97651]: from='client.25490 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:56.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:56 smithi022 ceph-mon[97651]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:56.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:56 smithi022 ceph-mon[97651]: pgmap v347: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-20T21:48:56.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:56 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2093152591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:56.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:56 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/873883039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:56.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.523+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/491194883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58101fc0 msgr2=0x7f2a58102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.523+0000 7f2a5d3ff700 1 --2- 172.21.15.22:0/491194883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58101fc0 0x7f2a58102420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0099c0 tx=0x7f2a4c009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.523+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/491194883 shutdown_connections 2023-09-20T21:48:56.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.523+0000 7f2a5d3ff700 1 --2- 172.21.15.22:0/491194883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a58102960 0x7f2a58109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.523+0000 7f2a5d3ff700 1 --2- 172.21.15.22:0/491194883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a58100dd0 0x7f2a581011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.523+0000 7f2a5d3ff700 1 --2- 172.21.15.22:0/491194883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58101fc0 0x7f2a58102420 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.523+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/491194883 >> 172.21.15.22:0/491194883 conn(0x7f2a580fc320 msgr2=0x7f2a580fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:56.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.523+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/491194883 shutdown_connections 2023-09-20T21:48:56.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.523+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/491194883 wait complete. 2023-09-20T21:48:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.524+0000 7f2a5d3ff700 1 Processor -- start 2023-09-20T21:48:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.524+0000 7f2a5d3ff700 1 -- start start 2023-09-20T21:48:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.524+0000 7f2a5d3ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58100dd0 0x7f2a58196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.524+0000 7f2a5d3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a58101fc0 0x7f2a58196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.524+0000 7f2a5d3ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a58102960 0x7f2a5819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.524+0000 7f2a5d3ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5819e620 con 0x7f2a58100dd0 2023-09-20T21:48:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.524+0000 7f2a5d3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5819e760 con 0x7f2a58101fc0 2023-09-20T21:48:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.524+0000 7f2a5d3ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5819e8f0 con 0x7f2a58102960 2023-09-20T21:48:56.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.524+0000 7f2a56ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58100dd0 0x7f2a58196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.525+0000 7f2a577fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a58102960 0x7f2a5819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.525+0000 7f2a56ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58100dd0 0x7f2a58196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45234/0 (socket says 172.21.15.22:45234) 2023-09-20T21:48:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.525+0000 7f2a56ffd700 1 -- 172.21.15.22:0/635944465 learned_addr learned my addr 172.21.15.22:0/635944465 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.525+0000 7f2a567fc700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a58101fc0 0x7f2a58196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.525+0000 7f2a577fe700 1 -- 172.21.15.22:0/635944465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a58101fc0 msgr2=0x7f2a58196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.525+0000 7f2a577fe700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a58101fc0 0x7f2a58196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.525+0000 7f2a577fe700 1 -- 172.21.15.22:0/635944465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58100dd0 msgr2=0x7f2a58196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.525+0000 7f2a577fe700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58100dd0 0x7f2a58196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.525+0000 7f2a577fe700 1 -- 172.21.15.22:0/635944465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48009520 con 0x7f2a58102960 2023-09-20T21:48:56.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.525+0000 7f2a577fe700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a58102960 0x7f2a5819e050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000efc0 tx=0x7f2a5000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.526+0000 7f2a3ffff700 1 -- 172.21.15.22:0/635944465 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50018070 con 0x7f2a58102960 2023-09-20T21:48:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.526+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/635944465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c0095f0 con 0x7f2a58102960 2023-09-20T21:48:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.526+0000 7f2a3ffff700 1 -- 172.21.15.22:0/635944465 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a5000f040 con 0x7f2a58102960 2023-09-20T21:48:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.526+0000 7f2a3ffff700 1 -- 172.21.15.22:0/635944465 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5000cd40 con 0x7f2a58102960 2023-09-20T21:48:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.526+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/635944465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5819ef40 con 0x7f2a58102960 2023-09-20T21:48:56.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.527+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/635944465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a58107d50 con 0x7f2a58102960 2023-09-20T21:48:56.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.529+0000 7f2a3ffff700 1 -- 172.21.15.22:0/635944465 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2a50003a00 con 0x7f2a58102960 2023-09-20T21:48:56.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.530+0000 7f2a3ffff700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2a400a1380 0x7f2a400a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.530+0000 7f2a3ffff700 1 -- 172.21.15.22:0/635944465 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f2a500cd4c0 con 0x7f2a58102960 2023-09-20T21:48:56.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.530+0000 7f2a56ffd700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2a400a1380 0x7f2a400a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.531+0000 7f2a56ffd700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2a400a1380 0x7f2a400a3840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2a48005950 tx=0x7f2a48009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:56.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.532+0000 7f2a3ffff700 1 -- 172.21.15.22:0/635944465 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2a50096830 con 0x7f2a58102960 2023-09-20T21:48:56.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:56 smithi123 ceph-mon[134433]: from='client.25490 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:56.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:56 smithi123 ceph-mon[134433]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:56.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:56 smithi123 ceph-mon[134433]: pgmap v347: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-20T21:48:56.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:56 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2093152591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:56.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:56 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/873883039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:56.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:56 smithi089 ceph-mon[134090]: from='client.25490 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:56.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:56 smithi089 ceph-mon[134090]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-09-20T21:48:56.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:56 smithi089 ceph-mon[134090]: pgmap v347: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-20T21:48:56.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:56 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2093152591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:56.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:56 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/873883039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T21:48:56.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.744+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/635944465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a5810c810 con 0x7f2a58102960 2023-09-20T21:48:56.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.745+0000 7f2a3ffff700 1 -- 172.21.15.22:0/635944465 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a50099e50 con 0x7f2a58102960 2023-09-20T21:48:56.746 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T21:48:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.747+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/635944465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2a400a1380 msgr2=0x7f2a400a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2a400a1380 0x7f2a400a3840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2a48005950 tx=0x7f2a48009260 comp rx=0 tx=0).stop 2023-09-20T21:48:56.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/635944465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a58102960 msgr2=0x7f2a5819e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a58102960 0x7f2a5819e050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000efc0 tx=0x7f2a5000c3a0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/635944465 shutdown_connections 2023-09-20T21:48:56.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a58102960 0x7f2a5819e050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a58100dd0 0x7f2a58196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2a400a1380 0x7f2a400a3840 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 --2- 172.21.15.22:0/635944465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2a58101fc0 0x7f2a58196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/635944465 >> 172.21.15.22:0/635944465 conn(0x7f2a580fc320 msgr2=0x7f2a580fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/635944465 shutdown_connections 2023-09-20T21:48:56.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.748+0000 7f2a5d3ff700 1 -- 172.21.15.22:0/635944465 wait complete. 2023-09-20T21:48:56.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T21:48:56.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.976+0000 7f7133fff700 1 -- 172.21.15.22:0/2314784265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f712c101fc0 msgr2=0x7f712c102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.976+0000 7f7133fff700 1 --2- 172.21.15.22:0/2314784265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f712c101fc0 0x7f712c102420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f711c009920 tx=0x7f711c009c30 comp rx=0 tx=0).stop 2023-09-20T21:48:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.976+0000 7f7133fff700 1 -- 172.21.15.22:0/2314784265 shutdown_connections 2023-09-20T21:48:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.976+0000 7f7133fff700 1 --2- 172.21.15.22:0/2314784265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f712c102960 0x7f712c109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.976+0000 7f7133fff700 1 --2- 172.21.15.22:0/2314784265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c100dd0 0x7f712c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.976+0000 7f7133fff700 1 --2- 172.21.15.22:0/2314784265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f712c101fc0 0x7f712c102420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.976+0000 7f7133fff700 1 -- 172.21.15.22:0/2314784265 >> 172.21.15.22:0/2314784265 conn(0x7f712c0fc320 msgr2=0x7f712c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.976+0000 7f7133fff700 1 -- 172.21.15.22:0/2314784265 shutdown_connections 2023-09-20T21:48:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.976+0000 7f7133fff700 1 -- 172.21.15.22:0/2314784265 wait complete. 2023-09-20T21:48:56.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.977+0000 7f7133fff700 1 Processor -- start 2023-09-20T21:48:56.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.977+0000 7f7133fff700 1 -- start start 2023-09-20T21:48:56.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.977+0000 7f7133fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f712c100dd0 0x7f712c196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.977+0000 7f7133fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c101fc0 0x7f712c1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.977+0000 7f7133fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f712c102960 0x7f712c19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f7133fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c19e6f0 con 0x7f712c100dd0 2023-09-20T21:48:56.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f7133fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c19e830 con 0x7f712c102960 2023-09-20T21:48:56.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f7133fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c19e9c0 con 0x7f712c101fc0 2023-09-20T21:48:56.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f7131d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f712c100dd0 0x7f712c196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f713259c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f712c102960 0x7f712c19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f713159a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c101fc0 0x7f712c1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f7131d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f712c100dd0 0x7f712c196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45242/0 (socket says 172.21.15.22:45242) 2023-09-20T21:48:56.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f713159a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c101fc0 0x7f712c1970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:42602/0 (socket says 172.21.15.22:42602) 2023-09-20T21:48:56.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f7131d9b700 1 -- 172.21.15.22:0/1790327587 learned_addr learned my addr 172.21.15.22:0/1790327587 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:56.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f713259c700 1 -- 172.21.15.22:0/1790327587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c101fc0 msgr2=0x7f712c1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f713259c700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c101fc0 0x7f712c1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f713259c700 1 -- 172.21.15.22:0/1790327587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f712c100dd0 msgr2=0x7f712c196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:56.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f713259c700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f712c100dd0 0x7f712c196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:56.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f713259c700 1 -- 172.21.15.22:0/1790327587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7124009d00 con 0x7f712c102960 2023-09-20T21:48:56.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.978+0000 7f7131d9b700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f712c100dd0 0x7f712c196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:56.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.979+0000 7f713259c700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f712c102960 0x7f712c19e120 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7124009cd0 tx=0x7f712400c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:56.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.979+0000 7f7122ffd700 1 -- 172.21.15.22:0/1790327587 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124017070 con 0x7f712c102960 2023-09-20T21:48:56.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.979+0000 7f7133fff700 1 -- 172.21.15.22:0/1790327587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c009520 con 0x7f712c102960 2023-09-20T21:48:56.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.979+0000 7f7122ffd700 1 -- 172.21.15.22:0/1790327587 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712400f040 con 0x7f712c102960 2023-09-20T21:48:56.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.979+0000 7f7122ffd700 1 -- 172.21.15.22:0/1790327587 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400e050 con 0x7f712c102960 2023-09-20T21:48:56.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.979+0000 7f7133fff700 1 -- 172.21.15.22:0/1790327587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c19efc0 con 0x7f712c102960 2023-09-20T21:48:56.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.980+0000 7f7133fff700 1 -- 172.21.15.22:0/1790327587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c108490 con 0x7f712c102960 2023-09-20T21:48:56.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.982+0000 7f7122ffd700 1 -- 172.21.15.22:0/1790327587 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f712400cd10 con 0x7f712c102960 2023-09-20T21:48:56.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.983+0000 7f7122ffd700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71180a1300 0x7f71180a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:56.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.983+0000 7f7122ffd700 1 -- 172.21.15.22:0/1790327587 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f71240cd3a0 con 0x7f712c102960 2023-09-20T21:48:56.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.984+0000 7f7131d9b700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71180a1300 0x7f71180a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:56.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.984+0000 7f7131d9b700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71180a1300 0x7f71180a37c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7128005950 tx=0x7f7128009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:56.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:56.985+0000 7f7122ffd700 1 -- 172.21.15.22:0/1790327587 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f7124096680 con 0x7f712c102960 2023-09-20T21:48:57.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.195+0000 7f7133fff700 1 -- 172.21.15.22:0/1790327587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f712c10c7f0 con 0x7f712c102960 2023-09-20T21:48:57.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.195+0000 7f7122ffd700 1 -- 172.21.15.22:0/1790327587 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7124099ca0 con 0x7f712c102960 2023-09-20T21:48:57.198 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T21:48:57.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 -- 172.21.15.22:0/1790327587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71180a1300 msgr2=0x7f71180a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:57.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71180a1300 0x7f71180a37c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7128005950 tx=0x7f7128009260 comp rx=0 tx=0).stop 2023-09-20T21:48:57.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 -- 172.21.15.22:0/1790327587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f712c102960 msgr2=0x7f712c19e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:57.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f712c102960 0x7f712c19e120 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7124009cd0 tx=0x7f712400c620 comp rx=0 tx=0).stop 2023-09-20T21:48:57.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 -- 172.21.15.22:0/1790327587 shutdown_connections 2023-09-20T21:48:57.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f71180a1300 0x7f71180a37c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f712c102960 0x7f712c19e120 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f712c100dd0 0x7f712c196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 --2- 172.21.15.22:0/1790327587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c101fc0 0x7f712c1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 -- 172.21.15.22:0/1790327587 >> 172.21.15.22:0/1790327587 conn(0x7f712c0fc320 msgr2=0x7f712c1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:57.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 -- 172.21.15.22:0/1790327587 shutdown_connections 2023-09-20T21:48:57.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.198+0000 7f7133fff700 1 -- 172.21.15.22:0/1790327587 wait complete. 2023-09-20T21:48:57.217 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:48:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.422+0000 7fa7febff700 1 -- 172.21.15.22:0/4255793277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f8073840 msgr2=0x7fa7f8073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.422+0000 7fa7febff700 1 --2- 172.21.15.22:0/4255793277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f8073840 0x7fa7f8073cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40098f0 tx=0x7fa7e4009c00 comp rx=0 tx=0).stop 2023-09-20T21:48:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.422+0000 7fa7febff700 1 -- 172.21.15.22:0/4255793277 shutdown_connections 2023-09-20T21:48:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.422+0000 7fa7febff700 1 --2- 172.21.15.22:0/4255793277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7f8108ca0 0x7fa7f810b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.422+0000 7fa7febff700 1 --2- 172.21.15.22:0/4255793277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7f8074e20 0x7fa7f8073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.422+0000 7fa7febff700 1 --2- 172.21.15.22:0/4255793277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f8073840 0x7fa7f8073cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.422+0000 7fa7febff700 1 -- 172.21.15.22:0/4255793277 >> 172.21.15.22:0/4255793277 conn(0x7fa7f80fc310 msgr2=0x7fa7f80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.422+0000 7fa7febff700 1 -- 172.21.15.22:0/4255793277 shutdown_connections 2023-09-20T21:48:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.422+0000 7fa7febff700 1 -- 172.21.15.22:0/4255793277 wait complete. 2023-09-20T21:48:57.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.424+0000 7fa7febff700 1 Processor -- start 2023-09-20T21:48:57.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.424+0000 7fa7febff700 1 -- start start 2023-09-20T21:48:57.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.424+0000 7fa7febff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f8073840 0x7fa7f8198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:57.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.424+0000 7fa7febff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7f8074e20 0x7fa7f8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:57.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.424+0000 7fa7febff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7f8108ca0 0x7fa7f81a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:57.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.424+0000 7fa7febff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f81a0830 con 0x7fa7f8108ca0 2023-09-20T21:48:57.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.424+0000 7fa7febff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f81a09a0 con 0x7fa7f8074e20 2023-09-20T21:48:57.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.424+0000 7fa7febff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f81a0b60 con 0x7fa7f8073840 2023-09-20T21:48:57.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7fc99b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f8073840 0x7fa7f8198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:57.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7fc99b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f8073840 0x7fa7f8198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:42622/0 (socket says 172.21.15.22:42622) 2023-09-20T21:48:57.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7fc99b700 1 -- 172.21.15.22:0/2139377608 learned_addr learned my addr 172.21.15.22:0/2139377608 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:57.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7fd19c700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7f8108ca0 0x7fa7f81a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:57.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7f7fff700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7f8074e20 0x7fa7f8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:57.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7fc99b700 1 -- 172.21.15.22:0/2139377608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7f8074e20 msgr2=0x7fa7f8199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:57.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7fc99b700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7f8074e20 0x7fa7f8199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7fc99b700 1 -- 172.21.15.22:0/2139377608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7f8108ca0 msgr2=0x7fa7f81a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7fc99b700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7f8108ca0 0x7fa7f81a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7fc99b700 1 -- 172.21.15.22:0/2139377608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec0095f0 con 0x7fa7f8073840 2023-09-20T21:48:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7fc99b700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f8073840 0x7fa7f8198cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00ec10 tx=0x7fa7ec00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.425+0000 7fa7f5ffb700 1 -- 172.21.15.22:0/2139377608 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00cc90 con 0x7fa7f8073840 2023-09-20T21:48:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.426+0000 7fa7febff700 1 -- 172.21.15.22:0/2139377608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e4009520 con 0x7fa7f8073840 2023-09-20T21:48:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.426+0000 7fa7f5ffb700 1 -- 172.21.15.22:0/2139377608 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7ec0044e0 con 0x7fa7f8073840 2023-09-20T21:48:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.426+0000 7fa7f5ffb700 1 -- 172.21.15.22:0/2139377608 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec010430 con 0x7fa7f8073840 2023-09-20T21:48:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.426+0000 7fa7febff700 1 -- 172.21.15.22:0/2139377608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f81a1190 con 0x7fa7f8073840 2023-09-20T21:48:57.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.427+0000 7fa7febff700 1 -- 172.21.15.22:0/2139377608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f8107350 con 0x7fa7f8073840 2023-09-20T21:48:57.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.429+0000 7fa7f5ffb700 1 -- 172.21.15.22:0/2139377608 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa7ec003fb0 con 0x7fa7f8073840 2023-09-20T21:48:57.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.430+0000 7fa7f5ffb700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7e80a1350 0x7fa7e80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:57.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.430+0000 7fa7f5ffb700 1 -- 172.21.15.22:0/2139377608 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa7ec014070 con 0x7fa7f8073840 2023-09-20T21:48:57.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.431+0000 7fa7f7fff700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7e80a1350 0x7fa7e80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:57.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.431+0000 7fa7f7fff700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7e80a1350 0x7fa7e80a3810 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40098c0 tx=0x7fa7e4019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:57.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.432+0000 7fa7f5ffb700 1 -- 172.21.15.22:0/2139377608 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa7ec096860 con 0x7fa7f8073840 2023-09-20T21:48:57.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:57 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2670658643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:57.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:57 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/635944465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:57.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:57 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1790327587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:57.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:57 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2670658643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:57.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:57 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/635944465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:57.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:57 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1790327587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:57.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:57 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2670658643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T21:48:57.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:57 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/635944465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T21:48:57.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:57 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1790327587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T21:48:57.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.696+0000 7fa7febff700 1 -- 172.21.15.22:0/2139377608 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7f810e7d0 con 0x7fa7f8073840 2023-09-20T21:48:57.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.697+0000 7fa7f5ffb700 1 -- 172.21.15.22:0/2139377608 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11361 (secure 0 0 0) 0x7fa7ec099e80 con 0x7fa7f8073840 2023-09-20T21:48:57.698 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:57.698 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:47:09.607119+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T21:48:57.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 -- 172.21.15.22:0/2139377608 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7e80a1350 msgr2=0x7fa7e80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:57.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7e80a1350 0x7fa7e80a3810 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40098c0 tx=0x7fa7e4019040 comp rx=0 tx=0).stop 2023-09-20T21:48:57.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 -- 172.21.15.22:0/2139377608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f8073840 msgr2=0x7fa7f8198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:57.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f8073840 0x7fa7f8198cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00ec10 tx=0x7fa7ec00ef20 comp rx=0 tx=0).stop 2023-09-20T21:48:57.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 -- 172.21.15.22:0/2139377608 shutdown_connections 2023-09-20T21:48:57.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7f8108ca0 0x7fa7f81a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7f8073840 0x7fa7f8198cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa7e80a1350 0x7fa7e80a3810 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 --2- 172.21.15.22:0/2139377608 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa7f8074e20 0x7fa7f8199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 -- 172.21.15.22:0/2139377608 >> 172.21.15.22:0/2139377608 conn(0x7fa7f80fc310 msgr2=0x7fa7f80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:57.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 -- 172.21.15.22:0/2139377608 shutdown_connections 2023-09-20T21:48:57.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.700+0000 7fa7febff700 1 -- 172.21.15.22:0/2139377608 wait complete. 2023-09-20T21:48:57.703 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2023-09-20T21:48:57.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T21:48:57.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.923+0000 7ffa55dff700 1 -- 172.21.15.22:0/4280556190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa50100de0 msgr2=0x7ffa50101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:57.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.923+0000 7ffa55dff700 1 --2- 172.21.15.22:0/4280556190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa50100de0 0x7ffa50101200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffa400099c0 tx=0x7ffa40009cd0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.923+0000 7ffa55dff700 1 -- 172.21.15.22:0/4280556190 shutdown_connections 2023-09-20T21:48:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.923+0000 7ffa55dff700 1 --2- 172.21.15.22:0/4280556190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa50102970 0x7ffa50109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.923+0000 7ffa55dff700 1 --2- 172.21.15.22:0/4280556190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa50100de0 0x7ffa50101200 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.923+0000 7ffa55dff700 1 --2- 172.21.15.22:0/4280556190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50101fd0 0x7ffa50102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.923+0000 7ffa55dff700 1 -- 172.21.15.22:0/4280556190 >> 172.21.15.22:0/4280556190 conn(0x7ffa500fc310 msgr2=0x7ffa500fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.923+0000 7ffa55dff700 1 -- 172.21.15.22:0/4280556190 shutdown_connections 2023-09-20T21:48:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.923+0000 7ffa55dff700 1 -- 172.21.15.22:0/4280556190 wait complete. 2023-09-20T21:48:57.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.924+0000 7ffa55dff700 1 Processor -- start 2023-09-20T21:48:57.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa55dff700 1 -- start start 2023-09-20T21:48:57.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa55dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa50100de0 0x7ffa50196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:57.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa55dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50101fd0 0x7ffa50197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:57.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa55dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa50102970 0x7ffa5019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:57.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa55dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5019e650 con 0x7ffa50102970 2023-09-20T21:48:57.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa55dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5019e790 con 0x7ffa50100de0 2023-09-20T21:48:57.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa55dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5019e920 con 0x7ffa50101fd0 2023-09-20T21:48:57.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa4ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa50102970 0x7ffa5019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:57.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa4ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa50102970 0x7ffa5019e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45274/0 (socket says 172.21.15.22:45274) 2023-09-20T21:48:57.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa4ffff700 1 -- 172.21.15.22:0/159023937 learned_addr learned my addr 172.21.15.22:0/159023937 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:57.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa4effd700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50101fd0 0x7ffa50197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:57.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.925+0000 7ffa4f7fe700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa50100de0 0x7ffa50196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:57.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa4ffff700 1 -- 172.21.15.22:0/159023937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50101fd0 msgr2=0x7ffa50197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:57.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa4ffff700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50101fd0 0x7ffa50197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa4ffff700 1 -- 172.21.15.22:0/159023937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa50100de0 msgr2=0x7ffa50196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:57.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa4ffff700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa50100de0 0x7ffa50196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:57.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa4ffff700 1 -- 172.21.15.22:0/159023937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400095f0 con 0x7ffa50102970 2023-09-20T21:48:57.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa4effd700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50101fd0 0x7ffa50197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:57.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa4ffff700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa50102970 0x7ffa5019e080 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800ef30 tx=0x7ffa4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:57.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa4cff9700 1 -- 172.21.15.22:0/159023937 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48018070 con 0x7ffa50102970 2023-09-20T21:48:57.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa4cff9700 1 -- 172.21.15.22:0/159023937 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa4800f040 con 0x7ffa50102970 2023-09-20T21:48:57.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa4cff9700 1 -- 172.21.15.22:0/159023937 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4800cd00 con 0x7ffa50102970 2023-09-20T21:48:57.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa55dff700 1 -- 172.21.15.22:0/159023937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5019eba0 con 0x7ffa50102970 2023-09-20T21:48:57.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.926+0000 7ffa55dff700 1 -- 172.21.15.22:0/159023937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5019f060 con 0x7ffa50102970 2023-09-20T21:48:57.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.927+0000 7ffa55dff700 1 -- 172.21.15.22:0/159023937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5010c640 con 0x7ffa50102970 2023-09-20T21:48:57.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.929+0000 7ffa4cff9700 1 -- 172.21.15.22:0/159023937 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ffa48003a00 con 0x7ffa50102970 2023-09-20T21:48:57.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.931+0000 7ffa4cff9700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffa380a1350 0x7ffa380a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:57.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.931+0000 7ffa4cff9700 1 -- 172.21.15.22:0/159023937 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ffa480ccb00 con 0x7ffa50102970 2023-09-20T21:48:57.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.931+0000 7ffa4f7fe700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffa380a1350 0x7ffa380a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:57.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.932+0000 7ffa4f7fe700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffa380a1350 0x7ffa380a3810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009990 tx=0x7ffa40005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:57.935+0000 7ffa4cff9700 1 -- 172.21.15.22:0/159023937 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ffa480cee60 con 0x7ffa50102970 2023-09-20T21:48:58.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.151+0000 7ffa55dff700 1 -- 172.21.15.22:0/159023937 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa5010c880 con 0x7ffa50102970 2023-09-20T21:48:58.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.152+0000 7ffa4cff9700 1 -- 172.21.15.22:0/159023937 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7ffa48095e70 con 0x7ffa50102970 2023-09-20T21:48:58.153 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:48:58.153 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T21:48:58.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 -- 172.21.15.22:0/159023937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffa380a1350 msgr2=0x7ffa380a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffa380a1350 0x7ffa380a3810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009990 tx=0x7ffa40005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 -- 172.21.15.22:0/159023937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa50102970 msgr2=0x7ffa5019e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa50102970 0x7ffa5019e080 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800ef30 tx=0x7ffa4800c3a0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 -- 172.21.15.22:0/159023937 shutdown_connections 2023-09-20T21:48:58.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa50102970 0x7ffa5019e080 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ffa380a1350 0x7ffa380a3810 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa50100de0 0x7ffa50196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 --2- 172.21.15.22:0/159023937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50101fd0 0x7ffa50197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 -- 172.21.15.22:0/159023937 >> 172.21.15.22:0/159023937 conn(0x7ffa500fc310 msgr2=0x7ffa500fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 -- 172.21.15.22:0/159023937 shutdown_connections 2023-09-20T21:48:58.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.155+0000 7ffa55dff700 1 -- 172.21.15.22:0/159023937 wait complete. 2023-09-20T21:48:58.175 DEBUG:tasks.fwd_scrub:Yielding 2023-09-20T21:48:58.176 INFO:teuthology.run_tasks:Running task exec... 2023-09-20T21:48:58.186 INFO:teuthology.task.exec:Executing custom commands... 2023-09-20T21:48:58.186 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi022.front.sepia.ceph.com 2023-09-20T21:48:58.187 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-09-20T21:48:58.187 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-09-20T21:48:58.187 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-20T21:48:58.188 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-20T21:48:58.288 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:48:58.329 INFO:teuthology.orchestra.run.smithi022.stdout:Filesystem Size Used Avail Use% Mounted on 2023-09-20T21:48:58.330 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6789,172.21.15.89:6789,172.21.15.123:6789:/ 1.1T 332M 1.1T 1% /tmp/cephfs 2023-09-20T21:48:58.334 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-09-20T21:48:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.409+0000 7fae26bff700 1 -- 172.21.15.22:0/4202851056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae20074e20 msgr2=0x7fae20073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.409+0000 7fae26bff700 1 --2- 172.21.15.22:0/4202851056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae20074e20 0x7fae20073270 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae140098f0 tx=0x7fae14009c00 comp rx=0 tx=0).stop 2023-09-20T21:48:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.409+0000 7fae26bff700 1 -- 172.21.15.22:0/4202851056 shutdown_connections 2023-09-20T21:48:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.409+0000 7fae26bff700 1 --2- 172.21.15.22:0/4202851056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae2010d3a0 0x7fae2010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.409+0000 7fae26bff700 1 --2- 172.21.15.22:0/4202851056 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae20074e20 0x7fae20073270 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.409+0000 7fae26bff700 1 --2- 172.21.15.22:0/4202851056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae20073840 0x7fae20073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.409+0000 7fae26bff700 1 -- 172.21.15.22:0/4202851056 >> 172.21.15.22:0/4202851056 conn(0x7fae200fc320 msgr2=0x7fae200fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.409+0000 7fae26bff700 1 -- 172.21.15.22:0/4202851056 shutdown_connections 2023-09-20T21:48:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.409+0000 7fae26bff700 1 -- 172.21.15.22:0/4202851056 wait complete. 2023-09-20T21:48:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.411+0000 7fae26bff700 1 Processor -- start 2023-09-20T21:48:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.411+0000 7fae26bff700 1 -- start start 2023-09-20T21:48:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.411+0000 7fae26bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae20073840 0x7fae2019d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae26bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae20074e20 0x7fae2019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae26bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae2010d3a0 0x7fae201a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae26bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae201a4d00 con 0x7fae2010d3a0 2023-09-20T21:48:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae26bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae201a4e40 con 0x7fae20073840 2023-09-20T21:48:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae26bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae201a4fd0 con 0x7fae20074e20 2023-09-20T21:48:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae2519c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae2010d3a0 0x7fae201a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae2499b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae20073840 0x7fae2019d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae2519c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae2010d3a0 0x7fae201a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45298/0 (socket says 172.21.15.22:45298) 2023-09-20T21:48:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae2519c700 1 -- 172.21.15.22:0/2757786365 learned_addr learned my addr 172.21.15.22:0/2757786365 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae1ffff700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae20074e20 0x7fae2019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae2499b700 1 -- 172.21.15.22:0/2757786365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae20074e20 msgr2=0x7fae2019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae2499b700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae20074e20 0x7fae2019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae2499b700 1 -- 172.21.15.22:0/2757786365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae2010d3a0 msgr2=0x7fae201a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae2499b700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae2010d3a0 0x7fae201a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae2499b700 1 -- 172.21.15.22:0/2757786365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14009520 con 0x7fae20073840 2023-09-20T21:48:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.412+0000 7fae2519c700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae2010d3a0 0x7fae201a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.413+0000 7fae2499b700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae20073840 0x7fae2019d180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae14009230 tx=0x7fae1400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.413+0000 7fae1dffb700 1 -- 172.21.15.22:0/2757786365 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1401c070 con 0x7fae20073840 2023-09-20T21:48:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.413+0000 7fae26bff700 1 -- 172.21.15.22:0/2757786365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae201a5250 con 0x7fae20073840 2023-09-20T21:48:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.413+0000 7fae1dffb700 1 -- 172.21.15.22:0/2757786365 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae1400f7d0 con 0x7fae20073840 2023-09-20T21:48:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.413+0000 7fae1dffb700 1 -- 172.21.15.22:0/2757786365 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14003a00 con 0x7fae20073840 2023-09-20T21:48:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.413+0000 7fae26bff700 1 -- 172.21.15.22:0/2757786365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae201a56d0 con 0x7fae20073840 2023-09-20T21:48:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.417+0000 7fae1dffb700 1 -- 172.21.15.22:0/2757786365 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fae14020030 con 0x7fae20073840 2023-09-20T21:48:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.418+0000 7fae3a4d8b80 1 Processor -- start 2023-09-20T21:48:58.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.418+0000 7fae3a4d8b80 1 -- start start 2023-09-20T21:48:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.418+0000 7fae1dffb700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fae100a1380 0x7fae100a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.418+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.418+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.418+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2aec60 0x55611f2af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.418+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f221d20 con 0x55611f2ad770 2023-09-20T21:48:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.418+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f205cf0 con 0x55611f2ae0d0 2023-09-20T21:48:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.418+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f203610 con 0x55611f2aec60 2023-09-20T21:48:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.418+0000 7fae1dffb700 1 -- 172.21.15.22:0/2757786365 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fae140cddd0 con 0x7fae20073840 2023-09-20T21:48:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae2499b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae2519c700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fae100a1380 0x7fae100a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae2519c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2aec60 0x55611f2af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae1ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae2519c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2aec60 0x55611f2af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:42674/0 (socket says 172.21.15.22:42674) 2023-09-20T21:48:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae1ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2ae660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38354/0 (socket says 172.21.15.22:38354) 2023-09-20T21:48:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae2519c700 1 -- 172.21.15.22:0/4118738818 learned_addr learned my addr 172.21.15.22:0/4118738818 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:58.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae2519c700 1 -- 172.21.15.22:0/4118738818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 msgr2=0x55611f2ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae2519c700 1 --2- 172.21.15.22:0/4118738818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae2519c700 1 -- 172.21.15.22:0/4118738818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 msgr2=0x55611f2adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae2519c700 1 --2- 172.21.15.22:0/4118738818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.419+0000 7fae2519c700 1 -- 172.21.15.22:0/4118738818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55611f029e60 con 0x55611f2aec60 2023-09-20T21:48:58.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.420+0000 7fae2519c700 1 --2- 172.21.15.22:0/4118738818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2aec60 0x55611f2af0e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fae1800e570 tx=0x7fae1800e880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dacbe9231741ffaf server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.420+0000 7fae2519c700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fae100a1380 0x7fae100a3840 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae1800ebd0 tx=0x7fae18010000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.420+0000 7fae0affd700 1 -- 172.21.15.22:0/4118738818 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1801bcf0 con 0x55611f2aec60 2023-09-20T21:48:58.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.420+0000 7fae0affd700 1 -- 172.21.15.22:0/4118738818 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae1801f070 con 0x55611f2aec60 2023-09-20T21:48:58.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.420+0000 7fae0affd700 1 -- 172.21.15.22:0/4118738818 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180243d0 con 0x55611f2aec60 2023-09-20T21:48:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.421+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/4118738818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2aec60 msgr2=0x55611f2af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.421+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/4118738818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2aec60 0x55611f2af0e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fae1800e570 tx=0x7fae1800e880 comp rx=0 tx=0).stop 2023-09-20T21:48:58.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.421+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/4118738818 shutdown_connections 2023-09-20T21:48:58.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.421+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/4118738818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2aec60 0x55611f2af0e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.421+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/4118738818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.421+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/4118738818 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.421+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/4118738818 >> 172.21.15.22:0/4118738818 conn(0x55611f1da080 msgr2=0x55611f2a8250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.421+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/4118738818 shutdown_connections 2023-09-20T21:48:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.421+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/4118738818 wait complete. 2023-09-20T21:48:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.422+0000 7fae3a4d8b80 1 -- start start 2023-09-20T21:48:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.422+0000 7fae3a4d8b80 1 Processor -- start 2023-09-20T21:48:58.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.422+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.422+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.422+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 0x55611f2be750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.422+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f1fef00 con 0x55611f2ad770 2023-09-20T21:48:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.422+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f19b7c0 con 0x55611f2aec60 2023-09-20T21:48:58.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.422+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f19b460 con 0x55611f2ae0d0 2023-09-20T21:48:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.422+0000 7fae3a4d8b80 10 client.0 _ll_register_callbacks cb 0x55611f0e5b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae2499b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae2519c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 0x55611f2be750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae1ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae2499b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45320/0 (socket says 172.21.15.22:45320) 2023-09-20T21:48:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae2499b700 1 -- 172.21.15.22:0/693300210 learned_addr learned my addr 172.21.15.22:0/693300210 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae1ffff700 1 -- 172.21.15.22:0/693300210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 msgr2=0x55611f2be750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae1ffff700 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 0x55611f2be750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae1ffff700 1 -- 172.21.15.22:0/693300210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 msgr2=0x55611f2b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae1ffff700 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae1ffff700 1 -- 172.21.15.22:0/693300210 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae18008bd0 con 0x55611f2ae0d0 2023-09-20T21:48:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae2519c700 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 0x55611f2be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae2499b700 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fae1ffff700 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2b7710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55611f2a9090 tx=0x7fae0c011a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f53f5c285093bc0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.423+0000 7fadfffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:48:58.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fadfffff700 1 -- 172.21.15.22:0/693300210 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c016070 con 0x55611f2ae0d0 2023-09-20T21:48:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/693300210 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55611f16fc30 con 0x55611f2ae0d0 2023-09-20T21:48:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fadfffff700 1 -- 172.21.15.22:0/693300210 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fae0c004320 con 0x55611f2ae0d0 2023-09-20T21:48:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fadfffff700 1 -- 172.21.15.22:0/693300210 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae0c0044e0 con 0x55611f2ae0d0 2023-09-20T21:48:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fae3a4d8b80 10 client.25540 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:58.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/693300210 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55611f012ce0 con 0x55611f2ae0d0 2023-09-20T21:48:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fadfffff700 1 -- 172.21.15.22:0/693300210 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55611f16fc30 con 0x55611f2ae0d0 2023-09-20T21:48:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fadfffff700 1 -- 172.21.15.22:0/693300210 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fae0c005570 con 0x55611f2ae0d0 2023-09-20T21:48:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fae3a4d8b80 10 client.25540 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fae3a4d8b80 10 client.25540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:48:58.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.424+0000 7fae3a4d8b80 20 client.25540 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.425+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55611f2c2fc0 0x55611f2c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.425+0000 7fae3a4d8b80 4 client.25540 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.425+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/693300210 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55611f27c0a0 con 0x55611f2c2fc0 2023-09-20T21:48:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.425+0000 7fae2499b700 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55611f2c2fc0 0x55611f2c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.426+0000 7fadfffff700 10 client.25540 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:58.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.426+0000 7fae2499b700 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55611f2c2fc0 0x55611f2c53e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a474276179bcfc90 server_cookie=fa04a434a56268ea in_seq=0 out_seq=0 2023-09-20T21:48:58.438 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:58 smithi022 ceph-mon[97651]: pgmap v348: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-20T21:48:58.438 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:58 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2139377608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:58.438 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:58 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/159023937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:58.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.426+0000 7fadfffff700 1 -- 172.21.15.22:0/693300210 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55611f27c0a0 con 0x55611f2c2fc0 2023-09-20T21:48:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.426+0000 7fadfffff700 10 client.25540 handle_command_reply: tid=0 2023-09-20T21:48:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 client.25540 shutdown 2023-09-20T21:48:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/693300210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 msgr2=0x55611f2b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2b7710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55611f2a9090 tx=0x7fae0c011a10 comp rx=0 tx=0).stop 2023-09-20T21:48:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/693300210 shutdown_connections 2023-09-20T21:48:58.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55611f2c2fc0 0x55611f2c53e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 0x55611f2be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fadfffff700 0 client.25540 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:58.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/693300210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2b7710 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/693300210 >> 172.21.15.22:0/693300210 conn(0x55611f1da080 msgr2=0x55611f2a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/693300210 shutdown_connections 2023-09-20T21:48:58.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/693300210 wait complete. 2023-09-20T21:48:58.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.427+0000 7fae3a4d8b80 20 client.25540 trim_cache size 0 max 16384 2023-09-20T21:48:58.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.446+0000 7fae3a4d8b80 1 Processor -- start 2023-09-20T21:48:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.446+0000 7fae3a4d8b80 1 -- start start 2023-09-20T21:48:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.446+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ad770 0x55611f2b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2aec60 0x55611f2b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f07f9a0 con 0x55611f2aec60 2023-09-20T21:48:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f2b6680 con 0x55611f2ae0d0 2023-09-20T21:48:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f2b6d40 con 0x55611f2ad770 2023-09-20T21:48:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae1ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae1ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2be710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38370/0 (socket says 172.21.15.22:38370) 2023-09-20T21:48:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae1ffff700 1 -- 172.21.15.22:0/1620246988 learned_addr learned my addr 172.21.15.22:0/1620246988 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae2519c700 1 --2- 172.21.15.22:0/1620246988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2aec60 0x55611f2b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae2499b700 1 --2- 172.21.15.22:0/1620246988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ad770 0x55611f2b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae1ffff700 1 -- 172.21.15.22:0/1620246988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ad770 msgr2=0x55611f2b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae1ffff700 1 --2- 172.21.15.22:0/1620246988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ad770 0x55611f2b6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae1ffff700 1 -- 172.21.15.22:0/1620246988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2aec60 msgr2=0x55611f2b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae1ffff700 1 --2- 172.21.15.22:0/1620246988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2aec60 0x55611f2b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae1ffff700 1 -- 172.21.15.22:0/1620246988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55611f2a4630 con 0x55611f2ae0d0 2023-09-20T21:48:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae2499b700 1 --2- 172.21.15.22:0/1620246988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ad770 0x55611f2b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.447+0000 7fae1ffff700 1 --2- 172.21.15.22:0/1620246988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2be710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009770 tx=0x7fae0c011d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0e1c336e82af9a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.448+0000 7fae0b7fe700 1 -- 172.21.15.22:0/1620246988 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c016030 con 0x55611f2ae0d0 2023-09-20T21:48:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.448+0000 7fae0b7fe700 1 -- 172.21.15.22:0/1620246988 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae0c0056b0 con 0x55611f2ae0d0 2023-09-20T21:48:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.448+0000 7fae0b7fe700 1 -- 172.21.15.22:0/1620246988 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c029d20 con 0x55611f2ae0d0 2023-09-20T21:48:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1620246988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 msgr2=0x55611f2be710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/1620246988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2be710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009770 tx=0x7fae0c011d60 comp rx=0 tx=0).stop 2023-09-20T21:48:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1620246988 shutdown_connections 2023-09-20T21:48:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/1620246988 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2aec60 0x55611f2b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/1620246988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ad770 0x55611f2b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/1620246988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2ae0d0 0x55611f2be710 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1620246988 >> 172.21.15.22:0/1620246988 conn(0x55611f1da080 msgr2=0x55611f2b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1620246988 shutdown_connections 2023-09-20T21:48:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1620246988 wait complete. 2023-09-20T21:48:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 -- start start 2023-09-20T21:48:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.449+0000 7fae3a4d8b80 1 Processor -- start 2023-09-20T21:48:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2be6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2bccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae3a4d8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 0x55611f2bd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f2bd970 con 0x55611f2ad770 2023-09-20T21:48:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f2bdae0 con 0x55611f2aec60 2023-09-20T21:48:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae3a4d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55611f316de0 con 0x55611f2ae0d0 2023-09-20T21:48:58.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae3a4d8b80 10 client.0 _ll_register_callbacks cb 0x55611f0e5b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:58.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae1ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2bccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae1ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2bccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:42720/0 (socket says 172.21.15.22:42720) 2023-09-20T21:48:58.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae1ffff700 1 -- 172.21.15.22:0/1915687474 learned_addr learned my addr 172.21.15.22:0/1915687474 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:58.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae2519c700 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 0x55611f2bd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.450+0000 7fae2499b700 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2be6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.451+0000 7fae1ffff700 1 -- 172.21.15.22:0/1915687474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 msgr2=0x55611f2bd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.451+0000 7fae1ffff700 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 0x55611f2bd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.451+0000 7fae1ffff700 1 -- 172.21.15.22:0/1915687474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 msgr2=0x55611f2be6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.451+0000 7fae1ffff700 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2be6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.451+0000 7fae1ffff700 1 -- 172.21.15.22:0/1915687474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55611f012ce0 con 0x55611f2ae0d0 2023-09-20T21:48:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.451+0000 7fae1ffff700 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2bccc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0097c0 tx=0x7fae0c0296c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1c6312b6e768eae server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.451+0000 7fadfffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:48:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.451+0000 7fadfffff700 1 -- 172.21.15.22:0/1915687474 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c016040 con 0x55611f2ae0d0 2023-09-20T21:48:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.451+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1915687474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae1801f070 con 0x55611f2ae0d0 2023-09-20T21:48:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.451+0000 7fadfffff700 1 -- 172.21.15.22:0/1915687474 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fae0c017070 con 0x55611f2ae0d0 2023-09-20T21:48:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fadfffff700 1 -- 172.21.15.22:0/1915687474 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae0c029a20 con 0x55611f2ae0d0 2023-09-20T21:48:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fae3a4d8b80 10 client.25546 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1915687474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55611f0502d0 con 0x55611f2ae0d0 2023-09-20T21:48:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fadfffff700 1 -- 172.21.15.22:0/1915687474 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae1801f070 con 0x55611f2ae0d0 2023-09-20T21:48:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fadfffff700 1 -- 172.21.15.22:0/1915687474 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fae0c03b570 con 0x55611f2ae0d0 2023-09-20T21:48:58.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fae3a4d8b80 10 client.25546 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:58.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fae3a4d8b80 10 client.25546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:48:58.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fae3a4d8b80 20 client.25546 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:58.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55611f318450 0x55611f31a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fae3a4d8b80 4 client.25546 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-20T21:48:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1915687474 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55611f1f7ee0 con 0x55611f318450 2023-09-20T21:48:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.452+0000 7fae2499b700 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55611f318450 0x55611f31a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.453+0000 7fae2499b700 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55611f318450 0x55611f31a870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6330a532909a4616 server_cookie=46274aaa6291e1b2 in_seq=0 out_seq=0 2023-09-20T21:48:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.453+0000 7fadfffff700 10 client.25546 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fadfffff700 1 -- 172.21.15.22:0/1915687474 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55611f1f7ee0 con 0x55611f318450 2023-09-20T21:48:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fadfffff700 10 client.25546 handle_command_reply: tid=0 2023-09-20T21:48:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 client.25546 shutdown 2023-09-20T21:48:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1915687474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 msgr2=0x55611f2bccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2bccc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0097c0 tx=0x7fae0c0296c0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1915687474 shutdown_connections 2023-09-20T21:48:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55611f318450 0x55611f31a870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55611f2aec60 0x55611f2bd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55611f2ad770 0x55611f2be6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fadfffff700 0 client.25546 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 --2- 172.21.15.22:0/1915687474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55611f2ae0d0 0x55611f2bccc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1915687474 >> 172.21.15.22:0/1915687474 conn(0x55611f1da080 msgr2=0x55611f2b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1915687474 shutdown_connections 2023-09-20T21:48:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 1 -- 172.21.15.22:0/1915687474 wait complete. 2023-09-20T21:48:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.454+0000 7fae3a4d8b80 20 client.25546 trim_cache size 0 max 16384 2023-09-20T21:48:58.467 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2023-09-20T21:48:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "b08bee00-9608-4acf-be65-ebce29d9ff14", 2023-09-20T21:48:58.468 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2023-09-20T21:48:58.468 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:58.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 -- 172.21.15.22:0/2757786365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fae100a1380 msgr2=0x7fae100a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fae100a1380 0x7fae100a3840 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fae1800ebd0 tx=0x7fae18010000 comp rx=0 tx=0).stop 2023-09-20T21:48:58.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 -- 172.21.15.22:0/2757786365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae20073840 msgr2=0x7fae2019d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae20073840 0x7fae2019d180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae14009230 tx=0x7fae1400b790 comp rx=0 tx=0).stop 2023-09-20T21:48:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 -- 172.21.15.22:0/2757786365 shutdown_connections 2023-09-20T21:48:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae2010d3a0 0x7fae201a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fae100a1380 0x7fae100a3840 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae20073840 0x7fae2019d180 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 --2- 172.21.15.22:0/2757786365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae20074e20 0x7fae2019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 -- 172.21.15.22:0/2757786365 >> 172.21.15.22:0/2757786365 conn(0x7fae200fc320 msgr2=0x7fae200fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 -- 172.21.15.22:0/2757786365 shutdown_connections 2023-09-20T21:48:58.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.455+0000 7fae26bff700 1 -- 172.21.15.22:0/2757786365 wait complete. 2023-09-20T21:48:58.474 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:48:58.492 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-09-20T21:48:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:58 smithi123 ceph-mon[134433]: pgmap v348: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-20T21:48:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:58 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2139377608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:58 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/159023937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:58 smithi089 ceph-mon[134090]: pgmap v348: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2023-09-20T21:48:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:58 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2139377608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:48:58.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:58 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/159023937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T21:48:58.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-09-20T21:48:58.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.685+0000 7f53227ff700 1 -- 172.21.15.22:0/3918376812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f531c074e20 msgr2=0x7f531c073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.685+0000 7f53227ff700 1 --2- 172.21.15.22:0/3918376812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f531c074e20 0x7f531c073270 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f530c009990 tx=0x7f530c009ca0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.685+0000 7f53227ff700 1 -- 172.21.15.22:0/3918376812 shutdown_connections 2023-09-20T21:48:58.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.685+0000 7f53227ff700 1 --2- 172.21.15.22:0/3918376812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c10d3a0 0x7f531c10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.686+0000 7f53227ff700 1 --2- 172.21.15.22:0/3918376812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f531c074e20 0x7f531c073270 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.686+0000 7f53227ff700 1 --2- 172.21.15.22:0/3918376812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c073840 0x7f531c073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.686+0000 7f53227ff700 1 -- 172.21.15.22:0/3918376812 >> 172.21.15.22:0/3918376812 conn(0x7f531c0fc320 msgr2=0x7f531c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.686+0000 7f53227ff700 1 -- 172.21.15.22:0/3918376812 shutdown_connections 2023-09-20T21:48:58.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.686+0000 7f53227ff700 1 -- 172.21.15.22:0/3918376812 wait complete. 2023-09-20T21:48:58.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f53227ff700 1 Processor -- start 2023-09-20T21:48:58.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f53227ff700 1 -- start start 2023-09-20T21:48:58.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f53227ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c073840 0x7f531c19d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f53227ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f531c074e20 0x7f531c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f53227ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c10d3a0 0x7f531c1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f53227ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c1a4c90 con 0x7f531c074e20 2023-09-20T21:48:58.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f53227ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c1a4dd0 con 0x7f531c10d3a0 2023-09-20T21:48:58.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f53227ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c1a4f60 con 0x7f531c073840 2023-09-20T21:48:58.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f531c074e20 0x7f531c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f531c074e20 0x7f531c19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45338/0 (socket says 172.21.15.22:45338) 2023-09-20T21:48:58.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f531b7fe700 1 -- 172.21.15.22:0/3306635413 learned_addr learned my addr 172.21.15.22:0/3306635413 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:58.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f531bfff700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c073840 0x7f531c19d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.687+0000 7f5320d9c700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c10d3a0 0x7f531c1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.688+0000 7f531b7fe700 1 -- 172.21.15.22:0/3306635413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c073840 msgr2=0x7f531c19d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.688+0000 7f531b7fe700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c073840 0x7f531c19d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.688+0000 7f531b7fe700 1 -- 172.21.15.22:0/3306635413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c10d3a0 msgr2=0x7f531c1a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.688+0000 7f531b7fe700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c10d3a0 0x7f531c1a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.688+0000 7f531b7fe700 1 -- 172.21.15.22:0/3306635413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0095f0 con 0x7f531c074e20 2023-09-20T21:48:58.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.688+0000 7f531bfff700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c073840 0x7f531c19d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:58.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.688+0000 7f531b7fe700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f531c074e20 0x7f531c19d650 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f531000e970 tx=0x7f531000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.688+0000 7f53197fa700 1 -- 172.21.15.22:0/3306635413 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310009790 con 0x7f531c074e20 2023-09-20T21:48:58.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.688+0000 7f53227ff700 1 -- 172.21.15.22:0/3306635413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c1a5240 con 0x7f531c074e20 2023-09-20T21:48:58.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.688+0000 7f53197fa700 1 -- 172.21.15.22:0/3306635413 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531000c7e0 con 0x7f531c074e20 2023-09-20T21:48:58.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.689+0000 7f53197fa700 1 -- 172.21.15.22:0/3306635413 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310003c70 con 0x7f531c074e20 2023-09-20T21:48:58.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.689+0000 7f53227ff700 1 -- 172.21.15.22:0/3306635413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c1a57a0 con 0x7f531c074e20 2023-09-20T21:48:58.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.693+0000 7f53361a4b80 1 Processor -- start 2023-09-20T21:48:58.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.693+0000 7f53197fa700 1 -- 172.21.15.22:0/3306635413 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f531000cc50 con 0x7f531c074e20 2023-09-20T21:48:58.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.693+0000 7f53361a4b80 1 -- start start 2023-09-20T21:48:58.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd86b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd875e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87be0 0x55a52bd88060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f53361a4b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bcfaca0 con 0x55a52bd87be0 2023-09-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f53361a4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bcdec70 con 0x55a52bd866f0 2023-09-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f53361a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bcdc590 con 0x55a52bd87050 2023-09-20T21:48:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87be0 0x55a52bd88060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f531bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f5320d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd86b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f531bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd875e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:42760/0 (socket says 172.21.15.22:42760) 2023-09-20T21:48:58.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87be0 0x55a52bd88060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45352/0 (socket says 172.21.15.22:45352) 2023-09-20T21:48:58.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.694+0000 7f531bfff700 1 -- 172.21.15.22:0/1409387959 learned_addr learned my addr 172.21.15.22:0/1409387959 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f5320d9c700 1 -- 172.21.15.22:0/1409387959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 msgr2=0x55a52bd875e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f5320d9c700 1 --2- 172.21.15.22:0/1409387959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd875e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f5320d9c700 1 -- 172.21.15.22:0/1409387959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87be0 msgr2=0x55a52bd88060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f5320d9c700 1 --2- 172.21.15.22:0/1409387959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87be0 0x55a52bd88060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f5320d9c700 1 -- 172.21.15.22:0/1409387959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a52bb02de0 con 0x55a52bd866f0 2023-09-20T21:48:58.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f531bfff700 1 --2- 172.21.15.22:0/1409387959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd875e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:58.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f5320d9c700 1 --2- 172.21.15.22:0/1409387959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd86b10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5314011580 tx=0x7f5314009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b08f736c718aa529 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f53197fa700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f53040a1380 0x7f53040a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f53027fc700 1 -- 172.21.15.22:0/1409387959 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531400e040 con 0x55a52bd866f0 2023-09-20T21:48:58.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f53027fc700 1 -- 172.21.15.22:0/1409387959 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531400f040 con 0x55a52bd866f0 2023-09-20T21:48:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.695+0000 7f53027fc700 1 -- 172.21.15.22:0/1409387959 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53140043f0 con 0x55a52bd866f0 2023-09-20T21:48:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.696+0000 7f53197fa700 1 -- 172.21.15.22:0/3306635413 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f5310014070 con 0x7f531c074e20 2023-09-20T21:48:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.696+0000 7f531bfff700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f53040a1380 0x7f53040a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.696+0000 7f53361a4b80 1 -- 172.21.15.22:0/1409387959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 msgr2=0x55a52bd86b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.696+0000 7f53361a4b80 1 --2- 172.21.15.22:0/1409387959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd86b10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5314011580 tx=0x7f5314009920 comp rx=0 tx=0).stop 2023-09-20T21:48:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.696+0000 7f531bfff700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f53040a1380 0x7f53040a3840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55a52bd82150 tx=0x7f530c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.696+0000 7f53361a4b80 1 -- 172.21.15.22:0/1409387959 shutdown_connections 2023-09-20T21:48:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.696+0000 7f53361a4b80 1 --2- 172.21.15.22:0/1409387959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87be0 0x55a52bd88060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.696+0000 7f53361a4b80 1 --2- 172.21.15.22:0/1409387959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd86b10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.696+0000 7f53361a4b80 1 --2- 172.21.15.22:0/1409387959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd875e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.697+0000 7f53361a4b80 1 -- 172.21.15.22:0/1409387959 >> 172.21.15.22:0/1409387959 conn(0x55a52bcb3000 msgr2=0x55a52bd811d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.697+0000 7f53361a4b80 1 -- 172.21.15.22:0/1409387959 shutdown_connections 2023-09-20T21:48:58.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.697+0000 7f53361a4b80 1 -- 172.21.15.22:0/1409387959 wait complete. 2023-09-20T21:48:58.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.697+0000 7f53361a4b80 1 -- start start 2023-09-20T21:48:58.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.697+0000 7f53361a4b80 1 Processor -- start 2023-09-20T21:48:58.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd90200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd90740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87be0 0x55a52bd97780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f53361a4b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bcd7e80 con 0x55a52bd866f0 2023-09-20T21:48:58.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f53361a4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bc74740 con 0x55a52bd87be0 2023-09-20T21:48:58.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f53361a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bc743e0 con 0x55a52bd87050 2023-09-20T21:48:58.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f53361a4b80 10 client.0 _ll_register_callbacks cb 0x55a52bbbea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:58.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f531bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd90200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f531bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd90200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45362/0 (socket says 172.21.15.22:45362) 2023-09-20T21:48:58.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f531bfff700 1 -- 172.21.15.22:0/744714348 learned_addr learned my addr 172.21.15.22:0/744714348 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:58.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.698+0000 7f531b7fe700 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd90740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f5320d9c700 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87be0 0x55a52bd97780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f531bfff700 1 -- 172.21.15.22:0/744714348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 msgr2=0x55a52bd90740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f531bfff700 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd90740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f531bfff700 1 -- 172.21.15.22:0/744714348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87be0 msgr2=0x55a52bd97780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f531bfff700 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87be0 0x55a52bd97780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f531bfff700 1 -- 172.21.15.22:0/744714348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c110140 con 0x55a52bd866f0 2023-09-20T21:48:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f531b7fe700 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd90740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f531bfff700 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd90200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b680 tx=0x7f530c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fad8a5b55f82947 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f52fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f52fb7fe700 1 -- 172.21.15.22:0/744714348 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c026870 con 0x55a52bd866f0 2023-09-20T21:48:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f53361a4b80 1 -- 172.21.15.22:0/744714348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a52bc48bb0 con 0x55a52bd866f0 2023-09-20T21:48:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.699+0000 7f52fb7fe700 1 -- 172.21.15.22:0/744714348 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f530c0269d0 con 0x55a52bd866f0 2023-09-20T21:48:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f52fb7fe700 1 -- 172.21.15.22:0/744714348 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f530c026b90 con 0x55a52bd866f0 2023-09-20T21:48:58.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f53361a4b80 10 client.15531 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:58.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f53361a4b80 1 -- 172.21.15.22:0/744714348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f5314011230 con 0x55a52bd866f0 2023-09-20T21:48:58.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f52fb7fe700 1 -- 172.21.15.22:0/744714348 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a52bc48bb0 con 0x55a52bd866f0 2023-09-20T21:48:58.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f52fb7fe700 1 -- 172.21.15.22:0/744714348 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f530c039070 con 0x55a52bd866f0 2023-09-20T21:48:58.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f53361a4b80 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:58.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f53361a4b80 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:48:58.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f53361a4b80 20 client.15531 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:58.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f53361a4b80 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a52bd9bea0 0x55a52bd9e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f53361a4b80 4 client.15531 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:48:58.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.700+0000 7f53361a4b80 1 -- 172.21.15.22:0/744714348 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a52bd55020 con 0x55a52bd9bea0 2023-09-20T21:48:58.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.701+0000 7f531b7fe700 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a52bd9bea0 0x55a52bd9e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.701+0000 7f531b7fe700 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a52bd9bea0 0x55a52bd9e2c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237ed3eca91bef92 server_cookie=f36ffeaeadb6b5c8 in_seq=0 out_seq=0 2023-09-20T21:48:58.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.701+0000 7f52fb7fe700 10 client.15531 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:58.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f52fb7fe700 1 -- 172.21.15.22:0/744714348 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a52bd55020 con 0x55a52bd9bea0 2023-09-20T21:48:58.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f52fb7fe700 10 client.15531 handle_command_reply: tid=0 2023-09-20T21:48:58.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 client.15531 shutdown 2023-09-20T21:48:58.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 -- 172.21.15.22:0/744714348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 msgr2=0x55a52bd90200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd90200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b680 tx=0x7f530c009310 comp rx=0 tx=0).stop 2023-09-20T21:48:58.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 -- 172.21.15.22:0/744714348 shutdown_connections 2023-09-20T21:48:58.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a52bd9bea0 0x55a52bd9e2c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87be0 0x55a52bd97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd90200 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 --2- 172.21.15.22:0/744714348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87050 0x55a52bd90740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f52fb7fe700 0 client.15531 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:58.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 -- 172.21.15.22:0/744714348 >> 172.21.15.22:0/744714348 conn(0x55a52bcb3000 msgr2=0x55a52bd80880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 -- 172.21.15.22:0/744714348 shutdown_connections 2023-09-20T21:48:58.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.702+0000 7f53361a4b80 1 -- 172.21.15.22:0/744714348 wait complete. 2023-09-20T21:48:58.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.703+0000 7f53361a4b80 20 client.15531 trim_cache size 0 max 16384 2023-09-20T21:48:58.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.721+0000 7f53361a4b80 1 Processor -- start 2023-09-20T21:48:58.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.721+0000 7f53361a4b80 1 -- start start 2023-09-20T21:48:58.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.721+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd87fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.721+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87050 0x55a52bd97740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 0x55a52bd8f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f53361a4b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bb58920 con 0x55a52bd87050 2023-09-20T21:48:58.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f53361a4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bd8f6b0 con 0x55a52bd866f0 2023-09-20T21:48:58.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f53361a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bd87a40 con 0x55a52bd87be0 2023-09-20T21:48:58.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f531bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd87fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f531bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd87fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38424/0 (socket says 172.21.15.22:38424) 2023-09-20T21:48:58.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f531bfff700 1 -- 172.21.15.22:0/612740626 learned_addr learned my addr 172.21.15.22:0/612740626 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:58.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f531b7fe700 1 --2- 172.21.15.22:0/612740626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87050 0x55a52bd97740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f5320d9c700 1 --2- 172.21.15.22:0/612740626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 0x55a52bd8f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f531bfff700 1 -- 172.21.15.22:0/612740626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 msgr2=0x55a52bd8f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f531bfff700 1 --2- 172.21.15.22:0/612740626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 0x55a52bd8f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f531bfff700 1 -- 172.21.15.22:0/612740626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87050 msgr2=0x55a52bd97740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f531bfff700 1 --2- 172.21.15.22:0/612740626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87050 0x55a52bd97740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f531bfff700 1 -- 172.21.15.22:0/612740626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a52baebc60 con 0x55a52bd866f0 2023-09-20T21:48:58.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.722+0000 7f5320d9c700 1 --2- 172.21.15.22:0/612740626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 0x55a52bd8f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:48:58.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.723+0000 7f531bfff700 1 --2- 172.21.15.22:0/612740626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd87fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f530c00fde0 tx=0x7f530c004e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110af2a287539d9b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.723+0000 7f5302ffd700 1 -- 172.21.15.22:0/612740626 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00f660 con 0x55a52bd866f0 2023-09-20T21:48:58.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.723+0000 7f5302ffd700 1 -- 172.21.15.22:0/612740626 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f530c01e040 con 0x55a52bd866f0 2023-09-20T21:48:58.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.723+0000 7f5302ffd700 1 -- 172.21.15.22:0/612740626 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c041440 con 0x55a52bd866f0 2023-09-20T21:48:58.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 -- 172.21.15.22:0/612740626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 msgr2=0x55a52bd87fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 --2- 172.21.15.22:0/612740626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd87fc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f530c00fde0 tx=0x7f530c004e80 comp rx=0 tx=0).stop 2023-09-20T21:48:58.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 -- 172.21.15.22:0/612740626 shutdown_connections 2023-09-20T21:48:58.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 --2- 172.21.15.22:0/612740626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 0x55a52bd8f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 --2- 172.21.15.22:0/612740626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd866f0 0x55a52bd87fc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 --2- 172.21.15.22:0/612740626 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd87050 0x55a52bd97740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 -- 172.21.15.22:0/612740626 >> 172.21.15.22:0/612740626 conn(0x55a52bcb3000 msgr2=0x55a52bd7d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 -- 172.21.15.22:0/612740626 shutdown_connections 2023-09-20T21:48:58.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 -- 172.21.15.22:0/612740626 wait complete. 2023-09-20T21:48:58.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 -- start start 2023-09-20T21:48:58.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.724+0000 7f53361a4b80 1 Processor -- start 2023-09-20T21:48:58.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd94a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87050 0x55a52bd94f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f53361a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 0x55a52bd95f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f53361a4b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bd964b0 con 0x55a52bd866f0 2023-09-20T21:48:58.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f53361a4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bd96620 con 0x55a52bd87050 2023-09-20T21:48:58.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f53361a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a52bd967e0 con 0x55a52bd87be0 2023-09-20T21:48:58.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f53361a4b80 10 client.0 _ll_register_callbacks cb 0x55a52bbbea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:48:58.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f531bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd94a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f531bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd94a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45386/0 (socket says 172.21.15.22:45386) 2023-09-20T21:48:58.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f531bfff700 1 -- 172.21.15.22:0/1330378395 learned_addr learned my addr 172.21.15.22:0/1330378395 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:48:58.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f531b7fe700 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87050 0x55a52bd94f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.725+0000 7f5320d9c700 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 0x55a52bd95f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f531bfff700 1 -- 172.21.15.22:0/1330378395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 msgr2=0x55a52bd95f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f531bfff700 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 0x55a52bd95f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f531bfff700 1 -- 172.21.15.22:0/1330378395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87050 msgr2=0x55a52bd94f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:48:58.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f531bfff700 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87050 0x55a52bd94f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f531bfff700 1 -- 172.21.15.22:0/1330378395 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5314011230 con 0x55a52bd866f0 2023-09-20T21:48:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f531b7fe700 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87050 0x55a52bd94f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:48:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f531bfff700 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd94a20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f530c00fd60 tx=0x7f530c005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab9ac0edcb81876 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:48:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f52fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:48:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f52fb7fe700 1 -- 172.21.15.22:0/1330378395 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00f870 con 0x55a52bd866f0 2023-09-20T21:48:58.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f53361a4b80 1 -- 172.21.15.22:0/1330378395 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f531400f040 con 0x55a52bd866f0 2023-09-20T21:48:58.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.726+0000 7f52fb7fe700 1 -- 172.21.15.22:0/1330378395 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f530c00f9d0 con 0x55a52bd866f0 2023-09-20T21:48:58.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f52fb7fe700 1 -- 172.21.15.22:0/1330378395 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f530c009290 con 0x55a52bd866f0 2023-09-20T21:48:58.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f53361a4b80 10 client.15537 fetch_fsmap learned FSMap version 28 2023-09-20T21:48:58.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f53361a4b80 1 -- 172.21.15.22:0/1330378395 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a52bb29250 con 0x55a52bd866f0 2023-09-20T21:48:58.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f52fb7fe700 1 -- 172.21.15.22:0/1330378395 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531400f040 con 0x55a52bd866f0 2023-09-20T21:48:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f52fb7fe700 1 -- 172.21.15.22:0/1330378395 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f530c004810 con 0x55a52bd866f0 2023-09-20T21:48:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f53361a4b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2023-09-20T21:48:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f53361a4b80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:48:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f53361a4b80 20 client.15537 populate_metadata read hostname 'smithi022' 2023-09-20T21:48:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f53361a4b80 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a52bdefd60 0x55a52bdf2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:48:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f53361a4b80 4 client.15537 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:48:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.727+0000 7f53361a4b80 1 -- 172.21.15.22:0/1330378395 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a52bcd0e60 con 0x55a52bdefd60 2023-09-20T21:48:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.728+0000 7f531b7fe700 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a52bdefd60 0x55a52bdf2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:48:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.728+0000 7f531b7fe700 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a52bdefd60 0x55a52bdf2180 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea737e8edf4d2a2 server_cookie=63ad2872ebc1f715 in_seq=0 out_seq=0 2023-09-20T21:48:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.728+0000 7f52fb7fe700 10 client.15537 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:58.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.728+0000 7f52fb7fe700 1 -- 172.21.15.22:0/1330378395 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a52bcd0e60 con 0x55a52bdefd60 2023-09-20T21:48:58.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.728+0000 7f52fb7fe700 10 client.15537 handle_command_reply: tid=0 2023-09-20T21:48:58.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.728+0000 7f53361a4b80 1 client.15537 shutdown 2023-09-20T21:48:58.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.728+0000 7f53361a4b80 1 -- 172.21.15.22:0/1330378395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 msgr2=0x55a52bd94a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.728+0000 7f53361a4b80 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd94a20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f530c00fd60 tx=0x7f530c005a50 comp rx=0 tx=0).stop 2023-09-20T21:48:58.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.729+0000 7f53361a4b80 1 -- 172.21.15.22:0/1330378395 shutdown_connections 2023-09-20T21:48:58.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.729+0000 7f53361a4b80 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a52bdefd60 0x55a52bdf2180 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.729+0000 7f53361a4b80 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a52bd87be0 0x55a52bd95f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.729+0000 7f53361a4b80 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a52bd866f0 0x55a52bd94a20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.729+0000 7f53361a4b80 1 --2- 172.21.15.22:0/1330378395 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a52bd87050 0x55a52bd94f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.729+0000 7f52fb7fe700 0 client.15537 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:48:58.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.729+0000 7f53361a4b80 1 -- 172.21.15.22:0/1330378395 >> 172.21.15.22:0/1330378395 conn(0x55a52bcb3000 msgr2=0x55a52bd8e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.729+0000 7f53361a4b80 1 -- 172.21.15.22:0/1330378395 shutdown_connections 2023-09-20T21:48:58.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.729+0000 7f53361a4b80 1 -- 172.21.15.22:0/1330378395 wait complete. 2023-09-20T21:48:58.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.729+0000 7f53361a4b80 20 client.15537 trim_cache size 0 max 16384 2023-09-20T21:48:58.744 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:48:58.744 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-20T21:48:58.744 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:48:58.744 INFO:teuthology.orchestra.run.smithi022.stdout: "b08bee00-9608-4acf-be65-ebce29d9ff14": { 2023-09-20T21:48:58.745 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:48:58.745 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "b08bee00-9608-4acf-be65-ebce29d9ff14", 2023-09-20T21:48:58.745 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:48:58.745 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:48:58.745 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:48:58.746 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:48:58.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 -- 172.21.15.22:0/3306635413 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f53040a1380 msgr2=0x7f53040a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f53040a1380 0x7f53040a3840 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55a52bd82150 tx=0x7f530c005fb0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 -- 172.21.15.22:0/3306635413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f531c074e20 msgr2=0x7f531c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:48:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f531c074e20 0x7f531c19d650 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f531000e970 tx=0x7f531000ec80 comp rx=0 tx=0).stop 2023-09-20T21:48:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 -- 172.21.15.22:0/3306635413 shutdown_connections 2023-09-20T21:48:58.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f53040a1380 0x7f53040a3840 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c10d3a0 0x7f531c1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f531c073840 0x7f531c19d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 --2- 172.21.15.22:0/3306635413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f531c074e20 0x7f531c19d650 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:48:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 -- 172.21.15.22:0/3306635413 >> 172.21.15.22:0/3306635413 conn(0x7f531c0fc320 msgr2=0x7f531c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:48:58.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 -- 172.21.15.22:0/3306635413 shutdown_connections 2023-09-20T21:48:58.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:48:58.730+0000 7f53227ff700 1 -- 172.21.15.22:0/3306635413 wait complete. 2023-09-20T21:48:58.751 INFO:tasks.cephfs.filesystem:scrub status for tag:b08bee00-9608-4acf-be65-ebce29d9ff14 - {'path': '/', 'tag': 'b08bee00-9608-4acf-be65-ebce29d9ff14', 'options': 'recursive,force'} 2023-09-20T21:48:59.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:48:59 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:48:59.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:48:59 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:48:59.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:48:59 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:49:00.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:00 smithi022 ceph-mon[97651]: scrub queued for path: / 2023-09-20T21:49:00.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:00 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:00.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:00 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:49:00.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:00 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:00.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:00 smithi022 ceph-mon[97651]: pgmap v349: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-20T21:49:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:00 smithi123 ceph-mon[134433]: scrub queued for path: / 2023-09-20T21:49:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:00 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:00.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:00 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:49:00.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:00 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:00.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:00 smithi123 ceph-mon[134433]: pgmap v349: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-20T21:49:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:00 smithi089 ceph-mon[134090]: scrub queued for path: / 2023-09-20T21:49:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:00 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:00 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:49:00.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:00 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:00.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:00 smithi089 ceph-mon[134090]: pgmap v349: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-20T21:49:02.264 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-09-20T21:49:02.372 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-09-20T21:49:02.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-09-20T21:49:02.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-20T21:49:02.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:02 smithi022 ceph-mon[97651]: pgmap v350: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-20T21:49:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:02 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:02 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:02 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:02 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.589 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/: 2023-09-20T21:49:02.589 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:02.590 INFO:teuthology.orchestra.run.smithi022.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:48 . 2023-09-20T21:49:02.590 INFO:teuthology.orchestra.run.smithi022.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:48 .. 2023-09-20T21:49:02.590 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:48 data 2023-09-20T21:49:02.590 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:02.591 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data: 2023-09-20T21:49:02.591 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:02.591 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:48 . 2023-09-20T21:49:02.591 INFO:teuthology.orchestra.run.smithi022.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:48 .. 2023-09-20T21:49:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:02 smithi123 ceph-mon[134433]: pgmap v350: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-20T21:49:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:02 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:02 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:02 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:02 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.595 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-09-20T21:49:02.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:02 smithi089 ceph-mon[134090]: pgmap v350: 129 pgs: 129 active+clean; 587 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-20T21:49:02.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:02 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:02 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:02 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:02 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:02.764 INFO:teuthology.orchestra.run.smithi022.stdout:[Service] 2023-09-20T21:49:02.765 INFO:teuthology.orchestra.run.smithi022.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-09-20T21:49:02.765 INFO:teuthology.orchestra.run.smithi022.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-09-20T21:49:02.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-09-20T21:49:02.963 INFO:teuthology.orchestra.run.smithi022.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-09-20T21:49:03.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:03 smithi089 ceph-mon[134090]: pgmap v351: 129 pgs: 129 active+clean; 593 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2023-09-20T21:49:03.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:03.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:03 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:03.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:03 smithi022 ceph-mon[97651]: pgmap v351: 129 pgs: 129 active+clean; 593 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2023-09-20T21:49:03.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:03.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:03 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:04.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:03 smithi123 ceph-mon[134433]: pgmap v351: 129 pgs: 129 active+clean; 593 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2023-09-20T21:49:04.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:04.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:03 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:04.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:04 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:49:04.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:04 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:49:04.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:04 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:04.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:04 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:49:04.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:04 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:49:04.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:04 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:05.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:04 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:49:05.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:04 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:49:05.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:04 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:49:05.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:05 smithi089 ceph-mon[134090]: pgmap v352: 129 pgs: 129 active+clean; 9.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 768 KiB/s wr, 12 op/s 2023-09-20T21:49:05.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:05 smithi022 ceph-mon[97651]: pgmap v352: 129 pgs: 129 active+clean; 9.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 768 KiB/s wr, 12 op/s 2023-09-20T21:49:06.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:05 smithi123 ceph-mon[134433]: pgmap v352: 129 pgs: 129 active+clean; 9.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 768 KiB/s wr, 12 op/s 2023-09-20T21:49:08.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:08 smithi022 ceph-mon[97651]: pgmap v353: 129 pgs: 129 active+clean; 20 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.6 MiB/s wr, 20 op/s 2023-09-20T21:49:08.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:08 smithi123 ceph-mon[134433]: pgmap v353: 129 pgs: 129 active+clean; 20 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.6 MiB/s wr, 20 op/s 2023-09-20T21:49:08.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:08 smithi089 ceph-mon[134090]: pgmap v353: 129 pgs: 129 active+clean; 20 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.6 MiB/s wr, 20 op/s 2023-09-20T21:49:10.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:10 smithi022 ceph-mon[97651]: pgmap v354: 129 pgs: 129 active+clean; 20 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.6 MiB/s wr, 20 op/s 2023-09-20T21:49:10.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:10 smithi123 ceph-mon[134433]: pgmap v354: 129 pgs: 129 active+clean; 20 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.6 MiB/s wr, 20 op/s 2023-09-20T21:49:10.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:10 smithi089 ceph-mon[134090]: pgmap v354: 129 pgs: 129 active+clean; 20 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.6 MiB/s wr, 20 op/s 2023-09-20T21:49:11.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:11 smithi022 ceph-mon[97651]: scrub summary: idle 2023-09-20T21:49:11.517 INFO:teuthology.orchestra.run.smithi022.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-09-20T21:49:11.524 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-20T21:49:11.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:11 smithi123 ceph-mon[134433]: scrub summary: idle 2023-09-20T21:49:11.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:11 smithi089 ceph-mon[134090]: scrub summary: idle 2023-09-20T21:49:11.638 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/: 2023-09-20T21:49:11.639 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.639 INFO:teuthology.orchestra.run.smithi022.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:48 . 2023-09-20T21:49:11.639 INFO:teuthology.orchestra.run.smithi022.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:48 .. 2023-09-20T21:49:11.639 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 data 2023-09-20T21:49:11.640 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.640 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data: 2023-09-20T21:49:11.640 INFO:teuthology.orchestra.run.smithi022.stdout:total 30 2023-09-20T21:49:11.640 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 . 2023-09-20T21:49:11.640 INFO:teuthology.orchestra.run.smithi022.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:48 .. 2023-09-20T21:49:11.641 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 base 2023-09-20T21:49:11.641 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 20 21:49 global 2023-09-20T21:49:11.641 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 log 2023-09-20T21:49:11.641 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_commit_ts 2023-09-20T21:49:11.641 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_dynshmem 2023-09-20T21:49:11.642 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Sep 20 21:49 pg_hba.conf 2023-09-20T21:49:11.642 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 20 21:49 pg_ident.conf 2023-09-20T21:49:11.642 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 pg_logical 2023-09-20T21:49:11.642 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 pg_multixact 2023-09-20T21:49:11.642 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 pg_notify 2023-09-20T21:49:11.643 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_replslot 2023-09-20T21:49:11.643 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_serial 2023-09-20T21:49:11.643 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_snapshots 2023-09-20T21:49:11.643 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_stat 2023-09-20T21:49:11.643 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_stat_tmp 2023-09-20T21:49:11.644 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 pg_subtrans 2023-09-20T21:49:11.644 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_tblspc 2023-09-20T21:49:11.644 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_twophase 2023-09-20T21:49:11.644 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 PG_VERSION 2023-09-20T21:49:11.644 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 pg_wal 2023-09-20T21:49:11.645 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 pg_xact 2023-09-20T21:49:11.645 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 20 21:49 postgresql.auto.conf 2023-09-20T21:49:11.645 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Sep 20 21:49 postgresql.conf 2023-09-20T21:49:11.645 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.645 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/base: 2023-09-20T21:49:11.645 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.646 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 . 2023-09-20T21:49:11.646 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.646 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 20 21:49 1 2023-09-20T21:49:11.646 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 20 21:49 12346 2023-09-20T21:49:11.646 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 20 21:49 12347 2023-09-20T21:49:11.647 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.647 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/base/1: 2023-09-20T21:49:11.647 INFO:teuthology.orchestra.run.smithi022.stdout:total 7233 2023-09-20T21:49:11.647 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 20 21:49 . 2023-09-20T21:49:11.647 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 .. 2023-09-20T21:49:11.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 112 2023-09-20T21:49:11.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 113 2023-09-20T21:49:11.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 20 21:49 12182 2023-09-20T21:49:11.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12182_fsm 2023-09-20T21:49:11.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12182_vm 2023-09-20T21:49:11.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12184 2023-09-20T21:49:11.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12186 2023-09-20T21:49:11.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187 2023-09-20T21:49:11.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12187_fsm 2023-09-20T21:49:11.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187_vm 2023-09-20T21:49:11.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12189 2023-09-20T21:49:11.650 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12191 2023-09-20T21:49:11.650 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192 2023-09-20T21:49:11.650 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12192_fsm 2023-09-20T21:49:11.650 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192_vm 2023-09-20T21:49:11.650 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12194 2023-09-20T21:49:11.651 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12196 2023-09-20T21:49:11.651 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197 2023-09-20T21:49:11.651 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12197_fsm 2023-09-20T21:49:11.651 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197_vm 2023-09-20T21:49:11.651 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12199 2023-09-20T21:49:11.652 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12201 2023-09-20T21:49:11.652 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202 2023-09-20T21:49:11.652 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12202_fsm 2023-09-20T21:49:11.652 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202_vm 2023-09-20T21:49:11.652 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12204 2023-09-20T21:49:11.653 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12206 2023-09-20T21:49:11.653 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207 2023-09-20T21:49:11.653 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12207_fsm 2023-09-20T21:49:11.653 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207_vm 2023-09-20T21:49:11.653 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12209 2023-09-20T21:49:11.653 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12211 2023-09-20T21:49:11.654 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12212 2023-09-20T21:49:11.654 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12214 2023-09-20T21:49:11.654 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12216 2023-09-20T21:49:11.654 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 20 21:49 1247 2023-09-20T21:49:11.654 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1247_fsm 2023-09-20T21:49:11.655 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1247_vm 2023-09-20T21:49:11.655 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 20 21:49 1249 2023-09-20T21:49:11.655 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1249_fsm 2023-09-20T21:49:11.655 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1249_vm 2023-09-20T21:49:11.655 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 20 21:49 1255 2023-09-20T21:49:11.656 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1255_fsm 2023-09-20T21:49:11.656 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1255_vm 2023-09-20T21:49:11.656 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 20 21:49 1259 2023-09-20T21:49:11.656 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1259_fsm 2023-09-20T21:49:11.656 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1259_vm 2023-09-20T21:49:11.656 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417 2023-09-20T21:49:11.657 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417_vm 2023-09-20T21:49:11.657 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418 2023-09-20T21:49:11.657 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418_vm 2023-09-20T21:49:11.657 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 174 2023-09-20T21:49:11.657 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 175 2023-09-20T21:49:11.658 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2187 2023-09-20T21:49:11.658 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224 2023-09-20T21:49:11.658 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224_vm 2023-09-20T21:49:11.658 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328 2023-09-20T21:49:11.658 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328_vm 2023-09-20T21:49:11.658 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336 2023-09-20T21:49:11.659 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336_vm 2023-09-20T21:49:11.659 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2337 2023-09-20T21:49:11.659 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2600 2023-09-20T21:49:11.659 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2600_fsm 2023-09-20T21:49:11.659 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2600_vm 2023-09-20T21:49:11.660 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601 2023-09-20T21:49:11.660 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2601_fsm 2023-09-20T21:49:11.660 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601_vm 2023-09-20T21:49:11.660 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2602 2023-09-20T21:49:11.660 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2602_fsm 2023-09-20T21:49:11.660 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2602_vm 2023-09-20T21:49:11.661 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2603 2023-09-20T21:49:11.661 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2603_fsm 2023-09-20T21:49:11.661 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2603_vm 2023-09-20T21:49:11.661 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604 2023-09-20T21:49:11.661 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604_vm 2023-09-20T21:49:11.662 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2605 2023-09-20T21:49:11.662 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2605_fsm 2023-09-20T21:49:11.662 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2605_vm 2023-09-20T21:49:11.662 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606 2023-09-20T21:49:11.662 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2606_fsm 2023-09-20T21:49:11.662 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606_vm 2023-09-20T21:49:11.663 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607 2023-09-20T21:49:11.663 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607_fsm 2023-09-20T21:49:11.663 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2607_vm 2023-09-20T21:49:11.663 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 20 21:49 2608 2023-09-20T21:49:11.663 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2608_fsm 2023-09-20T21:49:11.664 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2608_vm 2023-09-20T21:49:11.664 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 20 21:49 2609 2023-09-20T21:49:11.664 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2609_fsm 2023-09-20T21:49:11.664 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2609_vm 2023-09-20T21:49:11.664 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610 2023-09-20T21:49:11.664 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610_fsm 2023-09-20T21:49:11.665 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2610_vm 2023-09-20T21:49:11.665 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611 2023-09-20T21:49:11.665 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611_vm 2023-09-20T21:49:11.665 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612 2023-09-20T21:49:11.665 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2612_fsm 2023-09-20T21:49:11.666 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612_vm 2023-09-20T21:49:11.666 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613 2023-09-20T21:49:11.666 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613_vm 2023-09-20T21:49:11.666 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615 2023-09-20T21:49:11.666 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2615_fsm 2023-09-20T21:49:11.666 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615_vm 2023-09-20T21:49:11.667 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616 2023-09-20T21:49:11.667 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616_fsm 2023-09-20T21:49:11.667 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2616_vm 2023-09-20T21:49:11.667 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 20 21:49 2617 2023-09-20T21:49:11.667 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2617_fsm 2023-09-20T21:49:11.668 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2617_vm 2023-09-20T21:49:11.668 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 20 21:49 2618 2023-09-20T21:49:11.668 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2618_fsm 2023-09-20T21:49:11.668 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2618_vm 2023-09-20T21:49:11.668 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 20 21:49 2619 2023-09-20T21:49:11.668 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2619_fsm 2023-09-20T21:49:11.669 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2619_vm 2023-09-20T21:49:11.669 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620 2023-09-20T21:49:11.669 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620_vm 2023-09-20T21:49:11.669 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2650 2023-09-20T21:49:11.669 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2651 2023-09-20T21:49:11.670 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2652 2023-09-20T21:49:11.670 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2653 2023-09-20T21:49:11.670 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2654 2023-09-20T21:49:11.670 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2655 2023-09-20T21:49:11.670 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2656 2023-09-20T21:49:11.670 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2657 2023-09-20T21:49:11.671 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 20 21:49 2658 2023-09-20T21:49:11.671 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2659 2023-09-20T21:49:11.671 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2660 2023-09-20T21:49:11.671 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2661 2023-09-20T21:49:11.671 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2662 2023-09-20T21:49:11.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2663 2023-09-20T21:49:11.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2664 2023-09-20T21:49:11.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2665 2023-09-20T21:49:11.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2666 2023-09-20T21:49:11.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2667 2023-09-20T21:49:11.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2668 2023-09-20T21:49:11.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2669 2023-09-20T21:49:11.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2670 2023-09-20T21:49:11.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 20 21:49 2673 2023-09-20T21:49:11.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Sep 20 21:49 2674 2023-09-20T21:49:11.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 20 21:49 2675 2023-09-20T21:49:11.674 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2678 2023-09-20T21:49:11.674 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2679 2023-09-20T21:49:11.674 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2680 2023-09-20T21:49:11.674 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2681 2023-09-20T21:49:11.674 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2682 2023-09-20T21:49:11.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2683 2023-09-20T21:49:11.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2684 2023-09-20T21:49:11.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2685 2023-09-20T21:49:11.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2686 2023-09-20T21:49:11.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2687 2023-09-20T21:49:11.676 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2688 2023-09-20T21:49:11.676 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2689 2023-09-20T21:49:11.676 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2690 2023-09-20T21:49:11.676 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 20 21:49 2691 2023-09-20T21:49:11.676 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2692 2023-09-20T21:49:11.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2693 2023-09-20T21:49:11.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2696 2023-09-20T21:49:11.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2699 2023-09-20T21:49:11.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2701 2023-09-20T21:49:11.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2702 2023-09-20T21:49:11.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2703 2023-09-20T21:49:11.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2704 2023-09-20T21:49:11.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2753 2023-09-20T21:49:11.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2753_fsm 2023-09-20T21:49:11.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2753_vm 2023-09-20T21:49:11.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2754 2023-09-20T21:49:11.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2755 2023-09-20T21:49:11.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2756 2023-09-20T21:49:11.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2757 2023-09-20T21:49:11.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830 2023-09-20T21:49:11.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830_vm 2023-09-20T21:49:11.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2831 2023-09-20T21:49:11.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832 2023-09-20T21:49:11.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832_vm 2023-09-20T21:49:11.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2833 2023-09-20T21:49:11.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834 2023-09-20T21:49:11.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834_vm 2023-09-20T21:49:11.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2835 2023-09-20T21:49:11.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836 2023-09-20T21:49:11.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836_vm 2023-09-20T21:49:11.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2837 2023-09-20T21:49:11.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 20 21:49 2838 2023-09-20T21:49:11.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2838_fsm 2023-09-20T21:49:11.682 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2838_vm 2023-09-20T21:49:11.682 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2839 2023-09-20T21:49:11.682 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840 2023-09-20T21:49:11.682 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840_fsm 2023-09-20T21:49:11.682 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2840_vm 2023-09-20T21:49:11.683 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2841 2023-09-20T21:49:11.683 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995 2023-09-20T21:49:11.683 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995_vm 2023-09-20T21:49:11.683 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2996 2023-09-20T21:49:11.683 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079 2023-09-20T21:49:11.684 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3079_fsm 2023-09-20T21:49:11.684 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079_vm 2023-09-20T21:49:11.684 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3080 2023-09-20T21:49:11.684 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3081 2023-09-20T21:49:11.684 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3085 2023-09-20T21:49:11.684 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118 2023-09-20T21:49:11.685 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118_vm 2023-09-20T21:49:11.685 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3119 2023-09-20T21:49:11.685 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3164 2023-09-20T21:49:11.685 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256 2023-09-20T21:49:11.685 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256_vm 2023-09-20T21:49:11.686 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3257 2023-09-20T21:49:11.686 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3258 2023-09-20T21:49:11.686 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350 2023-09-20T21:49:11.686 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350_vm 2023-09-20T21:49:11.686 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3351 2023-09-20T21:49:11.687 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3379 2023-09-20T21:49:11.687 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3380 2023-09-20T21:49:11.687 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381 2023-09-20T21:49:11.687 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381_vm 2023-09-20T21:49:11.687 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3394 2023-09-20T21:49:11.688 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3394_fsm 2023-09-20T21:49:11.688 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3394_vm 2023-09-20T21:49:11.688 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3395 2023-09-20T21:49:11.688 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439 2023-09-20T21:49:11.688 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439_vm 2023-09-20T21:49:11.689 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3440 2023-09-20T21:49:11.689 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 3455 2023-09-20T21:49:11.689 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3456 2023-09-20T21:49:11.689 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3456_fsm 2023-09-20T21:49:11.689 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3456_vm 2023-09-20T21:49:11.689 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466 2023-09-20T21:49:11.690 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466_vm 2023-09-20T21:49:11.690 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3467 2023-09-20T21:49:11.690 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3468 2023-09-20T21:49:11.690 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501 2023-09-20T21:49:11.690 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501_vm 2023-09-20T21:49:11.691 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3502 2023-09-20T21:49:11.691 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3503 2023-09-20T21:49:11.691 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3534 2023-09-20T21:49:11.691 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541 2023-09-20T21:49:11.691 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3541_fsm 2023-09-20T21:49:11.692 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541_vm 2023-09-20T21:49:11.692 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3542 2023-09-20T21:49:11.692 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3574 2023-09-20T21:49:11.692 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3575 2023-09-20T21:49:11.692 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576 2023-09-20T21:49:11.692 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576_vm 2023-09-20T21:49:11.693 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596 2023-09-20T21:49:11.693 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596_vm 2023-09-20T21:49:11.693 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3597 2023-09-20T21:49:11.693 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598 2023-09-20T21:49:11.693 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598_vm 2023-09-20T21:49:11.694 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3599 2023-09-20T21:49:11.694 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600 2023-09-20T21:49:11.694 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3600_fsm 2023-09-20T21:49:11.694 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600_vm 2023-09-20T21:49:11.694 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601 2023-09-20T21:49:11.694 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3601_fsm 2023-09-20T21:49:11.695 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601_vm 2023-09-20T21:49:11.695 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602 2023-09-20T21:49:11.695 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3602_fsm 2023-09-20T21:49:11.695 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602_vm 2023-09-20T21:49:11.695 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3603 2023-09-20T21:49:11.696 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3603_fsm 2023-09-20T21:49:11.696 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3603_vm 2023-09-20T21:49:11.696 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3604 2023-09-20T21:49:11.696 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3605 2023-09-20T21:49:11.696 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3606 2023-09-20T21:49:11.699 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3607 2023-09-20T21:49:11.700 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3608 2023-09-20T21:49:11.700 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 3609 2023-09-20T21:49:11.700 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3712 2023-09-20T21:49:11.700 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764 2023-09-20T21:49:11.700 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3764_fsm 2023-09-20T21:49:11.701 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764_vm 2023-09-20T21:49:11.701 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3766 2023-09-20T21:49:11.701 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3767 2023-09-20T21:49:11.701 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3997 2023-09-20T21:49:11.701 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 5002 2023-09-20T21:49:11.702 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 548 2023-09-20T21:49:11.702 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 549 2023-09-20T21:49:11.702 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102 2023-09-20T21:49:11.702 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102_vm 2023-09-20T21:49:11.702 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104 2023-09-20T21:49:11.703 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104_vm 2023-09-20T21:49:11.703 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106 2023-09-20T21:49:11.703 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106_vm 2023-09-20T21:49:11.703 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6110 2023-09-20T21:49:11.703 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6111 2023-09-20T21:49:11.704 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6112 2023-09-20T21:49:11.704 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6113 2023-09-20T21:49:11.704 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6117 2023-09-20T21:49:11.704 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826 2023-09-20T21:49:11.704 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826_vm 2023-09-20T21:49:11.704 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 827 2023-09-20T21:49:11.705 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 828 2023-09-20T21:49:11.705 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 20 21:49 pg_filenode.map 2023-09-20T21:49:11.705 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 PG_VERSION 2023-09-20T21:49:11.705 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.705 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-09-20T21:49:11.706 INFO:teuthology.orchestra.run.smithi022.stdout:total 7233 2023-09-20T21:49:11.706 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 20 21:49 . 2023-09-20T21:49:11.706 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 .. 2023-09-20T21:49:11.706 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 112 2023-09-20T21:49:11.706 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 113 2023-09-20T21:49:11.707 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 20 21:49 12182 2023-09-20T21:49:11.707 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12182_fsm 2023-09-20T21:49:11.707 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12182_vm 2023-09-20T21:49:11.707 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12184 2023-09-20T21:49:11.707 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12186 2023-09-20T21:49:11.708 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187 2023-09-20T21:49:11.708 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12187_fsm 2023-09-20T21:49:11.708 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187_vm 2023-09-20T21:49:11.708 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12189 2023-09-20T21:49:11.708 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12191 2023-09-20T21:49:11.708 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192 2023-09-20T21:49:11.709 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12192_fsm 2023-09-20T21:49:11.709 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192_vm 2023-09-20T21:49:11.709 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12194 2023-09-20T21:49:11.709 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12196 2023-09-20T21:49:11.709 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197 2023-09-20T21:49:11.710 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12197_fsm 2023-09-20T21:49:11.710 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197_vm 2023-09-20T21:49:11.710 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12199 2023-09-20T21:49:11.710 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12201 2023-09-20T21:49:11.710 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202 2023-09-20T21:49:11.711 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12202_fsm 2023-09-20T21:49:11.711 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202_vm 2023-09-20T21:49:11.711 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12204 2023-09-20T21:49:11.711 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12206 2023-09-20T21:49:11.711 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207 2023-09-20T21:49:11.711 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12207_fsm 2023-09-20T21:49:11.712 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207_vm 2023-09-20T21:49:11.712 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12209 2023-09-20T21:49:11.712 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12211 2023-09-20T21:49:11.712 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12212 2023-09-20T21:49:11.712 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12214 2023-09-20T21:49:11.713 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12216 2023-09-20T21:49:11.713 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 20 21:49 1247 2023-09-20T21:49:11.713 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1247_fsm 2023-09-20T21:49:11.713 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1247_vm 2023-09-20T21:49:11.713 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 20 21:49 1249 2023-09-20T21:49:11.713 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1249_fsm 2023-09-20T21:49:11.714 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1249_vm 2023-09-20T21:49:11.714 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 20 21:49 1255 2023-09-20T21:49:11.714 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1255_fsm 2023-09-20T21:49:11.714 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1255_vm 2023-09-20T21:49:11.714 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 20 21:49 1259 2023-09-20T21:49:11.715 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1259_fsm 2023-09-20T21:49:11.715 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1259_vm 2023-09-20T21:49:11.715 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417 2023-09-20T21:49:11.715 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417_vm 2023-09-20T21:49:11.715 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418 2023-09-20T21:49:11.716 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418_vm 2023-09-20T21:49:11.716 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 174 2023-09-20T21:49:11.716 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 175 2023-09-20T21:49:11.716 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2187 2023-09-20T21:49:11.716 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224 2023-09-20T21:49:11.716 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224_vm 2023-09-20T21:49:11.717 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328 2023-09-20T21:49:11.717 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328_vm 2023-09-20T21:49:11.717 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336 2023-09-20T21:49:11.717 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336_vm 2023-09-20T21:49:11.717 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2337 2023-09-20T21:49:11.718 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2600 2023-09-20T21:49:11.718 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2600_fsm 2023-09-20T21:49:11.718 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2600_vm 2023-09-20T21:49:11.718 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601 2023-09-20T21:49:11.718 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2601_fsm 2023-09-20T21:49:11.718 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601_vm 2023-09-20T21:49:11.719 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2602 2023-09-20T21:49:11.719 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2602_fsm 2023-09-20T21:49:11.719 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2602_vm 2023-09-20T21:49:11.719 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2603 2023-09-20T21:49:11.719 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2603_fsm 2023-09-20T21:49:11.720 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2603_vm 2023-09-20T21:49:11.720 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604 2023-09-20T21:49:11.720 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604_vm 2023-09-20T21:49:11.720 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2605 2023-09-20T21:49:11.720 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2605_fsm 2023-09-20T21:49:11.721 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2605_vm 2023-09-20T21:49:11.721 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606 2023-09-20T21:49:11.721 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2606_fsm 2023-09-20T21:49:11.721 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606_vm 2023-09-20T21:49:11.721 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607 2023-09-20T21:49:11.721 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607_fsm 2023-09-20T21:49:11.722 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2607_vm 2023-09-20T21:49:11.722 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 20 21:49 2608 2023-09-20T21:49:11.722 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2608_fsm 2023-09-20T21:49:11.722 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2608_vm 2023-09-20T21:49:11.722 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 20 21:49 2609 2023-09-20T21:49:11.723 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2609_fsm 2023-09-20T21:49:11.723 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2609_vm 2023-09-20T21:49:11.723 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610 2023-09-20T21:49:11.723 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610_fsm 2023-09-20T21:49:11.723 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2610_vm 2023-09-20T21:49:11.723 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611 2023-09-20T21:49:11.724 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611_vm 2023-09-20T21:49:11.724 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612 2023-09-20T21:49:11.724 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2612_fsm 2023-09-20T21:49:11.724 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612_vm 2023-09-20T21:49:11.724 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613 2023-09-20T21:49:11.725 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613_vm 2023-09-20T21:49:11.725 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615 2023-09-20T21:49:11.725 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2615_fsm 2023-09-20T21:49:11.725 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615_vm 2023-09-20T21:49:11.725 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616 2023-09-20T21:49:11.725 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616_fsm 2023-09-20T21:49:11.726 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2616_vm 2023-09-20T21:49:11.726 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 20 21:49 2617 2023-09-20T21:49:11.726 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2617_fsm 2023-09-20T21:49:11.726 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2617_vm 2023-09-20T21:49:11.726 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 20 21:49 2618 2023-09-20T21:49:11.727 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2618_fsm 2023-09-20T21:49:11.727 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2618_vm 2023-09-20T21:49:11.727 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 20 21:49 2619 2023-09-20T21:49:11.727 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2619_fsm 2023-09-20T21:49:11.727 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2619_vm 2023-09-20T21:49:11.728 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620 2023-09-20T21:49:11.728 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620_vm 2023-09-20T21:49:11.728 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2650 2023-09-20T21:49:11.728 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2651 2023-09-20T21:49:11.728 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2652 2023-09-20T21:49:11.728 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2653 2023-09-20T21:49:11.729 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2654 2023-09-20T21:49:11.729 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2655 2023-09-20T21:49:11.729 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2656 2023-09-20T21:49:11.729 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2657 2023-09-20T21:49:11.729 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 20 21:49 2658 2023-09-20T21:49:11.730 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2659 2023-09-20T21:49:11.730 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2660 2023-09-20T21:49:11.730 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2661 2023-09-20T21:49:11.730 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2662 2023-09-20T21:49:11.730 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2663 2023-09-20T21:49:11.731 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2664 2023-09-20T21:49:11.731 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2665 2023-09-20T21:49:11.731 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2666 2023-09-20T21:49:11.731 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2667 2023-09-20T21:49:11.731 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2668 2023-09-20T21:49:11.731 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2669 2023-09-20T21:49:11.732 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2670 2023-09-20T21:49:11.732 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 20 21:49 2673 2023-09-20T21:49:11.732 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Sep 20 21:49 2674 2023-09-20T21:49:11.732 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 20 21:49 2675 2023-09-20T21:49:11.732 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2678 2023-09-20T21:49:11.733 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2679 2023-09-20T21:49:11.733 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2680 2023-09-20T21:49:11.733 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2681 2023-09-20T21:49:11.733 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2682 2023-09-20T21:49:11.733 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2683 2023-09-20T21:49:11.734 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2684 2023-09-20T21:49:11.734 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2685 2023-09-20T21:49:11.734 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2686 2023-09-20T21:49:11.734 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2687 2023-09-20T21:49:11.734 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2688 2023-09-20T21:49:11.735 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2689 2023-09-20T21:49:11.735 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2690 2023-09-20T21:49:11.735 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 20 21:49 2691 2023-09-20T21:49:11.735 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2692 2023-09-20T21:49:11.735 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2693 2023-09-20T21:49:11.736 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2696 2023-09-20T21:49:11.736 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2699 2023-09-20T21:49:11.736 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2701 2023-09-20T21:49:11.736 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2702 2023-09-20T21:49:11.736 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2703 2023-09-20T21:49:11.737 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2704 2023-09-20T21:49:11.737 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2753 2023-09-20T21:49:11.737 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2753_fsm 2023-09-20T21:49:11.737 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2753_vm 2023-09-20T21:49:11.737 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2754 2023-09-20T21:49:11.737 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2755 2023-09-20T21:49:11.738 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2756 2023-09-20T21:49:11.738 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2757 2023-09-20T21:49:11.738 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830 2023-09-20T21:49:11.738 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830_vm 2023-09-20T21:49:11.738 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2831 2023-09-20T21:49:11.739 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832 2023-09-20T21:49:11.739 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832_vm 2023-09-20T21:49:11.739 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2833 2023-09-20T21:49:11.739 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834 2023-09-20T21:49:11.739 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834_vm 2023-09-20T21:49:11.740 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2835 2023-09-20T21:49:11.740 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836 2023-09-20T21:49:11.740 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836_vm 2023-09-20T21:49:11.740 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2837 2023-09-20T21:49:11.740 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 20 21:49 2838 2023-09-20T21:49:11.741 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2838_fsm 2023-09-20T21:49:11.741 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2838_vm 2023-09-20T21:49:11.741 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2839 2023-09-20T21:49:11.741 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840 2023-09-20T21:49:11.741 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840_fsm 2023-09-20T21:49:11.741 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2840_vm 2023-09-20T21:49:11.742 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2841 2023-09-20T21:49:11.742 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995 2023-09-20T21:49:11.742 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995_vm 2023-09-20T21:49:11.742 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2996 2023-09-20T21:49:11.743 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079 2023-09-20T21:49:11.743 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3079_fsm 2023-09-20T21:49:11.743 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079_vm 2023-09-20T21:49:11.743 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3080 2023-09-20T21:49:11.743 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3081 2023-09-20T21:49:11.743 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3085 2023-09-20T21:49:11.744 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118 2023-09-20T21:49:11.744 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118_vm 2023-09-20T21:49:11.744 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3119 2023-09-20T21:49:11.744 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3164 2023-09-20T21:49:11.744 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256 2023-09-20T21:49:11.745 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256_vm 2023-09-20T21:49:11.745 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3257 2023-09-20T21:49:11.745 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3258 2023-09-20T21:49:11.745 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350 2023-09-20T21:49:11.745 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350_vm 2023-09-20T21:49:11.745 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3351 2023-09-20T21:49:11.746 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3379 2023-09-20T21:49:11.746 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3380 2023-09-20T21:49:11.746 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381 2023-09-20T21:49:11.746 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381_vm 2023-09-20T21:49:11.746 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3394 2023-09-20T21:49:11.747 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3394_fsm 2023-09-20T21:49:11.747 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3394_vm 2023-09-20T21:49:11.747 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3395 2023-09-20T21:49:11.747 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439 2023-09-20T21:49:11.747 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439_vm 2023-09-20T21:49:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3440 2023-09-20T21:49:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 3455 2023-09-20T21:49:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3456 2023-09-20T21:49:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3456_fsm 2023-09-20T21:49:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3456_vm 2023-09-20T21:49:11.748 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466 2023-09-20T21:49:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466_vm 2023-09-20T21:49:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3467 2023-09-20T21:49:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3468 2023-09-20T21:49:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501 2023-09-20T21:49:11.749 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501_vm 2023-09-20T21:49:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3502 2023-09-20T21:49:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3503 2023-09-20T21:49:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3534 2023-09-20T21:49:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541 2023-09-20T21:49:11.750 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3541_fsm 2023-09-20T21:49:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541_vm 2023-09-20T21:49:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3542 2023-09-20T21:49:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3574 2023-09-20T21:49:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3575 2023-09-20T21:49:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576 2023-09-20T21:49:11.751 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576_vm 2023-09-20T21:49:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596 2023-09-20T21:49:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596_vm 2023-09-20T21:49:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3597 2023-09-20T21:49:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598 2023-09-20T21:49:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598_vm 2023-09-20T21:49:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3599 2023-09-20T21:49:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600 2023-09-20T21:49:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3600_fsm 2023-09-20T21:49:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600_vm 2023-09-20T21:49:11.753 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601 2023-09-20T21:49:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3601_fsm 2023-09-20T21:49:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601_vm 2023-09-20T21:49:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602 2023-09-20T21:49:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3602_fsm 2023-09-20T21:49:11.754 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602_vm 2023-09-20T21:49:11.755 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3603 2023-09-20T21:49:11.755 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3603_fsm 2023-09-20T21:49:11.755 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3603_vm 2023-09-20T21:49:11.755 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3604 2023-09-20T21:49:11.755 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3605 2023-09-20T21:49:11.755 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3606 2023-09-20T21:49:11.756 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3607 2023-09-20T21:49:11.756 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3608 2023-09-20T21:49:11.756 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 3609 2023-09-20T21:49:11.756 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3712 2023-09-20T21:49:11.756 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764 2023-09-20T21:49:11.757 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3764_fsm 2023-09-20T21:49:11.757 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764_vm 2023-09-20T21:49:11.757 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3766 2023-09-20T21:49:11.757 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3767 2023-09-20T21:49:11.757 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3997 2023-09-20T21:49:11.758 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 5002 2023-09-20T21:49:11.758 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 548 2023-09-20T21:49:11.758 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 549 2023-09-20T21:49:11.758 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102 2023-09-20T21:49:11.758 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102_vm 2023-09-20T21:49:11.758 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104 2023-09-20T21:49:11.759 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104_vm 2023-09-20T21:49:11.759 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106 2023-09-20T21:49:11.759 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106_vm 2023-09-20T21:49:11.759 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6110 2023-09-20T21:49:11.759 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6111 2023-09-20T21:49:11.760 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6112 2023-09-20T21:49:11.760 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6113 2023-09-20T21:49:11.760 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6117 2023-09-20T21:49:11.760 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826 2023-09-20T21:49:11.760 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826_vm 2023-09-20T21:49:11.761 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 827 2023-09-20T21:49:11.761 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 828 2023-09-20T21:49:11.761 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 20 21:49 pg_filenode.map 2023-09-20T21:49:11.761 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 PG_VERSION 2023-09-20T21:49:11.761 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.761 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-09-20T21:49:11.762 INFO:teuthology.orchestra.run.smithi022.stdout:total 7233 2023-09-20T21:49:11.762 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 20 21:49 . 2023-09-20T21:49:11.762 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 .. 2023-09-20T21:49:11.762 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 112 2023-09-20T21:49:11.762 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 113 2023-09-20T21:49:11.765 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 20 21:49 12182 2023-09-20T21:49:11.765 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12182_fsm 2023-09-20T21:49:11.765 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12182_vm 2023-09-20T21:49:11.765 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12184 2023-09-20T21:49:11.765 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12186 2023-09-20T21:49:11.766 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187 2023-09-20T21:49:11.766 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12187_fsm 2023-09-20T21:49:11.766 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187_vm 2023-09-20T21:49:11.766 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12189 2023-09-20T21:49:11.766 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12191 2023-09-20T21:49:11.766 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192 2023-09-20T21:49:11.767 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12192_fsm 2023-09-20T21:49:11.767 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192_vm 2023-09-20T21:49:11.767 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12194 2023-09-20T21:49:11.767 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12196 2023-09-20T21:49:11.767 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197 2023-09-20T21:49:11.768 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12197_fsm 2023-09-20T21:49:11.768 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197_vm 2023-09-20T21:49:11.768 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12199 2023-09-20T21:49:11.768 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12201 2023-09-20T21:49:11.768 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202 2023-09-20T21:49:11.769 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12202_fsm 2023-09-20T21:49:11.769 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202_vm 2023-09-20T21:49:11.769 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12204 2023-09-20T21:49:11.769 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12206 2023-09-20T21:49:11.769 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207 2023-09-20T21:49:11.769 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12207_fsm 2023-09-20T21:49:11.770 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207_vm 2023-09-20T21:49:11.770 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12209 2023-09-20T21:49:11.770 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12211 2023-09-20T21:49:11.770 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12212 2023-09-20T21:49:11.770 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12214 2023-09-20T21:49:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12216 2023-09-20T21:49:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 20 21:49 1247 2023-09-20T21:49:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1247_fsm 2023-09-20T21:49:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1247_vm 2023-09-20T21:49:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 20 21:49 1249 2023-09-20T21:49:11.771 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1249_fsm 2023-09-20T21:49:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1249_vm 2023-09-20T21:49:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 20 21:49 1255 2023-09-20T21:49:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1255_fsm 2023-09-20T21:49:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1255_vm 2023-09-20T21:49:11.772 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 20 21:49 1259 2023-09-20T21:49:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1259_fsm 2023-09-20T21:49:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1259_vm 2023-09-20T21:49:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417 2023-09-20T21:49:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417_vm 2023-09-20T21:49:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418 2023-09-20T21:49:11.773 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418_vm 2023-09-20T21:49:11.774 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 174 2023-09-20T21:49:11.774 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 175 2023-09-20T21:49:11.774 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2187 2023-09-20T21:49:11.774 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224 2023-09-20T21:49:11.774 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224_vm 2023-09-20T21:49:11.775 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328 2023-09-20T21:49:11.775 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328_vm 2023-09-20T21:49:11.775 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336 2023-09-20T21:49:11.775 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336_vm 2023-09-20T21:49:11.775 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2337 2023-09-20T21:49:11.775 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2600 2023-09-20T21:49:11.776 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2600_fsm 2023-09-20T21:49:11.776 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2600_vm 2023-09-20T21:49:11.776 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601 2023-09-20T21:49:11.776 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2601_fsm 2023-09-20T21:49:11.776 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601_vm 2023-09-20T21:49:11.777 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2602 2023-09-20T21:49:11.777 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2602_fsm 2023-09-20T21:49:11.777 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2602_vm 2023-09-20T21:49:11.777 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2603 2023-09-20T21:49:11.777 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2603_fsm 2023-09-20T21:49:11.778 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2603_vm 2023-09-20T21:49:11.778 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604 2023-09-20T21:49:11.778 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604_vm 2023-09-20T21:49:11.778 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2605 2023-09-20T21:49:11.778 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2605_fsm 2023-09-20T21:49:11.778 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2605_vm 2023-09-20T21:49:11.779 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606 2023-09-20T21:49:11.779 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2606_fsm 2023-09-20T21:49:11.779 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606_vm 2023-09-20T21:49:11.779 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607 2023-09-20T21:49:11.779 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607_fsm 2023-09-20T21:49:11.780 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2607_vm 2023-09-20T21:49:11.780 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 20 21:49 2608 2023-09-20T21:49:11.780 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2608_fsm 2023-09-20T21:49:11.780 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2608_vm 2023-09-20T21:49:11.780 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 20 21:49 2609 2023-09-20T21:49:11.781 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2609_fsm 2023-09-20T21:49:11.781 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2609_vm 2023-09-20T21:49:11.781 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610 2023-09-20T21:49:11.781 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610_fsm 2023-09-20T21:49:11.781 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2610_vm 2023-09-20T21:49:11.782 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611 2023-09-20T21:49:11.782 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611_vm 2023-09-20T21:49:11.782 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612 2023-09-20T21:49:11.782 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2612_fsm 2023-09-20T21:49:11.782 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612_vm 2023-09-20T21:49:11.783 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613 2023-09-20T21:49:11.783 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613_vm 2023-09-20T21:49:11.783 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615 2023-09-20T21:49:11.783 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2615_fsm 2023-09-20T21:49:11.783 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615_vm 2023-09-20T21:49:11.783 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616 2023-09-20T21:49:11.784 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616_fsm 2023-09-20T21:49:11.784 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2616_vm 2023-09-20T21:49:11.784 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 20 21:49 2617 2023-09-20T21:49:11.784 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2617_fsm 2023-09-20T21:49:11.785 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2617_vm 2023-09-20T21:49:11.785 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 20 21:49 2618 2023-09-20T21:49:11.785 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2618_fsm 2023-09-20T21:49:11.785 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2618_vm 2023-09-20T21:49:11.785 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 20 21:49 2619 2023-09-20T21:49:11.785 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2619_fsm 2023-09-20T21:49:11.786 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2619_vm 2023-09-20T21:49:11.786 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620 2023-09-20T21:49:11.786 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620_vm 2023-09-20T21:49:11.786 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2650 2023-09-20T21:49:11.786 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2651 2023-09-20T21:49:11.787 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2652 2023-09-20T21:49:11.787 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2653 2023-09-20T21:49:11.787 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2654 2023-09-20T21:49:11.787 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2655 2023-09-20T21:49:11.787 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2656 2023-09-20T21:49:11.787 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2657 2023-09-20T21:49:11.788 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 20 21:49 2658 2023-09-20T21:49:11.788 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2659 2023-09-20T21:49:11.788 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2660 2023-09-20T21:49:11.788 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2661 2023-09-20T21:49:11.788 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2662 2023-09-20T21:49:11.789 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2663 2023-09-20T21:49:11.789 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2664 2023-09-20T21:49:11.789 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2665 2023-09-20T21:49:11.789 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2666 2023-09-20T21:49:11.789 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2667 2023-09-20T21:49:11.790 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2668 2023-09-20T21:49:11.790 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2669 2023-09-20T21:49:11.790 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2670 2023-09-20T21:49:11.790 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 20 21:49 2673 2023-09-20T21:49:11.790 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Sep 20 21:49 2674 2023-09-20T21:49:11.791 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 20 21:49 2675 2023-09-20T21:49:11.791 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2678 2023-09-20T21:49:11.791 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2679 2023-09-20T21:49:11.791 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2680 2023-09-20T21:49:11.791 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2681 2023-09-20T21:49:11.791 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2682 2023-09-20T21:49:11.792 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2683 2023-09-20T21:49:11.792 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2684 2023-09-20T21:49:11.792 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2685 2023-09-20T21:49:11.792 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2686 2023-09-20T21:49:11.792 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2687 2023-09-20T21:49:11.793 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2688 2023-09-20T21:49:11.793 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2689 2023-09-20T21:49:11.793 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2690 2023-09-20T21:49:11.793 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 20 21:49 2691 2023-09-20T21:49:11.793 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2692 2023-09-20T21:49:11.793 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2693 2023-09-20T21:49:11.794 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2696 2023-09-20T21:49:11.794 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2699 2023-09-20T21:49:11.794 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2701 2023-09-20T21:49:11.794 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2702 2023-09-20T21:49:11.794 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2703 2023-09-20T21:49:11.795 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2704 2023-09-20T21:49:11.795 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2753 2023-09-20T21:49:11.795 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2753_fsm 2023-09-20T21:49:11.795 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2753_vm 2023-09-20T21:49:11.795 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2754 2023-09-20T21:49:11.796 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2755 2023-09-20T21:49:11.796 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2756 2023-09-20T21:49:11.796 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2757 2023-09-20T21:49:11.796 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830 2023-09-20T21:49:11.796 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830_vm 2023-09-20T21:49:11.796 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2831 2023-09-20T21:49:11.797 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832 2023-09-20T21:49:11.797 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832_vm 2023-09-20T21:49:11.797 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2833 2023-09-20T21:49:11.797 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834 2023-09-20T21:49:11.797 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834_vm 2023-09-20T21:49:11.798 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2835 2023-09-20T21:49:11.798 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836 2023-09-20T21:49:11.798 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836_vm 2023-09-20T21:49:11.798 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2837 2023-09-20T21:49:11.798 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 20 21:49 2838 2023-09-20T21:49:11.799 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2838_fsm 2023-09-20T21:49:11.799 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2838_vm 2023-09-20T21:49:11.799 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2839 2023-09-20T21:49:11.799 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840 2023-09-20T21:49:11.799 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840_fsm 2023-09-20T21:49:11.800 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2840_vm 2023-09-20T21:49:11.800 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2841 2023-09-20T21:49:11.800 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995 2023-09-20T21:49:11.800 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995_vm 2023-09-20T21:49:11.800 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2996 2023-09-20T21:49:11.800 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079 2023-09-20T21:49:11.801 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3079_fsm 2023-09-20T21:49:11.801 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079_vm 2023-09-20T21:49:11.801 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3080 2023-09-20T21:49:11.801 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3081 2023-09-20T21:49:11.801 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3085 2023-09-20T21:49:11.802 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118 2023-09-20T21:49:11.802 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118_vm 2023-09-20T21:49:11.802 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3119 2023-09-20T21:49:11.802 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3164 2023-09-20T21:49:11.802 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256 2023-09-20T21:49:11.803 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256_vm 2023-09-20T21:49:11.803 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3257 2023-09-20T21:49:11.803 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3258 2023-09-20T21:49:11.803 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350 2023-09-20T21:49:11.803 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350_vm 2023-09-20T21:49:11.803 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3351 2023-09-20T21:49:11.804 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3379 2023-09-20T21:49:11.804 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3380 2023-09-20T21:49:11.804 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381 2023-09-20T21:49:11.804 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381_vm 2023-09-20T21:49:11.804 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3394 2023-09-20T21:49:11.805 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3394_fsm 2023-09-20T21:49:11.805 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3394_vm 2023-09-20T21:49:11.805 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3395 2023-09-20T21:49:11.805 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439 2023-09-20T21:49:11.805 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439_vm 2023-09-20T21:49:11.806 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3440 2023-09-20T21:49:11.806 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 3455 2023-09-20T21:49:11.806 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3456 2023-09-20T21:49:11.806 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3456_fsm 2023-09-20T21:49:11.806 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3456_vm 2023-09-20T21:49:11.807 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466 2023-09-20T21:49:11.807 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466_vm 2023-09-20T21:49:11.807 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3467 2023-09-20T21:49:11.807 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3468 2023-09-20T21:49:11.807 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501 2023-09-20T21:49:11.808 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501_vm 2023-09-20T21:49:11.808 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3502 2023-09-20T21:49:11.808 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3503 2023-09-20T21:49:11.808 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3534 2023-09-20T21:49:11.808 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541 2023-09-20T21:49:11.809 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3541_fsm 2023-09-20T21:49:11.809 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541_vm 2023-09-20T21:49:11.809 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3542 2023-09-20T21:49:11.809 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3574 2023-09-20T21:49:11.809 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3575 2023-09-20T21:49:11.809 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576 2023-09-20T21:49:11.810 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576_vm 2023-09-20T21:49:11.810 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596 2023-09-20T21:49:11.810 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596_vm 2023-09-20T21:49:11.810 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3597 2023-09-20T21:49:11.810 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598 2023-09-20T21:49:11.811 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598_vm 2023-09-20T21:49:11.811 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3599 2023-09-20T21:49:11.811 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600 2023-09-20T21:49:11.811 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3600_fsm 2023-09-20T21:49:11.811 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600_vm 2023-09-20T21:49:11.812 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601 2023-09-20T21:49:11.812 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3601_fsm 2023-09-20T21:49:11.812 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601_vm 2023-09-20T21:49:11.812 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602 2023-09-20T21:49:11.812 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3602_fsm 2023-09-20T21:49:11.812 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602_vm 2023-09-20T21:49:11.813 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3603 2023-09-20T21:49:11.813 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3603_fsm 2023-09-20T21:49:11.813 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3603_vm 2023-09-20T21:49:11.813 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3604 2023-09-20T21:49:11.813 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3605 2023-09-20T21:49:11.813 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3606 2023-09-20T21:49:11.814 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3607 2023-09-20T21:49:11.814 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3608 2023-09-20T21:49:11.814 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 3609 2023-09-20T21:49:11.814 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3712 2023-09-20T21:49:11.814 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764 2023-09-20T21:49:11.815 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3764_fsm 2023-09-20T21:49:11.815 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764_vm 2023-09-20T21:49:11.815 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3766 2023-09-20T21:49:11.815 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3767 2023-09-20T21:49:11.815 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3997 2023-09-20T21:49:11.816 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 5002 2023-09-20T21:49:11.816 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 548 2023-09-20T21:49:11.816 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 549 2023-09-20T21:49:11.816 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102 2023-09-20T21:49:11.816 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102_vm 2023-09-20T21:49:11.817 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104 2023-09-20T21:49:11.817 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104_vm 2023-09-20T21:49:11.817 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106 2023-09-20T21:49:11.817 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106_vm 2023-09-20T21:49:11.817 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6110 2023-09-20T21:49:11.817 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6111 2023-09-20T21:49:11.818 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6112 2023-09-20T21:49:11.818 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6113 2023-09-20T21:49:11.818 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6117 2023-09-20T21:49:11.818 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826 2023-09-20T21:49:11.818 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826_vm 2023-09-20T21:49:11.819 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 827 2023-09-20T21:49:11.819 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 828 2023-09-20T21:49:11.819 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 20 21:49 pg_filenode.map 2023-09-20T21:49:11.819 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 PG_VERSION 2023-09-20T21:49:11.819 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.820 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/global: 2023-09-20T21:49:11.820 INFO:teuthology.orchestra.run.smithi022.stdout:total 553 2023-09-20T21:49:11.820 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 20 21:49 . 2023-09-20T21:49:11.820 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.820 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1136 2023-09-20T21:49:11.821 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1136_fsm 2023-09-20T21:49:11.821 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1136_vm 2023-09-20T21:49:11.821 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 1137 2023-09-20T21:49:11.821 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1213 2023-09-20T21:49:11.821 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1213_fsm 2023-09-20T21:49:11.821 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1213_vm 2023-09-20T21:49:11.822 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1214 2023-09-20T21:49:11.822 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1214_fsm 2023-09-20T21:49:11.822 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1214_vm 2023-09-20T21:49:11.822 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 1232 2023-09-20T21:49:11.822 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 1233 2023-09-20T21:49:11.823 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1260 2023-09-20T21:49:11.823 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1260_fsm 2023-09-20T21:49:11.823 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1260_vm 2023-09-20T21:49:11.823 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1261 2023-09-20T21:49:11.823 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1261_fsm 2023-09-20T21:49:11.824 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1261_vm 2023-09-20T21:49:11.824 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1262 2023-09-20T21:49:11.824 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1262_fsm 2023-09-20T21:49:11.824 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1262_vm 2023-09-20T21:49:11.824 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2396 2023-09-20T21:49:11.825 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2396_fsm 2023-09-20T21:49:11.825 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2396_vm 2023-09-20T21:49:11.825 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2397 2023-09-20T21:49:11.825 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2671 2023-09-20T21:49:11.825 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2672 2023-09-20T21:49:11.825 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2676 2023-09-20T21:49:11.826 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2677 2023-09-20T21:49:11.826 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2694 2023-09-20T21:49:11.826 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2695 2023-09-20T21:49:11.826 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2697 2023-09-20T21:49:11.826 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2698 2023-09-20T21:49:11.827 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2846 2023-09-20T21:49:11.827 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2846_vm 2023-09-20T21:49:11.827 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2847 2023-09-20T21:49:11.827 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2964 2023-09-20T21:49:11.827 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2964_vm 2023-09-20T21:49:11.828 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2965 2023-09-20T21:49:11.828 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2966 2023-09-20T21:49:11.828 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2966_vm 2023-09-20T21:49:11.828 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2967 2023-09-20T21:49:11.828 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3592 2023-09-20T21:49:11.828 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3592_vm 2023-09-20T21:49:11.829 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3593 2023-09-20T21:49:11.829 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 4060 2023-09-20T21:49:11.829 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 4060_vm 2023-09-20T21:49:11.829 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 4061 2023-09-20T21:49:11.829 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6000 2023-09-20T21:49:11.830 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6000_vm 2023-09-20T21:49:11.830 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6001 2023-09-20T21:49:11.830 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6002 2023-09-20T21:49:11.830 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6100 2023-09-20T21:49:11.830 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6100_vm 2023-09-20T21:49:11.831 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6114 2023-09-20T21:49:11.831 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6115 2023-09-20T21:49:11.831 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 pg_control 2023-09-20T21:49:11.831 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 20 21:49 pg_filenode.map 2023-09-20T21:49:11.831 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.832 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/log: 2023-09-20T21:49:11.832 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.832 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.832 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.832 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.833 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-09-20T21:49:11.833 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.833 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.833 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.833 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.834 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-09-20T21:49:11.834 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.834 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.834 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.834 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.835 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-09-20T21:49:11.835 INFO:teuthology.orchestra.run.smithi022.stdout:total 1 2023-09-20T21:49:11.835 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 . 2023-09-20T21:49:11.835 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.835 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 mappings 2023-09-20T21:49:11.836 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 20 21:49 replorigin_checkpoint 2023-09-20T21:49:11.836 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 snapshots 2023-09-20T21:49:11.836 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.836 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-09-20T21:49:11.836 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.837 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.837 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 .. 2023-09-20T21:49:11.837 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.837 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-09-20T21:49:11.837 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.838 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.838 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 .. 2023-09-20T21:49:11.838 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.838 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-09-20T21:49:11.838 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.839 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 . 2023-09-20T21:49:11.839 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.839 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 members 2023-09-20T21:49:11.839 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 offsets 2023-09-20T21:49:11.839 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.840 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-09-20T21:49:11.840 INFO:teuthology.orchestra.run.smithi022.stdout:total 8 2023-09-20T21:49:11.840 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 . 2023-09-20T21:49:11.840 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 .. 2023-09-20T21:49:11.840 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 0000 2023-09-20T21:49:11.840 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.841 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-09-20T21:49:11.841 INFO:teuthology.orchestra.run.smithi022.stdout:total 8 2023-09-20T21:49:11.841 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 . 2023-09-20T21:49:11.841 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 .. 2023-09-20T21:49:11.841 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 0000 2023-09-20T21:49:11.842 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.842 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-09-20T21:49:11.842 INFO:teuthology.orchestra.run.smithi022.stdout:total 8 2023-09-20T21:49:11.842 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 . 2023-09-20T21:49:11.842 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.843 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 0000 2023-09-20T21:49:11.843 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.843 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-09-20T21:49:11.843 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.843 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.843 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.844 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.844 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-09-20T21:49:11.844 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.844 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.844 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.845 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.845 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-09-20T21:49:11.845 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.845 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.845 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.846 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.846 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-09-20T21:49:11.846 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.846 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.846 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.846 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.847 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-09-20T21:49:11.847 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.847 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.847 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.847 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.848 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-09-20T21:49:11.848 INFO:teuthology.orchestra.run.smithi022.stdout:total 8 2023-09-20T21:49:11.848 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 . 2023-09-20T21:49:11.848 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.848 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 0000 2023-09-20T21:49:11.849 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.849 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-09-20T21:49:11.849 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.849 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.849 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.850 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.850 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-09-20T21:49:11.850 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.850 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.850 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.851 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.851 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-09-20T21:49:11.851 INFO:teuthology.orchestra.run.smithi022.stdout:total 16384 2023-09-20T21:49:11.851 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 . 2023-09-20T21:49:11.851 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.852 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:49 000000010000000000000001 2023-09-20T21:49:11.852 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 archive_status 2023-09-20T21:49:11.852 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.852 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-09-20T21:49:11.852 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T21:49:11.853 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T21:49:11.853 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 .. 2023-09-20T21:49:11.853 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:11.853 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-09-20T21:49:11.853 INFO:teuthology.orchestra.run.smithi022.stdout:total 8 2023-09-20T21:49:11.853 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 . 2023-09-20T21:49:11.854 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 20 21:49 .. 2023-09-20T21:49:11.854 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 0000 2023-09-20T21:49:11.854 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-09-20T21:49:12.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2023-09-20T21:49:12.208 INFO:teuthology.orchestra.run.smithi022.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-09-20T21:49:12.215 INFO:teuthology.orchestra.run.smithi022.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-09-20T21:49:12.219 INFO:teuthology.orchestra.run.smithi022.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-09-20T21:49:12.223 INFO:teuthology.orchestra.run.smithi022.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-09-20T21:49:12.252 INFO:teuthology.orchestra.run.smithi022.stderr:creating tables... 2023-09-20T21:49:12.364 INFO:teuthology.orchestra.run.smithi022.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.11 s, remaining 3.39 s) 2023-09-20T21:49:12.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:12 smithi022 ceph-mon[97651]: pgmap v355: 129 pgs: 129 active+clean; 21 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2023-09-20T21:49:12.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:12 smithi123 ceph-mon[134433]: pgmap v355: 129 pgs: 129 active+clean; 21 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2023-09-20T21:49:12.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:12 smithi089 ceph-mon[134090]: pgmap v355: 129 pgs: 129 active+clean; 21 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2023-09-20T21:49:12.696 INFO:teuthology.orchestra.run.smithi022.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.44 s, remaining 6.62 s) 2023-09-20T21:49:12.979 INFO:teuthology.orchestra.run.smithi022.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.72 s, remaining 7.00 s) 2023-09-20T21:49:13.143 INFO:teuthology.orchestra.run.smithi022.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.89 s, remaining 6.22 s) 2023-09-20T21:49:13.408 INFO:teuthology.orchestra.run.smithi022.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.15 s, remaining 6.23 s) 2023-09-20T21:49:13.734 INFO:teuthology.orchestra.run.smithi022.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.48 s, remaining 6.41 s) 2023-09-20T21:49:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.59 s, remaining 5.67 s) 2023-09-20T21:49:13.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:13 smithi089 ceph-mon[134090]: pgmap v356: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.2 MiB/s wr, 73 op/s 2023-09-20T21:49:13.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:13 smithi022 ceph-mon[97651]: pgmap v356: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.2 MiB/s wr, 73 op/s 2023-09-20T21:49:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:13 smithi123 ceph-mon[134433]: pgmap v356: 129 pgs: 129 active+clean; 26 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 2.2 MiB/s wr, 73 op/s 2023-09-20T21:49:14.199 INFO:teuthology.orchestra.run.smithi022.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.94 s, remaining 5.83 s) 2023-09-20T21:49:14.490 INFO:teuthology.orchestra.run.smithi022.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.23 s, remaining 5.71 s) 2023-09-20T21:49:14.597 INFO:teuthology.orchestra.run.smithi022.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.34 s, remaining 5.15 s) 2023-09-20T21:49:14.965 INFO:teuthology.orchestra.run.smithi022.stderr:1100000 of 3200000 tuples (34%) done (elapsed 2.71 s, remaining 5.17 s) 2023-09-20T21:49:15.074 INFO:teuthology.orchestra.run.smithi022.stderr:1200000 of 3200000 tuples (37%) done (elapsed 2.82 s, remaining 4.70 s) 2023-09-20T21:49:15.392 INFO:teuthology.orchestra.run.smithi022.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.14 s, remaining 4.58 s) 2023-09-20T21:49:15.645 INFO:teuthology.orchestra.run.smithi022.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.39 s, remaining 4.36 s) 2023-09-20T21:49:15.802 INFO:teuthology.orchestra.run.smithi022.stderr:1500000 of 3200000 tuples (46%) done (elapsed 3.55 s, remaining 4.02 s) 2023-09-20T21:49:16.113 INFO:teuthology.orchestra.run.smithi022.stderr:1600000 of 3200000 tuples (50%) done (elapsed 3.86 s, remaining 3.86 s) 2023-09-20T21:49:16.451 INFO:teuthology.orchestra.run.smithi022.stderr:1700000 of 3200000 tuples (53%) done (elapsed 4.20 s, remaining 3.70 s) 2023-09-20T21:49:16.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:16 smithi022 ceph-mon[97651]: pgmap v357: 129 pgs: 129 active+clean; 62 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 7.6 MiB/s wr, 104 op/s 2023-09-20T21:49:16.553 INFO:teuthology.orchestra.run.smithi022.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.30 s, remaining 3.34 s) 2023-09-20T21:49:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:16 smithi123 ceph-mon[134433]: pgmap v357: 129 pgs: 129 active+clean; 62 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 7.6 MiB/s wr, 104 op/s 2023-09-20T21:49:16.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:16 smithi089 ceph-mon[134090]: pgmap v357: 129 pgs: 129 active+clean; 62 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 7.6 MiB/s wr, 104 op/s 2023-09-20T21:49:16.935 INFO:teuthology.orchestra.run.smithi022.stderr:1900000 of 3200000 tuples (59%) done (elapsed 4.68 s, remaining 3.20 s) 2023-09-20T21:49:17.210 INFO:teuthology.orchestra.run.smithi022.stderr:2000000 of 3200000 tuples (62%) done (elapsed 4.95 s, remaining 2.97 s) 2023-09-20T21:49:17.319 INFO:teuthology.orchestra.run.smithi022.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.06 s, remaining 2.65 s) 2023-09-20T21:49:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2200000 of 3200000 tuples (68%) done (elapsed 5.43 s, remaining 2.47 s) 2023-09-20T21:49:17.784 INFO:teuthology.orchestra.run.smithi022.stderr:2300000 of 3200000 tuples (71%) done (elapsed 5.53 s, remaining 2.16 s) 2023-09-20T21:49:18.093 INFO:teuthology.orchestra.run.smithi022.stderr:2400000 of 3200000 tuples (75%) done (elapsed 5.84 s, remaining 1.95 s) 2023-09-20T21:49:18.429 INFO:teuthology.orchestra.run.smithi022.stderr:2500000 of 3200000 tuples (78%) done (elapsed 6.17 s, remaining 1.73 s) 2023-09-20T21:49:18.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:18 smithi022 ceph-mon[97651]: pgmap v358: 129 pgs: 129 active+clean; 142 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 20 MiB/s wr, 157 op/s 2023-09-20T21:49:18.538 INFO:teuthology.orchestra.run.smithi022.stderr:2600000 of 3200000 tuples (81%) done (elapsed 6.28 s, remaining 1.45 s) 2023-09-20T21:49:18.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:18 smithi123 ceph-mon[134433]: pgmap v358: 129 pgs: 129 active+clean; 142 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 20 MiB/s wr, 157 op/s 2023-09-20T21:49:18.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:18 smithi089 ceph-mon[134090]: pgmap v358: 129 pgs: 129 active+clean; 142 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 20 MiB/s wr, 157 op/s 2023-09-20T21:49:18.908 INFO:teuthology.orchestra.run.smithi022.stderr:2700000 of 3200000 tuples (84%) done (elapsed 6.65 s, remaining 1.23 s) 2023-09-20T21:49:19.061 INFO:teuthology.orchestra.run.smithi022.stderr:2800000 of 3200000 tuples (87%) done (elapsed 6.81 s, remaining 0.97 s) 2023-09-20T21:49:19.337 INFO:teuthology.orchestra.run.smithi022.stderr:2900000 of 3200000 tuples (90%) done (elapsed 7.08 s, remaining 0.73 s) 2023-09-20T21:49:19.650 INFO:teuthology.orchestra.run.smithi022.stderr:3000000 of 3200000 tuples (93%) done (elapsed 7.39 s, remaining 0.49 s) 2023-09-20T21:49:19.818 INFO:teuthology.orchestra.run.smithi022.stderr:3100000 of 3200000 tuples (96%) done (elapsed 7.56 s, remaining 0.24 s) 2023-09-20T21:49:20.089 INFO:teuthology.orchestra.run.smithi022.stderr:3200000 of 3200000 tuples (100%) done (elapsed 7.83 s, remaining 0.00 s) 2023-09-20T21:49:20.174 INFO:teuthology.orchestra.run.smithi022.stderr:vacuum... 2023-09-20T21:49:20.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:20 smithi022 ceph-mon[97651]: pgmap v359: 129 pgs: 129 active+clean; 218 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 32 MiB/s wr, 154 op/s 2023-09-20T21:49:20.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:20 smithi123 ceph-mon[134433]: pgmap v359: 129 pgs: 129 active+clean; 218 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 32 MiB/s wr, 154 op/s 2023-09-20T21:49:20.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:20 smithi089 ceph-mon[134090]: pgmap v359: 129 pgs: 129 active+clean; 218 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 32 MiB/s wr, 154 op/s 2023-09-20T21:49:21.123 INFO:teuthology.orchestra.run.smithi022.stderr:set primary keys... 2023-09-20T21:49:22.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:22 smithi022 ceph-mon[97651]: pgmap v360: 129 pgs: 129 active+clean; 323 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 50 MiB/s wr, 166 op/s 2023-09-20T21:49:22.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:22 smithi123 ceph-mon[134433]: pgmap v360: 129 pgs: 129 active+clean; 323 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 50 MiB/s wr, 166 op/s 2023-09-20T21:49:22.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:22 smithi089 ceph-mon[134090]: pgmap v360: 129 pgs: 129 active+clean; 323 MiB data, 915 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 50 MiB/s wr, 166 op/s 2023-09-20T21:49:23.436 INFO:teuthology.orchestra.run.smithi022.stderr:done. 2023-09-20T21:49:23.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-09-20T21:49:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:starting vacuum...end. 2023-09-20T21:49:23.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:23 smithi022 ceph-mon[97651]: pgmap v361: 129 pgs: 129 active+clean; 343 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 53 MiB/s wr, 160 op/s 2023-09-20T21:49:23.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:23 smithi089 ceph-mon[134090]: pgmap v361: 129 pgs: 129 active+clean; 343 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 53 MiB/s wr, 160 op/s 2023-09-20T21:49:24.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:23 smithi123 ceph-mon[134433]: pgmap v361: 129 pgs: 129 active+clean; 343 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 53 MiB/s wr, 160 op/s 2023-09-20T21:49:26.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:26 smithi022 ceph-mon[97651]: pgmap v362: 129 pgs: 129 active+clean; 423 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 64 MiB/s wr, 126 op/s 2023-09-20T21:49:26.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:26 smithi123 ceph-mon[134433]: pgmap v362: 129 pgs: 129 active+clean; 423 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 64 MiB/s wr, 126 op/s 2023-09-20T21:49:26.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:26 smithi089 ceph-mon[134090]: pgmap v362: 129 pgs: 129 active+clean; 423 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 64 MiB/s wr, 126 op/s 2023-09-20T21:49:27.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:27 smithi022 ceph-mon[97651]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:49:27.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:27 smithi123 ceph-mon[134433]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:49:27.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:27 smithi089 ceph-mon[134090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:49:28.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:28 smithi022 ceph-mon[97651]: pgmap v363: 129 pgs: 129 active+clean; 614 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 70 MiB/s wr, 141 op/s 2023-09-20T21:49:28.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 5.0 s, 1782.8 tps, lat 53.146 ms stddev 64.453 2023-09-20T21:49:28.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:28 smithi123 ceph-mon[134433]: pgmap v363: 129 pgs: 129 active+clean; 614 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 70 MiB/s wr, 141 op/s 2023-09-20T21:49:28.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:28 smithi089 ceph-mon[134090]: pgmap v363: 129 pgs: 129 active+clean; 614 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 70 MiB/s wr, 141 op/s 2023-09-20T21:49:28.751 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:49:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.097+0000 7fd8d79ff700 1 -- 172.21.15.22:0/3637355174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8d00ff640 msgr2=0x7fd8d0101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.097+0000 7fd8d79ff700 1 --2- 172.21.15.22:0/3637355174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8d00ff640 0x7fd8d0101a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009990 tx=0x7fd8c0009ca0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.099+0000 7fd8d79ff700 1 -- 172.21.15.22:0/3637355174 shutdown_connections 2023-09-20T21:49:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.099+0000 7fd8d79ff700 1 --2- 172.21.15.22:0/3637355174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8d0104980 0x7fd8d0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.099+0000 7fd8d79ff700 1 --2- 172.21.15.22:0/3637355174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8d00ff640 0x7fd8d0101a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.099+0000 7fd8d79ff700 1 --2- 172.21.15.22:0/3637355174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d0101fb0 0x7fd8d0104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.099+0000 7fd8d79ff700 1 -- 172.21.15.22:0/3637355174 >> 172.21.15.22:0/3637355174 conn(0x7fd8d00fb240 msgr2=0x7fd8d00fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:29.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.102+0000 7fd8d79ff700 1 -- 172.21.15.22:0/3637355174 shutdown_connections 2023-09-20T21:49:29.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.103+0000 7fd8d79ff700 1 -- 172.21.15.22:0/3637355174 wait complete. 2023-09-20T21:49:29.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.104+0000 7fd8d79ff700 1 Processor -- start 2023-09-20T21:49:29.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.105+0000 7fd8d79ff700 1 -- start start 2023-09-20T21:49:29.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.106+0000 7fd8d79ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d00ff640 0x7fd8d019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.106+0000 7fd8d79ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8d0101fb0 0x7fd8d019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.106+0000 7fd8d79ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8d0104980 0x7fd8d01a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.106+0000 7fd8d79ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d01a4c20 con 0x7fd8d0104980 2023-09-20T21:49:29.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.106+0000 7fd8d79ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d01a4d60 con 0x7fd8d0101fb0 2023-09-20T21:49:29.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.106+0000 7fd8d79ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d01a4ef0 con 0x7fd8d00ff640 2023-09-20T21:49:29.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d579b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d00ff640 0x7fd8d019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8d0101fb0 0x7fd8d019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d579b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d00ff640 0x7fd8d019d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43074/0 (socket says 172.21.15.22:43074) 2023-09-20T21:49:29.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d579b700 1 -- 172.21.15.22:0/1738322359 learned_addr learned my addr 172.21.15.22:0/1738322359 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:29.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d5f9c700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8d0104980 0x7fd8d01a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d4f9a700 1 -- 172.21.15.22:0/1738322359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d00ff640 msgr2=0x7fd8d019d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d00ff640 0x7fd8d019d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d4f9a700 1 -- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8d0104980 msgr2=0x7fd8d01a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8d0104980 0x7fd8d01a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d4f9a700 1 -- 172.21.15.22:0/1738322359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00095f0 con 0x7fd8d0101fb0 2023-09-20T21:49:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.107+0000 7fd8d5f9c700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8d0104980 0x7fd8d01a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:49:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.108+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8d0101fb0 0x7fd8d019d5e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00e9f0 tx=0x7fd8cc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.108+0000 7fd8c67fc700 1 -- 172.21.15.22:0/1738322359 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc00eee0 con 0x7fd8d0101fb0 2023-09-20T21:49:29.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.108+0000 7fd8d79ff700 1 -- 172.21.15.22:0/1738322359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d01a51d0 con 0x7fd8d0101fb0 2023-09-20T21:49:29.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.108+0000 7fd8d79ff700 1 -- 172.21.15.22:0/1738322359 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d01a5730 con 0x7fd8d0101fb0 2023-09-20T21:49:29.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.108+0000 7fd8c67fc700 1 -- 172.21.15.22:0/1738322359 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8cc00cbd0 con 0x7fd8d0101fb0 2023-09-20T21:49:29.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.108+0000 7fd8c67fc700 1 -- 172.21.15.22:0/1738322359 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc018680 con 0x7fd8d0101fb0 2023-09-20T21:49:29.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.116+0000 7fd8c67fc700 1 -- 172.21.15.22:0/1738322359 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fd8cc0187e0 con 0x7fd8d0101fb0 2023-09-20T21:49:29.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.117+0000 7fd8eb2aab80 1 Processor -- start 2023-09-20T21:49:29.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.117+0000 7fd8eb2aab80 1 -- start start 2023-09-20T21:49:29.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.118+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf792cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.118+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf792d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.118+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 0x557bf792e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.118+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf78a0ca0 con 0x557bf792d050 2023-09-20T21:49:29.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.118+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf7884c70 con 0x557bf792dbe0 2023-09-20T21:49:29.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.118+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf7882590 con 0x557bf792c6f0 2023-09-20T21:49:29.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.118+0000 7fd8d5f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf792cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.118+0000 7fd8d5f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf792cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43076/0 (socket says 172.21.15.22:43076) 2023-09-20T21:49:29.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.118+0000 7fd8d5f9c700 1 -- 172.21.15.22:0/2793501178 learned_addr learned my addr 172.21.15.22:0/2793501178 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:29.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.119+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/2793501178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 0x557bf792e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.119+0000 7fd8d579b700 1 --2- 172.21.15.22:0/2793501178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf792d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.119+0000 7fd8d5f9c700 1 -- 172.21.15.22:0/2793501178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 msgr2=0x557bf792e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:29.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.119+0000 7fd8d5f9c700 1 --2- 172.21.15.22:0/2793501178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 0x557bf792e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.119+0000 7fd8d5f9c700 1 -- 172.21.15.22:0/2793501178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 msgr2=0x557bf792d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:29.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.119+0000 7fd8d5f9c700 1 --2- 172.21.15.22:0/2793501178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf792d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.119+0000 7fd8d5f9c700 1 -- 172.21.15.22:0/2793501178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bf76a8de0 con 0x557bf792c6f0 2023-09-20T21:49:29.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.119+0000 7fd8c67fc700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd8bc0a1400 0x7fd8bc0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.120+0000 7fd8c67fc700 1 -- 172.21.15.22:0/1738322359 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fd8cc014070 con 0x7fd8d0101fb0 2023-09-20T21:49:29.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.120+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/2793501178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 0x557bf792e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:49:29.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.120+0000 7fd8d579b700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd8bc0a1400 0x7fd8bc0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.121+0000 7fd8d5f9c700 1 --2- 172.21.15.22:0/2793501178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf792cb10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800d300 tx=0x7fd8c800d610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95029155e31f6940 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:29.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.121+0000 7fd8bb7fe700 1 -- 172.21.15.22:0/2793501178 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800dea0 con 0x557bf792c6f0 2023-09-20T21:49:29.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.122+0000 7fd8bb7fe700 1 -- 172.21.15.22:0/2793501178 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8c8012ab0 con 0x557bf792c6f0 2023-09-20T21:49:29.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.122+0000 7fd8bb7fe700 1 -- 172.21.15.22:0/2793501178 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8011b60 con 0x557bf792c6f0 2023-09-20T21:49:29.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.122+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2793501178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 msgr2=0x557bf792cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.122+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2793501178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf792cb10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800d300 tx=0x7fd8c800d610 comp rx=0 tx=0).stop 2023-09-20T21:49:29.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.122+0000 7fd8d579b700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd8bc0a1400 0x7fd8bc0a38c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x557bf7928150 tx=0x7fd8c000b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:29.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.123+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2793501178 shutdown_connections 2023-09-20T21:49:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.123+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2793501178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 0x557bf792e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.123+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2793501178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf792cb10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.123+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2793501178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf792d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.123+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2793501178 >> 172.21.15.22:0/2793501178 conn(0x557bf7859000 msgr2=0x557bf79271d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.123+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2793501178 shutdown_connections 2023-09-20T21:49:29.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.123+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2793501178 wait complete. 2023-09-20T21:49:29.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.124+0000 7fd8eb2aab80 1 -- start start 2023-09-20T21:49:29.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.124+0000 7fd8eb2aab80 1 Processor -- start 2023-09-20T21:49:29.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.125+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf7936270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.125+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf79367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.125+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 0x557bf793d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.125+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf787de80 con 0x557bf792d050 2023-09-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.125+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf781a740 con 0x557bf792dbe0 2023-09-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.125+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf781a3e0 con 0x557bf792c6f0 2023-09-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.125+0000 7fd8eb2aab80 10 client.0 _ll_register_callbacks cb 0x557bf7764a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.126+0000 7fd8d579b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf7936270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.126+0000 7fd8d579b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf7936270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43080/0 (socket says 172.21.15.22:43080) 2023-09-20T21:49:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.126+0000 7fd8d579b700 1 -- 172.21.15.22:0/1529864511 learned_addr learned my addr 172.21.15.22:0/1529864511 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.126+0000 7fd8d5f9c700 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 0x557bf793d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.128+0000 7fd8d579b700 1 -- 172.21.15.22:0/1529864511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 msgr2=0x557bf793d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:29.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.128+0000 7fd8d579b700 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 0x557bf793d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.128+0000 7fd8d579b700 1 -- 172.21.15.22:0/1529864511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 msgr2=0x557bf79367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:29.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.128+0000 7fd8d579b700 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf79367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.128+0000 7fd8d579b700 1 -- 172.21.15.22:0/1529864511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d01a5730 con 0x557bf792c6f0 2023-09-20T21:49:29.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.129+0000 7fd8d579b700 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf7936270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0000c00 tx=0x7fd8c0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a835859882d86150 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:29.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.129+0000 7fd8b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:49:29.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.129+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/1529864511 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0026660 con 0x557bf792c6f0 2023-09-20T21:49:29.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.129+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/1529864511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bf77eebb0 con 0x557bf792c6f0 2023-09-20T21:49:29.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.129+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/1529864511 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fd8c00267c0 con 0x557bf792c6f0 2023-09-20T21:49:29.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.129+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/1529864511 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c0026980 con 0x557bf792c6f0 2023-09-20T21:49:29.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.129+0000 7fd8eb2aab80 10 client.25576 fetch_fsmap learned FSMap version 29 2023-09-20T21:49:29.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.130+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/1529864511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fd8c8007de0 con 0x557bf792c6f0 2023-09-20T21:49:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.130+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/1529864511 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557bf77eebb0 con 0x557bf792c6f0 2023-09-20T21:49:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.131+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/1529864511 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd8c000fbf0 con 0x557bf792c6f0 2023-09-20T21:49:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.131+0000 7fd8eb2aab80 10 client.25576 fetch_fsmap finished waiting for FSMap version 29 2023-09-20T21:49:29.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.131+0000 7fd8eb2aab80 10 client.25576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:49:29.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.131+0000 7fd8eb2aab80 20 client.25576 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:29.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.131+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557bf7952f90 0x557bf79553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.131+0000 7fd8eb2aab80 4 client.25576 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:49:29.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.131+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/1529864511 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bf78fb020 con 0x557bf7952f90 2023-09-20T21:49:29.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.132+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557bf7952f90 0x557bf79553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.132+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557bf7952f90 0x557bf79553b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20cefb6766a2005 server_cookie=38d7ebd8ec1af555 in_seq=0 out_seq=0 2023-09-20T21:49:29.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.132+0000 7fd8b8ff9700 10 client.25576 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:29.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.133+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/1529864511 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557bf78fb020 con 0x557bf7952f90 2023-09-20T21:49:29.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.133+0000 7fd8b8ff9700 10 client.25576 handle_command_reply: tid=0 2023-09-20T21:49:29.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.133+0000 7fd8eb2aab80 1 client.25576 shutdown 2023-09-20T21:49:29.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/1529864511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 msgr2=0x557bf7936270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf7936270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0000c00 tx=0x7fd8c0009310 comp rx=0 tx=0).stop 2023-09-20T21:49:29.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/1529864511 shutdown_connections 2023-09-20T21:49:29.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557bf7952f90 0x557bf79553b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792dbe0 0x557bf793d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792c6f0 0x557bf7936270 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/1529864511 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf79367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/1529864511 >> 172.21.15.22:0/1529864511 conn(0x557bf7859000 msgr2=0x557bf7926880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:29.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8b8ff9700 0 client.25576 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:29.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/1529864511 shutdown_connections 2023-09-20T21:49:29.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.136+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/1529864511 wait complete. 2023-09-20T21:49:29.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.137+0000 7fd8eb2aab80 20 client.25576 trim_cache size 0 max 16384 2023-09-20T21:49:29.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.170+0000 7fd8eb2aab80 1 Processor -- start 2023-09-20T21:49:29.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.170+0000 7fd8eb2aab80 1 -- start start 2023-09-20T21:49:29.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.170+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf7859000 0x557bf7954fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.170+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792c6f0 0x557bf793d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.170+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf7934f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.170+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf76fe920 con 0x557bf792d050 2023-09-20T21:49:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.170+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf79285c0 con 0x557bf792c6f0 2023-09-20T21:49:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.170+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf792e460 con 0x557bf7859000 2023-09-20T21:49:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.171+0000 7fd8d4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792c6f0 0x557bf793d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.171+0000 7fd8d4f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792c6f0 0x557bf793d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38008/0 (socket says 172.21.15.22:38008) 2023-09-20T21:49:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.171+0000 7fd8d4f9a700 1 -- 172.21.15.22:0/2657921853 learned_addr learned my addr 172.21.15.22:0/2657921853 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.171+0000 7fd8d5f9c700 1 --2- 172.21.15.22:0/2657921853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf7934f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.171+0000 7fd8d4f9a700 1 -- 172.21.15.22:0/2657921853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf7859000 msgr2=0x557bf7954fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:49:29.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.171+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/2657921853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf7859000 0x557bf7954fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.171+0000 7fd8d4f9a700 1 -- 172.21.15.22:0/2657921853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 msgr2=0x557bf7934f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:29.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.171+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/2657921853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf7934f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.172+0000 7fd8d4f9a700 1 -- 172.21.15.22:0/2657921853 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bf7691c60 con 0x557bf792c6f0 2023-09-20T21:49:29.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.172+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/2657921853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792c6f0 0x557bf793d780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc096a90 tx=0x7fd8cc0ca700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8172f5013ed6dbd server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:29.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.172+0000 7fd8bbfff700 1 -- 172.21.15.22:0/2657921853 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc09a360 con 0x557bf792c6f0 2023-09-20T21:49:29.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.172+0000 7fd8bbfff700 1 -- 172.21.15.22:0/2657921853 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8cc09a0d0 con 0x557bf792c6f0 2023-09-20T21:49:29.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.172+0000 7fd8bbfff700 1 -- 172.21.15.22:0/2657921853 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc09e120 con 0x557bf792c6f0 2023-09-20T21:49:29.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.173+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2657921853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792c6f0 msgr2=0x557bf793d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.173+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2657921853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792c6f0 0x557bf793d780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc096a90 tx=0x7fd8cc0ca700 comp rx=0 tx=0).stop 2023-09-20T21:49:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.174+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2657921853 shutdown_connections 2023-09-20T21:49:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.174+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2657921853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792d050 0x557bf7934f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.174+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2657921853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf7859000 0x557bf7954fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.174+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2657921853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf792c6f0 0x557bf793d780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.174+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2657921853 >> 172.21.15.22:0/2657921853 conn(0x557bf7927500 msgr2=0x557bf7926880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.176+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2657921853 shutdown_connections 2023-09-20T21:49:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.176+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2657921853 wait complete. 2023-09-20T21:49:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.176+0000 7fd8eb2aab80 1 -- start start 2023-09-20T21:49:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.177+0000 7fd8eb2aab80 1 Processor -- start 2023-09-20T21:49:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf7859000 0x557bf793be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792c6f0 0x557bf793c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8eb2aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792d050 0x557bf793c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf793daa0 con 0x557bf792c6f0 2023-09-20T21:49:29.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf79a6df0 con 0x557bf7859000 2023-09-20T21:49:29.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8eb2aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf79b7f40 con 0x557bf792d050 2023-09-20T21:49:29.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8eb2aab80 10 client.0 _ll_register_callbacks cb 0x557bf7764a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:29.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8d579b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf7859000 0x557bf793be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8d579b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf7859000 0x557bf793be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38022/0 (socket says 172.21.15.22:38022) 2023-09-20T21:49:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8d579b700 1 -- 172.21.15.22:0/2325431561 learned_addr learned my addr 172.21.15.22:0/2325431561 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792c6f0 0x557bf793c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8d579b700 1 -- 172.21.15.22:0/2325431561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792d050 msgr2=0x557bf793c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8d5f9c700 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792d050 0x557bf793c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.178+0000 7fd8d579b700 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792d050 0x557bf793c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.179+0000 7fd8d579b700 1 -- 172.21.15.22:0/2325431561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792c6f0 msgr2=0x557bf793c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:29.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.179+0000 7fd8d579b700 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792c6f0 0x557bf793c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.179+0000 7fd8d579b700 1 -- 172.21.15.22:0/2325431561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d01a51d0 con 0x557bf7859000 2023-09-20T21:49:29.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.179+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792c6f0 0x557bf793c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:49:29.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.179+0000 7fd8d5f9c700 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792d050 0x557bf793c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:49:29.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.179+0000 7fd8d579b700 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf7859000 0x557bf793be20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0006010 tx=0x7fd8c0005a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2175e7bd01e3c3f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:29.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.179+0000 7fd8b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:49:29.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.179+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/2325431561 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0031e70 con 0x557bf7859000 2023-09-20T21:49:29.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.179+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/2325431561 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fd8c000f460 con 0x557bf7859000 2023-09-20T21:49:29.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.179+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2325431561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8c8012ab0 con 0x557bf7859000 2023-09-20T21:49:29.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.180+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/2325431561 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c000f640 con 0x557bf7859000 2023-09-20T21:49:29.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.181+0000 7fd8eb2aab80 10 client.25592 fetch_fsmap learned FSMap version 29 2023-09-20T21:49:29.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.181+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2325431561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557bf76cf250 con 0x557bf7859000 2023-09-20T21:49:29.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.181+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/2325431561 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8c8012ab0 con 0x557bf7859000 2023-09-20T21:49:29.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.181+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/2325431561 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd8c0049070 con 0x557bf7859000 2023-09-20T21:49:29.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.182+0000 7fd8eb2aab80 10 client.25592 fetch_fsmap finished waiting for FSMap version 29 2023-09-20T21:49:29.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.182+0000 7fd8eb2aab80 10 client.25592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:49:29.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.182+0000 7fd8eb2aab80 20 client.25592 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:29.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.182+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557bf792dbe0 0x557bf79bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.182+0000 7fd8eb2aab80 4 client.25592 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:49:29.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.182+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2325431561 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557bf7876e60 con 0x557bf792dbe0 2023-09-20T21:49:29.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.183+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557bf792dbe0 0x557bf79bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.184+0000 7fd8d4f9a700 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557bf792dbe0 0x557bf79bb4b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4108752220638b22 server_cookie=e241641fdd26c9d2 in_seq=0 out_seq=0 2023-09-20T21:49:29.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.184+0000 7fd8b8ff9700 10 client.25592 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:29.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.184+0000 7fd8b8ff9700 1 -- 172.21.15.22:0/2325431561 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557bf7876e60 con 0x557bf792dbe0 2023-09-20T21:49:29.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.184+0000 7fd8b8ff9700 10 client.25592 handle_command_reply: tid=0 2023-09-20T21:49:29.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.184+0000 7fd8eb2aab80 1 client.25592 shutdown 2023-09-20T21:49:29.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.184+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2325431561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf7859000 msgr2=0x557bf793be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.185+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf7859000 0x557bf793be20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0006010 tx=0x7fd8c0005a70 comp rx=0 tx=0).stop 2023-09-20T21:49:29.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.185+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2325431561 shutdown_connections 2023-09-20T21:49:29.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.185+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557bf792dbe0 0x557bf79bb4b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.185+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557bf792d050 0x557bf793c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.185+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557bf7859000 0x557bf793be20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.185+0000 7fd8eb2aab80 1 --2- 172.21.15.22:0/2325431561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557bf792c6f0 0x557bf793c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.185+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2325431561 >> 172.21.15.22:0/2325431561 conn(0x557bf7927500 msgr2=0x557bf7954fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:29.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.185+0000 7fd8b8ff9700 0 client.25592 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:29.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.185+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2325431561 shutdown_connections 2023-09-20T21:49:29.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.187+0000 7fd8eb2aab80 1 -- 172.21.15.22:0/2325431561 wait complete. 2023-09-20T21:49:29.194 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:49:29.194 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2023-09-20T21:49:29.194 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2023-09-20T21:49:29.195 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:49:29.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.187+0000 7fd8eb2aab80 20 client.25592 trim_cache size 0 max 16384 2023-09-20T21:49:29.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.188+0000 7fd8d79ff700 1 -- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd8bc0a1400 msgr2=0x7fd8bc0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.188+0000 7fd8d79ff700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd8bc0a1400 0x7fd8bc0a38c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x557bf7928150 tx=0x7fd8c000b260 comp rx=0 tx=0).stop 2023-09-20T21:49:29.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.188+0000 7fd8d79ff700 1 -- 172.21.15.22:0/1738322359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8d0101fb0 msgr2=0x7fd8d019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.188+0000 7fd8d79ff700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8d0101fb0 0x7fd8d019d5e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00e9f0 tx=0x7fd8cc00ed00 comp rx=0 tx=0).stop 2023-09-20T21:49:29.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.189+0000 7fd8d79ff700 1 -- 172.21.15.22:0/1738322359 shutdown_connections 2023-09-20T21:49:29.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.189+0000 7fd8d79ff700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8d0104980 0x7fd8d01a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.189+0000 7fd8d79ff700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8d00ff640 0x7fd8d019d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.189+0000 7fd8d79ff700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd8bc0a1400 0x7fd8bc0a38c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.189+0000 7fd8d79ff700 1 --2- 172.21.15.22:0/1738322359 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd8d0101fb0 0x7fd8d019d5e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.189+0000 7fd8d79ff700 1 -- 172.21.15.22:0/1738322359 >> 172.21.15.22:0/1738322359 conn(0x7fd8d00fb240 msgr2=0x7fd8d0102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:29.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.189+0000 7fd8d79ff700 1 -- 172.21.15.22:0/1738322359 shutdown_connections 2023-09-20T21:49:29.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.189+0000 7fd8d79ff700 1 -- 172.21.15.22:0/1738322359 wait complete. 2023-09-20T21:49:29.217 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-20T21:49:29.217 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:49:29.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.517+0000 7f5fafdff700 1 -- 172.21.15.22:0/233167851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fa810d9b0 msgr2=0x7f5fa810fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.517+0000 7f5fafdff700 1 --2- 172.21.15.22:0/233167851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fa810d9b0 0x7f5fa810fe40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c210 tx=0x7f5fa000c520 comp rx=0 tx=0).stop 2023-09-20T21:49:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.517+0000 7f5fafdff700 1 -- 172.21.15.22:0/233167851 shutdown_connections 2023-09-20T21:49:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.517+0000 7f5fafdff700 1 --2- 172.21.15.22:0/233167851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fa810d9b0 0x7f5fa810fe40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.517+0000 7f5fafdff700 1 --2- 172.21.15.22:0/233167851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa8108670 0x7f5fa810aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.517+0000 7f5fafdff700 1 --2- 172.21.15.22:0/233167851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fa810afa0 0x7f5fa810d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.517+0000 7f5fafdff700 1 -- 172.21.15.22:0/233167851 >> 172.21.15.22:0/233167851 conn(0x7f5fa806df80 msgr2=0x7f5fa80703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.517+0000 7f5fafdff700 1 -- 172.21.15.22:0/233167851 shutdown_connections 2023-09-20T21:49:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.517+0000 7f5fafdff700 1 -- 172.21.15.22:0/233167851 wait complete. 2023-09-20T21:49:29.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.519+0000 7f5fafdff700 1 Processor -- start 2023-09-20T21:49:29.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.519+0000 7f5fafdff700 1 -- start start 2023-09-20T21:49:29.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.519+0000 7f5fafdff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fa8108670 0x7f5fa819d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fafdff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa810afa0 0x7f5fa819d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fafdff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fa810d9b0 0x7f5fa81a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fafdff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81a4d40 con 0x7f5fa8108670 2023-09-20T21:49:29.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fafdff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81a4eb0 con 0x7f5fa810d9b0 2023-09-20T21:49:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fafdff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81a5070 con 0x7f5fa810afa0 2023-09-20T21:49:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fad39a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa810afa0 0x7f5fa819d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fad39a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa810afa0 0x7f5fa819d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43124/0 (socket says 172.21.15.22:43124) 2023-09-20T21:49:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fad39a700 1 -- 172.21.15.22:0/815894758 learned_addr learned my addr 172.21.15.22:0/815894758 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fae39c700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fa810d9b0 0x7f5fa81a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fadb9b700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fa8108670 0x7f5fa819d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.520+0000 7f5fad39a700 1 -- 172.21.15.22:0/815894758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fa810d9b0 msgr2=0x7f5fa81a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.521+0000 7f5fad39a700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fa810d9b0 0x7f5fa81a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.521+0000 7f5fad39a700 1 -- 172.21.15.22:0/815894758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fa8108670 msgr2=0x7f5fa819d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.521+0000 7f5fad39a700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fa8108670 0x7f5fa819d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:29.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.521+0000 7f5fad39a700 1 -- 172.21.15.22:0/815894758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa000ae10 con 0x7f5fa810afa0 2023-09-20T21:49:29.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.521+0000 7f5fae39c700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fa810d9b0 0x7f5fa81a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:49:29.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.521+0000 7f5fad39a700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa810afa0 0x7f5fa819d700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800e9a0 tx=0x7f5f9800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:29.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.521+0000 7f5f9effd700 1 -- 172.21.15.22:0/815894758 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98009790 con 0x7f5fa810afa0 2023-09-20T21:49:29.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.521+0000 7f5fafdff700 1 -- 172.21.15.22:0/815894758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa81a52f0 con 0x7f5fa810afa0 2023-09-20T21:49:29.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.522+0000 7f5f9effd700 1 -- 172.21.15.22:0/815894758 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f9800cbd0 con 0x7f5fa810afa0 2023-09-20T21:49:29.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.522+0000 7f5f9effd700 1 -- 172.21.15.22:0/815894758 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98018680 con 0x7f5fa810afa0 2023-09-20T21:49:29.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.522+0000 7f5fafdff700 1 -- 172.21.15.22:0/815894758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa81a5850 con 0x7f5fa810afa0 2023-09-20T21:49:29.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.524+0000 7f5fafdff700 1 -- 172.21.15.22:0/815894758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa8114e40 con 0x7f5fa810afa0 2023-09-20T21:49:29.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.526+0000 7f5f9effd700 1 -- 172.21.15.22:0/815894758 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5f9800cd40 con 0x7f5fa810afa0 2023-09-20T21:49:29.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.528+0000 7f5f9effd700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5f940a1400 0x7f5f940a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.528+0000 7f5f9effd700 1 -- 172.21.15.22:0/815894758 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f5f98014070 con 0x7f5fa810afa0 2023-09-20T21:49:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.529+0000 7f5fadb9b700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5f940a1400 0x7f5f940a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.530+0000 7f5fadb9b700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5f940a1400 0x7f5f940a38c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8072f00 tx=0x7f5fa4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:29.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:29.533+0000 7f5f9effd700 1 -- 172.21.15.22:0/815894758 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5f98096670 con 0x7f5fa810afa0 2023-09-20T21:49:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.020+0000 7f5fafdff700 1 -- 172.21.15.22:0/815894758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fa810ef10 con 0x7f5fa810afa0 2023-09-20T21:49:30.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.025+0000 7f5f9effd700 1 -- 172.21.15.22:0/815894758 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11364 (secure 0 0 0) 0x7f5f98099c90 con 0x7f5fa810afa0 2023-09-20T21:49:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:49:30.028 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:49:25.256179+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T21:49:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.029+0000 7f5fafdff700 1 -- 172.21.15.22:0/815894758 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5f940a1400 msgr2=0x7f5f940a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.029+0000 7f5fafdff700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5f940a1400 0x7f5f940a38c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8072f00 tx=0x7f5fa4009260 comp rx=0 tx=0).stop 2023-09-20T21:49:30.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.029+0000 7f5fafdff700 1 -- 172.21.15.22:0/815894758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa810afa0 msgr2=0x7f5fa819d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.029+0000 7f5fafdff700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa810afa0 0x7f5fa819d700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800e9a0 tx=0x7f5f9800ed60 comp rx=0 tx=0).stop 2023-09-20T21:49:30.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.031+0000 7f5fafdff700 1 -- 172.21.15.22:0/815894758 shutdown_connections 2023-09-20T21:49:30.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.031+0000 7f5fafdff700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5f940a1400 0x7f5f940a38c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.031+0000 7f5fafdff700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5fa810d9b0 0x7f5fa81a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.031+0000 7f5fafdff700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fa8108670 0x7f5fa819d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.031+0000 7f5fafdff700 1 --2- 172.21.15.22:0/815894758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fa810afa0 0x7f5fa819d700 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.031+0000 7f5fafdff700 1 -- 172.21.15.22:0/815894758 >> 172.21.15.22:0/815894758 conn(0x7f5fa806df80 msgr2=0x7f5fa8112560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.037+0000 7f5fafdff700 1 -- 172.21.15.22:0/815894758 shutdown_connections 2023-09-20T21:49:30.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.037+0000 7f5fafdff700 1 -- 172.21.15.22:0/815894758 wait complete. 2023-09-20T21:49:30.041 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 29 2023-09-20T21:49:30.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-20T21:49:30.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.410+0000 7f24a5fff700 1 -- 172.21.15.22:0/2692526470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24a0107400 msgr2=0x7f24a0107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.410+0000 7f24a5fff700 1 --2- 172.21.15.22:0/2692526470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24a0107400 0x7f24a0107820 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2490009920 tx=0x7f2490009c30 comp rx=0 tx=0).stop 2023-09-20T21:49:30.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.411+0000 7f24a5fff700 1 -- 172.21.15.22:0/2692526470 shutdown_connections 2023-09-20T21:49:30.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.411+0000 7f24a5fff700 1 --2- 172.21.15.22:0/2692526470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a0108f90 0x7f24a010f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.411+0000 7f24a5fff700 1 --2- 172.21.15.22:0/2692526470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24a0107400 0x7f24a0107820 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.411+0000 7f24a5fff700 1 --2- 172.21.15.22:0/2692526470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24a01085f0 0x7f24a0108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.411+0000 7f24a5fff700 1 -- 172.21.15.22:0/2692526470 >> 172.21.15.22:0/2692526470 conn(0x7f24a00763a0 msgr2=0x7f24a00787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.411+0000 7f24a5fff700 1 -- 172.21.15.22:0/2692526470 shutdown_connections 2023-09-20T21:49:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.411+0000 7f24a5fff700 1 -- 172.21.15.22:0/2692526470 wait complete. 2023-09-20T21:49:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.412+0000 7f24a5fff700 1 Processor -- start 2023-09-20T21:49:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.412+0000 7f24a5fff700 1 -- start start 2023-09-20T21:49:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.412+0000 7f24a5fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24a0107400 0x7f24a019d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.412+0000 7f24a5fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24a01085f0 0x7f24a019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.412+0000 7f24a5fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a0108f90 0x7f24a01a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.412+0000 7f24a5fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01a4d20 con 0x7f24a01085f0 2023-09-20T21:49:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.412+0000 7f24a5fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01a4e60 con 0x7f24a0107400 2023-09-20T21:49:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f24a5fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a01a4ff0 con 0x7f24a0108f90 2023-09-20T21:49:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f2497fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24a01085f0 0x7f24a019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f249f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24a0107400 0x7f24a019d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f249f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24a0107400 0x7f24a019d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38062/0 (socket says 172.21.15.22:38062) 2023-09-20T21:49:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f249f7fe700 1 -- 172.21.15.22:0/4046796587 learned_addr learned my addr 172.21.15.22:0/4046796587 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f249ffff700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a0108f90 0x7f24a01a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f249f7fe700 1 -- 172.21.15.22:0/4046796587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a0108f90 msgr2=0x7f24a01a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f249f7fe700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a0108f90 0x7f24a01a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f249f7fe700 1 -- 172.21.15.22:0/4046796587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24a01085f0 msgr2=0x7f24a019d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f249f7fe700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24a01085f0 0x7f24a019d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f249f7fe700 1 -- 172.21.15.22:0/4046796587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880095f0 con 0x7f24a0107400 2023-09-20T21:49:30.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.413+0000 7f2497fff700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24a01085f0 0x7f24a019d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:49:30.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.414+0000 7f249f7fe700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24a0107400 0x7f24a019d1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2490009230 tx=0x7f249000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.414+0000 7f249d7fa700 1 -- 172.21.15.22:0/4046796587 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249001c070 con 0x7f24a0107400 2023-09-20T21:49:30.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.414+0000 7f249d7fa700 1 -- 172.21.15.22:0/4046796587 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f249000f7d0 con 0x7f24a0107400 2023-09-20T21:49:30.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.414+0000 7f24a5fff700 1 -- 172.21.15.22:0/4046796587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2490009520 con 0x7f24a0107400 2023-09-20T21:49:30.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.414+0000 7f249d7fa700 1 -- 172.21.15.22:0/4046796587 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490004a00 con 0x7f24a0107400 2023-09-20T21:49:30.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.414+0000 7f24a5fff700 1 -- 172.21.15.22:0/4046796587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a01a54b0 con 0x7f24a0107400 2023-09-20T21:49:30.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.419+0000 7f24b99e6b80 1 Processor -- start 2023-09-20T21:49:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.419+0000 7f249d7fa700 1 -- 172.21.15.22:0/4046796587 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2490020070 con 0x7f24a0107400 2023-09-20T21:49:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.419+0000 7f24b99e6b80 1 -- start start 2023-09-20T21:49:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.420+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd2bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.420+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 0x561febd2c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.420+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 0x561febd2d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.420+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febc9fca0 con 0x561febd2cbe0 2023-09-20T21:49:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.420+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febc83c70 con 0x561febd2b6f0 2023-09-20T21:49:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.420+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febc81590 con 0x561febd2c050 2023-09-20T21:49:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.420+0000 7f249ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd2bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.420+0000 7f249f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 0x561febd2c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.420+0000 7f249ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd2bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38066/0 (socket says 172.21.15.22:38066) 2023-09-20T21:49:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.420+0000 7f249ffff700 1 -- 172.21.15.22:0/40087695 learned_addr learned my addr 172.21.15.22:0/40087695 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.421+0000 7f249ffff700 1 -- 172.21.15.22:0/40087695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 msgr2=0x561febd2c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.421+0000 7f2497fff700 1 --2- 172.21.15.22:0/40087695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 0x561febd2d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.421+0000 7f249ffff700 1 --2- 172.21.15.22:0/40087695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 0x561febd2c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.421+0000 7f249ffff700 1 -- 172.21.15.22:0/40087695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 msgr2=0x561febd2d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.421+0000 7f249ffff700 1 --2- 172.21.15.22:0/40087695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 0x561febd2d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.421+0000 7f249ffff700 1 -- 172.21.15.22:0/40087695 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561febaa7de0 con 0x561febd2b6f0 2023-09-20T21:49:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.421+0000 7f2497fff700 1 --2- 172.21.15.22:0/40087695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 0x561febd2d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:49:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.421+0000 7f249f7fe700 1 --2- 172.21.15.22:0/40087695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 0x561febd2c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:49:30.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.422+0000 7f249d7fa700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f248c0a1380 0x7f248c0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.423+0000 7f249f7fe700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f248c0a1380 0x7f248c0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.423+0000 7f249d7fa700 1 -- 172.21.15.22:0/4046796587 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f24900cdd10 con 0x7f24a0107400 2023-09-20T21:49:30.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.424+0000 7f249f7fe700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f248c0a1380 0x7f248c0a3840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x561febd27010 tx=0x7f2490009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.427+0000 7f249ffff700 1 --2- 172.21.15.22:0/40087695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd2bb10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f24980098f0 tx=0x7f2498009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e74f70b983a94186 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.427+0000 7f2495ffb700 1 -- 172.21.15.22:0/40087695 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498019070 con 0x561febd2b6f0 2023-09-20T21:49:30.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.427+0000 7f2495ffb700 1 -- 172.21.15.22:0/40087695 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f249800f040 con 0x561febd2b6f0 2023-09-20T21:49:30.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.427+0000 7f2495ffb700 1 -- 172.21.15.22:0/40087695 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498007c00 con 0x561febd2b6f0 2023-09-20T21:49:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.428+0000 7f24b99e6b80 1 -- 172.21.15.22:0/40087695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 msgr2=0x561febd2bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.428+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/40087695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd2bb10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f24980098f0 tx=0x7f2498009c00 comp rx=0 tx=0).stop 2023-09-20T21:49:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.428+0000 7f24b99e6b80 1 -- 172.21.15.22:0/40087695 shutdown_connections 2023-09-20T21:49:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.428+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/40087695 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 0x561febd2d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.428+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/40087695 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd2bb10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.428+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/40087695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 0x561febd2c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.428+0000 7f24b99e6b80 1 -- 172.21.15.22:0/40087695 >> 172.21.15.22:0/40087695 conn(0x561febc58000 msgr2=0x561febd261d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.428+0000 7f24b99e6b80 1 -- 172.21.15.22:0/40087695 shutdown_connections 2023-09-20T21:49:30.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.428+0000 7f24b99e6b80 1 -- 172.21.15.22:0/40087695 wait complete. 2023-09-20T21:49:30.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.429+0000 7f24b99e6b80 1 -- start start 2023-09-20T21:49:30.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.429+0000 7f24b99e6b80 1 Processor -- start 2023-09-20T21:49:30.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.430+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd35160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.430+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 0x561febd356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.430+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 0x561febd3c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.430+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febc7ce80 con 0x561febd2cbe0 2023-09-20T21:49:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.430+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febc19740 con 0x561febd2b6f0 2023-09-20T21:49:30.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.430+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febc193e0 con 0x561febd2c050 2023-09-20T21:49:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.430+0000 7f24b99e6b80 10 client.0 _ll_register_callbacks cb 0x561febb63a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:30.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.431+0000 7f249ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd35160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.431+0000 7f249ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd35160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38078/0 (socket says 172.21.15.22:38078) 2023-09-20T21:49:30.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.431+0000 7f249ffff700 1 -- 172.21.15.22:0/2545456561 learned_addr learned my addr 172.21.15.22:0/2545456561 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:30.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.431+0000 7f2497fff700 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 0x561febd356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.432+0000 7f249ffff700 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 0x561febd3c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.432+0000 7f2497fff700 1 -- 172.21.15.22:0/2545456561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 msgr2=0x561febd35160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.432+0000 7f2497fff700 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd35160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.432+0000 7f2497fff700 1 -- 172.21.15.22:0/2545456561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 msgr2=0x561febd3c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.432+0000 7f2497fff700 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 0x561febd3c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.432+0000 7f2497fff700 1 -- 172.21.15.22:0/2545456561 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2498003c50 con 0x561febd2c050 2023-09-20T21:49:30.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.433+0000 7f2497fff700 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 0x561febd356a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2488013d60 tx=0x7f24880116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8ee2b3ab4faede server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.433+0000 7f247effd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:49:30.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.436+0000 7f247effd700 1 -- 172.21.15.22:0/2545456561 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488004a30 con 0x561febd2c050 2023-09-20T21:49:30.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.436+0000 7f247effd700 1 -- 172.21.15.22:0/2545456561 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f2488004b90 con 0x561febd2c050 2023-09-20T21:49:30.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.437+0000 7f24b99e6b80 1 -- 172.21.15.22:0/2545456561 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561febbedbb0 con 0x561febd2c050 2023-09-20T21:49:30.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.439+0000 7f247effd700 1 -- 172.21.15.22:0/2545456561 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2488004d50 con 0x561febd2c050 2023-09-20T21:49:30.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.439+0000 7f24b99e6b80 10 client.25591 fetch_fsmap learned FSMap version 29 2023-09-20T21:49:30.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.439+0000 7f24b99e6b80 1 -- 172.21.15.22:0/2545456561 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561feba90c60 con 0x561febd2c050 2023-09-20T21:49:30.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.440+0000 7f247effd700 1 -- 172.21.15.22:0/2545456561 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561febbedbb0 con 0x561febd2c050 2023-09-20T21:49:30.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.441+0000 7f247effd700 1 -- 172.21.15.22:0/2545456561 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f248801b780 con 0x561febd2c050 2023-09-20T21:49:30.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.441+0000 7f24b99e6b80 10 client.25591 fetch_fsmap finished waiting for FSMap version 29 2023-09-20T21:49:30.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.441+0000 7f24b99e6b80 10 client.25591 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.j 2023-09-20T21:49:30.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.441+0000 7f24b99e6b80 20 client.25591 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:30.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.441+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x561febd41000 0x561febd43420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.441+0000 7f24b99e6b80 4 client.25591 mds_command: new command op to 24473 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:49:30.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.441+0000 7f24b99e6b80 1 -- 172.21.15.22:0/2545456561 --> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561febcfa020 con 0x561febd41000 2023-09-20T21:49:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.453+0000 7f249ffff700 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x561febd41000 0x561febd43420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.454+0000 7f249ffff700 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x561febd41000 0x561febd43420 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ebeb68efa8c2a7dc server_cookie=9905b2d2a62a1280 in_seq=0 out_seq=0 2023-09-20T21:49:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.455+0000 7f247effd700 10 client.25591 ms_handle_connect on v2:172.21.15.22:6834/56798476 2023-09-20T21:49:30.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.456+0000 7f247effd700 1 -- 172.21.15.22:0/2545456561 <== mds.4 v2:172.21.15.22:6834/56798476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561febcfa020 con 0x561febd41000 2023-09-20T21:49:30.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.456+0000 7f247effd700 10 client.25591 handle_command_reply: tid=0 2023-09-20T21:49:30.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.457+0000 7f24b99e6b80 1 client.25591 shutdown 2023-09-20T21:49:30.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.461+0000 7f24b99e6b80 1 -- 172.21.15.22:0/2545456561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 msgr2=0x561febd356a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.461+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 0x561febd356a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2488013d60 tx=0x7f24880116c0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.461+0000 7f24b99e6b80 1 -- 172.21.15.22:0/2545456561 shutdown_connections 2023-09-20T21:49:30.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.461+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x561febd41000 0x561febd43420 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.461+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2cbe0 0x561febd3c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.461+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2b6f0 0x561febd35160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.461+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/2545456561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2c050 0x561febd356a0 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2488013d60 tx=0x7f24880116c0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.461+0000 7f24b99e6b80 1 -- 172.21.15.22:0/2545456561 >> 172.21.15.22:0/2545456561 conn(0x561febc58000 msgr2=0x561febd25880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.461+0000 7f247effd700 0 client.25591 ms_handle_reset on v2:172.21.15.22:6834/56798476 2023-09-20T21:49:30.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.463+0000 7f24b99e6b80 1 -- 172.21.15.22:0/2545456561 shutdown_connections 2023-09-20T21:49:30.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.463+0000 7f24b99e6b80 1 -- 172.21.15.22:0/2545456561 wait complete. 2023-09-20T21:49:30.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.471+0000 7f24b99e6b80 20 client.25591 trim_cache size 0 max 16384 2023-09-20T21:49:30.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:30 smithi022 ceph-mon[97651]: pgmap v364: 129 pgs: 129 active+clean; 833 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 MiB/s wr, 156 op/s 2023-09-20T21:49:30.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:30 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/815894758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:49:30.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.496+0000 7f24b99e6b80 1 Processor -- start 2023-09-20T21:49:30.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.496+0000 7f24b99e6b80 1 -- start start 2023-09-20T21:49:30.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.497+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2b6f0 0x561febd33000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.497+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2cbe0 0x561febd3c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.497+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd222e0 0x561febd2fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.497+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febafd920 con 0x561febd2b6f0 2023-09-20T21:49:30.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.497+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febbedbe0 con 0x561febd2cbe0 2023-09-20T21:49:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.497+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febd34610 con 0x561febd222e0 2023-09-20T21:49:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.497+0000 7f249ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd222e0 0x561febd2fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.497+0000 7f249ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd222e0 0x561febd2fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43172/0 (socket says 172.21.15.22:43172) 2023-09-20T21:49:30.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.497+0000 7f249ffff700 1 -- 172.21.15.22:0/3524128653 learned_addr learned my addr 172.21.15.22:0/3524128653 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:30.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.498+0000 7f249ffff700 1 -- 172.21.15.22:0/3524128653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2cbe0 msgr2=0x561febd3c600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:49:30.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.498+0000 7f249ffff700 1 --2- 172.21.15.22:0/3524128653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2cbe0 0x561febd3c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.498+0000 7f249ffff700 1 -- 172.21.15.22:0/3524128653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2b6f0 msgr2=0x561febd33000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.498+0000 7f249ffff700 1 --2- 172.21.15.22:0/3524128653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2b6f0 0x561febd33000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.498+0000 7f249ffff700 1 -- 172.21.15.22:0/3524128653 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561febace250 con 0x561febd222e0 2023-09-20T21:49:30.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.498+0000 7f249ffff700 1 --2- 172.21.15.22:0/3524128653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd222e0 0x561febd2fb60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f24980076e0 tx=0x7f2498009100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fe98ed13d98df72 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.499+0000 7f24967fc700 1 -- 172.21.15.22:0/3524128653 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498027040 con 0x561febd222e0 2023-09-20T21:49:30.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.499+0000 7f24967fc700 1 -- 172.21.15.22:0/3524128653 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2498007a00 con 0x561febd222e0 2023-09-20T21:49:30.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.499+0000 7f24967fc700 1 -- 172.21.15.22:0/3524128653 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498019040 con 0x561febd222e0 2023-09-20T21:49:30.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.499+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3524128653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd222e0 msgr2=0x561febd2fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.499+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/3524128653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd222e0 0x561febd2fb60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f24980076e0 tx=0x7f2498009100 comp rx=0 tx=0).stop 2023-09-20T21:49:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.500+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3524128653 shutdown_connections 2023-09-20T21:49:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.500+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/3524128653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd222e0 0x561febd2fb60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.500+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/3524128653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd2b6f0 0x561febd33000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.500+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/3524128653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2cbe0 0x561febd3c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.500+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3524128653 >> 172.21.15.22:0/3524128653 conn(0x561febc58000 msgr2=0x561febd43010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.500+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3524128653 shutdown_connections 2023-09-20T21:49:30.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.500+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3524128653 wait complete. 2023-09-20T21:49:30.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.500+0000 7f24b99e6b80 1 -- start start 2023-09-20T21:49:30.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.501+0000 7f24b99e6b80 1 Processor -- start 2023-09-20T21:49:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.501+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd222e0 0x561febd38610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.501+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2b6f0 0x561febd38b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f24b99e6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2cbe0 0x561febd3bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febd3c0b0 con 0x561febd222e0 2023-09-20T21:49:30.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febd3c220 con 0x561febd2cbe0 2023-09-20T21:49:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f24b99e6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561febd3c390 con 0x561febd2b6f0 2023-09-20T21:49:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f24b99e6b80 10 client.0 _ll_register_callbacks cb 0x561febb63a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f2497fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2b6f0 0x561febd38b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f2497fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2b6f0 0x561febd38b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43178/0 (socket says 172.21.15.22:43178) 2023-09-20T21:49:30.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f2497fff700 1 -- 172.21.15.22:0/3322749267 learned_addr learned my addr 172.21.15.22:0/3322749267 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:30.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f249ffff700 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd222e0 0x561febd38610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f249ffff700 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2cbe0 0x561febd3bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f2497fff700 1 -- 172.21.15.22:0/3322749267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2cbe0 msgr2=0x561febd3bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f2497fff700 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2cbe0 0x561febd3bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f2497fff700 1 -- 172.21.15.22:0/3322749267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd222e0 msgr2=0x561febd38610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f2497fff700 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd222e0 0x561febd38610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f2497fff700 1 -- 172.21.15.22:0/3322749267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561febaa7de0 con 0x561febd2b6f0 2023-09-20T21:49:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.502+0000 7f249ffff700 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2cbe0 0x561febd3bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:49:30.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.503+0000 7f2497fff700 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2b6f0 0x561febd38b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f248801bbe0 tx=0x7f248801b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ec275b366843ba server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.503+0000 7f247effd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:49:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.503+0000 7f247effd700 1 -- 172.21.15.22:0/3322749267 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488009380 con 0x561febd2b6f0 2023-09-20T21:49:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.503+0000 7f247effd700 1 -- 172.21.15.22:0/3322749267 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f2488000e40 con 0x561febd2b6f0 2023-09-20T21:49:30.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.504+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3322749267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f249800f040 con 0x561febd2b6f0 2023-09-20T21:49:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.504+0000 7f247effd700 1 -- 172.21.15.22:0/3322749267 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f248801b8e0 con 0x561febd2b6f0 2023-09-20T21:49:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.504+0000 7f24b99e6b80 10 client.25597 fetch_fsmap learned FSMap version 29 2023-09-20T21:49:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.504+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3322749267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561febaa62b0 con 0x561febd2b6f0 2023-09-20T21:49:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.505+0000 7f247effd700 1 -- 172.21.15.22:0/3322749267 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f249800f040 con 0x561febd2b6f0 2023-09-20T21:49:30.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.505+0000 7f247effd700 1 -- 172.21.15.22:0/3322749267 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f24880049c0 con 0x561febd2b6f0 2023-09-20T21:49:30.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.505+0000 7f24b99e6b80 10 client.25597 fetch_fsmap finished waiting for FSMap version 29 2023-09-20T21:49:30.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.505+0000 7f24b99e6b80 10 client.25597 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.j 2023-09-20T21:49:30.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.505+0000 7f24b99e6b80 20 client.25597 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:30.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.505+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x561febdb7ff0 0x561febdba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.505+0000 7f24b99e6b80 4 client.25597 mds_command: new command op to 24473 tid=0[{"prefix": "damage ls"}] 2023-09-20T21:49:30.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.505+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3322749267 --> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561febc75e60 con 0x561febdb7ff0 2023-09-20T21:49:30.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.507+0000 7f249ffff700 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x561febdb7ff0 0x561febdba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.508+0000 7f249ffff700 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x561febdb7ff0 0x561febdba410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3451ca24fa4859d server_cookie=44f210bacc8992ba in_seq=0 out_seq=0 2023-09-20T21:49:30.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.508+0000 7f247effd700 10 client.25597 ms_handle_connect on v2:172.21.15.22:6834/56798476 2023-09-20T21:49:30.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.508+0000 7f247effd700 1 -- 172.21.15.22:0/3322749267 <== mds.4 v2:172.21.15.22:6834/56798476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561febc75e60 con 0x561febdb7ff0 2023-09-20T21:49:30.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.508+0000 7f247effd700 10 client.25597 handle_command_reply: tid=0 2023-09-20T21:49:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.508+0000 7f24b99e6b80 1 client.25597 shutdown 2023-09-20T21:49:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.509+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3322749267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2b6f0 msgr2=0x561febd38b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.509+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2b6f0 0x561febd38b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f248801bbe0 tx=0x7f248801b640 comp rx=0 tx=0).stop 2023-09-20T21:49:30.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.509+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3322749267 shutdown_connections 2023-09-20T21:49:30.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.509+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561febd2cbe0 0x561febd3bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.509+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x561febdb7ff0 0x561febdba410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.509+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561febd222e0 0x561febd38610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.509+0000 7f24b99e6b80 1 --2- 172.21.15.22:0/3322749267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561febd2b6f0 0x561febd38b50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.509+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3322749267 >> 172.21.15.22:0/3322749267 conn(0x561febc58000 msgr2=0x561febd33000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.509+0000 7f247effd700 0 client.25597 ms_handle_reset on v2:172.21.15.22:6834/56798476 2023-09-20T21:49:30.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.510+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3322749267 shutdown_connections 2023-09-20T21:49:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.510+0000 7f24b99e6b80 1 -- 172.21.15.22:0/3322749267 wait complete. 2023-09-20T21:49:30.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.510+0000 7f24b99e6b80 20 client.25597 trim_cache size 0 max 16384 2023-09-20T21:49:30.521 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2023-09-20T21:49:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.513+0000 7f24a5fff700 1 -- 172.21.15.22:0/4046796587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f248c0a1380 msgr2=0x7f248c0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.513+0000 7f24a5fff700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f248c0a1380 0x7f248c0a3840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x561febd27010 tx=0x7f2490009260 comp rx=0 tx=0).stop 2023-09-20T21:49:30.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.513+0000 7f24a5fff700 1 -- 172.21.15.22:0/4046796587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24a0107400 msgr2=0x7f24a019d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.513+0000 7f24a5fff700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24a0107400 0x7f24a019d1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2490009230 tx=0x7f249000b790 comp rx=0 tx=0).stop 2023-09-20T21:49:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.513+0000 7f24a5fff700 1 -- 172.21.15.22:0/4046796587 shutdown_connections 2023-09-20T21:49:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.513+0000 7f24a5fff700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a0108f90 0x7f24a01a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.513+0000 7f24a5fff700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f248c0a1380 0x7f248c0a3840 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.513+0000 7f24a5fff700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24a0107400 0x7f24a019d1a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.513+0000 7f24a5fff700 1 --2- 172.21.15.22:0/4046796587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24a01085f0 0x7f24a019d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.513+0000 7f24a5fff700 1 -- 172.21.15.22:0/4046796587 >> 172.21.15.22:0/4046796587 conn(0x7f24a00763a0 msgr2=0x7f24a00786e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.514+0000 7f24a5fff700 1 -- 172.21.15.22:0/4046796587 shutdown_connections 2023-09-20T21:49:30.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.514+0000 7f24a5fff700 1 -- 172.21.15.22:0/4046796587 wait complete. 2023-09-20T21:49:30.545 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-20T21:49:30.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:30 smithi123 ceph-mon[134433]: pgmap v364: 129 pgs: 129 active+clean; 833 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 MiB/s wr, 156 op/s 2023-09-20T21:49:30.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:30 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/815894758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:49:30.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:30 smithi089 ceph-mon[134090]: pgmap v364: 129 pgs: 129 active+clean; 833 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 MiB/s wr, 156 op/s 2023-09-20T21:49:30.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:30 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/815894758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:49:30.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.926+0000 7fd2309ff700 1 -- 172.21.15.22:0/3042286059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c10d120 msgr2=0x7fd22c10d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.926+0000 7fd2309ff700 1 --2- 172.21.15.22:0/3042286059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c10d120 0x7fd22c10d5a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd220009990 tx=0x7fd220009ca0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.926+0000 7fd2309ff700 1 -- 172.21.15.22:0/3042286059 shutdown_connections 2023-09-20T21:49:30.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.926+0000 7fd2309ff700 1 --2- 172.21.15.22:0/3042286059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c10d120 0x7fd22c10d5a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.926+0000 7fd2309ff700 1 --2- 172.21.15.22:0/3042286059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd22c107de0 0x7fd22c10a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.926+0000 7fd2309ff700 1 --2- 172.21.15.22:0/3042286059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c10a750 0x7fd22c10cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.926+0000 7fd2309ff700 1 -- 172.21.15.22:0/3042286059 >> 172.21.15.22:0/3042286059 conn(0x7fd22c06db90 msgr2=0x7fd22c06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.926+0000 7fd2309ff700 1 -- 172.21.15.22:0/3042286059 shutdown_connections 2023-09-20T21:49:30.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.927+0000 7fd2309ff700 1 -- 172.21.15.22:0/3042286059 wait complete. 2023-09-20T21:49:30.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.928+0000 7fd2309ff700 1 Processor -- start 2023-09-20T21:49:30.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.929+0000 7fd2309ff700 1 -- start start 2023-09-20T21:49:30.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.929+0000 7fd2309ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd22c107de0 0x7fd22c114330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.929+0000 7fd2309ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c10a750 0x7fd22c114870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.929+0000 7fd2309ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c10d120 0x7fd22c1b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.929+0000 7fd2309ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c117950 con 0x7fd22c107de0 2023-09-20T21:49:30.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.929+0000 7fd2309ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c117ac0 con 0x7fd22c10d120 2023-09-20T21:49:30.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.929+0000 7fd2309ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c1b6170 con 0x7fd22c10a750 2023-09-20T21:49:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.930+0000 7fd22ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c10d120 0x7fd22c1b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.930+0000 7fd22ad9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c10d120 0x7fd22c1b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38098/0 (socket says 172.21.15.22:38098) 2023-09-20T21:49:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.930+0000 7fd22ad9d700 1 -- 172.21.15.22:0/3860599180 learned_addr learned my addr 172.21.15.22:0/3860599180 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.930+0000 7fd229d9b700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c10a750 0x7fd22c114870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.930+0000 7fd22a59c700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd22c107de0 0x7fd22c114330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.931+0000 7fd229d9b700 1 -- 172.21.15.22:0/3860599180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c10d120 msgr2=0x7fd22c1b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.931+0000 7fd229d9b700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c10d120 0x7fd22c1b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.931+0000 7fd229d9b700 1 -- 172.21.15.22:0/3860599180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd22c107de0 msgr2=0x7fd22c114330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.931+0000 7fd229d9b700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd22c107de0 0x7fd22c114330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.931+0000 7fd229d9b700 1 -- 172.21.15.22:0/3860599180 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2200095f0 con 0x7fd22c10a750 2023-09-20T21:49:30.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.936+0000 7fd229d9b700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c10a750 0x7fd22c114870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd21c006fd0 tx=0x7fd21c00b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.936+0000 7fd21b7fe700 1 -- 172.21.15.22:0/3860599180 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c011830 con 0x7fd22c10a750 2023-09-20T21:49:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.936+0000 7fd21b7fe700 1 -- 172.21.15.22:0/3860599180 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd21c011e50 con 0x7fd22c10a750 2023-09-20T21:49:30.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.936+0000 7fd21b7fe700 1 -- 172.21.15.22:0/3860599180 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c010d60 con 0x7fd22c10a750 2023-09-20T21:49:30.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.937+0000 7fd2309ff700 1 -- 172.21.15.22:0/3860599180 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22c1b6370 con 0x7fd22c10a750 2023-09-20T21:49:30.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.937+0000 7fd2309ff700 1 -- 172.21.15.22:0/3860599180 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c1b68b0 con 0x7fd22c10a750 2023-09-20T21:49:30.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.942+0000 7fd21b7fe700 1 -- 172.21.15.22:0/3860599180 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fd21c010380 con 0x7fd22c10a750 2023-09-20T21:49:30.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.943+0000 7fd2443eab80 1 Processor -- start 2023-09-20T21:49:30.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.943+0000 7fd2443eab80 1 -- start start 2023-09-20T21:49:30.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.944+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1d6f0 0x5589ebe1db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.944+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe1e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.944+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1ebe0 0x5589ebe1f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.944+0000 7fd2443eab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebd91ca0 con 0x5589ebe1ebe0 2023-09-20T21:49:30.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.944+0000 7fd2443eab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebd75c70 con 0x5589ebe1e050 2023-09-20T21:49:30.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.944+0000 7fd2443eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebd73590 con 0x5589ebe1d6f0 2023-09-20T21:49:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.944+0000 7fd22ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1d6f0 0x5589ebe1db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.944+0000 7fd22ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1d6f0 0x5589ebe1db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43210/0 (socket says 172.21.15.22:43210) 2023-09-20T21:49:30.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.944+0000 7fd22ad9d700 1 -- 172.21.15.22:0/2285561452 learned_addr learned my addr 172.21.15.22:0/2285561452 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:30.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.945+0000 7fd22a59c700 1 --2- 172.21.15.22:0/2285561452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe1e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.945+0000 7fd22ad9d700 1 -- 172.21.15.22:0/2285561452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 msgr2=0x5589ebe1e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.945+0000 7fd21b7fe700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd2140a1500 0x7fd2140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.945+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/2285561452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe1e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.945+0000 7fd22ad9d700 1 -- 172.21.15.22:0/2285561452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1ebe0 msgr2=0x5589ebe1f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.945+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/2285561452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1ebe0 0x5589ebe1f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.945+0000 7fd22ad9d700 1 -- 172.21.15.22:0/2285561452 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589ebb99de0 con 0x5589ebe1d6f0 2023-09-20T21:49:30.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.945+0000 7fd21b7fe700 1 -- 172.21.15.22:0/3860599180 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fd21c0cc4f0 con 0x7fd22c10a750 2023-09-20T21:49:30.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.945+0000 7fd22a59c700 1 --2- 172.21.15.22:0/2285561452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe1e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:49:30.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.949+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/2285561452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1d6f0 0x5589ebe1db10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd22000be30 tx=0x7fd22000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1d4db573402dc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.950+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd2140a1500 0x7fd2140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.950+0000 7fd218ff9700 1 -- 172.21.15.22:0/2285561452 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22001d030 con 0x5589ebe1d6f0 2023-09-20T21:49:30.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.950+0000 7fd218ff9700 1 -- 172.21.15.22:0/2285561452 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd22001e040 con 0x5589ebe1d6f0 2023-09-20T21:49:30.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.950+0000 7fd218ff9700 1 -- 172.21.15.22:0/2285561452 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2200236c0 con 0x5589ebe1d6f0 2023-09-20T21:49:30.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.951+0000 7fd2443eab80 1 -- 172.21.15.22:0/2285561452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1d6f0 msgr2=0x5589ebe1db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.951+0000 7fd2443eab80 1 --2- 172.21.15.22:0/2285561452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1d6f0 0x5589ebe1db10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd22000be30 tx=0x7fd22000bf10 comp rx=0 tx=0).stop 2023-09-20T21:49:30.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.951+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd2140a1500 0x7fd2140a39c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd220026000 tx=0x7fd22000f740 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.951+0000 7fd2443eab80 1 -- 172.21.15.22:0/2285561452 shutdown_connections 2023-09-20T21:49:30.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.951+0000 7fd2443eab80 1 --2- 172.21.15.22:0/2285561452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1ebe0 0x5589ebe1f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.951+0000 7fd2443eab80 1 --2- 172.21.15.22:0/2285561452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1d6f0 0x5589ebe1db10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.951+0000 7fd2443eab80 1 --2- 172.21.15.22:0/2285561452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe1e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.951+0000 7fd2443eab80 1 -- 172.21.15.22:0/2285561452 >> 172.21.15.22:0/2285561452 conn(0x5589ebd4a000 msgr2=0x5589ebe181d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.951+0000 7fd2443eab80 1 -- 172.21.15.22:0/2285561452 shutdown_connections 2023-09-20T21:49:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.951+0000 7fd2443eab80 1 -- 172.21.15.22:0/2285561452 wait complete. 2023-09-20T21:49:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.952+0000 7fd2443eab80 1 -- start start 2023-09-20T21:49:30.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.952+0000 7fd2443eab80 1 Processor -- start 2023-09-20T21:49:30.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.954+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 0x5589ebe271f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.954+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1e050 0x5589ebe27730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.954+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe2e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.954+0000 7fd2443eab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebd6ee80 con 0x5589ebe1e050 2023-09-20T21:49:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.954+0000 7fd2443eab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebd0b740 con 0x5589ebe1d6f0 2023-09-20T21:49:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.954+0000 7fd2443eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebd0b3e0 con 0x5589ebe1ebe0 2023-09-20T21:49:30.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.954+0000 7fd2443eab80 10 client.0 _ll_register_callbacks cb 0x5589ebc55a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.954+0000 7fd22a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 0x5589ebe271f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.954+0000 7fd22a59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 0x5589ebe271f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38112/0 (socket says 172.21.15.22:38112) 2023-09-20T21:49:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd22a59c700 1 -- 172.21.15.22:0/2293924874 learned_addr learned my addr 172.21.15.22:0/2293924874 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe2e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd22a59c700 1 -- 172.21.15.22:0/2293924874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 msgr2=0x5589ebe2e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd22a59c700 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe2e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd22a59c700 1 -- 172.21.15.22:0/2293924874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1e050 msgr2=0x5589ebe27730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd22a59c700 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1e050 0x5589ebe27730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd22a59c700 1 -- 172.21.15.22:0/2293924874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c10e120 con 0x5589ebe1d6f0 2023-09-20T21:49:30.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd22a59c700 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 0x5589ebe271f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd22400bf00 tx=0x7fd22400a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c869997c7814718d server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe2e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:49:30.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd211ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:49:30.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.955+0000 7fd211ffb700 1 -- 172.21.15.22:0/2293924874 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2240155f0 con 0x5589ebe1d6f0 2023-09-20T21:49:30.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.956+0000 7fd211ffb700 1 -- 172.21.15.22:0/2293924874 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fd224015750 con 0x5589ebe1d6f0 2023-09-20T21:49:30.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.956+0000 7fd2443eab80 1 -- 172.21.15.22:0/2293924874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589ebcdfbb0 con 0x5589ebe1d6f0 2023-09-20T21:49:30.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.956+0000 7fd211ffb700 1 -- 172.21.15.22:0/2293924874 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd224009e70 con 0x5589ebe1d6f0 2023-09-20T21:49:30.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.956+0000 7fd2443eab80 10 client.25622 fetch_fsmap learned FSMap version 29 2023-09-20T21:49:30.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.956+0000 7fd2443eab80 1 -- 172.21.15.22:0/2293924874 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5589ebb82c60 con 0x5589ebe1d6f0 2023-09-20T21:49:30.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.956+0000 7fd211ffb700 1 -- 172.21.15.22:0/2293924874 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589ebcdfbb0 con 0x5589ebe1d6f0 2023-09-20T21:49:30.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.957+0000 7fd211ffb700 1 -- 172.21.15.22:0/2293924874 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd224014070 con 0x5589ebe1d6f0 2023-09-20T21:49:30.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.957+0000 7fd2443eab80 10 client.25622 fetch_fsmap finished waiting for FSMap version 29 2023-09-20T21:49:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.957+0000 7fd2443eab80 10 client.25622 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-09-20T21:49:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.957+0000 7fd2443eab80 20 client.25622 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.957+0000 7fd2443eab80 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x5589ebe33050 0x5589ebe35470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.957+0000 7fd2443eab80 4 client.25622 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:49:30.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.957+0000 7fd2443eab80 1 -- 172.21.15.22:0/2293924874 --> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589ebdec020 con 0x5589ebe33050 2023-09-20T21:49:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.960+0000 7fd229d9b700 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x5589ebe33050 0x5589ebe35470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.961+0000 7fd229d9b700 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x5589ebe33050 0x5589ebe35470 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9013ce5bc85519ed server_cookie=d31408e5445b6bd8 in_seq=0 out_seq=0 2023-09-20T21:49:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.961+0000 7fd211ffb700 10 client.25622 ms_handle_connect on v2:172.21.15.123:6834/835271994 2023-09-20T21:49:30.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.962+0000 7fd211ffb700 1 -- 172.21.15.22:0/2293924874 <== mds.3 v2:172.21.15.123:6834/835271994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5589ebdec020 con 0x5589ebe33050 2023-09-20T21:49:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.962+0000 7fd211ffb700 10 client.25622 handle_command_reply: tid=0 2023-09-20T21:49:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.962+0000 7fd2443eab80 1 client.25622 shutdown 2023-09-20T21:49:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.964+0000 7fd2443eab80 1 -- 172.21.15.22:0/2293924874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 msgr2=0x5589ebe271f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.964+0000 7fd2443eab80 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 0x5589ebe271f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd22400bf00 tx=0x7fd22400a6c0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.965+0000 7fd2443eab80 1 -- 172.21.15.22:0/2293924874 shutdown_connections 2023-09-20T21:49:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.965+0000 7fd2443eab80 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x5589ebe33050 0x5589ebe35470 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.965+0000 7fd2443eab80 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe2e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.965+0000 7fd2443eab80 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 0x5589ebe271f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.965+0000 7fd2443eab80 1 --2- 172.21.15.22:0/2293924874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1e050 0x5589ebe27730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.965+0000 7fd2443eab80 1 -- 172.21.15.22:0/2293924874 >> 172.21.15.22:0/2293924874 conn(0x5589ebd4a000 msgr2=0x5589ebe17880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.965+0000 7fd211ffb700 0 client.25622 ms_handle_reset on v2:172.21.15.123:6834/835271994 2023-09-20T21:49:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.965+0000 7fd2443eab80 1 -- 172.21.15.22:0/2293924874 shutdown_connections 2023-09-20T21:49:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.965+0000 7fd2443eab80 1 -- 172.21.15.22:0/2293924874 wait complete. 2023-09-20T21:49:30.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.965+0000 7fd2443eab80 20 client.25622 trim_cache size 0 max 16384 2023-09-20T21:49:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.989+0000 7fd2443eab80 1 Processor -- start 2023-09-20T21:49:30.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.989+0000 7fd2443eab80 1 -- start start 2023-09-20T21:49:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.989+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 0x5589ebe1efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.989+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1e050 0x5589ebe2e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.989+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.989+0000 7fd2443eab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebbef920 con 0x5589ebe1e050 2023-09-20T21:49:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.989+0000 7fd2443eab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebe2ec00 con 0x5589ebe1d6f0 2023-09-20T21:49:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.989+0000 7fd2443eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebe10220 con 0x5589ebe1ebe0 2023-09-20T21:49:30.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.991+0000 7fd22ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.991+0000 7fd22ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe181a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43216/0 (socket says 172.21.15.22:43216) 2023-09-20T21:49:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.991+0000 7fd22ad9d700 1 -- 172.21.15.22:0/377723417 learned_addr learned my addr 172.21.15.22:0/377723417 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:30.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.992+0000 7fd22ad9d700 1 -- 172.21.15.22:0/377723417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 msgr2=0x5589ebe1efc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:49:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.992+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/377723417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 0x5589ebe1efc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.992+0000 7fd22ad9d700 1 -- 172.21.15.22:0/377723417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1e050 msgr2=0x5589ebe2e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.992+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/377723417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1e050 0x5589ebe2e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.992+0000 7fd22ad9d700 1 -- 172.21.15.22:0/377723417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589ebbc0250 con 0x5589ebe1ebe0 2023-09-20T21:49:30.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.992+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/377723417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe181a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd220027040 tx=0x7fd220031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791bfccc54757d16 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:30.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.993+0000 7fd2197fa700 1 -- 172.21.15.22:0/377723417 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22001d050 con 0x5589ebe1ebe0 2023-09-20T21:49:30.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.993+0000 7fd2197fa700 1 -- 172.21.15.22:0/377723417 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd220023e20 con 0x5589ebe1ebe0 2023-09-20T21:49:30.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.993+0000 7fd2197fa700 1 -- 172.21.15.22:0/377723417 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220023460 con 0x5589ebe1ebe0 2023-09-20T21:49:30.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.994+0000 7fd2443eab80 1 -- 172.21.15.22:0/377723417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 msgr2=0x5589ebe181a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:30.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.994+0000 7fd2443eab80 1 --2- 172.21.15.22:0/377723417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe181a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd220027040 tx=0x7fd220031000 comp rx=0 tx=0).stop 2023-09-20T21:49:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.994+0000 7fd2443eab80 1 -- 172.21.15.22:0/377723417 shutdown_connections 2023-09-20T21:49:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.994+0000 7fd2443eab80 1 --2- 172.21.15.22:0/377723417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe181a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.994+0000 7fd2443eab80 1 --2- 172.21.15.22:0/377723417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1d6f0 0x5589ebe1efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.994+0000 7fd2443eab80 1 --2- 172.21.15.22:0/377723417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1e050 0x5589ebe2e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:30.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.994+0000 7fd2443eab80 1 -- 172.21.15.22:0/377723417 >> 172.21.15.22:0/377723417 conn(0x5589ebd4a000 msgr2=0x5589ebe142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:30.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.994+0000 7fd2443eab80 1 -- 172.21.15.22:0/377723417 shutdown_connections 2023-09-20T21:49:30.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.994+0000 7fd2443eab80 1 -- 172.21.15.22:0/377723417 wait complete. 2023-09-20T21:49:30.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.995+0000 7fd2443eab80 1 -- start start 2023-09-20T21:49:30.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.996+0000 7fd2443eab80 1 Processor -- start 2023-09-20T21:49:30.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.996+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1d6f0 0x5589ebe2bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.996+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe2c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.996+0000 7fd2443eab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe2d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:30.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.996+0000 7fd2443eab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebe2da70 con 0x5589ebe1d6f0 2023-09-20T21:49:31.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.996+0000 7fd2443eab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebe2dbe0 con 0x5589ebe1e050 2023-09-20T21:49:31.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.996+0000 7fd2443eab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ebe2dda0 con 0x5589ebe1ebe0 2023-09-20T21:49:31.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.996+0000 7fd2443eab80 10 client.0 _ll_register_callbacks cb 0x5589ebc55a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:31.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.997+0000 7fd229d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe2c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.997+0000 7fd229d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe2c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38114/0 (socket says 172.21.15.22:38114) 2023-09-20T21:49:31.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.997+0000 7fd229d9b700 1 -- 172.21.15.22:0/4127964953 learned_addr learned my addr 172.21.15.22:0/4127964953 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:31.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.997+0000 7fd22ad9d700 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe2d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.997+0000 7fd229d9b700 1 -- 172.21.15.22:0/4127964953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 msgr2=0x5589ebe2d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.997+0000 7fd229d9b700 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe2d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.997+0000 7fd229d9b700 1 -- 172.21.15.22:0/4127964953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1d6f0 msgr2=0x5589ebe2bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.997+0000 7fd229d9b700 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1d6f0 0x5589ebe2bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.997+0000 7fd229d9b700 1 -- 172.21.15.22:0/4127964953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589ebb99de0 con 0x5589ebe1e050 2023-09-20T21:49:31.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.998+0000 7fd229d9b700 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe2c520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00ba80 tx=0x7fd21c099c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7824bb95069d3b8d server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.998+0000 7fd211ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:49:31.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.998+0000 7fd211ffb700 1 -- 172.21.15.22:0/4127964953 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0cc040 con 0x5589ebe1e050 2023-09-20T21:49:31.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.998+0000 7fd2443eab80 1 -- 172.21.15.22:0/4127964953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd22001e040 con 0x5589ebe1e050 2023-09-20T21:49:31.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.998+0000 7fd211ffb700 1 -- 172.21.15.22:0/4127964953 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fd21c099490 con 0x5589ebe1e050 2023-09-20T21:49:31.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.998+0000 7fd211ffb700 1 -- 172.21.15.22:0/4127964953 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd21c09d2c0 con 0x5589ebe1e050 2023-09-20T21:49:31.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.998+0000 7fd2443eab80 10 client.25625 fetch_fsmap learned FSMap version 29 2023-09-20T21:49:31.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.998+0000 7fd2443eab80 1 -- 172.21.15.22:0/4127964953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5589ebb982b0 con 0x5589ebe1e050 2023-09-20T21:49:31.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.999+0000 7fd211ffb700 1 -- 172.21.15.22:0/4127964953 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd21c07e020 con 0x5589ebe1e050 2023-09-20T21:49:31.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.999+0000 7fd211ffb700 1 -- 172.21.15.22:0/4127964953 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) v1 ==== 7898+0+0 (secure 0 0 0) 0x7fd21c099780 con 0x5589ebe1e050 2023-09-20T21:49:31.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.999+0000 7fd2443eab80 10 client.25625 fetch_fsmap finished waiting for FSMap version 29 2023-09-20T21:49:31.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.999+0000 7fd2443eab80 10 client.25625 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-09-20T21:49:31.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.999+0000 7fd2443eab80 20 client.25625 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:31.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.999+0000 7fd2443eab80 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x5589ebe87970 0x5589ebe89d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.999+0000 7fd2443eab80 4 client.25625 mds_command: new command op to 24487 tid=0[{"prefix": "damage ls"}] 2023-09-20T21:49:31.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.999+0000 7fd2443eab80 1 -- 172.21.15.22:0/4127964953 --> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5589ebd67e60 con 0x5589ebe87970 2023-09-20T21:49:31.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:30.999+0000 7fd22a59c700 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x5589ebe87970 0x5589ebe89d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.000+0000 7fd22a59c700 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x5589ebe87970 0x5589ebe89d90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7a7697d4162805a server_cookie=b1e349953a5d8960 in_seq=0 out_seq=0 2023-09-20T21:49:31.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.000+0000 7fd211ffb700 10 client.25625 ms_handle_connect on v2:172.21.15.123:6834/835271994 2023-09-20T21:49:31.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.000+0000 7fd211ffb700 1 -- 172.21.15.22:0/4127964953 <== mds.3 v2:172.21.15.123:6834/835271994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5589ebd67e60 con 0x5589ebe87970 2023-09-20T21:49:31.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.000+0000 7fd211ffb700 10 client.25625 handle_command_reply: tid=0 2023-09-20T21:49:31.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.000+0000 7fd2443eab80 1 client.25625 shutdown 2023-09-20T21:49:31.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.000+0000 7fd2443eab80 1 -- 172.21.15.22:0/4127964953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 msgr2=0x5589ebe2c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.000+0000 7fd2443eab80 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe2c520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00ba80 tx=0x7fd21c099c40 comp rx=0 tx=0).stop 2023-09-20T21:49:31.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.001+0000 7fd2443eab80 1 -- 172.21.15.22:0/4127964953 shutdown_connections 2023-09-20T21:49:31.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.001+0000 7fd2443eab80 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x5589ebe87970 0x5589ebe89d90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.001+0000 7fd2443eab80 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589ebe1ebe0 0x5589ebe2d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.001+0000 7fd2443eab80 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5589ebe1d6f0 0x5589ebe2bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.001+0000 7fd2443eab80 1 --2- 172.21.15.22:0/4127964953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589ebe1e050 0x5589ebe2c520 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.001+0000 7fd2443eab80 1 -- 172.21.15.22:0/4127964953 >> 172.21.15.22:0/4127964953 conn(0x5589ebd4a000 msgr2=0x5589ebe18030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.001+0000 7fd211ffb700 0 client.25625 ms_handle_reset on v2:172.21.15.123:6834/835271994 2023-09-20T21:49:31.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.002+0000 7fd2443eab80 1 -- 172.21.15.22:0/4127964953 shutdown_connections 2023-09-20T21:49:31.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.002+0000 7fd2443eab80 1 -- 172.21.15.22:0/4127964953 wait complete. 2023-09-20T21:49:31.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.003+0000 7fd2443eab80 20 client.25625 trim_cache size 0 max 16384 2023-09-20T21:49:31.010 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2023-09-20T21:49:31.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 -- 172.21.15.22:0/3860599180 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd2140a1500 msgr2=0x7fd2140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd2140a1500 0x7fd2140a39c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd220026000 tx=0x7fd22000f740 comp rx=0 tx=0).stop 2023-09-20T21:49:31.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 -- 172.21.15.22:0/3860599180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c10a750 msgr2=0x7fd22c114870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c10a750 0x7fd22c114870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd21c006fd0 tx=0x7fd21c00b730 comp rx=0 tx=0).stop 2023-09-20T21:49:31.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 -- 172.21.15.22:0/3860599180 shutdown_connections 2023-09-20T21:49:31.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fd2140a1500 0x7fd2140a39c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c10d120 0x7fd22c1b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd22c107de0 0x7fd22c114330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 --2- 172.21.15.22:0/3860599180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c10a750 0x7fd22c114870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 -- 172.21.15.22:0/3860599180 >> 172.21.15.22:0/3860599180 conn(0x7fd22c06db90 msgr2=0x7fd22c108a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.008+0000 7fd2309ff700 1 -- 172.21.15.22:0/3860599180 shutdown_connections 2023-09-20T21:49:31.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.009+0000 7fd2309ff700 1 -- 172.21.15.22:0/3860599180 wait complete. 2023-09-20T21:49:31.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-20T21:49:31.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.405+0000 7f1555bff700 1 -- 172.21.15.22:0/1333064531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1550103000 msgr2=0x7f1550103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.405+0000 7f1555bff700 1 --2- 172.21.15.22:0/1333064531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1550103000 0x7f1550103420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f15400099c0 tx=0x7f1540009cd0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.405+0000 7f1555bff700 1 -- 172.21.15.22:0/1333064531 shutdown_connections 2023-09-20T21:49:31.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.405+0000 7f1555bff700 1 --2- 172.21.15.22:0/1333064531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1550104b90 0x7f155010b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.405+0000 7f1555bff700 1 --2- 172.21.15.22:0/1333064531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1550103000 0x7f1550103420 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.405+0000 7f1555bff700 1 --2- 172.21.15.22:0/1333064531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f15501041f0 0x7f1550104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.405+0000 7f1555bff700 1 -- 172.21.15.22:0/1333064531 >> 172.21.15.22:0/1333064531 conn(0x7f15500fe590 msgr2=0x7f15501009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.405+0000 7f1555bff700 1 -- 172.21.15.22:0/1333064531 shutdown_connections 2023-09-20T21:49:31.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.406+0000 7f1555bff700 1 -- 172.21.15.22:0/1333064531 wait complete. 2023-09-20T21:49:31.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.407+0000 7f1555bff700 1 Processor -- start 2023-09-20T21:49:31.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.407+0000 7f1555bff700 1 -- start start 2023-09-20T21:49:31.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.409+0000 7f1555bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1550103000 0x7f1550198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.409+0000 7f1555bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15501041f0 0x7f1550199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.409+0000 7f1555bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1550104b90 0x7f15501a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.409+0000 7f1555bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15501a0880 con 0x7f1550103000 2023-09-20T21:49:31.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.409+0000 7f1555bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15501a09f0 con 0x7f1550104b90 2023-09-20T21:49:31.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.409+0000 7f1555bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15501a0bb0 con 0x7f15501041f0 2023-09-20T21:49:31.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.409+0000 7f154effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15501041f0 0x7f1550199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.409+0000 7f154effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15501041f0 0x7f1550199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43244/0 (socket says 172.21.15.22:43244) 2023-09-20T21:49:31.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.409+0000 7f154effd700 1 -- 172.21.15.22:0/2451067656 learned_addr learned my addr 172.21.15.22:0/2451067656 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:31.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.411+0000 7f154ffff700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1550104b90 0x7f15501a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.411+0000 7f154effd700 1 -- 172.21.15.22:0/2451067656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1550104b90 msgr2=0x7f15501a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.411+0000 7f154effd700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1550104b90 0x7f15501a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.411+0000 7f154effd700 1 -- 172.21.15.22:0/2451067656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1550103000 msgr2=0x7f1550198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.411+0000 7f154effd700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1550103000 0x7f1550198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.411+0000 7f154effd700 1 -- 172.21.15.22:0/2451067656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1544009520 con 0x7f15501041f0 2023-09-20T21:49:31.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.413+0000 7f154effd700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15501041f0 0x7f1550199280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1544009f90 tx=0x7f154400ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.413+0000 7f154cff9700 1 -- 172.21.15.22:0/2451067656 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544004220 con 0x7f15501041f0 2023-09-20T21:49:31.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.413+0000 7f154cff9700 1 -- 172.21.15.22:0/2451067656 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1544004380 con 0x7f15501041f0 2023-09-20T21:49:31.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.413+0000 7f154cff9700 1 -- 172.21.15.22:0/2451067656 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544003ac0 con 0x7f15501041f0 2023-09-20T21:49:31.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.413+0000 7f1555bff700 1 -- 172.21.15.22:0/2451067656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15400095f0 con 0x7f15501041f0 2023-09-20T21:49:31.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.414+0000 7f1555bff700 1 -- 172.21.15.22:0/2451067656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15500755b0 con 0x7f15501041f0 2023-09-20T21:49:31.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.420+0000 7f156955eb80 1 Processor -- start 2023-09-20T21:49:31.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.427+0000 7f154cff9700 1 -- 172.21.15.22:0/2451067656 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1544003c40 con 0x7f15501041f0 2023-09-20T21:49:31.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.427+0000 7f156955eb80 1 -- start start 2023-09-20T21:49:31.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.429+0000 7f154cff9700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f15380a1300 0x7f15380a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.430+0000 7f154cff9700 1 -- 172.21.15.22:0/2451067656 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f15440d58b0 con 0x7f15501041f0 2023-09-20T21:49:31.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.430+0000 7f156955eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.430+0000 7f156955eb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70050 0x55e194f705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.430+0000 7f156955eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70be0 0x55e194f71060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.430+0000 7f156955eb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194ee3ca0 con 0x55e194f70050 2023-09-20T21:49:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.430+0000 7f156955eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194ec7c70 con 0x55e194f6f6f0 2023-09-20T21:49:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.430+0000 7f156955eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194ec5590 con 0x55e194f70be0 2023-09-20T21:49:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.431+0000 7f154f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.431+0000 7f154f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38152/0 (socket says 172.21.15.22:38152) 2023-09-20T21:49:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.431+0000 7f154f7fe700 1 -- 172.21.15.22:0/133938563 learned_addr learned my addr 172.21.15.22:0/133938563 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.431+0000 7f154effd700 1 --2- 172.21.15.22:0/133938563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70050 0x55e194f705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.432+0000 7f154effd700 1 -- 172.21.15.22:0/133938563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70be0 msgr2=0x55e194f71060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:49:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.432+0000 7f154effd700 1 --2- 172.21.15.22:0/133938563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70be0 0x55e194f71060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.432+0000 7f154effd700 1 -- 172.21.15.22:0/133938563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 msgr2=0x55e194f6fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.432+0000 7f154effd700 1 --2- 172.21.15.22:0/133938563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.432+0000 7f154effd700 1 -- 172.21.15.22:0/133938563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e194cebde0 con 0x55e194f70050 2023-09-20T21:49:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.433+0000 7f154effd700 1 --2- 172.21.15.22:0/133938563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70050 0x55e194f705e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f154400d010 tx=0x7f15440a22e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b413b8c225427aa server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.433+0000 7f1535ffb700 1 -- 172.21.15.22:0/133938563 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15440a28b0 con 0x55e194f70050 2023-09-20T21:49:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.433+0000 7f1535ffb700 1 -- 172.21.15.22:0/133938563 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15440a66e0 con 0x55e194f70050 2023-09-20T21:49:31.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.433+0000 7f1535ffb700 1 -- 172.21.15.22:0/133938563 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15440a2000 con 0x55e194f70050 2023-09-20T21:49:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.434+0000 7f156955eb80 1 -- 172.21.15.22:0/133938563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70050 msgr2=0x55e194f705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.434+0000 7f156955eb80 1 --2- 172.21.15.22:0/133938563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70050 0x55e194f705e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f154400d010 tx=0x7f15440a22e0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.436+0000 7f156955eb80 1 -- 172.21.15.22:0/133938563 shutdown_connections 2023-09-20T21:49:31.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.436+0000 7f156955eb80 1 --2- 172.21.15.22:0/133938563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70be0 0x55e194f71060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.436+0000 7f156955eb80 1 --2- 172.21.15.22:0/133938563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.436+0000 7f156955eb80 1 --2- 172.21.15.22:0/133938563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70050 0x55e194f705e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.436+0000 7f156955eb80 1 -- 172.21.15.22:0/133938563 >> 172.21.15.22:0/133938563 conn(0x55e194e9c000 msgr2=0x55e194f6a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.437+0000 7f156955eb80 1 -- 172.21.15.22:0/133938563 shutdown_connections 2023-09-20T21:49:31.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.437+0000 7f156955eb80 1 -- 172.21.15.22:0/133938563 wait complete. 2023-09-20T21:49:31.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.438+0000 7f156955eb80 1 -- start start 2023-09-20T21:49:31.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.438+0000 7f154ffff700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f15380a1300 0x7f15380a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.439+0000 7f156955eb80 1 Processor -- start 2023-09-20T21:49:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.444+0000 7f156955eb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f6f6f0 0x55e194f79180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.444+0000 7f156955eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 0x55e194f796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.444+0000 7f156955eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f70be0 0x55e194f80700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.444+0000 7f156955eb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194ec0e80 con 0x55e194f6f6f0 2023-09-20T21:49:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.444+0000 7f156955eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194e5d740 con 0x55e194f70be0 2023-09-20T21:49:31.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.444+0000 7f156955eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194e5d3e0 con 0x55e194f70050 2023-09-20T21:49:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.444+0000 7f156955eb80 10 client.0 _ll_register_callbacks cb 0x55e194da7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:31.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.445+0000 7f154ffff700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f15380a1300 0x7f15380a37c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1548009260 tx=0x7f154800b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.445+0000 7f154effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 0x55e194f796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.445+0000 7f154f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f6f6f0 0x55e194f79180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.445+0000 7f154f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f6f6f0 0x55e194f79180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45300/0 (socket says 172.21.15.22:45300) 2023-09-20T21:49:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.445+0000 7f154f7fe700 1 -- 172.21.15.22:0/2779594258 learned_addr learned my addr 172.21.15.22:0/2779594258 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.445+0000 7f154ffff700 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f70be0 0x55e194f80700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.446+0000 7f154ffff700 1 -- 172.21.15.22:0/2779594258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 msgr2=0x55e194f796c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.446+0000 7f154ffff700 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 0x55e194f796c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.446+0000 7f154ffff700 1 -- 172.21.15.22:0/2779594258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f6f6f0 msgr2=0x55e194f79180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.446+0000 7f154ffff700 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f6f6f0 0x55e194f79180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.446+0000 7f154ffff700 1 -- 172.21.15.22:0/2779594258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15500755b0 con 0x55e194f70be0 2023-09-20T21:49:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.446+0000 7f154effd700 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 0x55e194f796c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:49:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.446+0000 7f154f7fe700 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f6f6f0 0x55e194f79180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:49:31.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.446+0000 7f154ffff700 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f70be0 0x55e194f80700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1548007b70 tx=0x7f1548004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=160e0de9112e98a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.446+0000 7f152effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:49:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.447+0000 7f152effd700 1 -- 172.21.15.22:0/2779594258 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15480147e0 con 0x55e194f70be0 2023-09-20T21:49:31.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.447+0000 7f156955eb80 1 -- 172.21.15.22:0/2779594258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e194e31bb0 con 0x55e194f70be0 2023-09-20T21:49:31.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.447+0000 7f152effd700 1 -- 172.21.15.22:0/2779594258 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f15480093a0 con 0x55e194f70be0 2023-09-20T21:49:31.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.448+0000 7f152effd700 1 -- 172.21.15.22:0/2779594258 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1548016e10 con 0x55e194f70be0 2023-09-20T21:49:31.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.448+0000 7f156955eb80 10 client.25634 fetch_fsmap learned FSMap version 30 2023-09-20T21:49:31.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.448+0000 7f156955eb80 1 -- 172.21.15.22:0/2779594258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e194cd4c60 con 0x55e194f70be0 2023-09-20T21:49:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.448+0000 7f152effd700 1 -- 172.21.15.22:0/2779594258 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e194e31bb0 con 0x55e194f70be0 2023-09-20T21:49:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.448+0000 7f152effd700 1 -- 172.21.15.22:0/2779594258 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1548020c10 con 0x55e194f70be0 2023-09-20T21:49:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.449+0000 7f156955eb80 10 client.25634 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:49:31.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.449+0000 7f156955eb80 10 client.25634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-20T21:49:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.449+0000 7f156955eb80 20 client.25634 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.449+0000 7f156955eb80 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e194f84f60 0x55e194f87380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.449+0000 7f156955eb80 4 client.25634 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:49:31.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.449+0000 7f156955eb80 1 -- 172.21.15.22:0/2779594258 --> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e194f3e020 con 0x55e194f84f60 2023-09-20T21:49:31.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.454+0000 7f154f7fe700 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e194f84f60 0x55e194f87380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.456+0000 7f154f7fe700 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e194f84f60 0x55e194f87380 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afcaac7d06dc94a4 server_cookie=3acfd367d94213b0 in_seq=0 out_seq=0 2023-09-20T21:49:31.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.456+0000 7f152effd700 10 client.25634 ms_handle_connect on v2:172.21.15.89:6836/3887702690 2023-09-20T21:49:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.457+0000 7f152effd700 1 -- 172.21.15.22:0/2779594258 <== mds.2 v2:172.21.15.89:6836/3887702690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e194f3e020 con 0x55e194f84f60 2023-09-20T21:49:31.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.457+0000 7f152effd700 10 client.25634 handle_command_reply: tid=0 2023-09-20T21:49:31.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.457+0000 7f156955eb80 1 client.25634 shutdown 2023-09-20T21:49:31.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.458+0000 7f156955eb80 1 -- 172.21.15.22:0/2779594258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f70be0 msgr2=0x55e194f80700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.458+0000 7f156955eb80 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f70be0 0x55e194f80700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1548007b70 tx=0x7f1548004ad0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.458+0000 7f156955eb80 1 -- 172.21.15.22:0/2779594258 shutdown_connections 2023-09-20T21:49:31.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.458+0000 7f156955eb80 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e194f84f60 0x55e194f87380 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.458+0000 7f156955eb80 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f70be0 0x55e194f80700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.458+0000 7f156955eb80 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f6f6f0 0x55e194f79180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.458+0000 7f156955eb80 1 --2- 172.21.15.22:0/2779594258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 0x55e194f796c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.458+0000 7f156955eb80 1 -- 172.21.15.22:0/2779594258 >> 172.21.15.22:0/2779594258 conn(0x55e194e9c000 msgr2=0x55e194f69880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.459+0000 7f152effd700 0 client.25634 ms_handle_reset on v2:172.21.15.89:6836/3887702690 2023-09-20T21:49:31.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.459+0000 7f156955eb80 1 -- 172.21.15.22:0/2779594258 shutdown_connections 2023-09-20T21:49:31.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.460+0000 7f156955eb80 1 -- 172.21.15.22:0/2779594258 wait complete. 2023-09-20T21:49:31.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.460+0000 7f156955eb80 20 client.25634 trim_cache size 0 max 16384 2023-09-20T21:49:31.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.481+0000 7f156955eb80 1 Processor -- start 2023-09-20T21:49:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.483+0000 7f156955eb80 1 -- start start 2023-09-20T21:49:31.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.484+0000 7f156955eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.484+0000 7f156955eb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70050 0x55e194f806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.484+0000 7f156955eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70be0 0x55e194f77ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.484+0000 7f156955eb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194d41920 con 0x55e194f70050 2023-09-20T21:49:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.484+0000 7f156955eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194f6acf0 con 0x55e194f6f6f0 2023-09-20T21:49:31.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.484+0000 7f156955eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194f70430 con 0x55e194f70be0 2023-09-20T21:49:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.485+0000 7f154f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.485+0000 7f154f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38172/0 (socket says 172.21.15.22:38172) 2023-09-20T21:49:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.485+0000 7f154f7fe700 1 -- 172.21.15.22:0/3336061051 learned_addr learned my addr 172.21.15.22:0/3336061051 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.486+0000 7f154f7fe700 1 -- 172.21.15.22:0/3336061051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70be0 msgr2=0x55e194f77ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:49:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.486+0000 7f154f7fe700 1 --2- 172.21.15.22:0/3336061051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70be0 0x55e194f77ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.486+0000 7f154f7fe700 1 -- 172.21.15.22:0/3336061051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70050 msgr2=0x55e194f806c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.486+0000 7f154f7fe700 1 --2- 172.21.15.22:0/3336061051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70050 0x55e194f806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.486+0000 7f154f7fe700 1 -- 172.21.15.22:0/3336061051 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e194d12250 con 0x55e194f6f6f0 2023-09-20T21:49:31.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.486+0000 7f154f7fe700 1 --2- 172.21.15.22:0/3336061051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1540006010 tx=0x7f1540019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac05e7aebf5001e server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.487+0000 7f15367fc700 1 -- 172.21.15.22:0/3336061051 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154001e040 con 0x55e194f6f6f0 2023-09-20T21:49:31.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.487+0000 7f15367fc700 1 -- 172.21.15.22:0/3336061051 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f154002d070 con 0x55e194f6f6f0 2023-09-20T21:49:31.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.487+0000 7f15367fc700 1 -- 172.21.15.22:0/3336061051 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15400054d0 con 0x55e194f6f6f0 2023-09-20T21:49:31.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.488+0000 7f156955eb80 1 -- 172.21.15.22:0/3336061051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 msgr2=0x55e194f6fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.488+0000 7f156955eb80 1 --2- 172.21.15.22:0/3336061051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1540006010 tx=0x7f1540019040 comp rx=0 tx=0).stop 2023-09-20T21:49:31.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.500+0000 7f156955eb80 1 -- 172.21.15.22:0/3336061051 shutdown_connections 2023-09-20T21:49:31.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.500+0000 7f156955eb80 1 --2- 172.21.15.22:0/3336061051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70be0 0x55e194f77ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.500+0000 7f156955eb80 1 --2- 172.21.15.22:0/3336061051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f6fad0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.500+0000 7f156955eb80 1 --2- 172.21.15.22:0/3336061051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70050 0x55e194f806c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.500+0000 7f156955eb80 1 -- 172.21.15.22:0/3336061051 >> 172.21.15.22:0/3336061051 conn(0x55e194e9c000 msgr2=0x55e194f662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.504+0000 7f156955eb80 1 -- 172.21.15.22:0/3336061051 shutdown_connections 2023-09-20T21:49:31.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.504+0000 7f156955eb80 1 -- 172.21.15.22:0/3336061051 wait complete. 2023-09-20T21:49:31.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.505+0000 7f156955eb80 1 -- start start 2023-09-20T21:49:31.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.505+0000 7f156955eb80 1 Processor -- start 2023-09-20T21:49:31.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.509+0000 7f156955eb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f7bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.509+0000 7f156955eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 0x55e194f7c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.509+0000 7f156955eb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70be0 0x55e194f7f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.509+0000 7f156955eb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194f7f820 con 0x55e194f70be0 2023-09-20T21:49:31.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.509+0000 7f156955eb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194f7f990 con 0x55e194f6f6f0 2023-09-20T21:49:31.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.509+0000 7f156955eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e194f7fb00 con 0x55e194f70050 2023-09-20T21:49:31.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.509+0000 7f156955eb80 10 client.0 _ll_register_callbacks cb 0x55e194da7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:31.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.510+0000 7f154effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 0x55e194f7c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.510+0000 7f154effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 0x55e194f7c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43280/0 (socket says 172.21.15.22:43280) 2023-09-20T21:49:31.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.510+0000 7f154effd700 1 -- 172.21.15.22:0/3800024387 learned_addr learned my addr 172.21.15.22:0/3800024387 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:31.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.510+0000 7f154f7fe700 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f7bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.510+0000 7f154ffff700 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70be0 0x55e194f7f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.511+0000 7f154f7fe700 1 -- 172.21.15.22:0/3800024387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 msgr2=0x55e194f7c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.511+0000 7f154f7fe700 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 0x55e194f7c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.511+0000 7f154f7fe700 1 -- 172.21.15.22:0/3800024387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70be0 msgr2=0x55e194f7f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.511+0000 7f154f7fe700 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70be0 0x55e194f7f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.511+0000 7f154f7fe700 1 -- 172.21.15.22:0/3800024387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155010bd90 con 0x55e194f6f6f0 2023-09-20T21:49:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.511+0000 7f154f7fe700 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f7bd80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55e194f7a020 tx=0x7f1540005300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22fe7b41d6a2cca0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.511+0000 7f152effd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:49:31.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.512+0000 7f152effd700 1 -- 172.21.15.22:0/3800024387 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154001e070 con 0x55e194f6f6f0 2023-09-20T21:49:31.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.512+0000 7f152effd700 1 -- 172.21.15.22:0/3800024387 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f15400290b0 con 0x55e194f6f6f0 2023-09-20T21:49:31.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.512+0000 7f156955eb80 1 -- 172.21.15.22:0/3800024387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15440a66e0 con 0x55e194f6f6f0 2023-09-20T21:49:31.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.514+0000 7f152effd700 1 -- 172.21.15.22:0/3800024387 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1540005720 con 0x55e194f6f6f0 2023-09-20T21:49:31.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.518+0000 7f156955eb80 10 client.25640 fetch_fsmap learned FSMap version 30 2023-09-20T21:49:31.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.518+0000 7f156955eb80 1 -- 172.21.15.22:0/3800024387 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e194cea2b0 con 0x55e194f6f6f0 2023-09-20T21:49:31.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.518+0000 7f152effd700 1 -- 172.21.15.22:0/3800024387 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15440a66e0 con 0x55e194f6f6f0 2023-09-20T21:49:31.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.518+0000 7f152effd700 1 -- 172.21.15.22:0/3800024387 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1540032420 con 0x55e194f6f6f0 2023-09-20T21:49:31.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.518+0000 7f156955eb80 10 client.25640 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:49:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.518+0000 7f156955eb80 10 client.25640 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-20T21:49:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.518+0000 7f156955eb80 20 client.25640 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.518+0000 7f156955eb80 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e194fea080 0x55e194fec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.519+0000 7f156955eb80 4 client.25640 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-09-20T21:49:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.519+0000 7f156955eb80 1 -- 172.21.15.22:0/3800024387 --> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e194eb9e60 con 0x55e194fea080 2023-09-20T21:49:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.519+0000 7f154effd700 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e194fea080 0x55e194fec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.519+0000 7f152effd700 10 client.25640 ms_handle_connect on v2:172.21.15.89:6836/3887702690 2023-09-20T21:49:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.520+0000 7f154effd700 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e194fea080 0x55e194fec4a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d6e0a1350f84a92 server_cookie=28b40319e19fe091 in_seq=0 out_seq=0 2023-09-20T21:49:31.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.520+0000 7f152effd700 1 -- 172.21.15.22:0/3800024387 <== mds.2 v2:172.21.15.89:6836/3887702690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e194eb9e60 con 0x55e194fea080 2023-09-20T21:49:31.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.520+0000 7f152effd700 10 client.25640 handle_command_reply: tid=0 2023-09-20T21:49:31.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.520+0000 7f156955eb80 1 client.25640 shutdown 2023-09-20T21:49:31.526 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2023-09-20T21:49:31.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 1 -- 172.21.15.22:0/3800024387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 msgr2=0x55e194f7bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f7bd80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55e194f7a020 tx=0x7f1540005300 comp rx=0 tx=0).stop 2023-09-20T21:49:31.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 1 -- 172.21.15.22:0/3800024387 shutdown_connections 2023-09-20T21:49:31.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e194fea080 0x55e194fec4a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e194f70be0 0x55e194f7f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e194f6f6f0 0x55e194f7bd80 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55e194f7a020 tx=0x7f1540005300 comp rx=0 tx=0).stop 2023-09-20T21:49:31.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 1 --2- 172.21.15.22:0/3800024387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e194f70050 0x55e194f7c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 1 -- 172.21.15.22:0/3800024387 >> 172.21.15.22:0/3800024387 conn(0x55e194e9c000 msgr2=0x55e194f76fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f152effd700 0 client.25640 ms_handle_reset on v2:172.21.15.89:6836/3887702690 2023-09-20T21:49:31.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 1 -- 172.21.15.22:0/3800024387 shutdown_connections 2023-09-20T21:49:31.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 1 -- 172.21.15.22:0/3800024387 wait complete. 2023-09-20T21:49:31.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.522+0000 7f156955eb80 20 client.25640 trim_cache size 0 max 16384 2023-09-20T21:49:31.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.529+0000 7f1555bff700 1 -- 172.21.15.22:0/2451067656 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f15380a1300 msgr2=0x7f15380a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.529+0000 7f1555bff700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f15380a1300 0x7f15380a37c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1548009260 tx=0x7f154800b040 comp rx=0 tx=0).stop 2023-09-20T21:49:31.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.529+0000 7f1555bff700 1 -- 172.21.15.22:0/2451067656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15501041f0 msgr2=0x7f1550199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.529+0000 7f1555bff700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15501041f0 0x7f1550199280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1544009f90 tx=0x7f154400ee90 comp rx=0 tx=0).stop 2023-09-20T21:49:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.531+0000 7f1555bff700 1 -- 172.21.15.22:0/2451067656 shutdown_connections 2023-09-20T21:49:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.531+0000 7f1555bff700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f15380a1300 0x7f15380a37c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.531+0000 7f1555bff700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1550104b90 0x7f15501a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.531+0000 7f1555bff700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1550103000 0x7f1550198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.531+0000 7f1555bff700 1 --2- 172.21.15.22:0/2451067656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15501041f0 0x7f1550199280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.531+0000 7f1555bff700 1 -- 172.21.15.22:0/2451067656 >> 172.21.15.22:0/2451067656 conn(0x7f15500fe590 msgr2=0x7f1550100880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.535+0000 7f1555bff700 1 -- 172.21.15.22:0/2451067656 shutdown_connections 2023-09-20T21:49:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.535+0000 7f1555bff700 1 -- 172.21.15.22:0/2451067656 wait complete. 2023-09-20T21:49:31.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-20T21:49:31.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.892+0000 7f16bcdff700 1 -- 172.21.15.22:0/3209795886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b80ff640 msgr2=0x7f16b8101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.892+0000 7f16bcdff700 1 --2- 172.21.15.22:0/3209795886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b80ff640 0x7f16b8101a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009990 tx=0x7f16ac009ca0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.892+0000 7f16bcdff700 1 -- 172.21.15.22:0/3209795886 shutdown_connections 2023-09-20T21:49:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.892+0000 7f16bcdff700 1 --2- 172.21.15.22:0/3209795886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b8104980 0x7f16b8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.892+0000 7f16bcdff700 1 --2- 172.21.15.22:0/3209795886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b80ff640 0x7f16b8101a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.892+0000 7f16bcdff700 1 --2- 172.21.15.22:0/3209795886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b8101fb0 0x7f16b8104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.892+0000 7f16bcdff700 1 -- 172.21.15.22:0/3209795886 >> 172.21.15.22:0/3209795886 conn(0x7f16b80fb240 msgr2=0x7f16b80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.893+0000 7f16bcdff700 1 -- 172.21.15.22:0/3209795886 shutdown_connections 2023-09-20T21:49:31.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.893+0000 7f16bcdff700 1 -- 172.21.15.22:0/3209795886 wait complete. 2023-09-20T21:49:31.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.894+0000 7f16bcdff700 1 Processor -- start 2023-09-20T21:49:31.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.898+0000 7f16bcdff700 1 -- start start 2023-09-20T21:49:31.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.898+0000 7f16bcdff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b80ff640 0x7f16b806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.898+0000 7f16bcdff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b8101fb0 0x7f16b806fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.898+0000 7f16bcdff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b8104980 0x7f16b81ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.898+0000 7f16bcdff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8072c10 con 0x7f16b8104980 2023-09-20T21:49:31.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.898+0000 7f16bcdff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8072d50 con 0x7f16b8101fb0 2023-09-20T21:49:31.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.898+0000 7f16bcdff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8105c60 con 0x7f16b80ff640 2023-09-20T21:49:31.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.899+0000 7f16b6d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b8104980 0x7f16b81ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.899+0000 7f16b5d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b8101fb0 0x7f16b806fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.899+0000 7f16b6d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b8104980 0x7f16b81ad320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45342/0 (socket says 172.21.15.22:45342) 2023-09-20T21:49:31.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.899+0000 7f16b6d9d700 1 -- 172.21.15.22:0/1189831097 learned_addr learned my addr 172.21.15.22:0/1189831097 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:31.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.899+0000 7f16b659c700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b80ff640 0x7f16b806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.899+0000 7f16b5d9b700 1 -- 172.21.15.22:0/1189831097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b80ff640 msgr2=0x7f16b806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.899+0000 7f16b5d9b700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b80ff640 0x7f16b806f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.899+0000 7f16b5d9b700 1 -- 172.21.15.22:0/1189831097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b8104980 msgr2=0x7f16b81ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.899+0000 7f16b5d9b700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b8104980 0x7f16b81ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.899+0000 7f16b5d9b700 1 -- 172.21.15.22:0/1189831097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac0095f0 con 0x7f16b8101fb0 2023-09-20T21:49:31.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.900+0000 7f16b5d9b700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b8101fb0 0x7f16b806fb30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d950 tx=0x7f16a800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.900+0000 7f16a77fe700 1 -- 172.21.15.22:0/1189831097 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a800f890 con 0x7f16b8101fb0 2023-09-20T21:49:31.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.900+0000 7f16a77fe700 1 -- 172.21.15.22:0/1189831097 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a8010460 con 0x7f16b8101fb0 2023-09-20T21:49:31.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.900+0000 7f16a77fe700 1 -- 172.21.15.22:0/1189831097 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a800e610 con 0x7f16b8101fb0 2023-09-20T21:49:31.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.901+0000 7f16bcdff700 1 -- 172.21.15.22:0/1189831097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b8105f40 con 0x7f16b8101fb0 2023-09-20T21:49:31.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.901+0000 7f16bcdff700 1 -- 172.21.15.22:0/1189831097 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b8106400 con 0x7f16b8101fb0 2023-09-20T21:49:31.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.906+0000 7f16d0649b80 1 Processor -- start 2023-09-20T21:49:31.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.907+0000 7f16d0649b80 1 -- start start 2023-09-20T21:49:31.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.907+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a556f0 0x5638b5a55b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.907+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a56050 0x5638b5a565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.907+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a56be0 0x5638b5a57060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.907+0000 7f16d0649b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b59c9ca0 con 0x5638b5a56be0 2023-09-20T21:49:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.907+0000 7f16d0649b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b59adc70 con 0x5638b5a56050 2023-09-20T21:49:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.907+0000 7f16d0649b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b59ab590 con 0x5638b5a556f0 2023-09-20T21:49:31.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a556f0 0x5638b5a55b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a56050 0x5638b5a565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a556f0 0x5638b5a55b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43302/0 (socket says 172.21.15.22:43302) 2023-09-20T21:49:31.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a56050 0x5638b5a565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38206/0 (socket says 172.21.15.22:38206) 2023-09-20T21:49:31.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b6d9d700 1 -- 172.21.15.22:0/1767580016 learned_addr learned my addr 172.21.15.22:0/1767580016 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:31.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b6d9d700 1 -- 172.21.15.22:0/1767580016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a56050 msgr2=0x5638b5a565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b6d9d700 1 --2- 172.21.15.22:0/1767580016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a56050 0x5638b5a565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b6d9d700 1 -- 172.21.15.22:0/1767580016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a56be0 msgr2=0x5638b5a57060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:49:31.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b6d9d700 1 --2- 172.21.15.22:0/1767580016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a56be0 0x5638b5a57060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b6d9d700 1 -- 172.21.15.22:0/1767580016 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638b57d1de0 con 0x5638b5a556f0 2023-09-20T21:49:31.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.908+0000 7f16b6d9d700 1 --2- 172.21.15.22:0/1767580016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a556f0 0x5638b5a55b10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f16b000a730 tx=0x7f16b0007990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f82b217fcec3219 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.911+0000 7f16a77fe700 1 -- 172.21.15.22:0/1189831097 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f16a8010890 con 0x7f16b8101fb0 2023-09-20T21:49:31.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.915+0000 7f16a77fe700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f16a00a1400 0x7f16a00a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.916+0000 7f16a4ff9700 1 -- 172.21.15.22:0/1767580016 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0018070 con 0x5638b5a556f0 2023-09-20T21:49:31.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.916+0000 7f16a77fe700 1 -- 172.21.15.22:0/1189831097 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f16a80ccc20 con 0x7f16b8101fb0 2023-09-20T21:49:31.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.916+0000 7f16a4ff9700 1 -- 172.21.15.22:0/1767580016 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b0010040 con 0x5638b5a556f0 2023-09-20T21:49:31.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.916+0000 7f16a4ff9700 1 -- 172.21.15.22:0/1767580016 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b000f050 con 0x5638b5a556f0 2023-09-20T21:49:31.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.917+0000 7f16d0649b80 1 -- 172.21.15.22:0/1767580016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a556f0 msgr2=0x5638b5a55b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.917+0000 7f16d0649b80 1 --2- 172.21.15.22:0/1767580016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a556f0 0x5638b5a55b10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f16b000a730 tx=0x7f16b0007990 comp rx=0 tx=0).stop 2023-09-20T21:49:31.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.921+0000 7f16b659c700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f16a00a1400 0x7f16a00a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.923+0000 7f16d0649b80 1 -- 172.21.15.22:0/1767580016 shutdown_connections 2023-09-20T21:49:31.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.923+0000 7f16d0649b80 1 --2- 172.21.15.22:0/1767580016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a56be0 0x5638b5a57060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.923+0000 7f16d0649b80 1 --2- 172.21.15.22:0/1767580016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a556f0 0x5638b5a55b10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.923+0000 7f16d0649b80 1 --2- 172.21.15.22:0/1767580016 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a56050 0x5638b5a565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.923+0000 7f16d0649b80 1 -- 172.21.15.22:0/1767580016 >> 172.21.15.22:0/1767580016 conn(0x5638b5982000 msgr2=0x5638b5a501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.924+0000 7f16b659c700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f16a00a1400 0x7f16a00a38c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16ac003790 tx=0x7f16ac003680 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.929+0000 7f16d0649b80 1 -- 172.21.15.22:0/1767580016 shutdown_connections 2023-09-20T21:49:31.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.929+0000 7f16d0649b80 1 -- 172.21.15.22:0/1767580016 wait complete. 2023-09-20T21:49:31.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.930+0000 7f16d0649b80 1 -- start start 2023-09-20T21:49:31.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.930+0000 7f16d0649b80 1 Processor -- start 2023-09-20T21:49:31.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.931+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a5f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.931+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56050 0x5638b5a5f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.931+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a56be0 0x5638b5a667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.931+0000 7f16d0649b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b59a6e80 con 0x5638b5a56be0 2023-09-20T21:49:31.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.931+0000 7f16d0649b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5943740 con 0x5638b5a556f0 2023-09-20T21:49:31.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.931+0000 7f16d0649b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b59433e0 con 0x5638b5a56050 2023-09-20T21:49:31.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.931+0000 7f16d0649b80 10 client.0 _ll_register_callbacks cb 0x5638b588da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:31.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.934+0000 7f16b5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56050 0x5638b5a5f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.934+0000 7f16b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a5f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.934+0000 7f16b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a5f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38214/0 (socket says 172.21.15.22:38214) 2023-09-20T21:49:31.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.934+0000 7f16b659c700 1 -- 172.21.15.22:0/3654169982 learned_addr learned my addr 172.21.15.22:0/3654169982 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:31.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.935+0000 7f16b5d9b700 1 -- 172.21.15.22:0/3654169982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 msgr2=0x5638b5a5f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.935+0000 7f16b5d9b700 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a5f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.935+0000 7f16b5d9b700 1 -- 172.21.15.22:0/3654169982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a56be0 msgr2=0x5638b5a667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.935+0000 7f16b5d9b700 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a56be0 0x5638b5a667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.935+0000 7f16b5d9b700 1 -- 172.21.15.22:0/3654169982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b000a410 con 0x5638b5a56050 2023-09-20T21:49:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.935+0000 7f16b5d9b700 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56050 0x5638b5a5f7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f16a800dfb0 tx=0x7f16a8099520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48c32eea696ee085 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.935+0000 7f169dffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:49:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.936+0000 7f169dffb700 1 -- 172.21.15.22:0/3654169982 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a809d9f0 con 0x5638b5a56050 2023-09-20T21:49:31.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.936+0000 7f16d0649b80 1 -- 172.21.15.22:0/3654169982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638b5917bb0 con 0x5638b5a56050 2023-09-20T21:49:31.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.936+0000 7f169dffb700 1 -- 172.21.15.22:0/3654169982 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f16a80a1c30 con 0x5638b5a56050 2023-09-20T21:49:31.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.936+0000 7f169dffb700 1 -- 172.21.15.22:0/3654169982 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a80b9070 con 0x5638b5a56050 2023-09-20T21:49:31.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.938+0000 7f16d0649b80 10 client.25633 fetch_fsmap learned FSMap version 30 2023-09-20T21:49:31.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.938+0000 7f16d0649b80 1 -- 172.21.15.22:0/3654169982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5638b57bac60 con 0x5638b5a56050 2023-09-20T21:49:31.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.939+0000 7f169dffb700 1 -- 172.21.15.22:0/3654169982 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638b5917bb0 con 0x5638b5a56050 2023-09-20T21:49:31.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.939+0000 7f169dffb700 1 -- 172.21.15.22:0/3654169982 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f16a809abd0 con 0x5638b5a56050 2023-09-20T21:49:31.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.939+0000 7f16d0649b80 10 client.25633 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:49:31.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.939+0000 7f16d0649b80 10 client.25633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-20T21:49:31.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.939+0000 7f16d0649b80 20 client.25633 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:31.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.939+0000 7f16d0649b80 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5638b5a6afc0 0x5638b5a6d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.939+0000 7f16d0649b80 4 client.25633 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:49:31.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.939+0000 7f16d0649b80 1 -- 172.21.15.22:0/3654169982 --> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638b5a24020 con 0x5638b5a6afc0 2023-09-20T21:49:31.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.940+0000 7f16b659c700 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5638b5a6afc0 0x5638b5a6d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.941+0000 7f16b659c700 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5638b5a6afc0 0x5638b5a6d3e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e93638ffa8daf9b5 server_cookie=e6111201bd29b70f in_seq=0 out_seq=0 2023-09-20T21:49:31.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.942+0000 7f169dffb700 10 client.25633 ms_handle_connect on v2:172.21.15.22:6840/1806616417 2023-09-20T21:49:31.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.952+0000 7f169dffb700 1 -- 172.21.15.22:0/3654169982 <== mds.1 v2:172.21.15.22:6840/1806616417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5638b5a24020 con 0x5638b5a6afc0 2023-09-20T21:49:31.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.952+0000 7f169dffb700 10 client.25633 handle_command_reply: tid=0 2023-09-20T21:49:31.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.952+0000 7f16d0649b80 1 client.25633 shutdown 2023-09-20T21:49:31.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.953+0000 7f16d0649b80 1 -- 172.21.15.22:0/3654169982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56050 msgr2=0x5638b5a5f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.953+0000 7f16d0649b80 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56050 0x5638b5a5f7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f16a800dfb0 tx=0x7f16a8099520 comp rx=0 tx=0).stop 2023-09-20T21:49:31.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.954+0000 7f16d0649b80 1 -- 172.21.15.22:0/3654169982 shutdown_connections 2023-09-20T21:49:31.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.954+0000 7f16d0649b80 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5638b5a6afc0 0x5638b5a6d3e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.954+0000 7f16d0649b80 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a56be0 0x5638b5a667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.954+0000 7f16d0649b80 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a5f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.954+0000 7f16d0649b80 1 --2- 172.21.15.22:0/3654169982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56050 0x5638b5a5f7a0 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f16a800dfb0 tx=0x7f16a8099520 comp rx=0 tx=0).stop 2023-09-20T21:49:31.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.954+0000 7f16d0649b80 1 -- 172.21.15.22:0/3654169982 >> 172.21.15.22:0/3654169982 conn(0x5638b5982000 msgr2=0x5638b5a4f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.956+0000 7f169dffb700 0 client.25633 ms_handle_reset on v2:172.21.15.22:6840/1806616417 2023-09-20T21:49:31.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.958+0000 7f16d0649b80 1 -- 172.21.15.22:0/3654169982 shutdown_connections 2023-09-20T21:49:31.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.958+0000 7f16d0649b80 1 -- 172.21.15.22:0/3654169982 wait complete. 2023-09-20T21:49:31.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.958+0000 7f16d0649b80 20 client.25633 trim_cache size 0 max 16384 2023-09-20T21:49:31.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.990+0000 7f16d0649b80 1 Processor -- start 2023-09-20T21:49:31.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.990+0000 7f16d0649b80 1 -- start start 2023-09-20T21:49:31.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.990+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a6cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.990+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56be0 0x5638b5a667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.990+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a6d510 0x5638b5a59bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.990+0000 7f16d0649b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5a5ee30 con 0x5638b5a6d510 2023-09-20T21:49:31.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.990+0000 7f16d0649b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5827920 con 0x5638b5a556f0 2023-09-20T21:49:31.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.990+0000 7f16d0649b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5a50ec0 con 0x5638b5a56be0 2023-09-20T21:49:31.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.991+0000 7f16b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a6cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:31.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.991+0000 7f16b659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a6cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38216/0 (socket says 172.21.15.22:38216) 2023-09-20T21:49:31.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.991+0000 7f16b659c700 1 -- 172.21.15.22:0/2824567022 learned_addr learned my addr 172.21.15.22:0/2824567022 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.991+0000 7f16b659c700 1 -- 172.21.15.22:0/2824567022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56be0 msgr2=0x5638b5a667a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:49:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.991+0000 7f16b659c700 1 --2- 172.21.15.22:0/2824567022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56be0 0x5638b5a667a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.992+0000 7f16b659c700 1 -- 172.21.15.22:0/2824567022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a6d510 msgr2=0x5638b5a59bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:31.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.992+0000 7f16b659c700 1 --2- 172.21.15.22:0/2824567022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a6d510 0x5638b5a59bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.992+0000 7f16b659c700 1 -- 172.21.15.22:0/2824567022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638b57f8250 con 0x5638b5a556f0 2023-09-20T21:49:31.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.992+0000 7f16b659c700 1 --2- 172.21.15.22:0/2824567022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a6cfd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009990 tx=0x7f16ac022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f41927816e807c74 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:31.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.992+0000 7f16a57fa700 1 -- 172.21.15.22:0/2824567022 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac01d040 con 0x5638b5a556f0 2023-09-20T21:49:31.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.992+0000 7f16a57fa700 1 -- 172.21.15.22:0/2824567022 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16ac01e040 con 0x5638b5a556f0 2023-09-20T21:49:31.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.992+0000 7f16a57fa700 1 -- 172.21.15.22:0/2824567022 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0336a0 con 0x5638b5a556f0 2023-09-20T21:49:31.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.993+0000 7f16d0649b80 1 -- 172.21.15.22:0/2824567022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 msgr2=0x5638b5a6cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:31.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.993+0000 7f16d0649b80 1 --2- 172.21.15.22:0/2824567022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a6cfd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009990 tx=0x7f16ac022000 comp rx=0 tx=0).stop 2023-09-20T21:49:31.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.993+0000 7f16d0649b80 1 -- 172.21.15.22:0/2824567022 shutdown_connections 2023-09-20T21:49:31.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.993+0000 7f16d0649b80 1 --2- 172.21.15.22:0/2824567022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a6d510 0x5638b5a59bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.993+0000 7f16d0649b80 1 --2- 172.21.15.22:0/2824567022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a556f0 0x5638b5a6cfd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.993+0000 7f16d0649b80 1 --2- 172.21.15.22:0/2824567022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56be0 0x5638b5a667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:31.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.993+0000 7f16d0649b80 1 -- 172.21.15.22:0/2824567022 >> 172.21.15.22:0/2824567022 conn(0x5638b5982000 msgr2=0x5638b5a67010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:31.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.994+0000 7f16d0649b80 1 -- 172.21.15.22:0/2824567022 shutdown_connections 2023-09-20T21:49:31.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.994+0000 7f16d0649b80 1 -- 172.21.15.22:0/2824567022 wait complete. 2023-09-20T21:49:31.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.994+0000 7f16d0649b80 1 -- start start 2023-09-20T21:49:31.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.994+0000 7f16d0649b80 1 Processor -- start 2023-09-20T21:49:31.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a556f0 0x5638b5a64680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56be0 0x5638b5a64bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16d0649b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a6d510 0x5638b5a65bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16d0649b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5a66110 con 0x5638b5a556f0 2023-09-20T21:49:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16d0649b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5a66280 con 0x5638b5a6d510 2023-09-20T21:49:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16d0649b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b5a66440 con 0x5638b5a56be0 2023-09-20T21:49:32.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16d0649b80 10 client.0 _ll_register_callbacks cb 0x5638b588da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:32.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16b6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a6d510 0x5638b5a65bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16b5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56be0 0x5638b5a64bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16b6d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a6d510 0x5638b5a65bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38226/0 (socket says 172.21.15.22:38226) 2023-09-20T21:49:32.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.995+0000 7f16b6d9d700 1 -- 172.21.15.22:0/924573857 learned_addr learned my addr 172.21.15.22:0/924573857 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:32.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.996+0000 7f16b5d9b700 1 -- 172.21.15.22:0/924573857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a6d510 msgr2=0x5638b5a65bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:32.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.996+0000 7f16b5d9b700 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a6d510 0x5638b5a65bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.996+0000 7f16b5d9b700 1 -- 172.21.15.22:0/924573857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a556f0 msgr2=0x5638b5a64680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:32.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.996+0000 7f16b5d9b700 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a556f0 0x5638b5a64680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.996+0000 7f16b5d9b700 1 -- 172.21.15.22:0/924573857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b8105930 con 0x5638b5a56be0 2023-09-20T21:49:32.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.999+0000 7f16b5d9b700 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56be0 0x5638b5a64bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16a8096630 tx=0x7f16a8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5eba5856bd49502 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:32.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.999+0000 7f169dffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:49:32.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.999+0000 7f169dffb700 1 -- 172.21.15.22:0/924573857 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a80b92b0 con 0x5638b5a56be0 2023-09-20T21:49:32.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.999+0000 7f169dffb700 1 -- 172.21.15.22:0/924573857 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f16a80a1c30 con 0x5638b5a56be0 2023-09-20T21:49:32.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:31.999+0000 7f16d0649b80 1 -- 172.21.15.22:0/924573857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16b0010040 con 0x5638b5a56be0 2023-09-20T21:49:32.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.001+0000 7f169dffb700 1 -- 172.21.15.22:0/924573857 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a80a1a30 con 0x5638b5a56be0 2023-09-20T21:49:32.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.001+0000 7f16d0649b80 10 client.25636 fetch_fsmap learned FSMap version 30 2023-09-20T21:49:32.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.001+0000 7f16d0649b80 1 -- 172.21.15.22:0/924573857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5638b57d02b0 con 0x5638b5a56be0 2023-09-20T21:49:32.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.003+0000 7f169dffb700 1 -- 172.21.15.22:0/924573857 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b0010040 con 0x5638b5a56be0 2023-09-20T21:49:32.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.003+0000 7f169dffb700 1 -- 172.21.15.22:0/924573857 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f16a8098b20 con 0x5638b5a56be0 2023-09-20T21:49:32.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.003+0000 7f16d0649b80 10 client.25636 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:49:32.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.003+0000 7f16d0649b80 10 client.25636 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-20T21:49:32.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.003+0000 7f16d0649b80 20 client.25636 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:32.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.004+0000 7f16d0649b80 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5638b5abfaf0 0x5638b5ac1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.004+0000 7f16d0649b80 4 client.25636 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-09-20T21:49:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.004+0000 7f16d0649b80 1 -- 172.21.15.22:0/924573857 --> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5638b599fe60 con 0x5638b5abfaf0 2023-09-20T21:49:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.004+0000 7f16b659c700 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5638b5abfaf0 0x5638b5ac1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.005+0000 7f16b659c700 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5638b5abfaf0 0x5638b5ac1f10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69d358a8bd778f00 server_cookie=6d6cec86de872c83 in_seq=0 out_seq=0 2023-09-20T21:49:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.005+0000 7f169dffb700 10 client.25636 ms_handle_connect on v2:172.21.15.22:6840/1806616417 2023-09-20T21:49:32.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f169dffb700 1 -- 172.21.15.22:0/924573857 <== mds.1 v2:172.21.15.22:6840/1806616417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5638b599fe60 con 0x5638b5abfaf0 2023-09-20T21:49:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f169dffb700 10 client.25636 handle_command_reply: tid=0 2023-09-20T21:49:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f16d0649b80 1 client.25636 shutdown 2023-09-20T21:49:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f16d0649b80 1 -- 172.21.15.22:0/924573857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56be0 msgr2=0x5638b5a64bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f16d0649b80 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56be0 0x5638b5a64bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16a8096630 tx=0x7f16a8010800 comp rx=0 tx=0).stop 2023-09-20T21:49:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f16d0649b80 1 -- 172.21.15.22:0/924573857 shutdown_connections 2023-09-20T21:49:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f16d0649b80 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5638b5a6d510 0x5638b5a65bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f16d0649b80 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x5638b5abfaf0 0x5638b5ac1f10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f16d0649b80 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638b5a556f0 0x5638b5a64680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f16d0649b80 1 --2- 172.21.15.22:0/924573857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638b5a56be0 0x5638b5a64bc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f16d0649b80 1 -- 172.21.15.22:0/924573857 >> 172.21.15.22:0/924573857 conn(0x5638b5982000 msgr2=0x5638b5a59bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:32.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.008+0000 7f169dffb700 0 client.25636 ms_handle_reset on v2:172.21.15.22:6840/1806616417 2023-09-20T21:49:32.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.009+0000 7f16d0649b80 1 -- 172.21.15.22:0/924573857 shutdown_connections 2023-09-20T21:49:32.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.009+0000 7f16d0649b80 1 -- 172.21.15.22:0/924573857 wait complete. 2023-09-20T21:49:32.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.009+0000 7f16d0649b80 20 client.25636 trim_cache size 0 max 16384 2023-09-20T21:49:32.012 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2023-09-20T21:49:32.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.010+0000 7f16bcdff700 1 -- 172.21.15.22:0/1189831097 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f16a00a1400 msgr2=0x7f16a00a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:32.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.010+0000 7f16bcdff700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f16a00a1400 0x7f16a00a38c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16ac003790 tx=0x7f16ac003680 comp rx=0 tx=0).stop 2023-09-20T21:49:32.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.010+0000 7f16bcdff700 1 -- 172.21.15.22:0/1189831097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b8101fb0 msgr2=0x7f16b806fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.010+0000 7f16bcdff700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b8101fb0 0x7f16b806fb30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d950 tx=0x7f16a800dc60 comp rx=0 tx=0).stop 2023-09-20T21:49:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.011+0000 7f16bcdff700 1 -- 172.21.15.22:0/1189831097 shutdown_connections 2023-09-20T21:49:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.011+0000 7f16bcdff700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16b8104980 0x7f16b81ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.011+0000 7f16bcdff700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b80ff640 0x7f16b806f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.011+0000 7f16bcdff700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f16a00a1400 0x7f16a00a38c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.011+0000 7f16bcdff700 1 --2- 172.21.15.22:0/1189831097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16b8101fb0 0x7f16b806fb30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.011+0000 7f16bcdff700 1 -- 172.21.15.22:0/1189831097 >> 172.21.15.22:0/1189831097 conn(0x7f16b80fb240 msgr2=0x7f16b8102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.011+0000 7f16bcdff700 1 -- 172.21.15.22:0/1189831097 shutdown_connections 2023-09-20T21:49:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.011+0000 7f16bcdff700 1 -- 172.21.15.22:0/1189831097 wait complete. 2023-09-20T21:49:32.042 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-20T21:49:32.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.404+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/3608965010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0a4104180 msgr2=0x7fc0a4106570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:32.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.404+0000 7fc0aa3ff700 1 --2- 172.21.15.22:0/3608965010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0a4104180 0x7fc0a4106570 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009990 tx=0x7fc08c009ca0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.404+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/3608965010 shutdown_connections 2023-09-20T21:49:32.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.404+0000 7fc0aa3ff700 1 --2- 172.21.15.22:0/3608965010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a4106ab0 0x7fc0a4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.404+0000 7fc0aa3ff700 1 --2- 172.21.15.22:0/3608965010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a4101850 0x7fc0a4103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.404+0000 7fc0aa3ff700 1 --2- 172.21.15.22:0/3608965010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0a4104180 0x7fc0a4106570 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.404+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/3608965010 >> 172.21.15.22:0/3608965010 conn(0x7fc0a40fb240 msgr2=0x7fc0a40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:32.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.408+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/3608965010 shutdown_connections 2023-09-20T21:49:32.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.408+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/3608965010 wait complete. 2023-09-20T21:49:32.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.409+0000 7fc0aa3ff700 1 Processor -- start 2023-09-20T21:49:32.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.412+0000 7fc0aa3ff700 1 -- start start 2023-09-20T21:49:32.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.412+0000 7fc0aa3ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a4101850 0x7fc0a41011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.412+0000 7fc0aa3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0a4104180 0x7fc0a40ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.412+0000 7fc0aa3ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a4106ab0 0x7fc0a40ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.412+0000 7fc0aa3ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a419a9a0 con 0x7fc0a4106ab0 2023-09-20T21:49:32.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.412+0000 7fc0aa3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a419ab10 con 0x7fc0a4104180 2023-09-20T21:49:32.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.412+0000 7fc0aa3ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a419acd0 con 0x7fc0a4101850 2023-09-20T21:49:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.413+0000 7fc0a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a4101850 0x7fc0a41011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.413+0000 7fc0a3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a4101850 0x7fc0a41011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43362/0 (socket says 172.21.15.22:43362) 2023-09-20T21:49:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.413+0000 7fc0a3fff700 1 -- 172.21.15.22:0/4042453408 learned_addr learned my addr 172.21.15.22:0/4042453408 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.413+0000 7fc0a3fff700 1 -- 172.21.15.22:0/4042453408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0a4104180 msgr2=0x7fc0a40ff800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:49:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.413+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0a4104180 0x7fc0a40ff800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.413+0000 7fc0a3fff700 1 -- 172.21.15.22:0/4042453408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a4106ab0 msgr2=0x7fc0a40ffd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:49:32.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.413+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a4106ab0 0x7fc0a40ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.413+0000 7fc0a3fff700 1 -- 172.21.15.22:0/4042453408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c0095f0 con 0x7fc0a4101850 2023-09-20T21:49:32.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.414+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a4101850 0x7fc0a41011c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc0940060b0 tx=0x7fc094007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:32.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.414+0000 7fc0a1ffb700 1 -- 172.21.15.22:0/4042453408 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09400de20 con 0x7fc0a4101850 2023-09-20T21:49:32.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.414+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/4042453408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a419afb0 con 0x7fc0a4101850 2023-09-20T21:49:32.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.414+0000 7fc0a1ffb700 1 -- 172.21.15.22:0/4042453408 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc09400f040 con 0x7fc0a4101850 2023-09-20T21:49:32.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.414+0000 7fc0a1ffb700 1 -- 172.21.15.22:0/4042453408 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0940146e0 con 0x7fc0a4101850 2023-09-20T21:49:32.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.414+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/4042453408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a419b510 con 0x7fc0a4101850 2023-09-20T21:49:32.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.420+0000 7fc0a1ffb700 1 -- 172.21.15.22:0/4042453408 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fc094015a30 con 0x7fc0a4101850 2023-09-20T21:49:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.428+0000 7fc0bddb3b80 1 Processor -- start 2023-09-20T21:49:32.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.428+0000 7fc0bddb3b80 1 -- start start 2023-09-20T21:49:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.429+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.429+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.429+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.429+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea526ca0 con 0x5555ea5b3050 2023-09-20T21:49:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.429+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea50ac70 con 0x5555ea5b26f0 2023-09-20T21:49:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.429+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea508590 con 0x5555ea5b3be0 2023-09-20T21:49:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.429+0000 7fc0a1ffb700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0900a1400 0x7fc0900a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.430+0000 7fc0a899c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.430+0000 7fc0a1ffb700 1 -- 172.21.15.22:0/4042453408 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc0940cc9c0 con 0x7fc0a4101850 2023-09-20T21:49:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.430+0000 7fc0a899c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38248/0 (socket says 172.21.15.22:38248) 2023-09-20T21:49:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.430+0000 7fc0a899c700 1 -- 172.21.15.22:0/3786396853 learned_addr learned my addr 172.21.15.22:0/3786396853 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.431+0000 7fc09bfff700 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.431+0000 7fc0a899c700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0900a1400 0x7fc0900a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.431+0000 7fc0a899c700 1 -- 172.21.15.22:0/3786396853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 msgr2=0x5555ea5b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.431+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.431+0000 7fc0a899c700 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.431+0000 7fc0a899c700 1 -- 172.21.15.22:0/3786396853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 msgr2=0x5555ea5b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.431+0000 7fc0a899c700 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.431+0000 7fc0a899c700 1 -- 172.21.15.22:0/3786396853 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555ea32ede0 con 0x5555ea5b26f0 2023-09-20T21:49:32.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.431+0000 7fc09bfff700 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:49:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.431+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:49:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.432+0000 7fc0a899c700 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5b2b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005fd0 tx=0x7fc09c0149e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2832828179ba7b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.432+0000 7fc09a7fc700 1 -- 172.21.15.22:0/3786396853 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c011e90 con 0x5555ea5b26f0 2023-09-20T21:49:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.432+0000 7fc09a7fc700 1 -- 172.21.15.22:0/3786396853 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc09c004cf0 con 0x5555ea5b26f0 2023-09-20T21:49:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.432+0000 7fc09a7fc700 1 -- 172.21.15.22:0/3786396853 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c01e460 con 0x5555ea5b26f0 2023-09-20T21:49:32.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.433+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/3786396853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 msgr2=0x5555ea5b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:32.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.433+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5b2b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005fd0 tx=0x7fc09c0149e0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.433+0000 7fc0a899c700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0900a1400 0x7fc0900a38c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00d000 tx=0x7fc09c019000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:32.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.433+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/3786396853 shutdown_connections 2023-09-20T21:49:32.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.433+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.433+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5b2b10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.433+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/3786396853 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.433+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/3786396853 >> 172.21.15.22:0/3786396853 conn(0x5555ea4df000 msgr2=0x5555ea5ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:32.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.435+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/3786396853 shutdown_connections 2023-09-20T21:49:32.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.436+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/3786396853 wait complete. 2023-09-20T21:49:32.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.437+0000 7fc0bddb3b80 1 -- start start 2023-09-20T21:49:32.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.437+0000 7fc0bddb3b80 1 Processor -- start 2023-09-20T21:49:32.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.440+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5bc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.440+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.440+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.440+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea503e80 con 0x5555ea5b3050 2023-09-20T21:49:32.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.440+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea4a0740 con 0x5555ea5b26f0 2023-09-20T21:49:32.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.440+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea4a03e0 con 0x5555ea5b3be0 2023-09-20T21:49:32.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.440+0000 7fc0bddb3b80 10 client.0 _ll_register_callbacks cb 0x5555ea3eaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:32.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.441+0000 7fc0a3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5bc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.441+0000 7fc0a3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5bc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38264/0 (socket says 172.21.15.22:38264) 2023-09-20T21:49:32.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.441+0000 7fc0a3fff700 1 -- 172.21.15.22:0/1578926690 learned_addr learned my addr 172.21.15.22:0/1578926690 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.441+0000 7fc0a3fff700 1 -- 172.21.15.22:0/1578926690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 msgr2=0x5555ea5c36e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:49:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.441+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5c36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.441+0000 7fc0a3fff700 1 -- 172.21.15.22:0/1578926690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 msgr2=0x5555ea5bc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.441+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.441+0000 7fc0a3fff700 1 -- 172.21.15.22:0/1578926690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a4109b10 con 0x5555ea5b26f0 2023-09-20T21:49:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.442+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5bc160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc094095e60 tx=0x7fc0940995d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f0ea9061abca7e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.442+0000 7fc0837fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:49:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.442+0000 7fc0837fe700 1 -- 172.21.15.22:0/1578926690 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094099960 con 0x5555ea5b26f0 2023-09-20T21:49:32.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.442+0000 7fc0837fe700 1 -- 172.21.15.22:0/1578926690 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc09409d790 con 0x5555ea5b26f0 2023-09-20T21:49:32.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.443+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1578926690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555ea474bb0 con 0x5555ea5b26f0 2023-09-20T21:49:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.446+0000 7fc0837fe700 1 -- 172.21.15.22:0/1578926690 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0940a19d0 con 0x5555ea5b26f0 2023-09-20T21:49:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.446+0000 7fc0bddb3b80 10 client.25664 fetch_fsmap learned FSMap version 30 2023-09-20T21:49:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.446+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1578926690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fc09c00ae30 con 0x5555ea5b26f0 2023-09-20T21:49:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.446+0000 7fc0837fe700 1 -- 172.21.15.22:0/1578926690 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555ea474bb0 con 0x5555ea5b26f0 2023-09-20T21:49:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.446+0000 7fc0837fe700 1 -- 172.21.15.22:0/1578926690 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc0940b7bb0 con 0x5555ea5b26f0 2023-09-20T21:49:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.446+0000 7fc0bddb3b80 10 client.25664 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:49:32.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.446+0000 7fc0bddb3b80 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:49:32.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.447+0000 7fc0bddb3b80 20 client.25664 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:32.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.447+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5555ea5c7fd0 0x5555ea5ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.447+0000 7fc0bddb3b80 4 client.25664 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:49:32.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.447+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1578926690 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555ea581020 con 0x5555ea5c7fd0 2023-09-20T21:49:32.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.447+0000 7fc09bfff700 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5555ea5c7fd0 0x5555ea5ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.448+0000 7fc0837fe700 10 client.25664 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:32.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.448+0000 7fc09bfff700 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5555ea5c7fd0 0x5555ea5ca410 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf5a88f6a444cddf server_cookie=1b6aec6523e0a39e in_seq=0 out_seq=0 2023-09-20T21:49:32.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.449+0000 7fc0837fe700 1 -- 172.21.15.22:0/1578926690 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5555ea581020 con 0x5555ea5c7fd0 2023-09-20T21:49:32.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.449+0000 7fc0837fe700 10 client.25664 handle_command_reply: tid=0 2023-09-20T21:49:32.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.449+0000 7fc0bddb3b80 1 client.25664 shutdown 2023-09-20T21:49:32.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.450+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1578926690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 msgr2=0x5555ea5bc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:32.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.450+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5bc160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc094095e60 tx=0x7fc0940995d0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.452+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1578926690 shutdown_connections 2023-09-20T21:49:32.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.452+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5555ea5c7fd0 0x5555ea5ca410 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.452+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.452+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b26f0 0x5555ea5bc160 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.452+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/1578926690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5bc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.452+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1578926690 >> 172.21.15.22:0/1578926690 conn(0x5555ea4df000 msgr2=0x5555ea5ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:32.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.452+0000 7fc0837fe700 0 client.25664 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:32.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.453+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1578926690 shutdown_connections 2023-09-20T21:49:32.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.453+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1578926690 wait complete. 2023-09-20T21:49:32.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.454+0000 7fc0bddb3b80 20 client.25664 trim_cache size 0 max 16384 2023-09-20T21:49:32.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.479+0000 7fc0bddb3b80 1 Processor -- start 2023-09-20T21:49:32.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.480+0000 7fc0bddb3b80 1 -- start start 2023-09-20T21:49:32.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.480+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b26f0 0x5555ea5c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.480+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3050 0x5555ea5c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.480+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.480+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea384920 con 0x5555ea5b26f0 2023-09-20T21:49:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.480+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea5babf0 con 0x5555ea5b3050 2023-09-20T21:49:32.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.480+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea5b4460 con 0x5555ea5b3be0 2023-09-20T21:49:32.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.481+0000 7fc0a3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b26f0 0x5555ea5c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.481+0000 7fc0a3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b26f0 0x5555ea5c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45396/0 (socket says 172.21.15.22:45396) 2023-09-20T21:49:32.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.481+0000 7fc0a3fff700 1 -- 172.21.15.22:0/1640106560 learned_addr learned my addr 172.21.15.22:0/1640106560 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:32.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.481+0000 7fc09bfff700 1 --2- 172.21.15.22:0/1640106560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3050 0x5555ea5c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.483 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.481+0000 7fc0a899c700 1 --2- 172.21.15.22:0/1640106560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.484+0000 7fc09bfff700 1 -- 172.21.15.22:0/1640106560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 msgr2=0x5555ea5b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.484+0000 7fc09bfff700 1 --2- 172.21.15.22:0/1640106560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.484+0000 7fc09bfff700 1 -- 172.21.15.22:0/1640106560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b26f0 msgr2=0x5555ea5c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.484+0000 7fc09bfff700 1 --2- 172.21.15.22:0/1640106560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b26f0 0x5555ea5c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.484+0000 7fc09bfff700 1 -- 172.21.15.22:0/1640106560 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555ea317c60 con 0x5555ea5b3050 2023-09-20T21:49:32.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.484+0000 7fc0a899c700 1 --2- 172.21.15.22:0/1640106560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:49:32.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.484+0000 7fc09bfff700 1 --2- 172.21.15.22:0/1640106560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3050 0x5555ea5c36a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc08c006010 tx=0x7fc08c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb28cf575be97925 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:32.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.484+0000 7fc09affd700 1 -- 172.21.15.22:0/1640106560 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c01d030 con 0x5555ea5b3050 2023-09-20T21:49:32.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.484+0000 7fc09affd700 1 -- 172.21.15.22:0/1640106560 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08c01e040 con 0x5555ea5b3050 2023-09-20T21:49:32.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.484+0000 7fc09affd700 1 -- 172.21.15.22:0/1640106560 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c024e70 con 0x5555ea5b3050 2023-09-20T21:49:32.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.485+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1640106560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3050 msgr2=0x5555ea5c36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:32.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.485+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/1640106560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3050 0x5555ea5c36a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc08c006010 tx=0x7fc08c009ca0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.486+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1640106560 shutdown_connections 2023-09-20T21:49:32.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.486+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/1640106560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b3be0 0x5555ea5b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.486+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/1640106560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b26f0 0x5555ea5c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.486+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/1640106560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3050 0x5555ea5c36a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.486+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1640106560 >> 172.21.15.22:0/1640106560 conn(0x5555ea4df000 msgr2=0x5555ea5ba0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:32.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.486+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1640106560 shutdown_connections 2023-09-20T21:49:32.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.486+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/1640106560 wait complete. 2023-09-20T21:49:32.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.487+0000 7fc0bddb3b80 1 -- start start 2023-09-20T21:49:32.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.487+0000 7fc0bddb3b80 1 Processor -- start 2023-09-20T21:49:32.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.488+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b26f0 0x5555ea5c1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.488+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.488+0000 7fc0bddb3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3be0 0x5555ea5c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.488+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea5c3100 con 0x5555ea5b3050 2023-09-20T21:49:32.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.488+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea5c3270 con 0x5555ea5b3be0 2023-09-20T21:49:32.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.488+0000 7fc0bddb3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555ea5c3430 con 0x5555ea5b26f0 2023-09-20T21:49:32.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.488+0000 7fc0bddb3b80 10 client.0 _ll_register_callbacks cb 0x5555ea3eaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:32.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.488+0000 7fc0a899c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3be0 0x5555ea5c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.488+0000 7fc0a899c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3be0 0x5555ea5c2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38284/0 (socket says 172.21.15.22:38284) 2023-09-20T21:49:32.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.488+0000 7fc0a899c700 1 -- 172.21.15.22:0/267389827 learned_addr learned my addr 172.21.15.22:0/267389827 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:32.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.489+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b26f0 0x5555ea5c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.489+0000 7fc0a899c700 1 -- 172.21.15.22:0/267389827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b26f0 msgr2=0x5555ea5c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.489+0000 7fc0a899c700 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b26f0 0x5555ea5c1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.489+0000 7fc0a899c700 1 -- 172.21.15.22:0/267389827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 msgr2=0x5555ea5c1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.489+0000 7fc0a899c700 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.489+0000 7fc0a899c700 1 -- 172.21.15.22:0/267389827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c00b7d0 con 0x5555ea5b3be0 2023-09-20T21:49:32.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.489+0000 7fc0a899c700 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3be0 0x5555ea5c2bc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc09c011ea0 tx=0x7fc09c011f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd1c5bd0ad6975a server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:32.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.490+0000 7fc0837fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:49:32.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.490+0000 7fc0837fe700 1 -- 172.21.15.22:0/267389827 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c01e460 con 0x5555ea5b3be0 2023-09-20T21:49:32.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.490+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/267389827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc09c004cf0 con 0x5555ea5b3be0 2023-09-20T21:49:32.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.490+0000 7fc0837fe700 1 -- 172.21.15.22:0/267389827 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc09c004a30 con 0x5555ea5b3be0 2023-09-20T21:49:32.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.490+0000 7fc0837fe700 1 -- 172.21.15.22:0/267389827 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc09c005030 con 0x5555ea5b3be0 2023-09-20T21:49:32.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.491+0000 7fc0bddb3b80 10 client.25670 fetch_fsmap learned FSMap version 30 2023-09-20T21:49:32.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.491+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/267389827 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5555ea355250 con 0x5555ea5b3be0 2023-09-20T21:49:32.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.491+0000 7fc0837fe700 1 -- 172.21.15.22:0/267389827 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc09c004cf0 con 0x5555ea5b3be0 2023-09-20T21:49:32.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.491+0000 7fc0837fe700 1 -- 172.21.15.22:0/267389827 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc09c01e750 con 0x5555ea5b3be0 2023-09-20T21:49:32.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.491+0000 7fc0bddb3b80 10 client.25670 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:49:32.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.491+0000 7fc0bddb3b80 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:49:32.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.491+0000 7fc0bddb3b80 20 client.25670 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:32.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.491+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5555ea61d150 0x5555ea61f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:32.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.491+0000 7fc0bddb3b80 4 client.25670 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-09-20T21:49:32.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.491+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/267389827 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5555ea4fce60 con 0x5555ea61d150 2023-09-20T21:49:32.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.492+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5555ea61d150 0x5555ea61f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:32.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.493+0000 7fc0a3fff700 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5555ea61d150 0x5555ea61f570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7c156278d1f4b2 server_cookie=261873a5863fc8b7 in_seq=0 out_seq=0 2023-09-20T21:49:32.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.493+0000 7fc0837fe700 10 client.25670 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:32.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.493+0000 7fc0837fe700 1 -- 172.21.15.22:0/267389827 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5555ea4fce60 con 0x5555ea61d150 2023-09-20T21:49:32.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.493+0000 7fc0837fe700 10 client.25670 handle_command_reply: tid=0 2023-09-20T21:49:32.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.493+0000 7fc0bddb3b80 1 client.25670 shutdown 2023-09-20T21:49:32.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.493+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/267389827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3be0 msgr2=0x5555ea5c2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:32.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.493+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3be0 0x5555ea5c2bc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc09c011ea0 tx=0x7fc09c011f80 comp rx=0 tx=0).stop 2023-09-20T21:49:32.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.495+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/267389827 shutdown_connections 2023-09-20T21:49:32.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.495+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5555ea61d150 0x5555ea61f570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.495+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5555ea5b3be0 0x5555ea5c2bc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.495+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5555ea5b26f0 0x5555ea5c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.495+0000 7fc0bddb3b80 1 --2- 172.21.15.22:0/267389827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555ea5b3050 0x5555ea5c1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.495+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/267389827 >> 172.21.15.22:0/267389827 conn(0x5555ea4df000 msgr2=0x5555ea5c9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:32.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.495+0000 7fc0837fe700 0 client.25670 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:32.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.495+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/267389827 shutdown_connections 2023-09-20T21:49:32.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.496+0000 7fc0bddb3b80 1 -- 172.21.15.22:0/267389827 wait complete. 2023-09-20T21:49:32.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.497+0000 7fc0bddb3b80 20 client.25670 trim_cache size 0 max 16384 2023-09-20T21:49:32.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:32 smithi022 ceph-mon[97651]: pgmap v365: 129 pgs: 129 active+clean; 976 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 63 MiB/s wr, 226 op/s 2023-09-20T21:49:32.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:32 smithi022 ceph-mon[97651]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:49:32.504 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2023-09-20T21:49:32.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.498+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/4042453408 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0900a1400 msgr2=0x7fc0900a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:32.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.498+0000 7fc0aa3ff700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0900a1400 0x7fc0900a38c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00d000 tx=0x7fc09c019000 comp rx=0 tx=0).stop 2023-09-20T21:49:32.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.498+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/4042453408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a4101850 msgr2=0x7fc0a41011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:32.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.498+0000 7fc0aa3ff700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a4101850 0x7fc0a41011c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc0940060b0 tx=0x7fc094007f90 comp rx=0 tx=0).stop 2023-09-20T21:49:32.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.500+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/4042453408 shutdown_connections 2023-09-20T21:49:32.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.500+0000 7fc0aa3ff700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0a4106ab0 0x7fc0a40ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.500+0000 7fc0aa3ff700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a4101850 0x7fc0a41011c0 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc0940060b0 tx=0x7fc094007f90 comp rx=0 tx=0).stop 2023-09-20T21:49:32.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.500+0000 7fc0aa3ff700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc0900a1400 0x7fc0900a38c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.500+0000 7fc0aa3ff700 1 --2- 172.21.15.22:0/4042453408 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0a4104180 0x7fc0a40ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:32.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.500+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/4042453408 >> 172.21.15.22:0/4042453408 conn(0x7fc0a40fb240 msgr2=0x7fc0a4102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:32.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.502+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/4042453408 shutdown_connections 2023-09-20T21:49:32.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:32.503+0000 7fc0aa3ff700 1 -- 172.21.15.22:0/4042453408 wait complete. 2023-09-20T21:49:32.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:32 smithi123 ceph-mon[134433]: pgmap v365: 129 pgs: 129 active+clean; 976 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 63 MiB/s wr, 226 op/s 2023-09-20T21:49:32.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:32 smithi123 ceph-mon[134433]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:49:32.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:32 smithi089 ceph-mon[134090]: pgmap v365: 129 pgs: 129 active+clean; 976 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 63 MiB/s wr, 226 op/s 2023-09-20T21:49:32.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:32 smithi089 ceph-mon[134090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:49:33.541 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-20T21:49:33.542 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-20T21:49:33.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 10.0 s, 1651.0 tps, lat 60.993 ms stddev 75.991 2023-09-20T21:49:33.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.922+0000 7f61751ff700 1 -- 172.21.15.22:0/2681514537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6170102940 msgr2=0x7f6170109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.922+0000 7f61751ff700 1 --2- 172.21.15.22:0/2681514537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6170102940 0x7f6170109260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f616800f400 tx=0x7f616800f710 comp rx=0 tx=0).stop 2023-09-20T21:49:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.923+0000 7f61751ff700 1 -- 172.21.15.22:0/2681514537 shutdown_connections 2023-09-20T21:49:33.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.923+0000 7f61751ff700 1 --2- 172.21.15.22:0/2681514537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6170102940 0x7f6170109260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.923+0000 7f61751ff700 1 --2- 172.21.15.22:0/2681514537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6170100e30 0x7f6170101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.923+0000 7f61751ff700 1 --2- 172.21.15.22:0/2681514537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6170101f80 0x7f6170102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.923+0000 7f61751ff700 1 -- 172.21.15.22:0/2681514537 >> 172.21.15.22:0/2681514537 conn(0x7f61700fc3c0 msgr2=0x7f61700fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:33.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.924+0000 7f61751ff700 1 -- 172.21.15.22:0/2681514537 shutdown_connections 2023-09-20T21:49:33.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.925+0000 7f61751ff700 1 -- 172.21.15.22:0/2681514537 wait complete. 2023-09-20T21:49:33.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.926+0000 7f61751ff700 1 Processor -- start 2023-09-20T21:49:33.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.927+0000 7f61751ff700 1 -- start start 2023-09-20T21:49:33.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.927+0000 7f61751ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6170100e30 0x7f6170194920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.927+0000 7f61751ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6170101f80 0x7f6170194e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.927+0000 7f616e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6170101f80 0x7f6170194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:33.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.927+0000 7f616e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6170101f80 0x7f6170194e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45406/0 (socket says 172.21.15.22:45406) 2023-09-20T21:49:33.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.927+0000 7f616ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6170100e30 0x7f6170194920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:33.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.928+0000 7f616ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6170100e30 0x7f6170194920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38296/0 (socket says 172.21.15.22:38296) 2023-09-20T21:49:33.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.930+0000 7f61751ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6170102940 0x7f617019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.930+0000 7f61751ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617019c4a0 con 0x7f6170101f80 2023-09-20T21:49:33.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.930+0000 7f61751ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617019c5e0 con 0x7f6170100e30 2023-09-20T21:49:33.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.930+0000 7f61751ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617019c770 con 0x7f6170102940 2023-09-20T21:49:33.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.930+0000 7f616e59c700 1 -- 172.21.15.22:0/3078921149 learned_addr learned my addr 172.21.15.22:0/3078921149 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:33.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.930+0000 7f616f59e700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6170102940 0x7f617019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:33.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.931+0000 7f616e59c700 1 -- 172.21.15.22:0/3078921149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6170102940 msgr2=0x7f617019bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:33.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.931+0000 7f616e59c700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6170102940 0x7f617019bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.931+0000 7f616e59c700 1 -- 172.21.15.22:0/3078921149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6170100e30 msgr2=0x7f6170194920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:33.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.931+0000 7f616e59c700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6170100e30 0x7f6170194920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.931+0000 7f616e59c700 1 -- 172.21.15.22:0/3078921149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616800f0b0 con 0x7f6170101f80 2023-09-20T21:49:33.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.931+0000 7f616e59c700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6170101f80 0x7f6170194e60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f616000e9c0 tx=0x7f616000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:33.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.932+0000 7f615ffff700 1 -- 172.21.15.22:0/3078921149 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160004a30 con 0x7f6170101f80 2023-09-20T21:49:33.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.932+0000 7f615ffff700 1 -- 172.21.15.22:0/3078921149 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f616000cc80 con 0x7f6170101f80 2023-09-20T21:49:33.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.932+0000 7f61751ff700 1 -- 172.21.15.22:0/3078921149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617019ca50 con 0x7f6170101f80 2023-09-20T21:49:33.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.932+0000 7f61751ff700 1 -- 172.21.15.22:0/3078921149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617019cfb0 con 0x7f6170101f80 2023-09-20T21:49:33.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.932+0000 7f615ffff700 1 -- 172.21.15.22:0/3078921149 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61600054e0 con 0x7f6170101f80 2023-09-20T21:49:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.938+0000 7f6188b2ab80 1 Processor -- start 2023-09-20T21:49:33.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.938+0000 7f6188b2ab80 1 -- start start 2023-09-20T21:49:33.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.939+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e95999ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.939+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599a0e0 0x557e9599a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.939+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e9599ac80 0x557e9599b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.939+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e958fcd20 con 0x557e9599a0e0 2023-09-20T21:49:33.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.939+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e958e0cf0 con 0x557e9599ac80 2023-09-20T21:49:33.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.939+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e958de610 con 0x557e95999780 2023-09-20T21:49:33.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.940+0000 7f616f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e95999ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:33.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.940+0000 7f616f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e95999ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43414/0 (socket says 172.21.15.22:43414) 2023-09-20T21:49:33.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.940+0000 7f616f59e700 1 -- 172.21.15.22:0/1905465602 learned_addr learned my addr 172.21.15.22:0/1905465602 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:33.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.940+0000 7f616ed9d700 1 --2- 172.21.15.22:0/1905465602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599a0e0 0x557e9599a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.940+0000 7f616f59e700 1 -- 172.21.15.22:0/1905465602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e9599ac80 msgr2=0x557e9599b100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:49:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.940+0000 7f616f59e700 1 --2- 172.21.15.22:0/1905465602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e9599ac80 0x557e9599b100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.940+0000 7f616f59e700 1 -- 172.21.15.22:0/1905465602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599a0e0 msgr2=0x557e9599a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:33.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.940+0000 7f616f59e700 1 --2- 172.21.15.22:0/1905465602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599a0e0 0x557e9599a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.940+0000 7f616f59e700 1 -- 172.21.15.22:0/1905465602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e95704e60 con 0x557e95999780 2023-09-20T21:49:33.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.941+0000 7f616f59e700 1 --2- 172.21.15.22:0/1905465602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e95999ba0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f61680189b0 tx=0x7f61680189e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ec4aba948dd342 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:33.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.944+0000 7f615d7fa700 1 -- 172.21.15.22:0/1905465602 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800b660 con 0x557e95999780 2023-09-20T21:49:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.944+0000 7f615d7fa700 1 -- 172.21.15.22:0/1905465602 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f616800b830 con 0x557e95999780 2023-09-20T21:49:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.944+0000 7f615d7fa700 1 -- 172.21.15.22:0/1905465602 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800d400 con 0x557e95999780 2023-09-20T21:49:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.945+0000 7f615ffff700 1 -- 172.21.15.22:0/3078921149 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f6160021440 con 0x7f6170101f80 2023-09-20T21:49:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.945+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1905465602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 msgr2=0x557e95999ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:33.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.945+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1905465602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e95999ba0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f61680189b0 tx=0x7f61680189e0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.946+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1905465602 shutdown_connections 2023-09-20T21:49:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.946+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1905465602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e9599ac80 0x557e9599b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.946+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1905465602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e95999ba0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.946+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1905465602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599a0e0 0x557e9599a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.946+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1905465602 >> 172.21.15.22:0/1905465602 conn(0x557e958b5080 msgr2=0x557e95983250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.946+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1905465602 shutdown_connections 2023-09-20T21:49:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.947+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1905465602 wait complete. 2023-09-20T21:49:33.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.947+0000 7f6188b2ab80 1 -- start start 2023-09-20T21:49:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.948+0000 7f6188b2ab80 1 Processor -- start 2023-09-20T21:49:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.948+0000 7f615ffff700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f61580a12e0 0x7f61580a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.949+0000 7f615ffff700 1 -- 172.21.15.22:0/3078921149 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6160014070 con 0x7f6170101f80 2023-09-20T21:49:33.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.949+0000 7f616f59e700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f61580a12e0 0x7f61580a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:33.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.950+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e959a3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.950+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e9599a0e0 0x557e959a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.950+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599ac80 0x557e959aa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.950+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e958d9f00 con 0x557e9599ac80 2023-09-20T21:49:33.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.950+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e958767c0 con 0x557e9599a0e0 2023-09-20T21:49:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.950+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e95876460 con 0x557e95999780 2023-09-20T21:49:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.950+0000 7f6188b2ab80 10 client.0 _ll_register_callbacks cb 0x557e957c0b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.951+0000 7f616e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e9599a0e0 0x557e959a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:33.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.951+0000 7f616e59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e9599a0e0 0x557e959a37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38312/0 (socket says 172.21.15.22:38312) 2023-09-20T21:49:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.951+0000 7f616f59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599ac80 0x557e959aa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.951+0000 7f616e59c700 1 -- 172.21.15.22:0/1807162681 learned_addr learned my addr 172.21.15.22:0/1807162681 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.951+0000 7f616f59e700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f61580a12e0 0x7f61580a37a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f616800dfd0 tx=0x7f616800dd30 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.951+0000 7f616f59e700 1 -- 172.21.15.22:0/1807162681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 msgr2=0x557e959a3260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:49:33.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.951+0000 7f616f59e700 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e959a3260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.951+0000 7f616f59e700 1 -- 172.21.15.22:0/1807162681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e9599a0e0 msgr2=0x557e959a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:33.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.951+0000 7f616f59e700 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e9599a0e0 0x557e959a37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.951+0000 7f616f59e700 1 -- 172.21.15.22:0/1807162681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6168018e70 con 0x557e9599ac80 2023-09-20T21:49:33.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.952+0000 7f616f59e700 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599ac80 0x557e959aa7e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6168032000 tx=0x7f616800fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fd279a5073f8fe server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:33.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.952+0000 7f61567fc700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:49:33.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.953+0000 7f61567fc700 1 -- 172.21.15.22:0/1807162681 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616802e070 con 0x557e9599ac80 2023-09-20T21:49:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.953+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1807162681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e9584ac30 con 0x557e9599ac80 2023-09-20T21:49:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.955+0000 7f61567fc700 1 -- 172.21.15.22:0/1807162681 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6168005070 con 0x557e9599ac80 2023-09-20T21:49:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.956+0000 7f61567fc700 1 -- 172.21.15.22:0/1807162681 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6168005230 con 0x557e9599ac80 2023-09-20T21:49:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.956+0000 7f6188b2ab80 10 client.15600 fetch_fsmap learned FSMap version 30 2023-09-20T21:49:33.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.956+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1807162681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557e956edce0 con 0x557e9599ac80 2023-09-20T21:49:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.957+0000 7f61567fc700 1 -- 172.21.15.22:0/1807162681 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e9584ac30 con 0x557e9599ac80 2023-09-20T21:49:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.957+0000 7f61567fc700 1 -- 172.21.15.22:0/1807162681 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f6168031070 con 0x557e9599ac80 2023-09-20T21:49:33.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.957+0000 7f6188b2ab80 10 client.15600 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:49:33.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.957+0000 7f6188b2ab80 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:49:33.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.957+0000 7f6188b2ab80 20 client.15600 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:33.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.957+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e959aee00 0x557e959b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.957+0000 7f6188b2ab80 4 client.15600 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:49:33.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.957+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1807162681 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e959570a0 con 0x557e959aee00 2023-09-20T21:49:33.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.958+0000 7f616ed9d700 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e959aee00 0x557e959b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:33.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.960+0000 7f616ed9d700 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e959aee00 0x557e959b1220 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc65062f68516 server_cookie=313c6745244edebf in_seq=0 out_seq=0 2023-09-20T21:49:33.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.960+0000 7f61567fc700 10 client.15600 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:33.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.961+0000 7f61567fc700 1 -- 172.21.15.22:0/1807162681 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e959570a0 con 0x557e959aee00 2023-09-20T21:49:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.961+0000 7f61567fc700 10 client.15600 handle_command_reply: tid=0 2023-09-20T21:49:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.961+0000 7f6188b2ab80 1 client.15600 shutdown 2023-09-20T21:49:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.962+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1807162681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599ac80 msgr2=0x557e959aa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.962+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599ac80 0x557e959aa7e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6168032000 tx=0x7f616800fa60 comp rx=0 tx=0).stop 2023-09-20T21:49:33.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.962+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1807162681 shutdown_connections 2023-09-20T21:49:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.962+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e959aee00 0x557e959b1220 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.962+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599ac80 0x557e959aa7e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.962+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e959a3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.962+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1807162681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e9599a0e0 0x557e959a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:33.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.962+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1807162681 >> 172.21.15.22:0/1807162681 conn(0x557e958b5080 msgr2=0x557e95982900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.963+0000 7f61567fc700 0 client.15600 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.963+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1807162681 shutdown_connections 2023-09-20T21:49:33.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.963+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1807162681 wait complete. 2023-09-20T21:49:33.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.963+0000 7f6188b2ab80 20 client.15600 trim_cache size 0 max 16384 2023-09-20T21:49:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.996+0000 7f6188b2ab80 1 Processor -- start 2023-09-20T21:49:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.997+0000 7f6188b2ab80 1 -- start start 2023-09-20T21:49:33.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.997+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 0x557e959b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.997+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e95999780 0x557e959aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.997+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e9599a0e0 0x557e959a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.997+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9575a9a0 con 0x557e95999780 2023-09-20T21:49:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.997+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e95983d40 con 0x557e958b5080 2023-09-20T21:49:33.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.997+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e959a2dd0 con 0x557e9599a0e0 2023-09-20T21:49:34.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.997+0000 7f616f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e9599a0e0 0x557e959a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.997+0000 7f616f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e9599a0e0 0x557e959a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43420/0 (socket says 172.21.15.22:43420) 2023-09-20T21:49:34.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.997+0000 7f616f59e700 1 -- 172.21.15.22:0/1400154846 learned_addr learned my addr 172.21.15.22:0/1400154846 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:34.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:33 smithi022 ceph-mon[97651]: pgmap v366: 129 pgs: 129 active+clean; 999 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 47 MiB/s wr, 253 op/s 2023-09-20T21:49:34.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.998+0000 7f616ed9d700 1 --2- 172.21.15.22:0/1400154846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 0x557e959b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.998+0000 7f616f59e700 1 -- 172.21.15.22:0/1400154846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 msgr2=0x557e959b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.998+0000 7f616f59e700 1 --2- 172.21.15.22:0/1400154846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 0x557e959b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.998+0000 7f616f59e700 1 -- 172.21.15.22:0/1400154846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e95999780 msgr2=0x557e959aa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.998+0000 7f616f59e700 1 --2- 172.21.15.22:0/1400154846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e95999780 0x557e959aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.998+0000 7f616f59e700 1 -- 172.21.15.22:0/1400154846 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e9597f630 con 0x557e9599a0e0 2023-09-20T21:49:34.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.999+0000 7f616f59e700 1 --2- 172.21.15.22:0/1400154846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e9599a0e0 0x557e959a1f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f616800b680 tx=0x7f6168034e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3c7d15d97e9194f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:34.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.999+0000 7f615dffb700 1 -- 172.21.15.22:0/1400154846 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800d7f0 con 0x557e9599a0e0 2023-09-20T21:49:34.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.999+0000 7f615dffb700 1 -- 172.21.15.22:0/1400154846 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61680075c0 con 0x557e9599a0e0 2023-09-20T21:49:34.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:33.999+0000 7f615dffb700 1 -- 172.21.15.22:0/1400154846 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168005330 con 0x557e9599a0e0 2023-09-20T21:49:34.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.000+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1400154846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e9599a0e0 msgr2=0x557e959a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.000+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1400154846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e9599a0e0 0x557e959a1f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f616800b680 tx=0x7f6168034e00 comp rx=0 tx=0).stop 2023-09-20T21:49:34.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.000+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1400154846 shutdown_connections 2023-09-20T21:49:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.000+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1400154846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e9599a0e0 0x557e959a1f10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.000+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1400154846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 0x557e959b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.000+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/1400154846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e95999780 0x557e959aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.000+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1400154846 >> 172.21.15.22:0/1400154846 conn(0x557e95983580 msgr2=0x557e959a10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:34.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.001+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1400154846 shutdown_connections 2023-09-20T21:49:34.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.001+0000 7f6188b2ab80 1 -- 172.21.15.22:0/1400154846 wait complete. 2023-09-20T21:49:34.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.001+0000 7f6188b2ab80 1 -- start start 2023-09-20T21:49:34.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.001+0000 7f6188b2ab80 1 Processor -- start 2023-09-20T21:49:34.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 0x557e959aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e959a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f6188b2ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599a0e0 0x557e959a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e959a96b0 con 0x557e9599a0e0 2023-09-20T21:49:34.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e959a9820 con 0x557e958b5080 2023-09-20T21:49:34.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f6188b2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e959a99e0 con 0x557e95999780 2023-09-20T21:49:34.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f6188b2ab80 10 client.0 _ll_register_callbacks cb 0x557e957c0b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:34.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f616ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 0x557e959aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f616ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 0x557e959aa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38332/0 (socket says 172.21.15.22:38332) 2023-09-20T21:49:34.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f616ed9d700 1 -- 172.21.15.22:0/575420744 learned_addr learned my addr 172.21.15.22:0/575420744 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:34.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.002+0000 7f616e59c700 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e959a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.003+0000 7f616e59c700 1 -- 172.21.15.22:0/575420744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 msgr2=0x557e959aa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.003+0000 7f616e59c700 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 0x557e959aa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.003+0000 7f616e59c700 1 -- 172.21.15.22:0/575420744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599a0e0 msgr2=0x557e959a90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.003+0000 7f616e59c700 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599a0e0 0x557e959a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.003+0000 7f616e59c700 1 -- 172.21.15.22:0/575420744 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e956edce0 con 0x557e95999780 2023-09-20T21:49:34.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.004+0000 7f616e59c700 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e959a8ba0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6160005950 tx=0x7f6160099e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825378f8630a30e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:34.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.004+0000 7f61567fc700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:49:34.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.004+0000 7f61567fc700 1 -- 172.21.15.22:0/575420744 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616009e290 con 0x557e95999780 2023-09-20T21:49:34.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.004+0000 7f61567fc700 1 -- 172.21.15.22:0/575420744 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f61600a24d0 con 0x557e95999780 2023-09-20T21:49:34.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.004+0000 7f6188b2ab80 1 -- 172.21.15.22:0/575420744 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f616800b830 con 0x557e95999780 2023-09-20T21:49:34.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.004+0000 7f61567fc700 1 -- 172.21.15.22:0/575420744 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6160005640 con 0x557e95999780 2023-09-20T21:49:34.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.005+0000 7f6188b2ab80 10 client.25666 fetch_fsmap learned FSMap version 30 2023-09-20T21:49:34.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.005+0000 7f6188b2ab80 1 -- 172.21.15.22:0/575420744 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557e9572b2d0 con 0x557e95999780 2023-09-20T21:49:34.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.006+0000 7f61567fc700 1 -- 172.21.15.22:0/575420744 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f616007c080 con 0x557e95999780 2023-09-20T21:49:34.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.006+0000 7f61567fc700 1 -- 172.21.15.22:0/575420744 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f61600a2140 con 0x557e95999780 2023-09-20T21:49:34.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.006+0000 7f6188b2ab80 10 client.25666 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:49:34.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.006+0000 7f6188b2ab80 10 client.25666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:49:34.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.006+0000 7f6188b2ab80 20 client.25666 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:34.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.006+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e9599ac80 0x557e95a065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.006+0000 7f6188b2ab80 4 client.25666 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-20T21:49:34.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.006+0000 7f6188b2ab80 1 -- 172.21.15.22:0/575420744 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557e958d2ee0 con 0x557e9599ac80 2023-09-20T21:49:34.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.006+0000 7f616ed9d700 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e9599ac80 0x557e95a065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.007+0000 7f616ed9d700 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e9599ac80 0x557e95a065e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa5ae4c061587415 server_cookie=9d486a298545e35 in_seq=0 out_seq=0 2023-09-20T21:49:34.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.007+0000 7f61567fc700 10 client.25666 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:34.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.009+0000 7f61567fc700 1 -- 172.21.15.22:0/575420744 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557e958d2ee0 con 0x557e9599ac80 2023-09-20T21:49:34.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.009+0000 7f61567fc700 10 client.25666 handle_command_reply: tid=0 2023-09-20T21:49:34.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.009+0000 7f6188b2ab80 1 client.25666 shutdown 2023-09-20T21:49:34.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.010+0000 7f6188b2ab80 1 -- 172.21.15.22:0/575420744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 msgr2=0x557e959a8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.010+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e959a8ba0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6160005950 tx=0x7f6160099e30 comp rx=0 tx=0).stop 2023-09-20T21:49:34.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.011+0000 7f6188b2ab80 1 -- 172.21.15.22:0/575420744 shutdown_connections 2023-09-20T21:49:34.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.011+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557e9599ac80 0x557e95a065e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.011+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e9599a0e0 0x557e959a90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.011+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557e958b5080 0x557e959aa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.011+0000 7f6188b2ab80 1 --2- 172.21.15.22:0/575420744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557e95999780 0x557e959a8ba0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.011+0000 7f6188b2ab80 1 -- 172.21.15.22:0/575420744 >> 172.21.15.22:0/575420744 conn(0x557e95983580 msgr2=0x557e959b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:34.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.011+0000 7f61567fc700 0 client.25666 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:34.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.012+0000 7f6188b2ab80 1 -- 172.21.15.22:0/575420744 shutdown_connections 2023-09-20T21:49:34.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.012+0000 7f6188b2ab80 1 -- 172.21.15.22:0/575420744 wait complete. 2023-09-20T21:49:34.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.013+0000 7f6188b2ab80 20 client.25666 trim_cache size 0 max 16384 2023-09-20T21:49:34.016 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:49:34.016 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2023-09-20T21:49:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "94b34214-ef80-4df0-8aab-02c689244dd6", 2023-09-20T21:49:34.017 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2023-09-20T21:49:34.017 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:49:34.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.018+0000 7f61751ff700 1 -- 172.21.15.22:0/3078921149 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f61580a12e0 msgr2=0x7f61580a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.018+0000 7f61751ff700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f61580a12e0 0x7f61580a37a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f616800dfd0 tx=0x7f616800dd30 comp rx=0 tx=0).stop 2023-09-20T21:49:34.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.018+0000 7f61751ff700 1 -- 172.21.15.22:0/3078921149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6170101f80 msgr2=0x7f6170194e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.018+0000 7f61751ff700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6170101f80 0x7f6170194e60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f616000e9c0 tx=0x7f616000ecd0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.019+0000 7f61751ff700 1 -- 172.21.15.22:0/3078921149 shutdown_connections 2023-09-20T21:49:34.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.019+0000 7f61751ff700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6170102940 0x7f617019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.019+0000 7f61751ff700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f61580a12e0 0x7f61580a37a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.019+0000 7f61751ff700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6170100e30 0x7f6170194920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.019+0000 7f61751ff700 1 --2- 172.21.15.22:0/3078921149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6170101f80 0x7f6170194e60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.019+0000 7f61751ff700 1 -- 172.21.15.22:0/3078921149 >> 172.21.15.22:0/3078921149 conn(0x7f61700fc3c0 msgr2=0x7f61700fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:34.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.020+0000 7f61751ff700 1 -- 172.21.15.22:0/3078921149 shutdown_connections 2023-09-20T21:49:34.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.020+0000 7f61751ff700 1 -- 172.21.15.22:0/3078921149 wait complete. 2023-09-20T21:49:34.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:49:34.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:33 smithi123 ceph-mon[134433]: pgmap v366: 129 pgs: 129 active+clean; 999 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 47 MiB/s wr, 253 op/s 2023-09-20T21:49:34.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:33 smithi089 ceph-mon[134090]: pgmap v366: 129 pgs: 129 active+clean; 999 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 47 MiB/s wr, 253 op/s 2023-09-20T21:49:34.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.437+0000 7fada3dff700 1 -- 172.21.15.22:0/3667033349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad9c101fb0 msgr2=0x7fad9c104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.437+0000 7fada3dff700 1 --2- 172.21.15.22:0/3667033349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad9c101fb0 0x7fad9c104440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009990 tx=0x7fad8c009ca0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.437+0000 7fada3dff700 1 -- 172.21.15.22:0/3667033349 shutdown_connections 2023-09-20T21:49:34.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.437+0000 7fada3dff700 1 --2- 172.21.15.22:0/3667033349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad9c104980 0x7fad9c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.437+0000 7fada3dff700 1 --2- 172.21.15.22:0/3667033349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c0ff640 0x7fad9c101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.437+0000 7fada3dff700 1 --2- 172.21.15.22:0/3667033349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad9c101fb0 0x7fad9c104440 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.437+0000 7fada3dff700 1 -- 172.21.15.22:0/3667033349 >> 172.21.15.22:0/3667033349 conn(0x7fad9c0fb240 msgr2=0x7fad9c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:34.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.437+0000 7fada3dff700 1 -- 172.21.15.22:0/3667033349 shutdown_connections 2023-09-20T21:49:34.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.438+0000 7fada3dff700 1 -- 172.21.15.22:0/3667033349 wait complete. 2023-09-20T21:49:34.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.439+0000 7fada3dff700 1 Processor -- start 2023-09-20T21:49:34.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.439+0000 7fada3dff700 1 -- start start 2023-09-20T21:49:34.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada3dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad9c0ff640 0x7fad9c19d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada3dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c101fb0 0x7fad9c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada3dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad9c104980 0x7fad9c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada3dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c1a4ca0 con 0x7fad9c104980 2023-09-20T21:49:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada3dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c1a4de0 con 0x7fad9c101fb0 2023-09-20T21:49:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada3dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c1a4f70 con 0x7fad9c0ff640 2023-09-20T21:49:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada1b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad9c0ff640 0x7fad9c19d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada1b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad9c0ff640 0x7fad9c19d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43448/0 (socket says 172.21.15.22:43448) 2023-09-20T21:49:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada1b9b700 1 -- 172.21.15.22:0/1533208929 learned_addr learned my addr 172.21.15.22:0/1533208929 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada139a700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c101fb0 0x7fad9c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada239c700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad9c104980 0x7fad9c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada1b9b700 1 -- 172.21.15.22:0/1533208929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c101fb0 msgr2=0x7fad9c19d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada1b9b700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c101fb0 0x7fad9c19d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada1b9b700 1 -- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad9c104980 msgr2=0x7fad9c1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada1b9b700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad9c104980 0x7fad9c1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.440+0000 7fada1b9b700 1 -- 172.21.15.22:0/1533208929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0095f0 con 0x7fad9c0ff640 2023-09-20T21:49:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.441+0000 7fada239c700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad9c104980 0x7fad9c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:49:34.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.441+0000 7fada1b9b700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad9c0ff640 0x7fad9c19d120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad9800e9a0 tx=0x7fad9800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.441+0000 7fad92ffd700 1 -- 172.21.15.22:0/1533208929 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98009790 con 0x7fad9c0ff640 2023-09-20T21:49:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.441+0000 7fad92ffd700 1 -- 172.21.15.22:0/1533208929 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad9800cbd0 con 0x7fad9c0ff640 2023-09-20T21:49:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.441+0000 7fad92ffd700 1 -- 172.21.15.22:0/1533208929 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98018680 con 0x7fad9c0ff640 2023-09-20T21:49:34.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.441+0000 7fada3dff700 1 -- 172.21.15.22:0/1533208929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c1a5250 con 0x7fad9c0ff640 2023-09-20T21:49:34.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.441+0000 7fada3dff700 1 -- 172.21.15.22:0/1533208929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c1a5710 con 0x7fad9c0ff640 2023-09-20T21:49:34.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.449+0000 7fadb7777b80 1 Processor -- start 2023-09-20T21:49:34.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.449+0000 7fadb7777b80 1 -- start start 2023-09-20T21:49:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.450+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.450+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2050 0x55b6fc9e25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.450+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9e3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.450+0000 7fadb7777b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc955ca0 con 0x55b6fc9e2be0 2023-09-20T21:49:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.450+0000 7fadb7777b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc939c70 con 0x55b6fc9e16f0 2023-09-20T21:49:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.450+0000 7fadb7777b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc937590 con 0x55b6fc9e2050 2023-09-20T21:49:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.450+0000 7fada239c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.450+0000 7fada239c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38354/0 (socket says 172.21.15.22:38354) 2023-09-20T21:49:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.450+0000 7fada239c700 1 -- 172.21.15.22:0/2566658129 learned_addr learned my addr 172.21.15.22:0/2566658129 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:34.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.451+0000 7fada239c700 1 -- 172.21.15.22:0/2566658129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2050 msgr2=0x55b6fc9e25e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:49:34.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.451+0000 7fada239c700 1 --2- 172.21.15.22:0/2566658129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2050 0x55b6fc9e25e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.451+0000 7fada239c700 1 -- 172.21.15.22:0/2566658129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2be0 msgr2=0x55b6fc9e3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.451+0000 7fada239c700 1 --2- 172.21.15.22:0/2566658129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.451+0000 7fada239c700 1 -- 172.21.15.22:0/2566658129 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6fc75dde0 con 0x55b6fc9e16f0 2023-09-20T21:49:34.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.451+0000 7fada239c700 1 --2- 172.21.15.22:0/2566658129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e1b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad94009400 tx=0x7fad94009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f015dc2195fa8e96 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:34.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.452+0000 7fad87fff700 1 -- 172.21.15.22:0/2566658129 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94014610 con 0x55b6fc9e16f0 2023-09-20T21:49:34.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.452+0000 7fad87fff700 1 -- 172.21.15.22:0/2566658129 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad9400f050 con 0x55b6fc9e16f0 2023-09-20T21:49:34.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.452+0000 7fad87fff700 1 -- 172.21.15.22:0/2566658129 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad940139a0 con 0x55b6fc9e16f0 2023-09-20T21:49:34.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.453+0000 7fadb7777b80 1 -- 172.21.15.22:0/2566658129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 msgr2=0x55b6fc9e1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.453+0000 7fadb7777b80 1 --2- 172.21.15.22:0/2566658129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e1b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad94009400 tx=0x7fad94009920 comp rx=0 tx=0).stop 2023-09-20T21:49:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.453+0000 7fadb7777b80 1 -- 172.21.15.22:0/2566658129 shutdown_connections 2023-09-20T21:49:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.453+0000 7fadb7777b80 1 --2- 172.21.15.22:0/2566658129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9e3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.453+0000 7fadb7777b80 1 --2- 172.21.15.22:0/2566658129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e1b10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.453+0000 7fadb7777b80 1 --2- 172.21.15.22:0/2566658129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2050 0x55b6fc9e25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.453+0000 7fadb7777b80 1 -- 172.21.15.22:0/2566658129 >> 172.21.15.22:0/2566658129 conn(0x55b6fc90e000 msgr2=0x55b6fc9dc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.453+0000 7fadb7777b80 1 -- 172.21.15.22:0/2566658129 shutdown_connections 2023-09-20T21:49:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.453+0000 7fadb7777b80 1 -- 172.21.15.22:0/2566658129 wait complete. 2023-09-20T21:49:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.453+0000 7fad92ffd700 1 -- 172.21.15.22:0/1533208929 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fad9800cd40 con 0x7fad9c0ff640 2023-09-20T21:49:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.454+0000 7fadb7777b80 1 -- start start 2023-09-20T21:49:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.454+0000 7fadb7777b80 1 Processor -- start 2023-09-20T21:49:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.455+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.455+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e2050 0x55b6fc9eb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.455+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9f2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.455+0000 7fadb7777b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc932e80 con 0x55b6fc9e16f0 2023-09-20T21:49:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.455+0000 7fadb7777b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc8cf740 con 0x55b6fc9e2050 2023-09-20T21:49:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.455+0000 7fadb7777b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc8cf3e0 con 0x55b6fc9e2be0 2023-09-20T21:49:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.455+0000 7fadb7777b80 10 client.0 _ll_register_callbacks cb 0x55b6fc819a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.456+0000 7fada239c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.456+0000 7fada239c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9eb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45466/0 (socket says 172.21.15.22:45466) 2023-09-20T21:49:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.456+0000 7fada239c700 1 -- 172.21.15.22:0/4233748840 learned_addr learned my addr 172.21.15.22:0/4233748840 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:34.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.456+0000 7fada139a700 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.456+0000 7fada1b9b700 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e2050 0x55b6fc9eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.456+0000 7fada139a700 1 -- 172.21.15.22:0/4233748840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e2050 msgr2=0x55b6fc9eb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.457+0000 7fada139a700 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e2050 0x55b6fc9eb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.457+0000 7fada139a700 1 -- 172.21.15.22:0/4233748840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e16f0 msgr2=0x55b6fc9eb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.457+0000 7fada139a700 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9eb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.457+0000 7fada139a700 1 -- 172.21.15.22:0/4233748840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad940090b0 con 0x55b6fc9e2be0 2023-09-20T21:49:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.457+0000 7fada239c700 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9eb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:49:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.457+0000 7fada1b9b700 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e2050 0x55b6fc9eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:49:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.457+0000 7fada139a700 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9f2870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad8c000c00 tx=0x7fad8c00bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0add07983ef320a server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.457+0000 7fad857fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:49:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.458+0000 7fad857fa700 1 -- 172.21.15.22:0/4233748840 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c026070 con 0x55b6fc9e2be0 2023-09-20T21:49:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.458+0000 7fad857fa700 1 -- 172.21.15.22:0/4233748840 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fad8c005630 con 0x55b6fc9e2be0 2023-09-20T21:49:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.460+0000 7fadb7777b80 1 -- 172.21.15.22:0/4233748840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6fc8a3bb0 con 0x55b6fc9e2be0 2023-09-20T21:49:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.460+0000 7fad92ffd700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fad880a1310 0x7fad880a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.460+0000 7fad857fa700 1 -- 172.21.15.22:0/4233748840 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad8c01d070 con 0x55b6fc9e2be0 2023-09-20T21:49:34.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.460+0000 7fad92ffd700 1 -- 172.21.15.22:0/1533208929 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fad98014070 con 0x7fad9c0ff640 2023-09-20T21:49:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.463+0000 7fada239c700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fad880a1310 0x7fad880a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.464+0000 7fadb7777b80 10 client.25675 fetch_fsmap learned FSMap version 30 2023-09-20T21:49:34.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.464+0000 7fadb7777b80 1 -- 172.21.15.22:0/4233748840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b6fc746c60 con 0x55b6fc9e2be0 2023-09-20T21:49:34.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.464+0000 7fada239c700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fad880a1310 0x7fad880a37d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fad940134b0 tx=0x7fad94009010 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:34.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.464+0000 7fad857fa700 1 -- 172.21.15.22:0/4233748840 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b6fc8a3bb0 con 0x55b6fc9e2be0 2023-09-20T21:49:34.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.464+0000 7fad857fa700 1 -- 172.21.15.22:0/4233748840 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fad8c029020 con 0x55b6fc9e2be0 2023-09-20T21:49:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.465+0000 7fadb7777b80 10 client.25675 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:49:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.465+0000 7fadb7777b80 10 client.25675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:49:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.465+0000 7fadb7777b80 20 client.25675 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.465+0000 7fadb7777b80 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55b6fc9f7080 0x55b6fc9f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.465+0000 7fadb7777b80 4 client.25675 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:49:34.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.465+0000 7fadb7777b80 1 -- 172.21.15.22:0/4233748840 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6fc9b0020 con 0x55b6fc9f7080 2023-09-20T21:49:34.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.465+0000 7fada1b9b700 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55b6fc9f7080 0x55b6fc9f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.466+0000 7fada1b9b700 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55b6fc9f7080 0x55b6fc9f94a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55927209acb17344 server_cookie=858e243fd7a54d64 in_seq=0 out_seq=0 2023-09-20T21:49:34.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.466+0000 7fad857fa700 10 client.25675 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:34.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.467+0000 7fad857fa700 1 -- 172.21.15.22:0/4233748840 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b6fc9b0020 con 0x55b6fc9f7080 2023-09-20T21:49:34.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.467+0000 7fad857fa700 10 client.25675 handle_command_reply: tid=0 2023-09-20T21:49:34.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.467+0000 7fadb7777b80 1 client.25675 shutdown 2023-09-20T21:49:34.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.467+0000 7fadb7777b80 1 -- 172.21.15.22:0/4233748840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 msgr2=0x55b6fc9f2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.467+0000 7fadb7777b80 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9f2870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad8c000c00 tx=0x7fad8c00bf80 comp rx=0 tx=0).stop 2023-09-20T21:49:34.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.468+0000 7fadb7777b80 1 -- 172.21.15.22:0/4233748840 shutdown_connections 2023-09-20T21:49:34.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.468+0000 7fadb7777b80 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55b6fc9f7080 0x55b6fc9f94a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.468+0000 7fadb7777b80 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9f2870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.468+0000 7fadb7777b80 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9eb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.468+0000 7fadb7777b80 1 --2- 172.21.15.22:0/4233748840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e2050 0x55b6fc9eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.468+0000 7fadb7777b80 1 -- 172.21.15.22:0/4233748840 >> 172.21.15.22:0/4233748840 conn(0x55b6fc90e000 msgr2=0x55b6fc9db880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:34.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.468+0000 7fad857fa700 0 client.25675 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:34.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.468+0000 7fadb7777b80 1 -- 172.21.15.22:0/4233748840 shutdown_connections 2023-09-20T21:49:34.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.468+0000 7fadb7777b80 1 -- 172.21.15.22:0/4233748840 wait complete. 2023-09-20T21:49:34.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.468+0000 7fadb7777b80 20 client.25675 trim_cache size 0 max 16384 2023-09-20T21:49:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.508+0000 7fadb7777b80 1 Processor -- start 2023-09-20T21:49:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fadb7777b80 1 -- start start 2023-09-20T21:49:34.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 0x55b6fc9f2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fadb7777b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc7b3920 con 0x55b6fc9e2050 2023-09-20T21:49:34.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fadb7777b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc9eae60 con 0x55b6fc9e16f0 2023-09-20T21:49:34.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fadb7777b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc9e2a70 con 0x55b6fc9e2be0 2023-09-20T21:49:34.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fada239c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fada139a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 0x55b6fc9f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fada1b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.509+0000 7fada239c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9e9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43468/0 (socket says 172.21.15.22:43468) 2023-09-20T21:49:34.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.510+0000 7fada139a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 0x55b6fc9f2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45474/0 (socket says 172.21.15.22:45474) 2023-09-20T21:49:34.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.510+0000 7fada139a700 1 -- 172.21.15.22:0/542684352 learned_addr learned my addr 172.21.15.22:0/542684352 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:34.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.510+0000 7fada1b9b700 1 -- 172.21.15.22:0/542684352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 msgr2=0x55b6fc9e9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.510+0000 7fada1b9b700 1 --2- 172.21.15.22:0/542684352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9e9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.510+0000 7fada1b9b700 1 -- 172.21.15.22:0/542684352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 msgr2=0x55b6fc9f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.510+0000 7fada1b9b700 1 --2- 172.21.15.22:0/542684352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 0x55b6fc9f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.510+0000 7fada1b9b700 1 -- 172.21.15.22:0/542684352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6fc784250 con 0x55b6fc9e16f0 2023-09-20T21:49:34.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.510+0000 7fada139a700 1 --2- 172.21.15.22:0/542684352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 0x55b6fc9f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:49:34.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.510+0000 7fada1b9b700 1 --2- 172.21.15.22:0/542684352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e2fc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fad980ca950 tx=0x7fad980cf440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8bef706ffb2f99 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:34.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.510+0000 7fad90bff700 1 -- 172.21.15.22:0/542684352 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9809a910 con 0x55b6fc9e16f0 2023-09-20T21:49:34.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.511+0000 7fad90bff700 1 -- 172.21.15.22:0/542684352 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad9809a210 con 0x55b6fc9e16f0 2023-09-20T21:49:34.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.511+0000 7fad90bff700 1 -- 172.21.15.22:0/542684352 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9809a6d0 con 0x55b6fc9e16f0 2023-09-20T21:49:34.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.512+0000 7fadb7777b80 1 -- 172.21.15.22:0/542684352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 msgr2=0x55b6fc9e2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.512+0000 7fadb7777b80 1 --2- 172.21.15.22:0/542684352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e2fc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fad980ca950 tx=0x7fad980cf440 comp rx=0 tx=0).stop 2023-09-20T21:49:34.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.512+0000 7fadb7777b80 1 -- 172.21.15.22:0/542684352 shutdown_connections 2023-09-20T21:49:34.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.512+0000 7fadb7777b80 1 --2- 172.21.15.22:0/542684352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9e9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.512+0000 7fadb7777b80 1 --2- 172.21.15.22:0/542684352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9e2fc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.512+0000 7fadb7777b80 1 --2- 172.21.15.22:0/542684352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 0x55b6fc9f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.512+0000 7fadb7777b80 1 -- 172.21.15.22:0/542684352 >> 172.21.15.22:0/542684352 conn(0x55b6fc90e000 msgr2=0x55b6fc9d82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.512+0000 7fadb7777b80 1 -- 172.21.15.22:0/542684352 shutdown_connections 2023-09-20T21:49:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.512+0000 7fadb7777b80 1 -- 172.21.15.22:0/542684352 wait complete. 2023-09-20T21:49:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.512+0000 7fadb7777b80 1 -- start start 2023-09-20T21:49:34.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.513+0000 7fadb7777b80 1 Processor -- start 2023-09-20T21:49:34.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.513+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.513+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 0x55b6fc9f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.513+0000 7fadb7777b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9f1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.513+0000 7fadb7777b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc9f21b0 con 0x55b6fc9e2050 2023-09-20T21:49:34.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.513+0000 7fadb7777b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc9f2320 con 0x55b6fc9e16f0 2023-09-20T21:49:34.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.513+0000 7fadb7777b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fc9f24e0 con 0x55b6fc9e2be0 2023-09-20T21:49:34.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fadb7777b80 10 client.0 _ll_register_callbacks cb 0x55b6fc819a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:49:34.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada1b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada139a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 0x55b6fc9f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada1b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:38366/0 (socket says 172.21.15.22:38366) 2023-09-20T21:49:34.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada239c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada1b9b700 1 -- 172.21.15.22:0/1809716527 learned_addr learned my addr 172.21.15.22:0/1809716527 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:49:34.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada1b9b700 1 -- 172.21.15.22:0/1809716527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 msgr2=0x55b6fc9f1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada1b9b700 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9f1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada1b9b700 1 -- 172.21.15.22:0/1809716527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 msgr2=0x55b6fc9f0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:49:34.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada1b9b700 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 0x55b6fc9f0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada1b9b700 1 -- 172.21.15.22:0/1809716527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6fc746c60 con 0x55b6fc9e16f0 2023-09-20T21:49:34.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fada1b9b700 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9f0720 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad98099f10 tx=0x7fad98099ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22de4cb7558478f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:49:34.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.514+0000 7fad857fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:49:34.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fad857fa700 1 -- 172.21.15.22:0/1809716527 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9809a910 con 0x55b6fc9e16f0 2023-09-20T21:49:34.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fad857fa700 1 -- 172.21.15.22:0/1809716527 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fad980cae40 con 0x55b6fc9e16f0 2023-09-20T21:49:34.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fadb7777b80 1 -- 172.21.15.22:0/1809716527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad9400f050 con 0x55b6fc9e16f0 2023-09-20T21:49:34.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fad857fa700 1 -- 172.21.15.22:0/1809716527 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad980a2280 con 0x55b6fc9e16f0 2023-09-20T21:49:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fadb7777b80 10 client.25700 fetch_fsmap learned FSMap version 30 2023-09-20T21:49:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fadb7777b80 1 -- 172.21.15.22:0/1809716527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fad9c1a5710 con 0x55b6fc9e16f0 2023-09-20T21:49:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fad857fa700 1 -- 172.21.15.22:0/1809716527 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad9400f050 con 0x55b6fc9e16f0 2023-09-20T21:49:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fad857fa700 1 -- 172.21.15.22:0/1809716527 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fad98096000 con 0x55b6fc9e16f0 2023-09-20T21:49:34.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fadb7777b80 10 client.25700 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:49:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fadb7777b80 10 client.25700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:49:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fadb7777b80 20 client.25700 populate_metadata read hostname 'smithi022' 2023-09-20T21:49:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fadb7777b80 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55b6fca4be50 0x55b6fca4e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:49:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fadb7777b80 4 client.25700 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:49:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.515+0000 7fadb7777b80 1 -- 172.21.15.22:0/1809716527 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b6fc92be60 con 0x55b6fca4be50 2023-09-20T21:49:34.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fada139a700 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55b6fca4be50 0x55b6fca4e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:49:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fada139a700 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55b6fca4be50 0x55b6fca4e270 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ee98ea2938115e4 server_cookie=26a9018e28a6c800 in_seq=0 out_seq=0 2023-09-20T21:49:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fad857fa700 10 client.25700 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fad857fa700 1 -- 172.21.15.22:0/1809716527 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b6fc92be60 con 0x55b6fca4be50 2023-09-20T21:49:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fad857fa700 10 client.25700 handle_command_reply: tid=0 2023-09-20T21:49:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fadb7777b80 1 client.25700 shutdown 2023-09-20T21:49:34.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fadb7777b80 1 -- 172.21.15.22:0/1809716527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 msgr2=0x55b6fc9f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fadb7777b80 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9f0720 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad98099f10 tx=0x7fad98099ca0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fadb7777b80 1 -- 172.21.15.22:0/1809716527 shutdown_connections 2023-09-20T21:49:34.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fadb7777b80 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55b6fca4be50 0x55b6fca4e270 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fadb7777b80 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6fc9e2be0 0x55b6fc9f1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fadb7777b80 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6fc9e16f0 0x55b6fc9f0720 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fadb7777b80 1 --2- 172.21.15.22:0/1809716527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6fc9e2050 0x55b6fc9f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fadb7777b80 1 -- 172.21.15.22:0/1809716527 >> 172.21.15.22:0/1809716527 conn(0x55b6fc90e000 msgr2=0x55b6fc9eb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.516+0000 7fad857fa700 0 client.25700 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:49:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.517+0000 7fadb7777b80 1 -- 172.21.15.22:0/1809716527 shutdown_connections 2023-09-20T21:49:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.517+0000 7fadb7777b80 1 -- 172.21.15.22:0/1809716527 wait complete. 2023-09-20T21:49:34.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.517+0000 7fadb7777b80 20 client.25700 trim_cache size 0 max 16384 2023-09-20T21:49:34.532 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:49:34.532 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-20T21:49:34.532 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:49:34.532 INFO:teuthology.orchestra.run.smithi022.stdout: "94b34214-ef80-4df0-8aab-02c689244dd6": { 2023-09-20T21:49:34.533 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:49:34.533 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "94b34214-ef80-4df0-8aab-02c689244dd6", 2023-09-20T21:49:34.533 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:49:34.533 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:49:34.533 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:49:34.533 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:49:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.517+0000 7fada3dff700 1 -- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fad880a1310 msgr2=0x7fad880a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.517+0000 7fada3dff700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fad880a1310 0x7fad880a37d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fad940134b0 tx=0x7fad94009010 comp rx=0 tx=0).stop 2023-09-20T21:49:34.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.517+0000 7fada3dff700 1 -- 172.21.15.22:0/1533208929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad9c0ff640 msgr2=0x7fad9c19d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:49:34.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.517+0000 7fada3dff700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad9c0ff640 0x7fad9c19d120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad9800e9a0 tx=0x7fad9800ecb0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.518+0000 7fada3dff700 1 -- 172.21.15.22:0/1533208929 shutdown_connections 2023-09-20T21:49:34.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.518+0000 7fada3dff700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad9c104980 0x7fad9c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.518+0000 7fada3dff700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad9c0ff640 0x7fad9c19d120 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.518+0000 7fada3dff700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fad880a1310 0x7fad880a37d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.518+0000 7fada3dff700 1 --2- 172.21.15.22:0/1533208929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad9c101fb0 0x7fad9c19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:49:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.518+0000 7fada3dff700 1 -- 172.21.15.22:0/1533208929 >> 172.21.15.22:0/1533208929 conn(0x7fad9c0fb240 msgr2=0x7fad9c1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:49:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.518+0000 7fada3dff700 1 -- 172.21.15.22:0/1533208929 shutdown_connections 2023-09-20T21:49:34.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:49:34.518+0000 7fada3dff700 1 -- 172.21.15.22:0/1533208929 wait complete. 2023-09-20T21:49:34.537 INFO:tasks.cephfs.filesystem:scrub status for tag:94b34214-ef80-4df0-8aab-02c689244dd6 - {'path': '/', 'tag': '94b34214-ef80-4df0-8aab-02c689244dd6', 'options': 'recursive,force'} 2023-09-20T21:49:34.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub queued for path: / 2023-09-20T21:49:34.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:49:35.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:49:35.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:49:35.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:49:35.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:49:35.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:49:35.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:34 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub queued for path: / 2023-09-20T21:49:35.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:49:35.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:49:35.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:49:35.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:49:35.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:49:35.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:49:35.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:34 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub queued for path: / 2023-09-20T21:49:35.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:49:35.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:49:35.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:49:35.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:49:35.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:49:35.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:49:35.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:35.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:34 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:35 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:49:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:35 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:49:36.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:35 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:36.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:35 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:36.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:35 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:36.008 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:35 smithi022 ceph-mon[97651]: pgmap v367: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 47 MiB/s wr, 378 op/s 2023-09-20T21:49:36.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:35 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:49:36.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:35 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:49:36.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:35 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:36.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:35 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:36.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:35 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:36.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:35 smithi123 ceph-mon[134433]: pgmap v367: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 47 MiB/s wr, 378 op/s 2023-09-20T21:49:36.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:35 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:49:36.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:35 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:49:36.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:35 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:36.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:35 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:36.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:35 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:49:36.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:35 smithi089 ceph-mon[134090]: pgmap v367: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 47 MiB/s wr, 378 op/s 2023-09-20T21:49:38.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 15.0 s, 1170.6 tps, lat 82.864 ms stddev 161.632 2023-09-20T21:49:38.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:38 smithi123 ceph-mon[134433]: pgmap v368: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 43 MiB/s wr, 526 op/s 2023-09-20T21:49:38.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:38 smithi089 ceph-mon[134090]: pgmap v368: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 43 MiB/s wr, 526 op/s 2023-09-20T21:49:38.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:38 smithi022 ceph-mon[97651]: pgmap v368: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 43 MiB/s wr, 526 op/s 2023-09-20T21:49:40.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:40 smithi123 ceph-mon[134433]: pgmap v369: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 51 MiB/s wr, 741 op/s 2023-09-20T21:49:40.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:40 smithi089 ceph-mon[134090]: pgmap v369: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 51 MiB/s wr, 741 op/s 2023-09-20T21:49:40.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:40 smithi022 ceph-mon[97651]: pgmap v369: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 51 MiB/s wr, 741 op/s 2023-09-20T21:49:41.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:41 smithi123 ceph-mon[134433]: scrub summary: idle 2023-09-20T21:49:41.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:41 smithi123 ceph-mon[134433]: scrub summary: idle 2023-09-20T21:49:41.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:41 smithi123 ceph-mon[134433]: scrub summary: idle 2023-09-20T21:49:41.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:41 smithi123 ceph-mon[134433]: scrub summary: idle 2023-09-20T21:49:41.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:41 smithi123 ceph-mon[134433]: scrub summary: idle 2023-09-20T21:49:41.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:41 smithi089 ceph-mon[134090]: scrub summary: idle 2023-09-20T21:49:41.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:41 smithi089 ceph-mon[134090]: scrub summary: idle 2023-09-20T21:49:41.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:41 smithi089 ceph-mon[134090]: scrub summary: idle 2023-09-20T21:49:41.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:41 smithi089 ceph-mon[134090]: scrub summary: idle 2023-09-20T21:49:41.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:41 smithi089 ceph-mon[134090]: scrub summary: idle 2023-09-20T21:49:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:41 smithi022 ceph-mon[97651]: scrub summary: idle 2023-09-20T21:49:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:41 smithi022 ceph-mon[97651]: scrub summary: idle 2023-09-20T21:49:41.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:41 smithi022 ceph-mon[97651]: scrub summary: idle 2023-09-20T21:49:41.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:41 smithi022 ceph-mon[97651]: scrub summary: idle 2023-09-20T21:49:41.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:41 smithi022 ceph-mon[97651]: scrub summary: idle 2023-09-20T21:49:42.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:42 smithi123 ceph-mon[134433]: pgmap v370: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 57 MiB/s wr, 907 op/s 2023-09-20T21:49:42.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:42 smithi089 ceph-mon[134090]: pgmap v370: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 57 MiB/s wr, 907 op/s 2023-09-20T21:49:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:42 smithi022 ceph-mon[97651]: pgmap v370: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 57 MiB/s wr, 907 op/s 2023-09-20T21:49:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 20.0 s, 1929.0 tps, lat 53.216 ms stddev 58.973 2023-09-20T21:49:43.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:43 smithi022 ceph-mon[97651]: pgmap v371: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 53 MiB/s wr, 936 op/s 2023-09-20T21:49:44.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:43 smithi123 ceph-mon[134433]: pgmap v371: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 53 MiB/s wr, 936 op/s 2023-09-20T21:49:44.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:43 smithi089 ceph-mon[134090]: pgmap v371: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 53 MiB/s wr, 936 op/s 2023-09-20T21:49:46.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:46 smithi022 ceph-mon[97651]: pgmap v372: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 MiB/s wr, 1.04k op/s 2023-09-20T21:49:46.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:46 smithi123 ceph-mon[134433]: pgmap v372: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 MiB/s wr, 1.04k op/s 2023-09-20T21:49:46.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:46 smithi089 ceph-mon[134090]: pgmap v372: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 MiB/s wr, 1.04k op/s 2023-09-20T21:49:48.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:48 smithi022 ceph-mon[97651]: pgmap v373: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 59 MiB/s wr, 1.11k op/s 2023-09-20T21:49:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 25.0 s, 1810.0 tps, lat 55.546 ms stddev 66.188 2023-09-20T21:49:48.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:48 smithi123 ceph-mon[134433]: pgmap v373: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 59 MiB/s wr, 1.11k op/s 2023-09-20T21:49:48.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:48 smithi089 ceph-mon[134090]: pgmap v373: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 59 MiB/s wr, 1.11k op/s 2023-09-20T21:49:50.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:50 smithi123 ceph-mon[134433]: pgmap v374: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 56 MiB/s wr, 1.18k op/s 2023-09-20T21:49:50.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:50 smithi089 ceph-mon[134090]: pgmap v374: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 56 MiB/s wr, 1.18k op/s 2023-09-20T21:49:50.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:50 smithi022 ceph-mon[97651]: pgmap v374: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 56 MiB/s wr, 1.18k op/s 2023-09-20T21:49:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:52 smithi123 ceph-mon[134433]: pgmap v375: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 MiB/s wr, 1.05k op/s 2023-09-20T21:49:52.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:52 smithi089 ceph-mon[134090]: pgmap v375: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 MiB/s wr, 1.05k op/s 2023-09-20T21:49:52.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:52 smithi022 ceph-mon[97651]: pgmap v375: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 MiB/s wr, 1.05k op/s 2023-09-20T21:49:53.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 30.0 s, 1952.8 tps, lat 51.152 ms stddev 60.431 2023-09-20T21:49:53.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:53 smithi022 ceph-mon[97651]: pgmap v376: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 879 op/s 2023-09-20T21:49:54.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:53 smithi123 ceph-mon[134433]: pgmap v376: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 879 op/s 2023-09-20T21:49:54.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:53 smithi089 ceph-mon[134090]: pgmap v376: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 879 op/s 2023-09-20T21:49:56.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:56 smithi022 ceph-mon[97651]: pgmap v377: 129 pgs: 129 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 MiB/s wr, 914 op/s 2023-09-20T21:49:56.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:56 smithi123 ceph-mon[134433]: pgmap v377: 129 pgs: 129 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 MiB/s wr, 914 op/s 2023-09-20T21:49:56.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:56 smithi089 ceph-mon[134090]: pgmap v377: 129 pgs: 129 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 27 MiB/s wr, 914 op/s 2023-09-20T21:49:58.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:49:58 smithi022 ceph-mon[97651]: pgmap v378: 129 pgs: 129 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 903 op/s 2023-09-20T21:49:58.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 35.0 s, 1896.2 tps, lat 52.593 ms stddev 65.074 2023-09-20T21:49:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:49:58 smithi123 ceph-mon[134433]: pgmap v378: 129 pgs: 129 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 903 op/s 2023-09-20T21:49:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:49:58 smithi089 ceph-mon[134090]: pgmap v378: 129 pgs: 129 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 903 op/s 2023-09-20T21:50:00.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:00 smithi022 ceph-mon[97651]: pgmap v379: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 832 op/s 2023-09-20T21:50:00.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:00 smithi022 ceph-mon[97651]: overall HEALTH_OK 2023-09-20T21:50:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:00 smithi123 ceph-mon[134433]: pgmap v379: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 832 op/s 2023-09-20T21:50:00.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:00 smithi123 ceph-mon[134433]: overall HEALTH_OK 2023-09-20T21:50:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:00 smithi089 ceph-mon[134090]: pgmap v379: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 25 MiB/s wr, 832 op/s 2023-09-20T21:50:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:00 smithi089 ceph-mon[134090]: overall HEALTH_OK 2023-09-20T21:50:02.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:02 smithi123 ceph-mon[134433]: pgmap v380: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 MiB/s wr, 762 op/s 2023-09-20T21:50:02.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:02 smithi089 ceph-mon[134090]: pgmap v380: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 MiB/s wr, 762 op/s 2023-09-20T21:50:02.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:02 smithi022 ceph-mon[97651]: pgmap v380: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 MiB/s wr, 762 op/s 2023-09-20T21:50:03.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 40.0 s, 1982.2 tps, lat 50.381 ms stddev 54.335 2023-09-20T21:50:03.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:03 smithi022 ceph-mon[97651]: pgmap v381: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 679 op/s 2023-09-20T21:50:04.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:03 smithi123 ceph-mon[134433]: pgmap v381: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 679 op/s 2023-09-20T21:50:04.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:03 smithi089 ceph-mon[134090]: pgmap v381: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 MiB/s wr, 679 op/s 2023-09-20T21:50:04.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:50:04.896 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:04 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:50:04.909 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:04 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:50:05.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:04 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:50:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.077+0000 7f1fc67ff700 1 -- 172.21.15.22:0/371858541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc0101850 msgr2=0x7f1fc0103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.077+0000 7f1fc67ff700 1 --2- 172.21.15.22:0/371858541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc0101850 0x7f1fc0103c40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000f490 tx=0x7f1fb000f7a0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.077+0000 7f1fc67ff700 1 -- 172.21.15.22:0/371858541 shutdown_connections 2023-09-20T21:50:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.077+0000 7f1fc67ff700 1 --2- 172.21.15.22:0/371858541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fc0106ab0 0x7f1fc0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.077+0000 7f1fc67ff700 1 --2- 172.21.15.22:0/371858541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc0101850 0x7f1fc0103c40 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000f490 tx=0x7f1fb000f7a0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.077+0000 7f1fc67ff700 1 --2- 172.21.15.22:0/371858541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc0104180 0x7f1fc0106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.077+0000 7f1fc67ff700 1 -- 172.21.15.22:0/371858541 >> 172.21.15.22:0/371858541 conn(0x7f1fc00fb240 msgr2=0x7f1fc00fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.078+0000 7f1fc67ff700 1 -- 172.21.15.22:0/371858541 shutdown_connections 2023-09-20T21:50:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.079+0000 7f1fc67ff700 1 -- 172.21.15.22:0/371858541 wait complete. 2023-09-20T21:50:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.081+0000 7f1fc67ff700 1 Processor -- start 2023-09-20T21:50:05.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.081+0000 7f1fc67ff700 1 -- start start 2023-09-20T21:50:05.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.082+0000 7f1fc67ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc0104180 0x7f1fc0194bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.082+0000 7f1fc67ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fc0106ab0 0x7f1fc0195130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.082+0000 7f1fc67ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc01ab340 0x7f1fc01ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.082+0000 7f1fc67ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc01adc70 con 0x7f1fc01ab340 2023-09-20T21:50:05.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.082+0000 7f1fc67ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc006a800 con 0x7f1fc0104180 2023-09-20T21:50:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.082+0000 7f1fc67ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc006a990 con 0x7f1fc0106ab0 2023-09-20T21:50:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.082+0000 7f1fc4d9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc01ab340 0x7f1fc01ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.082+0000 7f1fc4d9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc01ab340 0x7f1fc01ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47198/0 (socket says 172.21.15.22:47198) 2023-09-20T21:50:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.082+0000 7f1fc4d9c700 1 -- 172.21.15.22:0/4091500442 learned_addr learned my addr 172.21.15.22:0/4091500442 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:05.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.083+0000 7f1fb7fff700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fc0106ab0 0x7f1fc0195130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.084+0000 7f1fc4d9c700 1 -- 172.21.15.22:0/4091500442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fc0106ab0 msgr2=0x7f1fc0195130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.084+0000 7f1fc4d9c700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fc0106ab0 0x7f1fc0195130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.084+0000 7f1fc4d9c700 1 -- 172.21.15.22:0/4091500442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc0104180 msgr2=0x7f1fc0194bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-20T21:50:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.084+0000 7f1fc4d9c700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc0104180 0x7f1fc0194bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.084+0000 7f1fc4d9c700 1 -- 172.21.15.22:0/4091500442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb0009c40 con 0x7f1fc01ab340 2023-09-20T21:50:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.085+0000 7f1fc4d9c700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc01ab340 0x7f1fc01ad730 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800e970 tx=0x7f1fb800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:05.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.086+0000 7f1fbdffb700 1 -- 172.21.15.22:0/4091500442 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8004a30 con 0x7f1fc01ab340 2023-09-20T21:50:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.086+0000 7f1fbdffb700 1 -- 172.21.15.22:0/4091500442 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb800cc80 con 0x7f1fc01ab340 2023-09-20T21:50:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.086+0000 7f1fbdffb700 1 -- 172.21.15.22:0/4091500442 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80054e0 con 0x7f1fc01ab340 2023-09-20T21:50:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.087+0000 7f1fc67ff700 1 -- 172.21.15.22:0/4091500442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc006ac70 con 0x7f1fc01ab340 2023-09-20T21:50:05.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.087+0000 7f1fc67ff700 1 -- 172.21.15.22:0/4091500442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc006b130 con 0x7f1fc01ab340 2023-09-20T21:50:05.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.096+0000 7f1fbdffb700 1 -- 172.21.15.22:0/4091500442 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1fb800c7c0 con 0x7f1fc01ab340 2023-09-20T21:50:05.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.099+0000 7f1fbdffb700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1fac0a1500 0x7f1fac0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.099+0000 7f1fbdffb700 1 -- 172.21.15.22:0/4091500442 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1fb8014070 con 0x7f1fc01ab340 2023-09-20T21:50:05.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.099+0000 7f1fbffff700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1fac0a1500 0x7f1fac0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.103+0000 7f1fda078b80 1 Processor -- start 2023-09-20T21:50:05.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.103+0000 7f1fda078b80 1 -- start start 2023-09-20T21:50:05.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.104+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.104+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 0x55ed8f0c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.104+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.104+0000 7f1fda078b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8f03aca0 con 0x55ed8f0c66f0 2023-09-20T21:50:05.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.104+0000 7f1fda078b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8f01ec70 con 0x55ed8f0c7be0 2023-09-20T21:50:05.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.104+0000 7f1fda078b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8f01c590 con 0x55ed8f0c7050 2023-09-20T21:50:05.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.104+0000 7f1fb7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 0x55ed8f0c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.104+0000 7f1fb7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 0x55ed8f0c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:34792/0 (socket says 172.21.15.22:34792) 2023-09-20T21:50:05.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.104+0000 7f1fb7fff700 1 -- 172.21.15.22:0/2926425934 learned_addr learned my addr 172.21.15.22:0/2926425934 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:05.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.105+0000 7f1fb7fff700 1 -- 172.21.15.22:0/2926425934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c7be0 msgr2=0x55ed8f0c8060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:50:05.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.105+0000 7f1fb7fff700 1 --2- 172.21.15.22:0/2926425934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0c8060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.105+0000 7f1fb7fff700 1 -- 172.21.15.22:0/2926425934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c66f0 msgr2=0x55ed8f0c6b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:50:05.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.105+0000 7f1fb7fff700 1 --2- 172.21.15.22:0/2926425934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0c6b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.105+0000 7f1fb7fff700 1 -- 172.21.15.22:0/2926425934 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed8ee42de0 con 0x55ed8f0c7050 2023-09-20T21:50:05.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.106+0000 7f1fb7fff700 1 --2- 172.21.15.22:0/2926425934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 0x55ed8f0c75e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009af0 tx=0x7f1fa8009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68120c0fd1651df9 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:05.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.107+0000 7f1fb67fc700 1 -- 172.21.15.22:0/2926425934 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80095f0 con 0x55ed8f0c7050 2023-09-20T21:50:05.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.107+0000 7f1fb67fc700 1 -- 172.21.15.22:0/2926425934 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fa800ee50 con 0x55ed8f0c7050 2023-09-20T21:50:05.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.107+0000 7f1fb67fc700 1 -- 172.21.15.22:0/2926425934 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80123d0 con 0x55ed8f0c7050 2023-09-20T21:50:05.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.108+0000 7f1fda078b80 1 -- 172.21.15.22:0/2926425934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 msgr2=0x55ed8f0c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.108+0000 7f1fda078b80 1 --2- 172.21.15.22:0/2926425934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 0x55ed8f0c75e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009af0 tx=0x7f1fa8009e00 comp rx=0 tx=0).stop 2023-09-20T21:50:05.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.109+0000 7f1fda078b80 1 -- 172.21.15.22:0/2926425934 shutdown_connections 2023-09-20T21:50:05.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.109+0000 7f1fda078b80 1 --2- 172.21.15.22:0/2926425934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.109+0000 7f1fda078b80 1 --2- 172.21.15.22:0/2926425934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.109+0000 7f1fda078b80 1 --2- 172.21.15.22:0/2926425934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 0x55ed8f0c75e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.109+0000 7f1fda078b80 1 -- 172.21.15.22:0/2926425934 >> 172.21.15.22:0/2926425934 conn(0x55ed8eff3000 msgr2=0x55ed8f0c11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:05.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.109+0000 7f1fbffff700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1fac0a1500 0x7f1fac0a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009040 tx=0x7f1fb0000f40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:05.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.109+0000 7f1fda078b80 1 -- 172.21.15.22:0/2926425934 shutdown_connections 2023-09-20T21:50:05.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.109+0000 7f1fda078b80 1 -- 172.21.15.22:0/2926425934 wait complete. 2023-09-20T21:50:05.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.110+0000 7f1fda078b80 1 -- start start 2023-09-20T21:50:05.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.110+0000 7f1fda078b80 1 Processor -- start 2023-09-20T21:50:05.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.111+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.111+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.111+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0d7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.111+0000 7f1fda078b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8f017e80 con 0x55ed8f0c7050 2023-09-20T21:50:05.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.111+0000 7f1fda078b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8efb4740 con 0x55ed8f0c7be0 2023-09-20T21:50:05.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.111+0000 7f1fda078b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8efb43e0 con 0x55ed8f0c66f0 2023-09-20T21:50:05.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.111+0000 7f1fda078b80 10 client.0 _ll_register_callbacks cb 0x55ed8eefea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:05.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.112+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.112+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:34798/0 (socket says 172.21.15.22:34798) 2023-09-20T21:50:05.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.112+0000 7f1fbffff700 1 -- 172.21.15.22:0/346811165 learned_addr learned my addr 172.21.15.22:0/346811165 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:05.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.112+0000 7f1fbffff700 1 -- 172.21.15.22:0/346811165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c7be0 msgr2=0x55ed8f0d7810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:50:05.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.112+0000 7f1fbffff700 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0d7810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.112+0000 7f1fbffff700 1 -- 172.21.15.22:0/346811165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7050 msgr2=0x55ed8f0d07d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:50:05.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.112+0000 7f1fbffff700 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d07d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.112+0000 7f1fbffff700 1 -- 172.21.15.22:0/346811165 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa8009750 con 0x55ed8f0c66f0 2023-09-20T21:50:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.113+0000 7f1fbffff700 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d0290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000f490 tx=0x7f1fb000f7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc62fed01d052833 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:05.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.114+0000 7f1f9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:50:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.116+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/346811165 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb000f210 con 0x55ed8f0c66f0 2023-09-20T21:50:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.116+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/346811165 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1fb00050b0 con 0x55ed8f0c66f0 2023-09-20T21:50:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.116+0000 7f1fda078b80 1 -- 172.21.15.22:0/346811165 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed8ef88bb0 con 0x55ed8f0c66f0 2023-09-20T21:50:05.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.117+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/346811165 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb0005270 con 0x55ed8f0c66f0 2023-09-20T21:50:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.118+0000 7f1fda078b80 10 client.25693 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:05.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.119+0000 7f1fda078b80 1 -- 172.21.15.22:0/346811165 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ed8ee2bc60 con 0x55ed8f0c66f0 2023-09-20T21:50:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.129+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/346811165 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed8ef88bb0 con 0x55ed8f0c66f0 2023-09-20T21:50:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.129+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/346811165 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1fb001e070 con 0x55ed8f0c66f0 2023-09-20T21:50:05.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.133+0000 7f1fda078b80 10 client.25693 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.133+0000 7f1fda078b80 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:50:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.133+0000 7f1fda078b80 20 client.25693 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.133+0000 7f1fda078b80 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ed8f0dc110 0x55ed8f0de530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.133+0000 7f1fda078b80 4 client.25693 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:50:05.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.133+0000 7f1fda078b80 1 -- 172.21.15.22:0/346811165 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed8f095020 con 0x55ed8f0dc110 2023-09-20T21:50:05.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.135+0000 7f1fb7fff700 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ed8f0dc110 0x55ed8f0de530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.141+0000 7f1fb7fff700 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ed8f0dc110 0x55ed8f0de530 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e932a3d4539d18 server_cookie=ecbda12ff84fecd9 in_seq=0 out_seq=0 2023-09-20T21:50:05.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.142+0000 7f1f9f7fe700 10 client.25693 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:05.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.142+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/346811165 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ed8f095020 con 0x55ed8f0dc110 2023-09-20T21:50:05.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.142+0000 7f1f9f7fe700 10 client.25693 handle_command_reply: tid=0 2023-09-20T21:50:05.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.143+0000 7f1fda078b80 1 client.25693 shutdown 2023-09-20T21:50:05.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.145+0000 7f1fda078b80 1 -- 172.21.15.22:0/346811165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c66f0 msgr2=0x55ed8f0d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.145+0000 7f1fda078b80 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d0290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000f490 tx=0x7f1fb000f7a0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.148+0000 7f1fda078b80 1 -- 172.21.15.22:0/346811165 shutdown_connections 2023-09-20T21:50:05.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.148+0000 7f1fda078b80 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ed8f0dc110 0x55ed8f0de530 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.148+0000 7f1fda078b80 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0d7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.148+0000 7f1fda078b80 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d0290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.148+0000 7f1fda078b80 1 --2- 172.21.15.22:0/346811165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.148+0000 7f1fda078b80 1 -- 172.21.15.22:0/346811165 >> 172.21.15.22:0/346811165 conn(0x55ed8eff3000 msgr2=0x55ed8f0c0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:05.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.152+0000 7f1f9f7fe700 0 client.25693 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:05.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.154+0000 7f1fda078b80 1 -- 172.21.15.22:0/346811165 shutdown_connections 2023-09-20T21:50:05.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.177+0000 7f1fda078b80 1 -- 172.21.15.22:0/346811165 wait complete. 2023-09-20T21:50:05.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.179+0000 7f1fda078b80 20 client.25693 trim_cache size 0 max 16384 2023-09-20T21:50:05.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.232+0000 7f1fda078b80 1 Processor -- start 2023-09-20T21:50:05.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.236+0000 7f1fda078b80 1 -- start start 2023-09-20T21:50:05.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.236+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.236+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.236+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.236+0000 7f1fda078b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8ee98920 con 0x55ed8f0c7be0 2023-09-20T21:50:05.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.236+0000 7f1fda078b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8f0d9480 con 0x55ed8f0c66f0 2023-09-20T21:50:05.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.236+0000 7f1fda078b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8f0c7a40 con 0x55ed8f0c7050 2023-09-20T21:50:05.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.240+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.240+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:57270/0 (socket says 172.21.15.22:57270) 2023-09-20T21:50:05.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.240+0000 7f1fbffff700 1 -- 172.21.15.22:0/628425055 learned_addr learned my addr 172.21.15.22:0/628425055 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:05.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.241+0000 7f1fc4d9c700 1 --2- 172.21.15.22:0/628425055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.248+0000 7f1fc4d9c700 1 -- 172.21.15.22:0/628425055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 msgr2=0x55ed8f0d77d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:50:05.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.248+0000 7f1fc4d9c700 1 --2- 172.21.15.22:0/628425055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d77d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.248+0000 7f1fc4d9c700 1 -- 172.21.15.22:0/628425055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 msgr2=0x55ed8f0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:05.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.248+0000 7f1fc4d9c700 1 --2- 172.21.15.22:0/628425055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.248+0000 7f1fc4d9c700 1 -- 172.21.15.22:0/628425055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed8ee69250 con 0x55ed8f0c7be0 2023-09-20T21:50:05.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.249+0000 7f1fc4d9c700 1 --2- 172.21.15.22:0/628425055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0cac00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80ca7a0 tx=0x7f1fb80964a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2640eb4a4a37b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.251+0000 7f1fb6ffd700 1 -- 172.21.15.22:0/628425055 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb809a030 con 0x55ed8f0c7be0 2023-09-20T21:50:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.251+0000 7f1fb6ffd700 1 -- 172.21.15.22:0/628425055 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb809de60 con 0x55ed8f0c7be0 2023-09-20T21:50:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.251+0000 7f1fb6ffd700 1 -- 172.21.15.22:0/628425055 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8005640 con 0x55ed8f0c7be0 2023-09-20T21:50:05.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.252+0000 7f1fda078b80 1 -- 172.21.15.22:0/628425055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7be0 msgr2=0x55ed8f0cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.252+0000 7f1fda078b80 1 --2- 172.21.15.22:0/628425055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0cac00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80ca7a0 tx=0x7f1fb80964a0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.252+0000 7f1fda078b80 1 -- 172.21.15.22:0/628425055 shutdown_connections 2023-09-20T21:50:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.252+0000 7f1fda078b80 1 --2- 172.21.15.22:0/628425055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0cac00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.252+0000 7f1fda078b80 1 --2- 172.21.15.22:0/628425055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.252+0000 7f1fda078b80 1 --2- 172.21.15.22:0/628425055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.252+0000 7f1fda078b80 1 -- 172.21.15.22:0/628425055 >> 172.21.15.22:0/628425055 conn(0x55ed8eff3000 msgr2=0x55ed8f0de120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.265+0000 7f1fda078b80 1 -- 172.21.15.22:0/628425055 shutdown_connections 2023-09-20T21:50:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.265+0000 7f1fda078b80 1 -- 172.21.15.22:0/628425055 wait complete. 2023-09-20T21:50:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.266+0000 7f1fda078b80 1 -- start start 2023-09-20T21:50:05.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.266+0000 7f1fda078b80 1 Processor -- start 2023-09-20T21:50:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.267+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.267+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.267+0000 7f1fda078b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0d6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.267+0000 7f1fda078b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8f0d6f40 con 0x55ed8f0c7050 2023-09-20T21:50:05.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.267+0000 7f1fda078b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8f0d70b0 con 0x55ed8f0c66f0 2023-09-20T21:50:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.267+0000 7f1fda078b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed8f0d7270 con 0x55ed8f0c7be0 2023-09-20T21:50:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.267+0000 7f1fda078b80 10 client.0 _ll_register_callbacks cb 0x55ed8eefea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.267+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.268+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:57280/0 (socket says 172.21.15.22:57280) 2023-09-20T21:50:05.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.268+0000 7f1fbffff700 1 -- 172.21.15.22:0/2019006710 learned_addr learned my addr 172.21.15.22:0/2019006710 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:05.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.268+0000 7f1fb7fff700 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.268+0000 7f1fc4d9c700 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0d6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.268+0000 7f1fbffff700 1 -- 172.21.15.22:0/2019006710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7be0 msgr2=0x55ed8f0d6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:05.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.268+0000 7f1fbffff700 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0d6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.268+0000 7f1fbffff700 1 -- 172.21.15.22:0/2019006710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7050 msgr2=0x55ed8f0d59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:05.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.268+0000 7f1fbffff700 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.268+0000 7f1fbffff700 1 -- 172.21.15.22:0/2019006710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc006b130 con 0x55ed8f0c66f0 2023-09-20T21:50:05.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.269+0000 7f1fb7fff700 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:50:05.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.269+0000 7f1fbffff700 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d54b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00060b0 tx=0x7f1fb0022e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd17a14b40621812 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:05.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.269+0000 7f1f9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:50:05.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.269+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/2019006710 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0005820 con 0x55ed8f0c66f0 2023-09-20T21:50:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.269+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/2019006710 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1fb00227f0 con 0x55ed8f0c66f0 2023-09-20T21:50:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.270+0000 7f1fda078b80 1 -- 172.21.15.22:0/2019006710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fa800ee50 con 0x55ed8f0c66f0 2023-09-20T21:50:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.271+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/2019006710 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb0005980 con 0x55ed8f0c66f0 2023-09-20T21:50:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.271+0000 7f1fda078b80 10 client.25709 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:05.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.271+0000 7f1fda078b80 1 -- 172.21.15.22:0/2019006710 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ed8ee412b0 con 0x55ed8f0c66f0 2023-09-20T21:50:05.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.278+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/2019006710 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fa800ee50 con 0x55ed8f0c66f0 2023-09-20T21:50:05.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.278+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/2019006710 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f1fb00075a0 con 0x55ed8f0c66f0 2023-09-20T21:50:05.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.278+0000 7f1fda078b80 10 client.25709 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:05.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.278+0000 7f1fda078b80 10 client.25709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:50:05.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.278+0000 7f1fda078b80 20 client.25709 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:05.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.278+0000 7f1fda078b80 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ed8f130ab0 0x55ed8f132ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.278+0000 7f1fda078b80 4 client.25709 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:50:05.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.278+0000 7f1fda078b80 1 -- 172.21.15.22:0/2019006710 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ed8f010e60 con 0x55ed8f130ab0 2023-09-20T21:50:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.279+0000 7f1fb7fff700 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ed8f130ab0 0x55ed8f132ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.283+0000 7f1f9f7fe700 10 client.25709 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.283+0000 7f1fb7fff700 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ed8f130ab0 0x55ed8f132ed0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4396476beffdf37f server_cookie=552e080cad6e58bb in_seq=0 out_seq=0 2023-09-20T21:50:05.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.284+0000 7f1f9f7fe700 1 -- 172.21.15.22:0/2019006710 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ed8f010e60 con 0x55ed8f130ab0 2023-09-20T21:50:05.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.284+0000 7f1f9f7fe700 10 client.25709 handle_command_reply: tid=0 2023-09-20T21:50:05.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.284+0000 7f1fda078b80 1 client.25709 shutdown 2023-09-20T21:50:05.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.302+0000 7f1fda078b80 1 -- 172.21.15.22:0/2019006710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 msgr2=0x55ed8f0d54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.302+0000 7f1fda078b80 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d54b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00060b0 tx=0x7f1fb0022e30 comp rx=0 tx=0).stop 2023-09-20T21:50:05.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.303+0000 7f1fda078b80 1 -- 172.21.15.22:0/2019006710 shutdown_connections 2023-09-20T21:50:05.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.303+0000 7f1fda078b80 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ed8f130ab0 0x55ed8f132ed0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.303+0000 7f1fda078b80 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed8f0c7be0 0x55ed8f0d6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.303+0000 7f1fda078b80 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed8f0c66f0 0x55ed8f0d54b0 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00060b0 tx=0x7f1fb0022e30 comp rx=0 tx=0).stop 2023-09-20T21:50:05.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.303+0000 7f1fda078b80 1 --2- 172.21.15.22:0/2019006710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ed8f0c7050 0x55ed8f0d59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.303+0000 7f1fda078b80 1 -- 172.21.15.22:0/2019006710 >> 172.21.15.22:0/2019006710 conn(0x55ed8eff3000 msgr2=0x55ed8f0ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:05.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.303+0000 7f1f9f7fe700 0 client.25709 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:05.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.314+0000 7f1fda078b80 1 -- 172.21.15.22:0/2019006710 shutdown_connections 2023-09-20T21:50:05.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.315+0000 7f1fda078b80 1 -- 172.21.15.22:0/2019006710 wait complete. 2023-09-20T21:50:05.316 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:50:05.317 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2023-09-20T21:50:05.317 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2023-09-20T21:50:05.317 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:50:05.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.315+0000 7f1fda078b80 20 client.25709 trim_cache size 0 max 16384 2023-09-20T21:50:05.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.319+0000 7f1fc67ff700 1 -- 172.21.15.22:0/4091500442 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1fac0a1500 msgr2=0x7f1fac0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.319+0000 7f1fc67ff700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1fac0a1500 0x7f1fac0a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009040 tx=0x7f1fb0000f40 comp rx=0 tx=0).stop 2023-09-20T21:50:05.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.319+0000 7f1fc67ff700 1 -- 172.21.15.22:0/4091500442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc01ab340 msgr2=0x7f1fc01ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.319+0000 7f1fc67ff700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc01ab340 0x7f1fc01ad730 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800e970 tx=0x7f1fb800ec80 comp rx=0 tx=0).stop 2023-09-20T21:50:05.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.323+0000 7f1fc67ff700 1 -- 172.21.15.22:0/4091500442 shutdown_connections 2023-09-20T21:50:05.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.323+0000 7f1fc67ff700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1fc01ab340 0x7f1fc01ad730 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.323+0000 7f1fc67ff700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1fac0a1500 0x7f1fac0a39c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.323+0000 7f1fc67ff700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1fc0104180 0x7f1fc0194bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.323+0000 7f1fc67ff700 1 --2- 172.21.15.22:0/4091500442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fc0106ab0 0x7f1fc0195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.323+0000 7f1fc67ff700 1 -- 172.21.15.22:0/4091500442 >> 172.21.15.22:0/4091500442 conn(0x7f1fc00fb240 msgr2=0x7f1fc01024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:05.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.326+0000 7f1fc67ff700 1 -- 172.21.15.22:0/4091500442 shutdown_connections 2023-09-20T21:50:05.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.326+0000 7f1fc67ff700 1 -- 172.21.15.22:0/4091500442 wait complete. 2023-09-20T21:50:05.357 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-20T21:50:05.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T21:50:05.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.777+0000 7fdd98bff700 1 -- 172.21.15.22:0/229406162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd94107de0 msgr2=0x7fdd9410a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.777+0000 7fdd98bff700 1 --2- 172.21.15.22:0/229406162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd94107de0 0x7fdd9410a210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00b0a0 tx=0x7fdd8c00b3b0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.779+0000 7fdd98bff700 1 -- 172.21.15.22:0/229406162 shutdown_connections 2023-09-20T21:50:05.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.779+0000 7fdd98bff700 1 --2- 172.21.15.22:0/229406162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd9410d120 0x7fdd9410d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.779+0000 7fdd98bff700 1 --2- 172.21.15.22:0/229406162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd94107de0 0x7fdd9410a210 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.779+0000 7fdd98bff700 1 --2- 172.21.15.22:0/229406162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9410a750 0x7fdd9410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.779+0000 7fdd98bff700 1 -- 172.21.15.22:0/229406162 >> 172.21.15.22:0/229406162 conn(0x7fdd9406dbe0 msgr2=0x7fdd94070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:05.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.779+0000 7fdd98bff700 1 -- 172.21.15.22:0/229406162 shutdown_connections 2023-09-20T21:50:05.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.779+0000 7fdd98bff700 1 -- 172.21.15.22:0/229406162 wait complete. 2023-09-20T21:50:05.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.781+0000 7fdd98bff700 1 Processor -- start 2023-09-20T21:50:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.781+0000 7fdd98bff700 1 -- start start 2023-09-20T21:50:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.781+0000 7fdd98bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd94107de0 0x7fdd94114380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.781+0000 7fdd98bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9410a750 0x7fdd941148c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.782+0000 7fdd98bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd9410d120 0x7fdd941b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.782+0000 7fdd98bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd941179a0 con 0x7fdd9410d120 2023-09-20T21:50:05.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.782+0000 7fdd98bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd94117b10 con 0x7fdd94107de0 2023-09-20T21:50:05.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.782+0000 7fdd98bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd941b6050 con 0x7fdd9410a750 2023-09-20T21:50:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.783+0000 7fdd92d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd9410d120 0x7fdd941b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.783+0000 7fdd92d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd9410d120 0x7fdd941b5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47238/0 (socket says 172.21.15.22:47238) 2023-09-20T21:50:05.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.783+0000 7fdd92d9d700 1 -- 172.21.15.22:0/945591476 learned_addr learned my addr 172.21.15.22:0/945591476 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:05.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.783+0000 7fdd91d9b700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9410a750 0x7fdd941148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.783+0000 7fdd92d9d700 1 -- 172.21.15.22:0/945591476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9410a750 msgr2=0x7fdd941148c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.783+0000 7fdd92d9d700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9410a750 0x7fdd941148c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.783+0000 7fdd92d9d700 1 -- 172.21.15.22:0/945591476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd94107de0 msgr2=0x7fdd94114380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:05.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.783+0000 7fdd9259c700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd94107de0 0x7fdd94114380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.784+0000 7fdd92d9d700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd94107de0 0x7fdd94114380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.784+0000 7fdd92d9d700 1 -- 172.21.15.22:0/945591476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c009d00 con 0x7fdd9410d120 2023-09-20T21:50:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.784+0000 7fdd91d9b700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9410a750 0x7fdd941148c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:50:05.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.784+0000 7fdd9259c700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd94107de0 0x7fdd94114380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:50:05.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.784+0000 7fdd92d9d700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd9410d120 0x7fdd941b5b10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400e940 tx=0x7fdd8400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:05.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.788+0000 7fdd837fe700 1 -- 172.21.15.22:0/945591476 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd840097b0 con 0x7fdd9410d120 2023-09-20T21:50:05.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.788+0000 7fdd837fe700 1 -- 172.21.15.22:0/945591476 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd8400ee90 con 0x7fdd9410d120 2023-09-20T21:50:05.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.788+0000 7fdd837fe700 1 -- 172.21.15.22:0/945591476 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84005520 con 0x7fdd9410d120 2023-09-20T21:50:05.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.789+0000 7fdd98bff700 1 -- 172.21.15.22:0/945591476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd941b62b0 con 0x7fdd9410d120 2023-09-20T21:50:05.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.789+0000 7fdd98bff700 1 -- 172.21.15.22:0/945591476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd941b67c0 con 0x7fdd9410d120 2023-09-20T21:50:05.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.796+0000 7fdd98bff700 1 -- 172.21.15.22:0/945591476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd94067c40 con 0x7fdd9410d120 2023-09-20T21:50:05.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.797+0000 7fdd837fe700 1 -- 172.21.15.22:0/945591476 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fdd84004750 con 0x7fdd9410d120 2023-09-20T21:50:05.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.800+0000 7fdd837fe700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdd7c0a1500 0x7fdd7c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:05.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.800+0000 7fdd837fe700 1 -- 172.21.15.22:0/945591476 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fdd84014070 con 0x7fdd9410d120 2023-09-20T21:50:05.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.801+0000 7fdd9259c700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdd7c0a1500 0x7fdd7c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:05.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.803+0000 7fdd9259c700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdd7c0a1500 0x7fdd7c0a39c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009610 tx=0x7fdd8c0097e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:05.808+0000 7fdd837fe700 1 -- 172.21.15.22:0/945591476 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdd94067c40 con 0x7fdd9410d120 2023-09-20T21:50:06.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:05 smithi022 ceph-mon[97651]: pgmap v382: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 732 op/s 2023-09-20T21:50:06.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:05 smithi123 ceph-mon[134433]: pgmap v382: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 732 op/s 2023-09-20T21:50:06.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:05 smithi089 ceph-mon[134090]: pgmap v382: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 732 op/s 2023-09-20T21:50:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.312+0000 7fdd98bff700 1 -- 172.21.15.22:0/945591476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd9406f760 con 0x7fdd9410d120 2023-09-20T21:50:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.320+0000 7fdd837fe700 1 -- 172.21.15.22:0/945591476 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11368 (secure 0 0 0) 0x7fdd84096440 con 0x7fdd9410d120 2023-09-20T21:50:06.320 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T21:50:06.321 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T21:49:30.258841+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T21:50:06.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.328+0000 7fdd98bff700 1 -- 172.21.15.22:0/945591476 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdd7c0a1500 msgr2=0x7fdd7c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.328+0000 7fdd98bff700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdd7c0a1500 0x7fdd7c0a39c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009610 tx=0x7fdd8c0097e0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.328+0000 7fdd98bff700 1 -- 172.21.15.22:0/945591476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd9410d120 msgr2=0x7fdd941b5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.328+0000 7fdd98bff700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd9410d120 0x7fdd941b5b10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400e940 tx=0x7fdd8400ec50 comp rx=0 tx=0).stop 2023-09-20T21:50:06.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.329+0000 7fdd98bff700 1 -- 172.21.15.22:0/945591476 shutdown_connections 2023-09-20T21:50:06.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.329+0000 7fdd98bff700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd9410d120 0x7fdd941b5b10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.329+0000 7fdd98bff700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdd7c0a1500 0x7fdd7c0a39c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.329+0000 7fdd98bff700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdd94107de0 0x7fdd94114380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.329+0000 7fdd98bff700 1 --2- 172.21.15.22:0/945591476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9410a750 0x7fdd941148c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.329+0000 7fdd98bff700 1 -- 172.21.15.22:0/945591476 >> 172.21.15.22:0/945591476 conn(0x7fdd9406dbe0 msgr2=0x7fdd9410b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.333+0000 7fdd98bff700 1 -- 172.21.15.22:0/945591476 shutdown_connections 2023-09-20T21:50:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.333+0000 7fdd98bff700 1 -- 172.21.15.22:0/945591476 wait complete. 2023-09-20T21:50:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 30 2023-09-20T21:50:06.369 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-09-20T21:50:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.796+0000 7fc0957ff700 1 -- 172.21.15.22:0/3415780685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0900fec70 msgr2=0x7fc0900ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.796+0000 7fc0957ff700 1 --2- 172.21.15.22:0/3415780685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0900fec70 0x7fc0900ff090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc08001c0d0 tx=0x7fc08001c3e0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.796+0000 7fc0957ff700 1 -- 172.21.15.22:0/3415780685 shutdown_connections 2023-09-20T21:50:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.796+0000 7fc0957ff700 1 --2- 172.21.15.22:0/3415780685 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc090100760 0x7fc090105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.796+0000 7fc0957ff700 1 --2- 172.21.15.22:0/3415780685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0900fec70 0x7fc0900ff090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.796+0000 7fc0957ff700 1 --2- 172.21.15.22:0/3415780685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0900ffda0 0x7fc090100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.796+0000 7fc0957ff700 1 -- 172.21.15.22:0/3415780685 >> 172.21.15.22:0/3415780685 conn(0x7fc0900fa570 msgr2=0x7fc0900fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.796+0000 7fc0957ff700 1 -- 172.21.15.22:0/3415780685 shutdown_connections 2023-09-20T21:50:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.796+0000 7fc0957ff700 1 -- 172.21.15.22:0/3415780685 wait complete. 2023-09-20T21:50:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.798+0000 7fc0957ff700 1 Processor -- start 2023-09-20T21:50:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.799+0000 7fc0957ff700 1 -- start start 2023-09-20T21:50:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc0957ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0900fec70 0x7fc090194880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc0957ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0900ffda0 0x7fc090194dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc0957ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc090100760 0x7fc09019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc0957ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09019c400 con 0x7fc0900fec70 2023-09-20T21:50:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc0957ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09019c540 con 0x7fc0900ffda0 2023-09-20T21:50:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc0957ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09019c6d0 con 0x7fc090100760 2023-09-20T21:50:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc0867fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0900ffda0 0x7fc090194dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc0867fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0900ffda0 0x7fc090194dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42828/0 (socket says 172.21.15.22:42828) 2023-09-20T21:50:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc0867fc700 1 -- 172.21.15.22:0/1509501621 learned_addr learned my addr 172.21.15.22:0/1509501621 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:06.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc08effd700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0900fec70 0x7fc090194880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.802+0000 7fc08f7fe700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc090100760 0x7fc09019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.803+0000 7fc0867fc700 1 -- 172.21.15.22:0/1509501621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc090100760 msgr2=0x7fc09019be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.803+0000 7fc0867fc700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc090100760 0x7fc09019be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.803+0000 7fc0867fc700 1 -- 172.21.15.22:0/1509501621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0900fec70 msgr2=0x7fc090194880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.803+0000 7fc0867fc700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0900fec70 0x7fc090194880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.803+0000 7fc0867fc700 1 -- 172.21.15.22:0/1509501621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078009520 con 0x7fc0900ffda0 2023-09-20T21:50:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.803+0000 7fc0867fc700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0900ffda0 0x7fc090194dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc07800eb30 tx=0x7fc07800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.803+0000 7fc08cff9700 1 -- 172.21.15.22:0/1509501621 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078009880 con 0x7fc0900ffda0 2023-09-20T21:50:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.803+0000 7fc08cff9700 1 -- 172.21.15.22:0/1509501621 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0780044e0 con 0x7fc0900ffda0 2023-09-20T21:50:06.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.803+0000 7fc08cff9700 1 -- 172.21.15.22:0/1509501621 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078010430 con 0x7fc0900ffda0 2023-09-20T21:50:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.804+0000 7fc0957ff700 1 -- 172.21.15.22:0/1509501621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08001ad10 con 0x7fc0900ffda0 2023-09-20T21:50:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.804+0000 7fc0957ff700 1 -- 172.21.15.22:0/1509501621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09019cce0 con 0x7fc0900ffda0 2023-09-20T21:50:06.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.812+0000 7fc0a9147b80 1 Processor -- start 2023-09-20T21:50:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.812+0000 7fc0a9147b80 1 -- start start 2023-09-20T21:50:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.812+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 0x56202f945b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.812+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946050 0x56202f9465e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.812+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f947060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.812+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f8b9ca0 con 0x56202f946be0 2023-09-20T21:50:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.812+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f89dc70 con 0x56202f946050 2023-09-20T21:50:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.812+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f89b590 con 0x56202f9456f0 2023-09-20T21:50:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.813+0000 7fc08effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946050 0x56202f9465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.813+0000 7fc08effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946050 0x56202f9465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42844/0 (socket says 172.21.15.22:42844) 2023-09-20T21:50:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.813+0000 7fc08effd700 1 -- 172.21.15.22:0/1469745563 learned_addr learned my addr 172.21.15.22:0/1469745563 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.813+0000 7fc08f7fe700 1 --2- 172.21.15.22:0/1469745563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 0x56202f945b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.813+0000 7fc08effd700 1 -- 172.21.15.22:0/1469745563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 msgr2=0x56202f945b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.813+0000 7fc08effd700 1 --2- 172.21.15.22:0/1469745563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 0x56202f945b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.813+0000 7fc08effd700 1 -- 172.21.15.22:0/1469745563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 msgr2=0x56202f947060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:50:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.813+0000 7fc08effd700 1 --2- 172.21.15.22:0/1469745563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f947060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.813+0000 7fc08effd700 1 -- 172.21.15.22:0/1469745563 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56202f6c1de0 con 0x56202f946050 2023-09-20T21:50:06.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.813+0000 7fc08effd700 1 --2- 172.21.15.22:0/1469745563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946050 0x56202f9465e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc08001a710 tx=0x7fc08001a450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa2717db50860aae server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.816+0000 7fc0857fa700 1 -- 172.21.15.22:0/1469745563 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08001f050 con 0x56202f946050 2023-09-20T21:50:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.816+0000 7fc0857fa700 1 -- 172.21.15.22:0/1469745563 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08001dd70 con 0x56202f946050 2023-09-20T21:50:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.816+0000 7fc0857fa700 1 -- 172.21.15.22:0/1469745563 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080005430 con 0x56202f946050 2023-09-20T21:50:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.818+0000 7fc0a9147b80 1 -- 172.21.15.22:0/1469745563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946050 msgr2=0x56202f9465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.818+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/1469745563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946050 0x56202f9465e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc08001a710 tx=0x7fc08001a450 comp rx=0 tx=0).stop 2023-09-20T21:50:06.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.819+0000 7fc08cff9700 1 -- 172.21.15.22:0/1509501621 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fc078003ee0 con 0x7fc0900ffda0 2023-09-20T21:50:06.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.823+0000 7fc08cff9700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc07c0a1450 0x7fc07c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.823+0000 7fc08cff9700 1 -- 172.21.15.22:0/1509501621 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc078014070 con 0x7fc0900ffda0 2023-09-20T21:50:06.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.823+0000 7fc0a9147b80 1 -- 172.21.15.22:0/1469745563 shutdown_connections 2023-09-20T21:50:06.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.823+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/1469745563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f947060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.823+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/1469745563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 0x56202f945b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.823+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/1469745563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946050 0x56202f9465e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.823+0000 7fc0a9147b80 1 -- 172.21.15.22:0/1469745563 >> 172.21.15.22:0/1469745563 conn(0x56202f872000 msgr2=0x56202f9401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.823+0000 7fc08f7fe700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc07c0a1450 0x7fc07c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.823+0000 7fc0a9147b80 1 -- 172.21.15.22:0/1469745563 shutdown_connections 2023-09-20T21:50:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.823+0000 7fc0a9147b80 1 -- 172.21.15.22:0/1469745563 wait complete. 2023-09-20T21:50:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.824+0000 7fc0a9147b80 1 -- start start 2023-09-20T21:50:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.825+0000 7fc0a9147b80 1 Processor -- start 2023-09-20T21:50:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.825+0000 7fc08f7fe700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc07c0a1450 0x7fc07c0a3910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56202f9410b0 tx=0x7fc08800c590 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.825+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 0x56202f94f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.825+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946050 0x56202f94f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.825+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946be0 0x56202f9567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.825+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f896e80 con 0x56202f946050 2023-09-20T21:50:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.825+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f833740 con 0x56202f946be0 2023-09-20T21:50:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.825+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f8333e0 con 0x56202f9456f0 2023-09-20T21:50:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.825+0000 7fc0a9147b80 10 client.0 _ll_register_callbacks cb 0x56202f77da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.826+0000 7fc08f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946be0 0x56202f9567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.826+0000 7fc08f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946be0 0x56202f9567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42846/0 (socket says 172.21.15.22:42846) 2023-09-20T21:50:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.826+0000 7fc08f7fe700 1 -- 172.21.15.22:0/208007417 learned_addr learned my addr 172.21.15.22:0/208007417 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.826+0000 7fc08effd700 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 0x56202f94f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.827+0000 7fc08effd700 1 -- 172.21.15.22:0/208007417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946be0 msgr2=0x56202f9567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.827+0000 7fc08effd700 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946be0 0x56202f9567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.827+0000 7fc08effd700 1 -- 172.21.15.22:0/208007417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946050 msgr2=0x56202f94f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.827+0000 7fc08effd700 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946050 0x56202f94f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.827+0000 7fc08effd700 1 -- 172.21.15.22:0/208007417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc090105a30 con 0x56202f9456f0 2023-09-20T21:50:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.829+0000 7fc08effd700 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 0x56202f94f250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x56202f941150 tx=0x7fc080006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aec3465271879b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.829+0000 7fc0767fc700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:50:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.830+0000 7fc0767fc700 1 -- 172.21.15.22:0/208007417 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08001f040 con 0x56202f9456f0 2023-09-20T21:50:06.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.830+0000 7fc0767fc700 1 -- 172.21.15.22:0/208007417 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc080003c70 con 0x56202f9456f0 2023-09-20T21:50:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.830+0000 7fc0a9147b80 1 -- 172.21.15.22:0/208007417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56202f807bb0 con 0x56202f9456f0 2023-09-20T21:50:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.832+0000 7fc0767fc700 1 -- 172.21.15.22:0/208007417 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc080005380 con 0x56202f9456f0 2023-09-20T21:50:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.832+0000 7fc0a9147b80 10 client.25711 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.832+0000 7fc0a9147b80 1 -- 172.21.15.22:0/208007417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56202f6aac60 con 0x56202f9456f0 2023-09-20T21:50:06.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.833+0000 7fc0767fc700 1 -- 172.21.15.22:0/208007417 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56202f807bb0 con 0x56202f9456f0 2023-09-20T21:50:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.833+0000 7fc0767fc700 1 -- 172.21.15.22:0/208007417 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc08001c450 con 0x56202f9456f0 2023-09-20T21:50:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.833+0000 7fc0a9147b80 10 client.25711 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.833+0000 7fc0a9147b80 10 client.25711 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.j 2023-09-20T21:50:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.833+0000 7fc0a9147b80 20 client.25711 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:06.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.833+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x56202f95ab70 0x56202f95cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.833+0000 7fc0a9147b80 4 client.25711 mds_command: new command op to 24473 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:50:06.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.833+0000 7fc0a9147b80 1 -- 172.21.15.22:0/208007417 --> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56202f914020 con 0x56202f95ab70 2023-09-20T21:50:06.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.833+0000 7fc0867fc700 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x56202f95ab70 0x56202f95cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.834+0000 7fc0767fc700 10 client.25711 ms_handle_connect on v2:172.21.15.22:6834/56798476 2023-09-20T21:50:06.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.834+0000 7fc0867fc700 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x56202f95ab70 0x56202f95cf60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce636edd0a7c8713 server_cookie=5347a0a239e8f27f in_seq=0 out_seq=0 2023-09-20T21:50:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.835+0000 7fc0767fc700 1 -- 172.21.15.22:0/208007417 <== mds.4 v2:172.21.15.22:6834/56798476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56202f914020 con 0x56202f95ab70 2023-09-20T21:50:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.835+0000 7fc0767fc700 10 client.25711 handle_command_reply: tid=0 2023-09-20T21:50:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.835+0000 7fc0a9147b80 1 client.25711 shutdown 2023-09-20T21:50:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 1 -- 172.21.15.22:0/208007417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 msgr2=0x56202f94f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 0x56202f94f250 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x56202f941150 tx=0x7fc080006000 comp rx=0 tx=0).stop 2023-09-20T21:50:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 1 -- 172.21.15.22:0/208007417 shutdown_connections 2023-09-20T21:50:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f946be0 0x56202f9567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f9456f0 0x56202f94f250 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x56202f95ab70 0x56202f95cf60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/208007417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946050 0x56202f94f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0767fc700 0 client.25711 ms_handle_reset on v2:172.21.15.22:6834/56798476 2023-09-20T21:50:06.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 1 -- 172.21.15.22:0/208007417 >> 172.21.15.22:0/208007417 conn(0x56202f872000 msgr2=0x56202f93f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:06.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 1 -- 172.21.15.22:0/208007417 shutdown_connections 2023-09-20T21:50:06.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 1 -- 172.21.15.22:0/208007417 wait complete. 2023-09-20T21:50:06.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.836+0000 7fc0a9147b80 20 client.25711 trim_cache size 0 max 16384 2023-09-20T21:50:06.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.851+0000 7fc0a9147b80 1 Processor -- start 2023-09-20T21:50:06.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.851+0000 7fc0a9147b80 1 -- start start 2023-09-20T21:50:06.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.851+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f946fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.851+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 0x56202f956790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f94d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f717920 con 0x56202f946be0 2023-09-20T21:50:06.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f946a40 con 0x56202f9456f0 2023-09-20T21:50:06.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f914020 con 0x56202f946050 2023-09-20T21:50:06.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc08effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f946fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc08f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f94d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc08effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f946fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42848/0 (socket says 172.21.15.22:42848) 2023-09-20T21:50:06.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc08effd700 1 -- 172.21.15.22:0/1113196358 learned_addr learned my addr 172.21.15.22:0/1113196358 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:06.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc0867fc700 1 --2- 172.21.15.22:0/1113196358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 0x56202f956790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc08effd700 1 -- 172.21.15.22:0/1113196358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 msgr2=0x56202f956790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:06.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc08effd700 1 --2- 172.21.15.22:0/1113196358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 0x56202f956790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc08effd700 1 -- 172.21.15.22:0/1113196358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 msgr2=0x56202f94d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:06.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc08effd700 1 --2- 172.21.15.22:0/1113196358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f94d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc08effd700 1 -- 172.21.15.22:0/1113196358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56202f6e8250 con 0x56202f9456f0 2023-09-20T21:50:06.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc08f7fe700 1 --2- 172.21.15.22:0/1113196358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f94d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:50:06.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.852+0000 7fc0867fc700 1 --2- 172.21.15.22:0/1113196358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 0x56202f956790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:50:06.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.853+0000 7fc08effd700 1 --2- 172.21.15.22:0/1113196358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f946fc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc080018b50 tx=0x7fc0800060a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cda77b8526a226e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:06.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.853+0000 7fc085ffb700 1 -- 172.21.15.22:0/1113196358 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08001f040 con 0x56202f9456f0 2023-09-20T21:50:06.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.853+0000 7fc085ffb700 1 -- 172.21.15.22:0/1113196358 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08001db70 con 0x56202f9456f0 2023-09-20T21:50:06.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.853+0000 7fc085ffb700 1 -- 172.21.15.22:0/1113196358 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080005380 con 0x56202f9456f0 2023-09-20T21:50:06.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.853+0000 7fc0a9147b80 1 -- 172.21.15.22:0/1113196358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 msgr2=0x56202f946fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.853+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/1113196358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f946fc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc080018b50 tx=0x7fc0800060a0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.854+0000 7fc0a9147b80 1 -- 172.21.15.22:0/1113196358 shutdown_connections 2023-09-20T21:50:06.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.854+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/1113196358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f94d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.854+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/1113196358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f946fc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.854+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/1113196358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 0x56202f956790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.854+0000 7fc0a9147b80 1 -- 172.21.15.22:0/1113196358 >> 172.21.15.22:0/1113196358 conn(0x56202f872000 msgr2=0x56202f94de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:06.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.854+0000 7fc0a9147b80 1 -- 172.21.15.22:0/1113196358 shutdown_connections 2023-09-20T21:50:06.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.854+0000 7fc0a9147b80 1 -- 172.21.15.22:0/1113196358 wait complete. 2023-09-20T21:50:06.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.854+0000 7fc0a9147b80 1 -- start start 2023-09-20T21:50:06.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.854+0000 7fc0a9147b80 1 Processor -- start 2023-09-20T21:50:06.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.855+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f9544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.855+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 0x56202f954a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.855+0000 7fc0a9147b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f955a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.855+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f955f70 con 0x56202f946be0 2023-09-20T21:50:06.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.855+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f9560e0 con 0x56202f9456f0 2023-09-20T21:50:06.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.855+0000 7fc0a9147b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f9562a0 con 0x56202f946050 2023-09-20T21:50:06.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.855+0000 7fc0a9147b80 10 client.0 _ll_register_callbacks cb 0x56202f77da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:06.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.855+0000 7fc08effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f9544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.856+0000 7fc08effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f9544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42850/0 (socket says 172.21.15.22:42850) 2023-09-20T21:50:06.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.856+0000 7fc08effd700 1 -- 172.21.15.22:0/3055813485 learned_addr learned my addr 172.21.15.22:0/3055813485 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:06.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.856+0000 7fc0867fc700 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 0x56202f954a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.856+0000 7fc08effd700 1 -- 172.21.15.22:0/3055813485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 msgr2=0x56202f954a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:06.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.856+0000 7fc08effd700 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 0x56202f954a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.856+0000 7fc08effd700 1 -- 172.21.15.22:0/3055813485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 msgr2=0x56202f955a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:06.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.856+0000 7fc08effd700 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f955a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.856+0000 7fc08effd700 1 -- 172.21.15.22:0/3055813485 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56202f6aac60 con 0x56202f9456f0 2023-09-20T21:50:06.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.856+0000 7fc08effd700 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f9544e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc08004d040 tx=0x7fc0800356e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b44fae2d652c67cc server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:06.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.856+0000 7fc0767fc700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:50:06.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.858+0000 7fc0767fc700 1 -- 172.21.15.22:0/3055813485 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08001f070 con 0x56202f9456f0 2023-09-20T21:50:06.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.858+0000 7fc0767fc700 1 -- 172.21.15.22:0/3055813485 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc080005580 con 0x56202f9456f0 2023-09-20T21:50:06.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.858+0000 7fc0a9147b80 1 -- 172.21.15.22:0/3055813485 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc08001dd70 con 0x56202f9456f0 2023-09-20T21:50:06.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0767fc700 1 -- 172.21.15.22:0/3055813485 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc080035cb0 con 0x56202f9456f0 2023-09-20T21:50:06.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0a9147b80 10 client.25730 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:06.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0a9147b80 1 -- 172.21.15.22:0/3055813485 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56202f6c02b0 con 0x56202f9456f0 2023-09-20T21:50:06.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0767fc700 1 -- 172.21.15.22:0/3055813485 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc08001dd70 con 0x56202f9456f0 2023-09-20T21:50:06.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0767fc700 1 -- 172.21.15.22:0/3055813485 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fc08001c100 con 0x56202f9456f0 2023-09-20T21:50:06.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0a9147b80 10 client.25730 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:06.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0a9147b80 10 client.25730 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.j 2023-09-20T21:50:06.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0a9147b80 20 client.25730 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:06.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x56202f9af3c0 0x56202f9b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:06.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0a9147b80 4 client.25730 mds_command: new command op to 24473 tid=0[{"prefix": "damage ls"}] 2023-09-20T21:50:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.859+0000 7fc0a9147b80 1 -- 172.21.15.22:0/3055813485 --> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56202f88fe60 con 0x56202f9af3c0 2023-09-20T21:50:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.860+0000 7fc0867fc700 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x56202f9af3c0 0x56202f9b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.860+0000 7fc0867fc700 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x56202f9af3c0 0x56202f9b17e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f9ac5fcae0dc63eb server_cookie=1c101130557127f5 in_seq=0 out_seq=0 2023-09-20T21:50:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.860+0000 7fc0767fc700 10 client.25730 ms_handle_connect on v2:172.21.15.22:6834/56798476 2023-09-20T21:50:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.860+0000 7fc0767fc700 1 -- 172.21.15.22:0/3055813485 <== mds.4 v2:172.21.15.22:6834/56798476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56202f88fe60 con 0x56202f9af3c0 2023-09-20T21:50:06.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.860+0000 7fc0767fc700 10 client.25730 handle_command_reply: tid=0 2023-09-20T21:50:06.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 client.25730 shutdown 2023-09-20T21:50:06.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 -- 172.21.15.22:0/3055813485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 msgr2=0x56202f9544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f9544e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc08004d040 tx=0x7fc0800356e0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 -- 172.21.15.22:0/3055813485 shutdown_connections 2023-09-20T21:50:06.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.22:6834/56798476,v1:172.21.15.22:6835/56798476] conn(0x56202f9af3c0 0x56202f9b17e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56202f946be0 0x56202f955a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0767fc700 0 client.25730 ms_handle_reset on v2:172.21.15.22:6834/56798476 2023-09-20T21:50:06.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56202f9456f0 0x56202f9544e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 --2- 172.21.15.22:0/3055813485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56202f946050 0x56202f954a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 -- 172.21.15.22:0/3055813485 >> 172.21.15.22:0/3055813485 conn(0x56202f872000 msgr2=0x56202f94d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:06.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 -- 172.21.15.22:0/3055813485 shutdown_connections 2023-09-20T21:50:06.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 1 -- 172.21.15.22:0/3055813485 wait complete. 2023-09-20T21:50:06.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.861+0000 7fc0a9147b80 20 client.25730 trim_cache size 0 max 16384 2023-09-20T21:50:06.873 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2023-09-20T21:50:06.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 -- 172.21.15.22:0/1509501621 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc07c0a1450 msgr2=0x7fc07c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc07c0a1450 0x7fc07c0a3910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x56202f9410b0 tx=0x7fc08800c590 comp rx=0 tx=0).stop 2023-09-20T21:50:06.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 -- 172.21.15.22:0/1509501621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0900ffda0 msgr2=0x7fc090194dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:06.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0900ffda0 0x7fc090194dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc07800eb30 tx=0x7fc07800eef0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 -- 172.21.15.22:0/1509501621 shutdown_connections 2023-09-20T21:50:06.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc090100760 0x7fc09019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0900fec70 0x7fc090194880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc07c0a1450 0x7fc07c0a3910 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 --2- 172.21.15.22:0/1509501621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc0900ffda0 0x7fc090194dc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:06.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 -- 172.21.15.22:0/1509501621 >> 172.21.15.22:0/1509501621 conn(0x7fc0900fa570 msgr2=0x7fc0900fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:06.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 -- 172.21.15.22:0/1509501621 shutdown_connections 2023-09-20T21:50:06.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:06.862+0000 7fc0957ff700 1 -- 172.21.15.22:0/1509501621 wait complete. 2023-09-20T21:50:06.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-09-20T21:50:07.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:06 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:50:07.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:06 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:50:07.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:06 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/945591476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:50:07.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.164+0000 7f6056fff700 1 -- 172.21.15.22:0/282558668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6050107de0 msgr2=0x7f605010a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.164+0000 7f6056fff700 1 --2- 172.21.15.22:0/282558668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6050107de0 0x7f605010a210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6040009990 tx=0x7f6040009ca0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.164+0000 7f6056fff700 1 -- 172.21.15.22:0/282558668 shutdown_connections 2023-09-20T21:50:07.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.164+0000 7f6056fff700 1 --2- 172.21.15.22:0/282558668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f605010d120 0x7f605010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.164+0000 7f6056fff700 1 --2- 172.21.15.22:0/282558668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6050107de0 0x7f605010a210 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6040009990 tx=0x7f6040009ca0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.164+0000 7f6056fff700 1 --2- 172.21.15.22:0/282558668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f605010a750 0x7f605010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.164+0000 7f6056fff700 1 -- 172.21.15.22:0/282558668 >> 172.21.15.22:0/282558668 conn(0x7f605006db90 msgr2=0x7f605006ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.187+0000 7f6056fff700 1 -- 172.21.15.22:0/282558668 shutdown_connections 2023-09-20T21:50:07.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.192+0000 7f6056fff700 1 -- 172.21.15.22:0/282558668 wait complete. 2023-09-20T21:50:07.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.193+0000 7f6056fff700 1 Processor -- start 2023-09-20T21:50:07.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.200+0000 7f6056fff700 1 -- start start 2023-09-20T21:50:07.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.201+0000 7f6056fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f605010d120 0x7f60501146b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.201+0000 7f6056fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6050114bf0 0x7f60501b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.201+0000 7f6056fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6050115070 0x7f60501b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.201+0000 7f6056fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60501b67e0 con 0x7f605010d120 2023-09-20T21:50:07.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.201+0000 7f6056fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60501b6950 con 0x7f6050115070 2023-09-20T21:50:07.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.201+0000 7f6056fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60501b6b10 con 0x7f6050114bf0 2023-09-20T21:50:07.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.202+0000 7f6054d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f605010d120 0x7f60501146b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.202+0000 7f6054d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f605010d120 0x7f60501146b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40776/0 (socket says 172.21.15.22:40776) 2023-09-20T21:50:07.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.202+0000 7f6054d9b700 1 -- 172.21.15.22:0/3329179997 learned_addr learned my addr 172.21.15.22:0/3329179997 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:07.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.203+0000 7f6054d9b700 1 -- 172.21.15.22:0/3329179997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6050114bf0 msgr2=0x7f60501b3b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:50:07.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.203+0000 7f6054d9b700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6050114bf0 0x7f60501b3b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.203+0000 7f6054d9b700 1 -- 172.21.15.22:0/3329179997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6050115070 msgr2=0x7f60501b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.203+0000 7f6054d9b700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6050115070 0x7f60501b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.203+0000 7f6054d9b700 1 -- 172.21.15.22:0/3329179997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400095f0 con 0x7f605010d120 2023-09-20T21:50:07.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.210+0000 7f6054d9b700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f605010d120 0x7f60501146b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f604000b2e0 tx=0x7f60400057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.218+0000 7f604dffb700 1 -- 172.21.15.22:0/3329179997 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604001d070 con 0x7f605010d120 2023-09-20T21:50:07.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.218+0000 7f604dffb700 1 -- 172.21.15.22:0/3329179997 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f604000f460 con 0x7f605010d120 2023-09-20T21:50:07.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.218+0000 7f604dffb700 1 -- 172.21.15.22:0/3329179997 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040021670 con 0x7f605010d120 2023-09-20T21:50:07.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.224+0000 7f6056fff700 1 -- 172.21.15.22:0/3329179997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60501b6d90 con 0x7f605010d120 2023-09-20T21:50:07.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.224+0000 7f6056fff700 1 -- 172.21.15.22:0/3329179997 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60501b7160 con 0x7f605010d120 2023-09-20T21:50:07.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.230+0000 7f606a8ddb80 1 Processor -- start 2023-09-20T21:50:07.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.234+0000 7f606a8ddb80 1 -- start start 2023-09-20T21:50:07.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.234+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929432b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.234+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x5589294335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.234+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 0x558929434060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.234+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589293a6ca0 con 0x558929433050 2023-09-20T21:50:07.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.234+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892938ac70 con 0x558929433be0 2023-09-20T21:50:07.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.234+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929388590 con 0x5589294326f0 2023-09-20T21:50:07.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.241+0000 7f604ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 0x558929434060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.241+0000 7f604ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 0x558929434060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42874/0 (socket says 172.21.15.22:42874) 2023-09-20T21:50:07.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.241+0000 7f604ffff700 1 -- 172.21.15.22:0/2024213683 learned_addr learned my addr 172.21.15.22:0/2024213683 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:07.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.242+0000 7f605559c700 1 --2- 172.21.15.22:0/2024213683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929432b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:06 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:50:07.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:06 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:50:07.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:06 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/945591476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:50:07.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.248+0000 7f6054d9b700 1 --2- 172.21.15.22:0/2024213683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x5589294335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.250+0000 7f604ffff700 1 -- 172.21.15.22:0/2024213683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 msgr2=0x558929432b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.250+0000 7f604ffff700 1 --2- 172.21.15.22:0/2024213683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929432b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.250+0000 7f604ffff700 1 -- 172.21.15.22:0/2024213683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 msgr2=0x5589294335e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.250+0000 7f604ffff700 1 --2- 172.21.15.22:0/2024213683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x5589294335e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.250+0000 7f604ffff700 1 -- 172.21.15.22:0/2024213683 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589291aede0 con 0x558929433be0 2023-09-20T21:50:07.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.257+0000 7f604dffb700 1 -- 172.21.15.22:0/3329179997 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f604000f5d0 con 0x7f605010d120 2023-09-20T21:50:07.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.259+0000 7f604dffb700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60340a1500 0x7f60340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.259+0000 7f604dffb700 1 -- 172.21.15.22:0/3329179997 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f60400d37b0 con 0x7f605010d120 2023-09-20T21:50:07.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.262+0000 7f604ffff700 1 --2- 172.21.15.22:0/2024213683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 0x558929434060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f603c009760 tx=0x7f603c009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db62feb212f668a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.264+0000 7f605559c700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60340a1500 0x7f60340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.265+0000 7f6046ffd700 1 -- 172.21.15.22:0/2024213683 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c011830 con 0x558929433be0 2023-09-20T21:50:07.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.265+0000 7f6046ffd700 1 -- 172.21.15.22:0/2024213683 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f603c011e50 con 0x558929433be0 2023-09-20T21:50:07.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.265+0000 7f6046ffd700 1 -- 172.21.15.22:0/2024213683 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c00f2e0 con 0x558929433be0 2023-09-20T21:50:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.266+0000 7f606a8ddb80 1 -- 172.21.15.22:0/2024213683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 msgr2=0x558929434060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.266+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/2024213683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 0x558929434060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f603c009760 tx=0x7f603c009a70 comp rx=0 tx=0).stop 2023-09-20T21:50:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.266+0000 7f605559c700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60340a1500 0x7f60340a39c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6048009fd0 tx=0x7f604800b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.267+0000 7f606a8ddb80 1 -- 172.21.15.22:0/2024213683 shutdown_connections 2023-09-20T21:50:07.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.267+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/2024213683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 0x558929434060 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.267+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/2024213683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929432b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.267+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/2024213683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x5589294335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.267+0000 7f606a8ddb80 1 -- 172.21.15.22:0/2024213683 >> 172.21.15.22:0/2024213683 conn(0x55892935f000 msgr2=0x55892942d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.269+0000 7f606a8ddb80 1 -- 172.21.15.22:0/2024213683 shutdown_connections 2023-09-20T21:50:07.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.269+0000 7f606a8ddb80 1 -- 172.21.15.22:0/2024213683 wait complete. 2023-09-20T21:50:07.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.270+0000 7f606a8ddb80 1 -- start start 2023-09-20T21:50:07.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.270+0000 7f606a8ddb80 1 Processor -- start 2023-09-20T21:50:07.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.277+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589294326f0 0x55892943c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.277+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x55892943c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.277+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558929433be0 0x5589294437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.277+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929383e80 con 0x558929433050 2023-09-20T21:50:07.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.277+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929320740 con 0x5589294326f0 2023-09-20T21:50:07.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.277+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589293203e0 con 0x558929433be0 2023-09-20T21:50:07.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.277+0000 7f606a8ddb80 10 client.0 _ll_register_callbacks cb 0x55892926aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:07.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.278+0000 7f604ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x55892943c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.278+0000 7f604ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x55892943c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40784/0 (socket says 172.21.15.22:40784) 2023-09-20T21:50:07.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.278+0000 7f604ffff700 1 -- 172.21.15.22:0/3867038285 learned_addr learned my addr 172.21.15.22:0/3867038285 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:07.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.278+0000 7f6054d9b700 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589294326f0 0x55892943c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.279+0000 7f605559c700 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558929433be0 0x5589294437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.279+0000 7f6054d9b700 1 -- 172.21.15.22:0/3867038285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558929433be0 msgr2=0x5589294437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.279+0000 7f6054d9b700 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558929433be0 0x5589294437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.279+0000 7f6054d9b700 1 -- 172.21.15.22:0/3867038285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 msgr2=0x55892943c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.279+0000 7f6054d9b700 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x55892943c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.279+0000 7f6054d9b700 1 -- 172.21.15.22:0/3867038285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c009390 con 0x5589294326f0 2023-09-20T21:50:07.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.279+0000 7f604ffff700 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x55892943c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:50:07.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.287+0000 7f6054d9b700 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589294326f0 0x55892943c230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f604000fc90 tx=0x7f604000fb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b8b0ec41ee9b4b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.287+0000 7f6033fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:50:07.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.287+0000 7f6033fff700 1 -- 172.21.15.22:0/3867038285 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60400a4320 con 0x5589294326f0 2023-09-20T21:50:07.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.287+0000 7f6033fff700 1 -- 172.21.15.22:0/3867038285 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f60400a0e50 con 0x5589294326f0 2023-09-20T21:50:07.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.287+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3867038285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589292f4bb0 con 0x5589294326f0 2023-09-20T21:50:07.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.288+0000 7f6033fff700 1 -- 172.21.15.22:0/3867038285 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60400a0750 con 0x5589294326f0 2023-09-20T21:50:07.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.288+0000 7f606a8ddb80 10 client.25739 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:07.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.288+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3867038285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558929197c60 con 0x5589294326f0 2023-09-20T21:50:07.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.289+0000 7f6033fff700 1 -- 172.21.15.22:0/3867038285 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589292f4bb0 con 0x5589294326f0 2023-09-20T21:50:07.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.289+0000 7f6033fff700 1 -- 172.21.15.22:0/3867038285 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f60400a8330 con 0x5589294326f0 2023-09-20T21:50:07.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.289+0000 7f606a8ddb80 10 client.25739 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:07.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.289+0000 7f606a8ddb80 10 client.25739 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-09-20T21:50:07.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.289+0000 7f606a8ddb80 20 client.25739 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:07.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.289+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x558929447bf0 0x558929449fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.289+0000 7f606a8ddb80 4 client.25739 mds_command: new command op to 24487 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:50:07.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.289+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3867038285 --> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558929401020 con 0x558929447bf0 2023-09-20T21:50:07.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.292+0000 7f604ffff700 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x558929447bf0 0x558929449fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.295+0000 7f604ffff700 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x558929447bf0 0x558929449fe0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e05bd36657bc05c1 server_cookie=b6c2ed5293cc9a01 in_seq=0 out_seq=0 2023-09-20T21:50:07.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.295+0000 7f6033fff700 10 client.25739 ms_handle_connect on v2:172.21.15.123:6834/835271994 2023-09-20T21:50:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.296+0000 7f6033fff700 1 -- 172.21.15.22:0/3867038285 <== mds.3 v2:172.21.15.123:6834/835271994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558929401020 con 0x558929447bf0 2023-09-20T21:50:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.296+0000 7f6033fff700 10 client.25739 handle_command_reply: tid=0 2023-09-20T21:50:07.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.297+0000 7f606a8ddb80 1 client.25739 shutdown 2023-09-20T21:50:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.297+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3867038285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589294326f0 msgr2=0x55892943c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.297+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589294326f0 0x55892943c230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f604000fc90 tx=0x7f604000fb90 comp rx=0 tx=0).stop 2023-09-20T21:50:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.304+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3867038285 shutdown_connections 2023-09-20T21:50:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.304+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x558929447bf0 0x558929449fe0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.304+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558929433be0 0x5589294437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.304+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5589294326f0 0x55892943c230 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.304+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/3867038285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x55892943c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.304+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3867038285 >> 172.21.15.22:0/3867038285 conn(0x55892935f000 msgr2=0x55892942c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.305+0000 7f6033fff700 0 client.25739 ms_handle_reset on v2:172.21.15.123:6834/835271994 2023-09-20T21:50:07.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.307+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3867038285 shutdown_connections 2023-09-20T21:50:07.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.307+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3867038285 wait complete. 2023-09-20T21:50:07.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.309+0000 7f606a8ddb80 20 client.25739 trim_cache size 0 max 16384 2023-09-20T21:50:07.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.336+0000 7f606a8ddb80 1 Processor -- start 2023-09-20T21:50:07.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.337+0000 7f606a8ddb80 1 -- start start 2023-09-20T21:50:07.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.337+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929433fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.337+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x558929443770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.337+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 0x55892943a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.337+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892943be00 con 0x558929433050 2023-09-20T21:50:07.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.337+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929204920 con 0x558929433be0 2023-09-20T21:50:07.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.337+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892943b6e0 con 0x5589294326f0 2023-09-20T21:50:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.338+0000 7f6054d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929433fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.338+0000 7f6054d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929433fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:55182/0 (socket says 172.21.15.22:55182) 2023-09-20T21:50:07.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.338+0000 7f6054d9b700 1 -- 172.21.15.22:0/3649603839 learned_addr learned my addr 172.21.15.22:0/3649603839 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.338+0000 7f6054d9b700 1 -- 172.21.15.22:0/3649603839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 msgr2=0x55892943a070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:50:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.338+0000 7f6054d9b700 1 --2- 172.21.15.22:0/3649603839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 0x55892943a070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.338+0000 7f6054d9b700 1 -- 172.21.15.22:0/3649603839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 msgr2=0x558929443770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.338+0000 7f6054d9b700 1 --2- 172.21.15.22:0/3649603839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x558929443770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.338+0000 7f6054d9b700 1 -- 172.21.15.22:0/3649603839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589291d5250 con 0x5589294326f0 2023-09-20T21:50:07.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.338+0000 7f6054d9b700 1 --2- 172.21.15.22:0/3649603839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929433fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60400a06e0 tx=0x7f60400a0d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7ffab42e48d83e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.339+0000 7f60477fe700 1 -- 172.21.15.22:0/3649603839 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60400d0e00 con 0x5589294326f0 2023-09-20T21:50:07.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.339+0000 7f60477fe700 1 -- 172.21.15.22:0/3649603839 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60400a0e50 con 0x5589294326f0 2023-09-20T21:50:07.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.339+0000 7f60477fe700 1 -- 172.21.15.22:0/3649603839 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60400a6a30 con 0x5589294326f0 2023-09-20T21:50:07.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.340+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3649603839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 msgr2=0x558929433fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.340+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/3649603839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929433fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60400a06e0 tx=0x7f60400a0d10 comp rx=0 tx=0).stop 2023-09-20T21:50:07.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.340+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3649603839 shutdown_connections 2023-09-20T21:50:07.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.340+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/3649603839 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433be0 0x55892943a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.340+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/3649603839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929433fc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.340+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/3649603839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433050 0x558929443770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.340+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3649603839 >> 172.21.15.22:0/3649603839 conn(0x55892935f000 msgr2=0x5589294440a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.340+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3649603839 shutdown_connections 2023-09-20T21:50:07.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.340+0000 7f606a8ddb80 1 -- 172.21.15.22:0/3649603839 wait complete. 2023-09-20T21:50:07.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.340+0000 7f606a8ddb80 1 -- start start 2023-09-20T21:50:07.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.341+0000 7f606a8ddb80 1 Processor -- start 2023-09-20T21:50:07.348 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:06 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:50:07.348 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:06 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:50:07.348 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:06 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/945591476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T21:50:07.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.343+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929440f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.343+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433050 0x5589294414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.343+0000 7f606a8ddb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433be0 0x5589294424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.343+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589294429f0 con 0x558929433be0 2023-09-20T21:50:07.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.343+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929442b60 con 0x558929433050 2023-09-20T21:50:07.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.343+0000 7f606a8ddb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558929442d20 con 0x5589294326f0 2023-09-20T21:50:07.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.343+0000 7f606a8ddb80 10 client.0 _ll_register_callbacks cb 0x55892926aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:07.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.344+0000 7f604ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433050 0x5589294414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.344+0000 7f604ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433050 0x5589294414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42898/0 (socket says 172.21.15.22:42898) 2023-09-20T21:50:07.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.344+0000 7f604ffff700 1 -- 172.21.15.22:0/682352926 learned_addr learned my addr 172.21.15.22:0/682352926 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:07.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.345+0000 7f6054d9b700 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929440f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.345+0000 7f605559c700 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433be0 0x5589294424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.346+0000 7f6054d9b700 1 -- 172.21.15.22:0/682352926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433050 msgr2=0x5589294414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.346+0000 7f6054d9b700 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433050 0x5589294414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.346+0000 7f6054d9b700 1 -- 172.21.15.22:0/682352926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433be0 msgr2=0x5589294424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.346+0000 7f6054d9b700 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433be0 0x5589294424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.346+0000 7f6054d9b700 1 -- 172.21.15.22:0/682352926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558929197c60 con 0x5589294326f0 2023-09-20T21:50:07.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.347+0000 7f6054d9b700 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929440f60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60400a06e0 tx=0x7f604009c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=640b320ea5db928e server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.347+0000 7f6033fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:50:07.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.347+0000 7f6033fff700 1 -- 172.21.15.22:0/682352926 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60400a4320 con 0x5589294326f0 2023-09-20T21:50:07.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.347+0000 7f6033fff700 1 -- 172.21.15.22:0/682352926 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f60400a87c0 con 0x5589294326f0 2023-09-20T21:50:07.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.347+0000 7f606a8ddb80 1 -- 172.21.15.22:0/682352926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f603c011e50 con 0x5589294326f0 2023-09-20T21:50:07.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.350+0000 7f6033fff700 1 -- 172.21.15.22:0/682352926 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6040089b40 con 0x5589294326f0 2023-09-20T21:50:07.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.351+0000 7f606a8ddb80 10 client.25729 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:07.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.351+0000 7f606a8ddb80 1 -- 172.21.15.22:0/682352926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5589291ad2b0 con 0x5589294326f0 2023-09-20T21:50:07.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.355+0000 7f6033fff700 1 -- 172.21.15.22:0/682352926 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f603c011e50 con 0x5589294326f0 2023-09-20T21:50:07.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.355+0000 7f6033fff700 1 -- 172.21.15.22:0/682352926 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f604000ba70 con 0x5589294326f0 2023-09-20T21:50:07.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.356+0000 7f606a8ddb80 10 client.25729 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:07.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.356+0000 7f606a8ddb80 10 client.25729 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-09-20T21:50:07.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.356+0000 7f606a8ddb80 20 client.25729 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:07.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.356+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55892949bd60 0x55892949e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.356+0000 7f606a8ddb80 4 client.25729 mds_command: new command op to 24487 tid=0[{"prefix": "damage ls"}] 2023-09-20T21:50:07.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.356+0000 7f606a8ddb80 1 -- 172.21.15.22:0/682352926 --> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55892937ce60 con 0x55892949bd60 2023-09-20T21:50:07.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.356+0000 7f604ffff700 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55892949bd60 0x55892949e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.357+0000 7f604ffff700 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55892949bd60 0x55892949e150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=616bd101d51fdb1b server_cookie=859f3fcb1dafa649 in_seq=0 out_seq=0 2023-09-20T21:50:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.357+0000 7f6033fff700 10 client.25729 ms_handle_connect on v2:172.21.15.123:6834/835271994 2023-09-20T21:50:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.358+0000 7f6033fff700 1 -- 172.21.15.22:0/682352926 <== mds.3 v2:172.21.15.123:6834/835271994 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55892937ce60 con 0x55892949bd60 2023-09-20T21:50:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.358+0000 7f6033fff700 10 client.25729 handle_command_reply: tid=0 2023-09-20T21:50:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.358+0000 7f606a8ddb80 1 client.25729 shutdown 2023-09-20T21:50:07.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.358+0000 7f606a8ddb80 1 -- 172.21.15.22:0/682352926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 msgr2=0x558929440f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.358+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929440f60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60400a06e0 tx=0x7f604009c9b0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.363+0000 7f606a8ddb80 1 -- 172.21.15.22:0/682352926 shutdown_connections 2023-09-20T21:50:07.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.363+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558929433be0 0x5589294424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.363+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.123:6834/835271994,v1:172.21.15.123:6835/835271994] conn(0x55892949bd60 0x55892949e150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.363+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5589294326f0 0x558929440f60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.363+0000 7f606a8ddb80 1 --2- 172.21.15.22:0/682352926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558929433050 0x5589294414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.363+0000 7f606a8ddb80 1 -- 172.21.15.22:0/682352926 >> 172.21.15.22:0/682352926 conn(0x55892935f000 msgr2=0x55892943a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.363+0000 7f6033fff700 0 client.25729 ms_handle_reset on v2:172.21.15.123:6834/835271994 2023-09-20T21:50:07.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.363+0000 7f606a8ddb80 1 -- 172.21.15.22:0/682352926 shutdown_connections 2023-09-20T21:50:07.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.363+0000 7f606a8ddb80 1 -- 172.21.15.22:0/682352926 wait complete. 2023-09-20T21:50:07.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.364+0000 7f606a8ddb80 20 client.25729 trim_cache size 0 max 16384 2023-09-20T21:50:07.366 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2023-09-20T21:50:07.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.365+0000 7f6056fff700 1 -- 172.21.15.22:0/3329179997 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60340a1500 msgr2=0x7f60340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.365+0000 7f6056fff700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60340a1500 0x7f60340a39c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6048009fd0 tx=0x7f604800b040 comp rx=0 tx=0).stop 2023-09-20T21:50:07.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.365+0000 7f6056fff700 1 -- 172.21.15.22:0/3329179997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f605010d120 msgr2=0x7f60501146b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.365+0000 7f6056fff700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f605010d120 0x7f60501146b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f604000b2e0 tx=0x7f60400057d0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.366+0000 7f6056fff700 1 -- 172.21.15.22:0/3329179997 shutdown_connections 2023-09-20T21:50:07.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.366+0000 7f6056fff700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f60340a1500 0x7f60340a39c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.366+0000 7f6056fff700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6050115070 0x7f60501b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.366+0000 7f6056fff700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f605010d120 0x7f60501146b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.366+0000 7f6056fff700 1 --2- 172.21.15.22:0/3329179997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6050114bf0 0x7f60501b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.366+0000 7f6056fff700 1 -- 172.21.15.22:0/3329179997 >> 172.21.15.22:0/3329179997 conn(0x7f605006db90 msgr2=0x7f605010b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.366+0000 7f6056fff700 1 -- 172.21.15.22:0/3329179997 shutdown_connections 2023-09-20T21:50:07.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.366+0000 7f6056fff700 1 -- 172.21.15.22:0/3329179997 wait complete. 2023-09-20T21:50:07.399 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-09-20T21:50:07.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.715+0000 7efe2fbff700 1 -- 172.21.15.22:0/3621381251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe28101f90 msgr2=0x7efe28102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.715+0000 7efe2fbff700 1 --2- 172.21.15.22:0/3621381251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe28101f90 0x7efe28102410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efe18009990 tx=0x7efe18009ca0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.715+0000 7efe2fbff700 1 -- 172.21.15.22:0/3621381251 shutdown_connections 2023-09-20T21:50:07.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.715+0000 7efe2fbff700 1 --2- 172.21.15.22:0/3621381251 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe28102950 0x7efe28109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.715+0000 7efe2fbff700 1 --2- 172.21.15.22:0/3621381251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe28100e40 0x7efe28101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.715+0000 7efe2fbff700 1 --2- 172.21.15.22:0/3621381251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe28101f90 0x7efe28102410 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.715+0000 7efe2fbff700 1 -- 172.21.15.22:0/3621381251 >> 172.21.15.22:0/3621381251 conn(0x7efe280fc3b0 msgr2=0x7efe280fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.715+0000 7efe2fbff700 1 -- 172.21.15.22:0/3621381251 shutdown_connections 2023-09-20T21:50:07.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.715+0000 7efe2fbff700 1 -- 172.21.15.22:0/3621381251 wait complete. 2023-09-20T21:50:07.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.716+0000 7efe2fbff700 1 Processor -- start 2023-09-20T21:50:07.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.716+0000 7efe2fbff700 1 -- start start 2023-09-20T21:50:07.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2fbff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe28100e40 0x7efe28198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2fbff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe28101f90 0x7efe28199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2fbff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe28102950 0x7efe281a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2fbff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe281a0870 con 0x7efe28100e40 2023-09-20T21:50:07.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2fbff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe281a09e0 con 0x7efe28101f90 2023-09-20T21:50:07.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2fbff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe281a0ba0 con 0x7efe28102950 2023-09-20T21:50:07.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2d99b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe28100e40 0x7efe28198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2d99b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe28100e40 0x7efe28198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40820/0 (socket says 172.21.15.22:40820) 2023-09-20T21:50:07.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2d99b700 1 -- 172.21.15.22:0/1377071636 learned_addr learned my addr 172.21.15.22:0/1377071636 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:07.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2d19a700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe28101f90 0x7efe28199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2e19c700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe28102950 0x7efe281a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2d99b700 1 -- 172.21.15.22:0/1377071636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe28102950 msgr2=0x7efe281a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe28102950 0x7efe281a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2d99b700 1 -- 172.21.15.22:0/1377071636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe28101f90 msgr2=0x7efe28199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe28101f90 0x7efe28199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2d99b700 1 -- 172.21.15.22:0/1377071636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180095f0 con 0x7efe28100e40 2023-09-20T21:50:07.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.717+0000 7efe2d19a700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe28101f90 0x7efe28199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:50:07.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.718+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe28100e40 0x7efe28198d30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efe2400e9a0 tx=0x7efe2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.718+0000 7efe1effd700 1 -- 172.21.15.22:0/1377071636 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24009790 con 0x7efe28100e40 2023-09-20T21:50:07.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.718+0000 7efe1effd700 1 -- 172.21.15.22:0/1377071636 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe24010430 con 0x7efe28100e40 2023-09-20T21:50:07.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.718+0000 7efe2fbff700 1 -- 172.21.15.22:0/1377071636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe281a0e80 con 0x7efe28100e40 2023-09-20T21:50:07.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.718+0000 7efe1effd700 1 -- 172.21.15.22:0/1377071636 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24018680 con 0x7efe28100e40 2023-09-20T21:50:07.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.718+0000 7efe2fbff700 1 -- 172.21.15.22:0/1377071636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe281a12a0 con 0x7efe28100e40 2023-09-20T21:50:07.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.721+0000 7efe1effd700 1 -- 172.21.15.22:0/1377071636 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7efe240105a0 con 0x7efe28100e40 2023-09-20T21:50:07.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.723+0000 7efe435f9b80 1 Processor -- start 2023-09-20T21:50:07.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe435f9b80 1 -- start start 2023-09-20T21:50:07.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233aa6f0 0x55e5233aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233ab050 0x55e5233ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233abbe0 0x55e5233ac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe435f9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52331eca0 con 0x55e5233abbe0 2023-09-20T21:50:07.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe435f9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523302c70 con 0x55e5233aa6f0 2023-09-20T21:50:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe435f9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523300590 con 0x55e5233ab050 2023-09-20T21:50:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe1effd700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7efe140a1300 0x7efe140a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe2d19a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233abbe0 0x55e5233ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe1effd700 1 -- 172.21.15.22:0/1377071636 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7efe24014070 con 0x7efe28100e40 2023-09-20T21:50:07.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe2e19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233aa6f0 0x55e5233aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe2d99b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233ab050 0x55e5233ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe2d19a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233abbe0 0x55e5233ac060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40830/0 (socket says 172.21.15.22:40830) 2023-09-20T21:50:07.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.724+0000 7efe2d19a700 1 -- 172.21.15.22:0/1901574875 learned_addr learned my addr 172.21.15.22:0/1901574875 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:07.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe2e19c700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7efe140a1300 0x7efe140a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe2d99b700 1 -- 172.21.15.22:0/1901574875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233aa6f0 msgr2=0x55e5233aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1901574875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233aa6f0 0x55e5233aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe2d99b700 1 -- 172.21.15.22:0/1901574875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233abbe0 msgr2=0x55e5233ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1901574875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233abbe0 0x55e5233ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe2d99b700 1 -- 172.21.15.22:0/1901574875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e523126de0 con 0x55e5233ab050 2023-09-20T21:50:07.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe2d19a700 1 --2- 172.21.15.22:0/1901574875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233abbe0 0x55e5233ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:50:07.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1901574875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233ab050 0x55e5233ab5e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe24096620 tx=0x7efe2409a020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73d01e31f99e0f7e server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe2e19c700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7efe140a1300 0x7efe140a37c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55e5233a60b0 tx=0x7efe2000b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe13fff700 1 -- 172.21.15.22:0/1901574875 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2409c070 con 0x55e5233ab050 2023-09-20T21:50:07.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe13fff700 1 -- 172.21.15.22:0/1901574875 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe2409a120 con 0x55e5233ab050 2023-09-20T21:50:07.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.725+0000 7efe13fff700 1 -- 172.21.15.22:0/1901574875 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24099610 con 0x55e5233ab050 2023-09-20T21:50:07.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.726+0000 7efe435f9b80 1 -- 172.21.15.22:0/1901574875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233ab050 msgr2=0x55e5233ab5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.726+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1901574875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233ab050 0x55e5233ab5e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe24096620 tx=0x7efe2409a020 comp rx=0 tx=0).stop 2023-09-20T21:50:07.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.726+0000 7efe435f9b80 1 -- 172.21.15.22:0/1901574875 shutdown_connections 2023-09-20T21:50:07.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.726+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1901574875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233abbe0 0x55e5233ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.726+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1901574875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233aa6f0 0x55e5233aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.726+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1901574875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233ab050 0x55e5233ab5e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.726+0000 7efe435f9b80 1 -- 172.21.15.22:0/1901574875 >> 172.21.15.22:0/1901574875 conn(0x55e5232d7000 msgr2=0x55e5233a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.726+0000 7efe435f9b80 1 -- 172.21.15.22:0/1901574875 shutdown_connections 2023-09-20T21:50:07.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.726+0000 7efe435f9b80 1 -- 172.21.15.22:0/1901574875 wait complete. 2023-09-20T21:50:07.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.727+0000 7efe435f9b80 1 -- start start 2023-09-20T21:50:07.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.727+0000 7efe435f9b80 1 Processor -- start 2023-09-20T21:50:07.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.727+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233aa6f0 0x55e5233b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.727+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233ab050 0x55e5233b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.727+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233abbe0 0x55e5233bb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.727+0000 7efe435f9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5232fbe80 con 0x55e5233ab050 2023-09-20T21:50:07.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.727+0000 7efe435f9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e523298740 con 0x55e5233abbe0 2023-09-20T21:50:07.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.727+0000 7efe435f9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5232983e0 con 0x55e5233aa6f0 2023-09-20T21:50:07.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.727+0000 7efe435f9b80 10 client.0 _ll_register_callbacks cb 0x55e5231e2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:07.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2d99b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233aa6f0 0x55e5233b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2d19a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233ab050 0x55e5233b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2d99b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233aa6f0 0x55e5233b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:55198/0 (socket says 172.21.15.22:55198) 2023-09-20T21:50:07.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2d99b700 1 -- 172.21.15.22:0/1125024867 learned_addr learned my addr 172.21.15.22:0/1125024867 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:07.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2e19c700 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233abbe0 0x55e5233bb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2d99b700 1 -- 172.21.15.22:0/1125024867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233abbe0 msgr2=0x55e5233bb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233abbe0 0x55e5233bb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2d99b700 1 -- 172.21.15.22:0/1125024867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233ab050 msgr2=0x55e5233b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233ab050 0x55e5233b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2d99b700 1 -- 172.21.15.22:0/1125024867 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe28109ba0 con 0x55e5233aa6f0 2023-09-20T21:50:07.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2e19c700 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233abbe0 0x55e5233bb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:50:07.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233aa6f0 0x55e5233b4190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe2400e970 tx=0x7efe24010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e708527c0ba6df server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe117fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:50:07.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe117fa700 1 -- 172.21.15.22:0/1125024867 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2409c030 con 0x55e5233aa6f0 2023-09-20T21:50:07.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.728+0000 7efe435f9b80 1 -- 172.21.15.22:0/1125024867 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e52326cbb0 con 0x55e5233aa6f0 2023-09-20T21:50:07.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe117fa700 1 -- 172.21.15.22:0/1125024867 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7efe2409d070 con 0x55e5233aa6f0 2023-09-20T21:50:07.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe117fa700 1 -- 172.21.15.22:0/1125024867 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe2409df50 con 0x55e5233aa6f0 2023-09-20T21:50:07.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe435f9b80 10 client.25741 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:07.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe435f9b80 1 -- 172.21.15.22:0/1125024867 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7efe281a12a0 con 0x55e5233aa6f0 2023-09-20T21:50:07.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe117fa700 1 -- 172.21.15.22:0/1125024867 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e52326cbb0 con 0x55e5233aa6f0 2023-09-20T21:50:07.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe117fa700 1 -- 172.21.15.22:0/1125024867 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7efe24099110 con 0x55e5233aa6f0 2023-09-20T21:50:07.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe435f9b80 10 client.25741 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:07.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe435f9b80 10 client.25741 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-20T21:50:07.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe435f9b80 20 client.25741 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:07.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e5233bfba0 0x55e5233c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe435f9b80 4 client.25741 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:50:07.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.729+0000 7efe435f9b80 1 -- 172.21.15.22:0/1125024867 --> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e523379020 con 0x55e5233bfba0 2023-09-20T21:50:07.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.730+0000 7efe2d19a700 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e5233bfba0 0x55e5233c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.730+0000 7efe2d19a700 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e5233bfba0 0x55e5233c1fc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81b83efd477ffcbc server_cookie=a49b4c298cdff688 in_seq=0 out_seq=0 2023-09-20T21:50:07.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.730+0000 7efe117fa700 10 client.25741 ms_handle_connect on v2:172.21.15.89:6836/3887702690 2023-09-20T21:50:07.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe117fa700 1 -- 172.21.15.22:0/1125024867 <== mds.2 v2:172.21.15.89:6836/3887702690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e523379020 con 0x55e5233bfba0 2023-09-20T21:50:07.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe117fa700 10 client.25741 handle_command_reply: tid=0 2023-09-20T21:50:07.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 client.25741 shutdown 2023-09-20T21:50:07.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 -- 172.21.15.22:0/1125024867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233aa6f0 msgr2=0x55e5233b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233aa6f0 0x55e5233b4190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe2400e970 tx=0x7efe24010ab0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 -- 172.21.15.22:0/1125024867 shutdown_connections 2023-09-20T21:50:07.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e5233bfba0 0x55e5233c1fc0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233abbe0 0x55e5233bb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233aa6f0 0x55e5233b4190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1125024867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233ab050 0x55e5233b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 -- 172.21.15.22:0/1125024867 >> 172.21.15.22:0/1125024867 conn(0x55e5232d7000 msgr2=0x55e5233a4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe117fa700 0 client.25741 ms_handle_reset on v2:172.21.15.89:6836/3887702690 2023-09-20T21:50:07.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 -- 172.21.15.22:0/1125024867 shutdown_connections 2023-09-20T21:50:07.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 1 -- 172.21.15.22:0/1125024867 wait complete. 2023-09-20T21:50:07.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.731+0000 7efe435f9b80 20 client.25741 trim_cache size 0 max 16384 2023-09-20T21:50:07.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe435f9b80 1 Processor -- start 2023-09-20T21:50:07.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe435f9b80 1 -- start start 2023-09-20T21:50:07.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 0x55e5233bb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233abbe0 0x55e5233aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe435f9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52317c920 con 0x55e5233aa6f0 2023-09-20T21:50:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe435f9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5233aba70 con 0x55e5233ab050 2023-09-20T21:50:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe435f9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5233ab480 con 0x55e5233abbe0 2023-09-20T21:50:07.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe2d99b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe2d99b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40842/0 (socket says 172.21.15.22:40842) 2023-09-20T21:50:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe2d99b700 1 -- 172.21.15.22:0/2805712097 learned_addr learned my addr 172.21.15.22:0/2805712097 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.745+0000 7efe2e19c700 1 --2- 172.21.15.22:0/2805712097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233abbe0 0x55e5233aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.746+0000 7efe2d19a700 1 --2- 172.21.15.22:0/2805712097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 0x55e5233bb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.746+0000 7efe2d99b700 1 -- 172.21.15.22:0/2805712097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233abbe0 msgr2=0x55e5233aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.746+0000 7efe2d99b700 1 --2- 172.21.15.22:0/2805712097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233abbe0 0x55e5233aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.746+0000 7efe2d99b700 1 -- 172.21.15.22:0/2805712097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 msgr2=0x55e5233bb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.746+0000 7efe2d99b700 1 --2- 172.21.15.22:0/2805712097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 0x55e5233bb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.746+0000 7efe2d99b700 1 -- 172.21.15.22:0/2805712097 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e52310fc60 con 0x55e5233aa6f0 2023-09-20T21:50:07.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.746+0000 7efe2d99b700 1 --2- 172.21.15.22:0/2805712097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b2090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efe24099e20 tx=0x7efe240ca930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c40d2e956e54a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.746+0000 7efe1cbff700 1 -- 172.21.15.22:0/2805712097 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2409f040 con 0x55e5233aa6f0 2023-09-20T21:50:07.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.746+0000 7efe1cbff700 1 -- 172.21.15.22:0/2805712097 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe240a2190 con 0x55e5233aa6f0 2023-09-20T21:50:07.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.746+0000 7efe1cbff700 1 -- 172.21.15.22:0/2805712097 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240989e0 con 0x55e5233aa6f0 2023-09-20T21:50:07.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.747+0000 7efe435f9b80 1 -- 172.21.15.22:0/2805712097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 msgr2=0x55e5233b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.747+0000 7efe435f9b80 1 --2- 172.21.15.22:0/2805712097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b2090 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efe24099e20 tx=0x7efe240ca930 comp rx=0 tx=0).stop 2023-09-20T21:50:07.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.747+0000 7efe435f9b80 1 -- 172.21.15.22:0/2805712097 shutdown_connections 2023-09-20T21:50:07.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.747+0000 7efe435f9b80 1 --2- 172.21.15.22:0/2805712097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233abbe0 0x55e5233aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.747+0000 7efe435f9b80 1 --2- 172.21.15.22:0/2805712097 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b2090 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.747+0000 7efe435f9b80 1 --2- 172.21.15.22:0/2805712097 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 0x55e5233bb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.747+0000 7efe435f9b80 1 -- 172.21.15.22:0/2805712097 >> 172.21.15.22:0/2805712097 conn(0x55e5232d7000 msgr2=0x55e5233b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.747+0000 7efe435f9b80 1 -- 172.21.15.22:0/2805712097 shutdown_connections 2023-09-20T21:50:07.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.747+0000 7efe435f9b80 1 -- 172.21.15.22:0/2805712097 wait complete. 2023-09-20T21:50:07.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.747+0000 7efe435f9b80 1 -- start start 2023-09-20T21:50:07.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.748+0000 7efe435f9b80 1 Processor -- start 2023-09-20T21:50:07.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.748+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.748+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 0x55e5233b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.748+0000 7efe435f9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233abbe0 0x55e5233ba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.748+0000 7efe435f9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5233bacc0 con 0x55e5233aa6f0 2023-09-20T21:50:07.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe435f9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5233bae30 con 0x55e5233ab050 2023-09-20T21:50:07.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe435f9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5233baff0 con 0x55e5233abbe0 2023-09-20T21:50:07.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe435f9b80 10 client.0 _ll_register_callbacks cb 0x55e5231e2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:07.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d99b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d99b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40848/0 (socket says 172.21.15.22:40848) 2023-09-20T21:50:07.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d99b700 1 -- 172.21.15.22:0/1087819417 learned_addr learned my addr 172.21.15.22:0/1087819417 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:07.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d19a700 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 0x55e5233b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2e19c700 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233abbe0 0x55e5233ba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d19a700 1 -- 172.21.15.22:0/1087819417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233abbe0 msgr2=0x55e5233ba780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d19a700 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233abbe0 0x55e5233ba780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d19a700 1 -- 172.21.15.22:0/1087819417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 msgr2=0x55e5233b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:07.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d19a700 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d19a700 1 -- 172.21.15.22:0/1087819417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe281a12a0 con 0x55e5233ab050 2023-09-20T21:50:07.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:50:07.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe2d19a700 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 0x55e5233b9770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efe18000c00 tx=0x7efe18005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2201e586cd00d28f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:07.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.749+0000 7efe117fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:50:07.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.750+0000 7efe117fa700 1 -- 172.21.15.22:0/1087819417 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1801d040 con 0x55e5233ab050 2023-09-20T21:50:07.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.750+0000 7efe435f9b80 1 -- 172.21.15.22:0/1087819417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe2409a120 con 0x55e5233ab050 2023-09-20T21:50:07.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.750+0000 7efe117fa700 1 -- 172.21.15.22:0/1087819417 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7efe18029150 con 0x55e5233ab050 2023-09-20T21:50:07.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.750+0000 7efe117fa700 1 -- 172.21.15.22:0/1087819417 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe18029310 con 0x55e5233ab050 2023-09-20T21:50:07.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.750+0000 7efe435f9b80 10 client.25760 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:07.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.750+0000 7efe435f9b80 1 -- 172.21.15.22:0/1087819417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e52314d250 con 0x55e5233ab050 2023-09-20T21:50:07.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.750+0000 7efe117fa700 1 -- 172.21.15.22:0/1087819417 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe2409a120 con 0x55e5233ab050 2023-09-20T21:50:07.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.750+0000 7efe117fa700 1 -- 172.21.15.22:0/1087819417 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7efe18015030 con 0x55e5233ab050 2023-09-20T21:50:07.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.751+0000 7efe435f9b80 10 client.25760 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:07.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.751+0000 7efe435f9b80 10 client.25760 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-09-20T21:50:07.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.751+0000 7efe435f9b80 20 client.25760 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:07.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.751+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e523414610 0x55e523416a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:07.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.751+0000 7efe435f9b80 4 client.25760 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-09-20T21:50:07.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.751+0000 7efe435f9b80 1 -- 172.21.15.22:0/1087819417 --> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e5232f4e60 con 0x55e523414610 2023-09-20T21:50:07.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.751+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e523414610 0x55e523416a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:07.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.751+0000 7efe2d99b700 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e523414610 0x55e523416a00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=850f92926e83a9e1 server_cookie=1fb9520280895e6a in_seq=0 out_seq=0 2023-09-20T21:50:07.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.751+0000 7efe117fa700 10 client.25760 ms_handle_connect on v2:172.21.15.89:6836/3887702690 2023-09-20T21:50:07.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe117fa700 1 -- 172.21.15.22:0/1087819417 <== mds.2 v2:172.21.15.89:6836/3887702690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e5232f4e60 con 0x55e523414610 2023-09-20T21:50:07.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe117fa700 10 client.25760 handle_command_reply: tid=0 2023-09-20T21:50:07.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 client.25760 shutdown 2023-09-20T21:50:07.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 -- 172.21.15.22:0/1087819417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 msgr2=0x55e5233b9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 0x55e5233b9770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efe18000c00 tx=0x7efe18005810 comp rx=0 tx=0).stop 2023-09-20T21:50:07.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 -- 172.21.15.22:0/1087819417 shutdown_connections 2023-09-20T21:50:07.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.89:6836/3887702690,v1:172.21.15.89:6837/3887702690] conn(0x55e523414610 0x55e523416a00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5233abbe0 0x55e5233ba780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe117fa700 0 client.25760 ms_handle_reset on v2:172.21.15.89:6836/3887702690 2023-09-20T21:50:07.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5233aa6f0 0x55e5233b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 --2- 172.21.15.22:0/1087819417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e5233ab050 0x55e5233b9770 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 -- 172.21.15.22:0/1087819417 >> 172.21.15.22:0/1087819417 conn(0x55e5232d7000 msgr2=0x55e5233b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 -- 172.21.15.22:0/1087819417 shutdown_connections 2023-09-20T21:50:07.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 1 -- 172.21.15.22:0/1087819417 wait complete. 2023-09-20T21:50:07.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.752+0000 7efe435f9b80 20 client.25760 trim_cache size 0 max 16384 2023-09-20T21:50:07.786 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2023-09-20T21:50:07.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 -- 172.21.15.22:0/1377071636 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7efe140a1300 msgr2=0x7efe140a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7efe140a1300 0x7efe140a37c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55e5233a60b0 tx=0x7efe2000b040 comp rx=0 tx=0).stop 2023-09-20T21:50:07.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 -- 172.21.15.22:0/1377071636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe28100e40 msgr2=0x7efe28198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:07.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe28100e40 0x7efe28198d30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efe2400e9a0 tx=0x7efe2400ecb0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 -- 172.21.15.22:0/1377071636 shutdown_connections 2023-09-20T21:50:07.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe28102950 0x7efe281a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efe28100e40 0x7efe28198d30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7efe140a1300 0x7efe140a37c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 --2- 172.21.15.22:0/1377071636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe28101f90 0x7efe28199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 -- 172.21.15.22:0/1377071636 >> 172.21.15.22:0/1377071636 conn(0x7efe280fc3b0 msgr2=0x7efe28107950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 -- 172.21.15.22:0/1377071636 shutdown_connections 2023-09-20T21:50:07.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:07.753+0000 7efe2fbff700 1 -- 172.21.15.22:0/1377071636 wait complete. 2023-09-20T21:50:07.790 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-09-20T21:50:08.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.167+0000 7f9b367ff700 1 -- 172.21.15.22:0/2478728323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b30107de0 msgr2=0x7f9b3010a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.167+0000 7f9b367ff700 1 --2- 172.21.15.22:0/2478728323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b30107de0 0x7f9b3010a210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009f90 tx=0x7f9b2000b330 comp rx=0 tx=0).stop 2023-09-20T21:50:08.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.170+0000 7f9b367ff700 1 -- 172.21.15.22:0/2478728323 shutdown_connections 2023-09-20T21:50:08.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.170+0000 7f9b367ff700 1 --2- 172.21.15.22:0/2478728323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b3010d120 0x7f9b3010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.170+0000 7f9b367ff700 1 --2- 172.21.15.22:0/2478728323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b30107de0 0x7f9b3010a210 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.170+0000 7f9b367ff700 1 --2- 172.21.15.22:0/2478728323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3010a750 0x7f9b3010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.170+0000 7f9b367ff700 1 -- 172.21.15.22:0/2478728323 >> 172.21.15.22:0/2478728323 conn(0x7f9b3006db90 msgr2=0x7f9b3006ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.171+0000 7f9b367ff700 1 -- 172.21.15.22:0/2478728323 shutdown_connections 2023-09-20T21:50:08.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.171+0000 7f9b367ff700 1 -- 172.21.15.22:0/2478728323 wait complete. 2023-09-20T21:50:08.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.172+0000 7f9b367ff700 1 Processor -- start 2023-09-20T21:50:08.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.173+0000 7f9b367ff700 1 -- start start 2023-09-20T21:50:08.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.173+0000 7f9b367ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b30107de0 0x7f9b3019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.173+0000 7f9b367ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b3010a750 0x7f9b3019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.173+0000 7f9b367ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3010d120 0x7f9b301a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.173+0000 7f9b367ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b301a4c10 con 0x7f9b30107de0 2023-09-20T21:50:08.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.173+0000 7f9b367ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b301a4d50 con 0x7f9b3010a750 2023-09-20T21:50:08.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.173+0000 7f9b367ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b301a4ee0 con 0x7f9b3010d120 2023-09-20T21:50:08.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.173+0000 7f9b2ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b30107de0 0x7f9b3019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b2ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b30107de0 0x7f9b3019d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40868/0 (socket says 172.21.15.22:40868) 2023-09-20T21:50:08.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b2ffff700 1 -- 172.21.15.22:0/1193775172 learned_addr learned my addr 172.21.15.22:0/1193775172 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:08.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b34d9c700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3010d120 0x7f9b301a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b2f7fe700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b3010a750 0x7f9b3019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b2ffff700 1 -- 172.21.15.22:0/1193775172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3010d120 msgr2=0x7f9b301a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b2ffff700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3010d120 0x7f9b301a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b2ffff700 1 -- 172.21.15.22:0/1193775172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b3010a750 msgr2=0x7f9b3019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b2ffff700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b3010a750 0x7f9b3019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b2ffff700 1 -- 172.21.15.22:0/1193775172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b20009c40 con 0x7f9b30107de0 2023-09-20T21:50:08.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b34d9c700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3010d120 0x7f9b301a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:50:08.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.174+0000 7f9b2ffff700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b30107de0 0x7f9b3019d090 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9b20004c90 tx=0x7f9b20004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.175+0000 7f9b2d7fa700 1 -- 172.21.15.22:0/1193775172 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000e040 con 0x7f9b30107de0 2023-09-20T21:50:08.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.175+0000 7f9b2d7fa700 1 -- 172.21.15.22:0/1193775172 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b20007870 con 0x7f9b30107de0 2023-09-20T21:50:08.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.175+0000 7f9b2d7fa700 1 -- 172.21.15.22:0/1193775172 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b200052a0 con 0x7f9b30107de0 2023-09-20T21:50:08.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.175+0000 7f9b367ff700 1 -- 172.21.15.22:0/1193775172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b301a5160 con 0x7f9b30107de0 2023-09-20T21:50:08.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.176+0000 7f9b367ff700 1 -- 172.21.15.22:0/1193775172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b301a5590 con 0x7f9b30107de0 2023-09-20T21:50:08.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.185+0000 7f9b4a1e2b80 1 Processor -- start 2023-09-20T21:50:08.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.185+0000 7f9b4a1e2b80 1 -- start start 2023-09-20T21:50:08.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.185+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a5126f0 0x556f6a512b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.185+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a513050 0x556f6a5135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.185+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a514060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.185+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a486ca0 con 0x556f6a5126f0 2023-09-20T21:50:08.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.185+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a46ac70 con 0x556f6a513be0 2023-09-20T21:50:08.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.186+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a468590 con 0x556f6a513050 2023-09-20T21:50:08.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.186+0000 7f9b2f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a514060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.186+0000 7f9b34d9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a5126f0 0x556f6a512b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.186+0000 7f9b2f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a514060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42994/0 (socket says 172.21.15.22:42994) 2023-09-20T21:50:08.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.186+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/450778422 learned_addr learned my addr 172.21.15.22:0/450778422 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:08.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.186+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/450778422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a513050 msgr2=0x556f6a5135e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:50:08.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.186+0000 7f9b2f7fe700 1 --2- 172.21.15.22:0/450778422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a513050 0x556f6a5135e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.186+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/450778422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a5126f0 msgr2=0x556f6a512b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.186+0000 7f9b2f7fe700 1 --2- 172.21.15.22:0/450778422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a5126f0 0x556f6a512b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.186+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/450778422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f6a28ede0 con 0x556f6a513be0 2023-09-20T21:50:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.187+0000 7f9b2f7fe700 1 --2- 172.21.15.22:0/450778422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a514060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009d80 tx=0x7f9b24011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76e56b5dc443e560 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.189+0000 7f9b2d7fa700 1 -- 172.21.15.22:0/1193775172 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f9b2001d470 con 0x7f9b30107de0 2023-09-20T21:50:08.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.190+0000 7f9b167fc700 1 -- 172.21.15.22:0/450778422 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24004230 con 0x556f6a513be0 2023-09-20T21:50:08.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.190+0000 7f9b167fc700 1 -- 172.21.15.22:0/450778422 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b24004390 con 0x556f6a513be0 2023-09-20T21:50:08.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.190+0000 7f9b167fc700 1 -- 172.21.15.22:0/450778422 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24019840 con 0x556f6a513be0 2023-09-20T21:50:08.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.191+0000 7f9b2d7fa700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9b180c3410 0x7f9b180c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.191+0000 7f9b2d7fa700 1 -- 172.21.15.22:0/1193775172 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f9b200ce080 con 0x7f9b30107de0 2023-09-20T21:50:08.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.192+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/450778422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 msgr2=0x556f6a514060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.192+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/450778422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a514060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009d80 tx=0x7f9b24011720 comp rx=0 tx=0).stop 2023-09-20T21:50:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.193+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/450778422 shutdown_connections 2023-09-20T21:50:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.193+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/450778422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a514060 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.193+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/450778422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a5126f0 0x556f6a512b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.193+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/450778422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a513050 0x556f6a5135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.193+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/450778422 >> 172.21.15.22:0/450778422 conn(0x556f6a43f000 msgr2=0x556f6a50d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.193+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/450778422 shutdown_connections 2023-09-20T21:50:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.193+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/450778422 wait complete. 2023-09-20T21:50:08.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.194+0000 7f9b4a1e2b80 1 -- start start 2023-09-20T21:50:08.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.194+0000 7f9b4a1e2b80 1 Processor -- start 2023-09-20T21:50:08.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.194+0000 7f9b34d9c700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9b180c3410 0x7f9b180c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.195+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a51c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.195+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 0x556f6a51c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.195+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a5237f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.195+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a463e80 con 0x556f6a513050 2023-09-20T21:50:08.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.195+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a400740 con 0x556f6a513be0 2023-09-20T21:50:08.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.195+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a4003e0 con 0x556f6a5126f0 2023-09-20T21:50:08.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.195+0000 7f9b4a1e2b80 10 client.0 _ll_register_callbacks cb 0x556f6a34aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:08.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.196+0000 7f9b34d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a5237f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.196+0000 7f9b34d9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a5237f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43010/0 (socket says 172.21.15.22:43010) 2023-09-20T21:50:08.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.196+0000 7f9b34d9c700 1 -- 172.21.15.22:0/3820484805 learned_addr learned my addr 172.21.15.22:0/3820484805 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:08.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.196+0000 7f9b2ffff700 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a51c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.196+0000 7f9b34d9c700 1 -- 172.21.15.22:0/3820484805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 msgr2=0x556f6a51c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.196+0000 7f9b34d9c700 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a51c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.196+0000 7f9b34d9c700 1 -- 172.21.15.22:0/3820484805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 msgr2=0x556f6a51c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.196+0000 7f9b34d9c700 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 0x556f6a51c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.196+0000 7f9b34d9c700 1 -- 172.21.15.22:0/3820484805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b240099e0 con 0x556f6a513be0 2023-09-20T21:50:08.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.197+0000 7f9b34d9c700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9b180c3410 0x7f9b180c58d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9b2801afd0 tx=0x7f9b28013580 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.197+0000 7f9b34d9c700 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a5237f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800ff30 tx=0x7f9b280047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29af06ac9d02e16b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.197+0000 7f9b0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:50:08.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.197+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/3820484805 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2801fb10 con 0x556f6a513be0 2023-09-20T21:50:08.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.197+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/3820484805 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f9b2801fc70 con 0x556f6a513be0 2023-09-20T21:50:08.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.197+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/3820484805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f6a3d4bb0 con 0x556f6a513be0 2023-09-20T21:50:08.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.197+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/3820484805 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b28004020 con 0x556f6a513be0 2023-09-20T21:50:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.199+0000 7f9b4a1e2b80 10 client.25772 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:08.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.199+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/3820484805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556f6a277c60 con 0x556f6a513be0 2023-09-20T21:50:08.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.199+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/3820484805 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f6a3d4bb0 con 0x556f6a513be0 2023-09-20T21:50:08.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.199+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/3820484805 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9b28003a30 con 0x556f6a513be0 2023-09-20T21:50:08.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.199+0000 7f9b4a1e2b80 10 client.25772 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:08.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.199+0000 7f9b4a1e2b80 10 client.25772 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-20T21:50:08.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.199+0000 7f9b4a1e2b80 20 client.25772 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:08.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.199+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x556f6a5280e0 0x556f6a52a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.199+0000 7f9b4a1e2b80 4 client.25772 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:50:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.199+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/3820484805 --> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f6a4e1020 con 0x556f6a5280e0 2023-09-20T21:50:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.200+0000 7f9b2ffff700 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x556f6a5280e0 0x556f6a52a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.201+0000 7f9b2ffff700 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x556f6a5280e0 0x556f6a52a500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=627349e974274594 server_cookie=d9c59eb9135bfc58 in_seq=0 out_seq=0 2023-09-20T21:50:08.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.201+0000 7f9b0f7fe700 10 client.25772 ms_handle_connect on v2:172.21.15.22:6840/1806616417 2023-09-20T21:50:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.201+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/3820484805 <== mds.1 v2:172.21.15.22:6840/1806616417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f6a4e1020 con 0x556f6a5280e0 2023-09-20T21:50:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.201+0000 7f9b0f7fe700 10 client.25772 handle_command_reply: tid=0 2023-09-20T21:50:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b4a1e2b80 1 client.25772 shutdown 2023-09-20T21:50:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/3820484805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 msgr2=0x556f6a5237f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a5237f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800ff30 tx=0x7f9b280047c0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/3820484805 shutdown_connections 2023-09-20T21:50:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a5237f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a51c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x556f6a5280e0 0x556f6a52a500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/3820484805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 0x556f6a51c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/3820484805 >> 172.21.15.22:0/3820484805 conn(0x556f6a43f000 msgr2=0x556f6a50c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b0f7fe700 0 client.25772 ms_handle_reset on v2:172.21.15.22:6840/1806616417 2023-09-20T21:50:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.202+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/3820484805 shutdown_connections 2023-09-20T21:50:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.203+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/3820484805 wait complete. 2023-09-20T21:50:08.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.204+0000 7f9b4a1e2b80 20 client.25772 trim_cache size 0 max 16384 2023-09-20T21:50:08.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.227+0000 7f9b4a1e2b80 1 Processor -- start 2023-09-20T21:50:08.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.228+0000 7f9b4a1e2b80 1 -- start start 2023-09-20T21:50:08.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.228+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a513fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.228+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 0x556f6a50d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.228+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a5237b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.228+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a2e4920 con 0x556f6a513050 2023-09-20T21:50:08.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.228+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a513430 con 0x556f6a513be0 2023-09-20T21:50:08.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.228+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a50de50 con 0x556f6a5126f0 2023-09-20T21:50:08.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.229+0000 7f9b2f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 0x556f6a50d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.229+0000 7f9b2f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 0x556f6a50d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40890/0 (socket says 172.21.15.22:40890) 2023-09-20T21:50:08.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.229+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/1904839662 learned_addr learned my addr 172.21.15.22:0/1904839662 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:08.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.229+0000 7f9b34d9c700 1 --2- 172.21.15.22:0/1904839662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a5237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.229+0000 7f9b2ffff700 1 --2- 172.21.15.22:0/1904839662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a513fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.229+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/1904839662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 msgr2=0x556f6a513fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.229+0000 7f9b2f7fe700 1 --2- 172.21.15.22:0/1904839662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a513fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.229+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/1904839662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 msgr2=0x556f6a5237b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.229+0000 7f9b2f7fe700 1 --2- 172.21.15.22:0/1904839662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a5237b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.229+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/1904839662 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f6a2b5250 con 0x556f6a513050 2023-09-20T21:50:08.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.230+0000 7f9b2f7fe700 1 --2- 172.21.15.22:0/1904839662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 0x556f6a50d1a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9b24005950 tx=0x7f9b24005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf79dd7b55a9ecf server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.230+0000 7f9b16ffd700 1 -- 172.21.15.22:0/1904839662 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24005240 con 0x556f6a513050 2023-09-20T21:50:08.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.230+0000 7f9b16ffd700 1 -- 172.21.15.22:0/1904839662 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b24009ed0 con 0x556f6a513050 2023-09-20T21:50:08.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.230+0000 7f9b16ffd700 1 -- 172.21.15.22:0/1904839662 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b240196f0 con 0x556f6a513050 2023-09-20T21:50:08.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.231+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/1904839662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 msgr2=0x556f6a50d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.231+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/1904839662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 0x556f6a50d1a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9b24005950 tx=0x7f9b24005ce0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.231+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/1904839662 shutdown_connections 2023-09-20T21:50:08.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.231+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/1904839662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513be0 0x556f6a5237b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.231+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/1904839662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a513fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.231+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/1904839662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513050 0x556f6a50d1a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.231+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/1904839662 >> 172.21.15.22:0/1904839662 conn(0x556f6a43f000 msgr2=0x556f6a5092e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.232+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/1904839662 shutdown_connections 2023-09-20T21:50:08.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.232+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/1904839662 wait complete. 2023-09-20T21:50:08.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.232+0000 7f9b4a1e2b80 1 -- start start 2023-09-20T21:50:08.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.233+0000 7f9b4a1e2b80 1 Processor -- start 2023-09-20T21:50:08.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.233+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a51f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.233+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513050 0x556f6a51f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.233+0000 7f9b4a1e2b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513be0 0x556f6a522630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.234+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a522b70 con 0x556f6a513be0 2023-09-20T21:50:08.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.234+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a522ce0 con 0x556f6a513050 2023-09-20T21:50:08.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.234+0000 7f9b4a1e2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6a522ea0 con 0x556f6a5126f0 2023-09-20T21:50:08.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.234+0000 7f9b4a1e2b80 10 client.0 _ll_register_callbacks cb 0x556f6a34aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:08.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.234+0000 7f9b2ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a51f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.234+0000 7f9b2f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513050 0x556f6a51f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.234+0000 7f9b2f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513050 0x556f6a51f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43018/0 (socket says 172.21.15.22:43018) 2023-09-20T21:50:08.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.234+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/515968923 learned_addr learned my addr 172.21.15.22:0/515968923 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:08.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.234+0000 7f9b34d9c700 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513be0 0x556f6a522630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.235+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/515968923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 msgr2=0x556f6a51f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.235+0000 7f9b2f7fe700 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a51f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.235+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/515968923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513be0 msgr2=0x556f6a522630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.235+0000 7f9b2f7fe700 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513be0 0x556f6a522630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.235+0000 7f9b2f7fe700 1 -- 172.21.15.22:0/515968923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f6a28ede0 con 0x556f6a513050 2023-09-20T21:50:08.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.235+0000 7f9b2f7fe700 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513050 0x556f6a51f610 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9b24000c00 tx=0x7f9b24004fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d6f6abfe610a7c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.235+0000 7f9b0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:50:08.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.236+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/515968923 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24005600 con 0x556f6a513050 2023-09-20T21:50:08.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.236+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/515968923 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f9b24023460 con 0x556f6a513050 2023-09-20T21:50:08.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.236+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/515968923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b24004390 con 0x556f6a513050 2023-09-20T21:50:08.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/515968923 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b24023620 con 0x556f6a513050 2023-09-20T21:50:08.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b4a1e2b80 10 client.25778 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:08.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/515968923 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556f6a28d2b0 con 0x556f6a513050 2023-09-20T21:50:08.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/515968923 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b24004390 con 0x556f6a513050 2023-09-20T21:50:08.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/515968923 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9b24021040 con 0x556f6a513050 2023-09-20T21:50:08.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b4a1e2b80 10 client.25778 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:08.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b4a1e2b80 10 client.25778 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-09-20T21:50:08.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b4a1e2b80 20 client.25778 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:08.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x556f6a57c550 0x556f6a57e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b4a1e2b80 4 client.25778 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-09-20T21:50:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.238+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/515968923 --> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556f6a45ce60 con 0x556f6a57c550 2023-09-20T21:50:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.240+0000 7f9b2ffff700 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x556f6a57c550 0x556f6a57e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.240+0000 7f9b2ffff700 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x556f6a57c550 0x556f6a57e970 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b201fd14793d1b9 server_cookie=e93800a7128966a2 in_seq=0 out_seq=0 2023-09-20T21:50:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.240+0000 7f9b0f7fe700 10 client.25778 ms_handle_connect on v2:172.21.15.22:6840/1806616417 2023-09-20T21:50:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b0f7fe700 1 -- 172.21.15.22:0/515968923 <== mds.1 v2:172.21.15.22:6840/1806616417 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556f6a45ce60 con 0x556f6a57c550 2023-09-20T21:50:08.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b0f7fe700 10 client.25778 handle_command_reply: tid=0 2023-09-20T21:50:08.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b4a1e2b80 1 client.25778 shutdown 2023-09-20T21:50:08.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/515968923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513050 msgr2=0x556f6a51f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513050 0x556f6a51f610 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9b24000c00 tx=0x7f9b24004fd0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/515968923 shutdown_connections 2023-09-20T21:50:08.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.22:6840/1806616417,v1:172.21.15.22:6841/1806616417] conn(0x556f6a57c550 0x556f6a57e970 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556f6a513be0 0x556f6a522630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f6a5126f0 0x556f6a51f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b4a1e2b80 1 --2- 172.21.15.22:0/515968923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f6a513050 0x556f6a51f610 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/515968923 >> 172.21.15.22:0/515968923 conn(0x556f6a43f000 msgr2=0x556f6a5240e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b0f7fe700 0 client.25778 ms_handle_reset on v2:172.21.15.22:6840/1806616417 2023-09-20T21:50:08.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.241+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/515968923 shutdown_connections 2023-09-20T21:50:08.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b4a1e2b80 1 -- 172.21.15.22:0/515968923 wait complete. 2023-09-20T21:50:08.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b4a1e2b80 20 client.25778 trim_cache size 0 max 16384 2023-09-20T21:50:08.252 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2023-09-20T21:50:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b367ff700 1 -- 172.21.15.22:0/1193775172 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9b180c3410 msgr2=0x7f9b180c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b367ff700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9b180c3410 0x7f9b180c58d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9b2801afd0 tx=0x7f9b28013580 comp rx=0 tx=0).stop 2023-09-20T21:50:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b367ff700 1 -- 172.21.15.22:0/1193775172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b30107de0 msgr2=0x7f9b3019d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b367ff700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b30107de0 0x7f9b3019d090 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9b20004c90 tx=0x7f9b20004cc0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b367ff700 1 -- 172.21.15.22:0/1193775172 shutdown_connections 2023-09-20T21:50:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b367ff700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3010d120 0x7f9b301a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b367ff700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b30107de0 0x7f9b3019d090 secure :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9b20004c90 tx=0x7f9b20004cc0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b367ff700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f9b180c3410 0x7f9b180c58d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b367ff700 1 --2- 172.21.15.22:0/1193775172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b3010a750 0x7f9b3019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.242+0000 7f9b367ff700 1 -- 172.21.15.22:0/1193775172 >> 172.21.15.22:0/1193775172 conn(0x7f9b3006db90 msgr2=0x7f9b3006ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.243+0000 7f9b367ff700 1 -- 172.21.15.22:0/1193775172 shutdown_connections 2023-09-20T21:50:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.243+0000 7f9b367ff700 1 -- 172.21.15.22:0/1193775172 wait complete. 2023-09-20T21:50:08.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:07 smithi022 ceph-mon[97651]: pgmap v383: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 751 op/s 2023-09-20T21:50:08.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:50:08.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:50:08.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:07 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:50:08.265 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-09-20T21:50:08.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:07 smithi123 ceph-mon[134433]: pgmap v383: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 751 op/s 2023-09-20T21:50:08.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:50:08.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:50:08.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:07 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:50:08.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:07 smithi089 ceph-mon[134090]: pgmap v383: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 751 op/s 2023-09-20T21:50:08.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:50:08.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:50:08.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:07 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:50:08.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 45.0 s, 818.0 tps, lat 117.269 ms stddev 167.981 2023-09-20T21:50:08.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.575+0000 7f14937ff700 1 -- 172.21.15.22:0/1187080586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c104980 msgr2=0x7f148c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.575+0000 7f14937ff700 1 --2- 172.21.15.22:0/1187080586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c104980 0x7f148c104e00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1484009990 tx=0x7f1484009ca0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.575+0000 7f14937ff700 1 -- 172.21.15.22:0/1187080586 shutdown_connections 2023-09-20T21:50:08.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.575+0000 7f14937ff700 1 --2- 172.21.15.22:0/1187080586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c104980 0x7f148c104e00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.575+0000 7f14937ff700 1 --2- 172.21.15.22:0/1187080586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c0ff640 0x7f148c101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.575+0000 7f14937ff700 1 --2- 172.21.15.22:0/1187080586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c101fb0 0x7f148c104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.575+0000 7f14937ff700 1 -- 172.21.15.22:0/1187080586 >> 172.21.15.22:0/1187080586 conn(0x7f148c0fb240 msgr2=0x7f148c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.580+0000 7f14937ff700 1 -- 172.21.15.22:0/1187080586 shutdown_connections 2023-09-20T21:50:08.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.580+0000 7f14937ff700 1 -- 172.21.15.22:0/1187080586 wait complete. 2023-09-20T21:50:08.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.582+0000 7f14937ff700 1 Processor -- start 2023-09-20T21:50:08.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.582+0000 7f14937ff700 1 -- start start 2023-09-20T21:50:08.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.583+0000 7f14937ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c0ff640 0x7f148c06f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.583+0000 7f14937ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c101fb0 0x7f148c06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.583+0000 7f14937ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c104980 0x7f148c19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.583+0000 7f14937ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c072cc0 con 0x7f148c101fb0 2023-09-20T21:50:08.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.583+0000 7f14937ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c072e30 con 0x7f148c0ff640 2023-09-20T21:50:08.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.583+0000 7f14937ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c19c8d0 con 0x7f148c104980 2023-09-20T21:50:08.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.583+0000 7f1491d9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c104980 0x7f148c19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.583+0000 7f1491d9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c104980 0x7f148c19c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:55290/0 (socket says 172.21.15.22:55290) 2023-09-20T21:50:08.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.583+0000 7f1491d9c700 1 -- 172.21.15.22:0/1997369479 learned_addr learned my addr 172.21.15.22:0/1997369479 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:08.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.584+0000 7f1491d9c700 1 -- 172.21.15.22:0/1997369479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c0ff640 msgr2=0x7f148c06f630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-20T21:50:08.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.584+0000 7f1491d9c700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c0ff640 0x7f148c06f630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.584+0000 7f1491d9c700 1 -- 172.21.15.22:0/1997369479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c101fb0 msgr2=0x7f148c06fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.584+0000 7f1491d9c700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c101fb0 0x7f148c06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.584+0000 7f1491d9c700 1 -- 172.21.15.22:0/1997369479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14840095f0 con 0x7f148c104980 2023-09-20T21:50:08.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.585+0000 7f1491d9c700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c104980 0x7f148c19c390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f148400f690 tx=0x7f148400f770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.586+0000 7f148a7fc700 1 -- 172.21.15.22:0/1997369479 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148401d070 con 0x7f148c104980 2023-09-20T21:50:08.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.586+0000 7f148a7fc700 1 -- 172.21.15.22:0/1997369479 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f148400fb50 con 0x7f148c104980 2023-09-20T21:50:08.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.586+0000 7f148a7fc700 1 -- 172.21.15.22:0/1997369479 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484017650 con 0x7f148c104980 2023-09-20T21:50:08.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.588+0000 7f14937ff700 1 -- 172.21.15.22:0/1997369479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c19cb50 con 0x7f148c104980 2023-09-20T21:50:08.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.588+0000 7f14937ff700 1 -- 172.21.15.22:0/1997369479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c19d030 con 0x7f148c104980 2023-09-20T21:50:08.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.595+0000 7f14a7160b80 1 Processor -- start 2023-09-20T21:50:08.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.597+0000 7f148a7fc700 1 -- 172.21.15.22:0/1997369479 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f148400fcc0 con 0x7f148c104980 2023-09-20T21:50:08.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.600+0000 7f148a7fc700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f147c0a1450 0x7f147c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.600+0000 7f148a7fc700 1 -- 172.21.15.22:0/1997369479 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f14840ce650 con 0x7f148c104980 2023-09-20T21:50:08.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.600+0000 7f14a7160b80 1 -- start start 2023-09-20T21:50:08.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.601+0000 7f1490d9a700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f147c0a1450 0x7f147c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.601+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388bf6f0 0x561d388bfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.601+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388c0050 0x561d388c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.601+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0be0 0x561d388c1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.601+0000 7f14a7160b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38833ca0 con 0x561d388bf6f0 2023-09-20T21:50:08.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.601+0000 7f14a7160b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38817c70 con 0x561d388c0050 2023-09-20T21:50:08.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.601+0000 7f14a7160b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38815590 con 0x561d388c0be0 2023-09-20T21:50:08.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.601+0000 7f1491d9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0be0 0x561d388c1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.601+0000 7f1491d9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0be0 0x561d388c1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:55302/0 (socket says 172.21.15.22:55302) 2023-09-20T21:50:08.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.601+0000 7f1491d9c700 1 -- 172.21.15.22:0/3397695046 learned_addr learned my addr 172.21.15.22:0/3397695046 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:08.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.602+0000 7f1490d9a700 1 --2- 172.21.15.22:0/3397695046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388c0050 0x561d388c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.602+0000 7f1491d9c700 1 -- 172.21.15.22:0/3397695046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388c0050 msgr2=0x561d388c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.602+0000 7f1491d9c700 1 --2- 172.21.15.22:0/3397695046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388c0050 0x561d388c05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.602+0000 7f1491d9c700 1 -- 172.21.15.22:0/3397695046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388bf6f0 msgr2=0x561d388bfb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:50:08.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.602+0000 7f1491d9c700 1 --2- 172.21.15.22:0/3397695046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388bf6f0 0x561d388bfb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.602+0000 7f1491d9c700 1 -- 172.21.15.22:0/3397695046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d3863bde0 con 0x561d388c0be0 2023-09-20T21:50:08.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.602+0000 7f1491d9c700 1 --2- 172.21.15.22:0/3397695046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0be0 0x561d388c1060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14840cbe10 tx=0x7f1484003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a23867d9c186b6c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.602+0000 7f1490d9a700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f147c0a1450 0x7f147c0a3910 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f148c070750 tx=0x7f1478006cd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.608+0000 7f14777fe700 1 -- 172.21.15.22:0/3397695046 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148409f630 con 0x561d388c0be0 2023-09-20T21:50:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.608+0000 7f14777fe700 1 -- 172.21.15.22:0/3397695046 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14840a37e0 con 0x561d388c0be0 2023-09-20T21:50:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.608+0000 7f14777fe700 1 -- 172.21.15.22:0/3397695046 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14840a3350 con 0x561d388c0be0 2023-09-20T21:50:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.609+0000 7f14a7160b80 1 -- 172.21.15.22:0/3397695046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0be0 msgr2=0x561d388c1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.609+0000 7f14a7160b80 1 --2- 172.21.15.22:0/3397695046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0be0 0x561d388c1060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14840cbe10 tx=0x7f1484003680 comp rx=0 tx=0).stop 2023-09-20T21:50:08.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.610+0000 7f14a7160b80 1 -- 172.21.15.22:0/3397695046 shutdown_connections 2023-09-20T21:50:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.610+0000 7f14a7160b80 1 --2- 172.21.15.22:0/3397695046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0be0 0x561d388c1060 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.610+0000 7f14a7160b80 1 --2- 172.21.15.22:0/3397695046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388bf6f0 0x561d388bfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.610+0000 7f14a7160b80 1 --2- 172.21.15.22:0/3397695046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388c0050 0x561d388c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.610+0000 7f14a7160b80 1 -- 172.21.15.22:0/3397695046 >> 172.21.15.22:0/3397695046 conn(0x561d387ec000 msgr2=0x561d388ba1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.610+0000 7f14a7160b80 1 -- 172.21.15.22:0/3397695046 shutdown_connections 2023-09-20T21:50:08.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.611+0000 7f14a7160b80 1 -- 172.21.15.22:0/3397695046 wait complete. 2023-09-20T21:50:08.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.611+0000 7f14a7160b80 1 -- start start 2023-09-20T21:50:08.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.612+0000 7f14a7160b80 1 Processor -- start 2023-09-20T21:50:08.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.612+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.612+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 0x561d388c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.612+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 0x561d388d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.612+0000 7f14a7160b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38810e80 con 0x561d388c0be0 2023-09-20T21:50:08.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.612+0000 7f14a7160b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d387ad740 con 0x561d388bf6f0 2023-09-20T21:50:08.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.612+0000 7f14a7160b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d387ad3e0 con 0x561d388c0050 2023-09-20T21:50:08.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.612+0000 7f14a7160b80 10 client.0 _ll_register_callbacks cb 0x561d386f7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:08.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.613+0000 7f149159b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.613+0000 7f149159b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43060/0 (socket says 172.21.15.22:43060) 2023-09-20T21:50:08.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.613+0000 7f149159b700 1 -- 172.21.15.22:0/853077190 learned_addr learned my addr 172.21.15.22:0/853077190 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:08.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.613+0000 7f1490d9a700 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 0x561d388c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.614+0000 7f1490d9a700 1 -- 172.21.15.22:0/853077190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 msgr2=0x561d388c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.614+0000 7f1490d9a700 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.614+0000 7f1490d9a700 1 -- 172.21.15.22:0/853077190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 msgr2=0x561d388d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.614+0000 7f1490d9a700 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 0x561d388d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.614+0000 7f1490d9a700 1 -- 172.21.15.22:0/853077190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c19d030 con 0x561d388c0050 2023-09-20T21:50:08.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.614+0000 7f1490d9a700 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 0x561d388c9400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x561d388bf490 tx=0x7f14780047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a18a5f7c80e4e85 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.615+0000 7f1474ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:50:08.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.615+0000 7f1474ff9700 1 -- 172.21.15.22:0/853077190 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147801e430 con 0x561d388c0050 2023-09-20T21:50:08.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.615+0000 7f1474ff9700 1 -- 172.21.15.22:0/853077190 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f147801e590 con 0x561d388c0050 2023-09-20T21:50:08.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.615+0000 7f14a7160b80 1 -- 172.21.15.22:0/853077190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d38781bb0 con 0x561d388c0050 2023-09-20T21:50:08.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.616+0000 7f1474ff9700 1 -- 172.21.15.22:0/853077190 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f147801e750 con 0x561d388c0050 2023-09-20T21:50:08.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.616+0000 7f14a7160b80 10 client.25765 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:08.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.616+0000 7f14a7160b80 1 -- 172.21.15.22:0/853077190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561d38624c60 con 0x561d388c0050 2023-09-20T21:50:08.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.616+0000 7f1474ff9700 1 -- 172.21.15.22:0/853077190 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561d38781bb0 con 0x561d388c0050 2023-09-20T21:50:08.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.617+0000 7f1474ff9700 1 -- 172.21.15.22:0/853077190 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f147801d860 con 0x561d388c0050 2023-09-20T21:50:08.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.617+0000 7f14a7160b80 10 client.25765 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:08.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.617+0000 7f14a7160b80 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:50:08.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.617+0000 7f14a7160b80 20 client.25765 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:08.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.617+0000 7f14a7160b80 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561d388d4dc0 0x561d388d71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.617+0000 7f14a7160b80 4 client.25765 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:50:08.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.617+0000 7f14a7160b80 1 -- 172.21.15.22:0/853077190 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d3888e020 con 0x561d388d4dc0 2023-09-20T21:50:08.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.618+0000 7f149159b700 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561d388d4dc0 0x561d388d71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.619+0000 7f149159b700 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561d388d4dc0 0x561d388d71e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abcdc683f613ab93 server_cookie=4484a9963122143d in_seq=0 out_seq=0 2023-09-20T21:50:08.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.619+0000 7f1474ff9700 10 client.25765 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:08.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.620+0000 7f1474ff9700 1 -- 172.21.15.22:0/853077190 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561d3888e020 con 0x561d388d4dc0 2023-09-20T21:50:08.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.620+0000 7f1474ff9700 10 client.25765 handle_command_reply: tid=0 2023-09-20T21:50:08.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.620+0000 7f14a7160b80 1 client.25765 shutdown 2023-09-20T21:50:08.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.622+0000 7f14a7160b80 1 -- 172.21.15.22:0/853077190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 msgr2=0x561d388c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.622+0000 7f14a7160b80 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 0x561d388c9400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x561d388bf490 tx=0x7f14780047e0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.622+0000 7f14a7160b80 1 -- 172.21.15.22:0/853077190 shutdown_connections 2023-09-20T21:50:08.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.622+0000 7f14a7160b80 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561d388d4dc0 0x561d388d71e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.622+0000 7f14a7160b80 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 0x561d388d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.622+0000 7f14a7160b80 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.622+0000 7f1474ff9700 0 client.25765 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:08.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.622+0000 7f14a7160b80 1 --2- 172.21.15.22:0/853077190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 0x561d388c9400 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.622+0000 7f14a7160b80 1 -- 172.21.15.22:0/853077190 >> 172.21.15.22:0/853077190 conn(0x561d387ec000 msgr2=0x561d388b9880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.623+0000 7f14a7160b80 1 -- 172.21.15.22:0/853077190 shutdown_connections 2023-09-20T21:50:08.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.623+0000 7f14a7160b80 1 -- 172.21.15.22:0/853077190 wait complete. 2023-09-20T21:50:08.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.623+0000 7f14a7160b80 20 client.25765 trim_cache size 0 max 16384 2023-09-20T21:50:08.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.647+0000 7f14a7160b80 1 Processor -- start 2023-09-20T21:50:08.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.647+0000 7f14a7160b80 1 -- start start 2023-09-20T21:50:08.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.647+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.647+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 0x561d388d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.647+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 0x561d388c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.647+0000 7f14a7160b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d388c8a90 con 0x561d388c0be0 2023-09-20T21:50:08.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.647+0000 7f14a7160b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d38691920 con 0x561d388bf6f0 2023-09-20T21:50:08.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.647+0000 7f14a7160b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d388bbb90 con 0x561d388c0050 2023-09-20T21:50:08.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.648+0000 7f149159b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.648+0000 7f149159b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43076/0 (socket says 172.21.15.22:43076) 2023-09-20T21:50:08.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.648+0000 7f149159b700 1 -- 172.21.15.22:0/1016944415 learned_addr learned my addr 172.21.15.22:0/1016944415 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:08.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.648+0000 7f1490d9a700 1 --2- 172.21.15.22:0/1016944415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 0x561d388d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.648+0000 7f149159b700 1 -- 172.21.15.22:0/1016944415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 msgr2=0x561d388d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.648+0000 7f149159b700 1 --2- 172.21.15.22:0/1016944415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 0x561d388d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.648+0000 7f149159b700 1 -- 172.21.15.22:0/1016944415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 msgr2=0x561d388c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.648+0000 7f149159b700 1 --2- 172.21.15.22:0/1016944415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 0x561d388c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.648+0000 7f149159b700 1 -- 172.21.15.22:0/1016944415 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d38662250 con 0x561d388bf6f0 2023-09-20T21:50:08.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.649+0000 7f149159b700 1 --2- 172.21.15.22:0/1016944415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c0fc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x561d388d7ac0 tx=0x7f1480017ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60a120f7627397c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.649+0000 7f1477fff700 1 -- 172.21.15.22:0/1016944415 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480017270 con 0x561d388bf6f0 2023-09-20T21:50:08.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.649+0000 7f1477fff700 1 -- 172.21.15.22:0/1016944415 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14800173d0 con 0x561d388bf6f0 2023-09-20T21:50:08.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.649+0000 7f1477fff700 1 -- 172.21.15.22:0/1016944415 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480016780 con 0x561d388bf6f0 2023-09-20T21:50:08.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.650+0000 7f14a7160b80 1 -- 172.21.15.22:0/1016944415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 msgr2=0x561d388c0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.650+0000 7f14a7160b80 1 --2- 172.21.15.22:0/1016944415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c0fc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x561d388d7ac0 tx=0x7f1480017ce0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.650+0000 7f14a7160b80 1 -- 172.21.15.22:0/1016944415 shutdown_connections 2023-09-20T21:50:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.650+0000 7f14a7160b80 1 --2- 172.21.15.22:0/1016944415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 0x561d388c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.650+0000 7f14a7160b80 1 --2- 172.21.15.22:0/1016944415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388bf6f0 0x561d388c0fc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.650+0000 7f14a7160b80 1 --2- 172.21.15.22:0/1016944415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388c0050 0x561d388d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.650+0000 7f14a7160b80 1 -- 172.21.15.22:0/1016944415 >> 172.21.15.22:0/1016944415 conn(0x561d387ec000 msgr2=0x561d388c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.650+0000 7f14a7160b80 1 -- 172.21.15.22:0/1016944415 shutdown_connections 2023-09-20T21:50:08.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.651+0000 7f14a7160b80 1 -- 172.21.15.22:0/1016944415 wait complete. 2023-09-20T21:50:08.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.651+0000 7f14a7160b80 1 -- start start 2023-09-20T21:50:08.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.651+0000 7f14a7160b80 1 Processor -- start 2023-09-20T21:50:08.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.652+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388bf6f0 0x561d388cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.652+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388c0050 0x561d388d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.652+0000 7f14a7160b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 0x561d388ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.652+0000 7f14a7160b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d388ceb20 con 0x561d388c0be0 2023-09-20T21:50:08.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.652+0000 7f14a7160b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d388cec90 con 0x561d388c0050 2023-09-20T21:50:08.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.652+0000 7f14a7160b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d388cee50 con 0x561d388bf6f0 2023-09-20T21:50:08.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.652+0000 7f14a7160b80 10 client.0 _ll_register_callbacks cb 0x561d386f7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:08.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.653+0000 7f149159b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388bf6f0 0x561d388cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.653+0000 7f149159b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388bf6f0 0x561d388cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:55324/0 (socket says 172.21.15.22:55324) 2023-09-20T21:50:08.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.653+0000 7f149159b700 1 -- 172.21.15.22:0/2327225068 learned_addr learned my addr 172.21.15.22:0/2327225068 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:08.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.653+0000 7f149159b700 1 -- 172.21.15.22:0/2327225068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388c0050 msgr2=0x561d388d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.653+0000 7f149159b700 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388c0050 0x561d388d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.653+0000 7f149159b700 1 -- 172.21.15.22:0/2327225068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 msgr2=0x561d388ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:08.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.653+0000 7f149159b700 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 0x561d388ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.653+0000 7f149159b700 1 -- 172.21.15.22:0/2327225068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1480009940 con 0x561d388bf6f0 2023-09-20T21:50:08.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.655+0000 7f149159b700 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388bf6f0 0x561d388cfe30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1480020000 tx=0x7f1480014930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f7a5d8f17456fd server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:08.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.655+0000 7f1474ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:50:08.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.655+0000 7f1474ff9700 1 -- 172.21.15.22:0/2327225068 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480023650 con 0x561d388bf6f0 2023-09-20T21:50:08.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.655+0000 7f1474ff9700 1 -- 172.21.15.22:0/2327225068 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1480006e40 con 0x561d388bf6f0 2023-09-20T21:50:08.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.657+0000 7f14a7160b80 1 -- 172.21.15.22:0/2327225068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14840a37e0 con 0x561d388bf6f0 2023-09-20T21:50:08.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.657+0000 7f1474ff9700 1 -- 172.21.15.22:0/2327225068 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1480017540 con 0x561d388bf6f0 2023-09-20T21:50:08.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.657+0000 7f14a7160b80 10 client.25771 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:08.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.657+0000 7f14a7160b80 1 -- 172.21.15.22:0/2327225068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561d3863a2b0 con 0x561d388bf6f0 2023-09-20T21:50:08.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.658+0000 7f1474ff9700 1 -- 172.21.15.22:0/2327225068 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14840a37e0 con 0x561d388bf6f0 2023-09-20T21:50:08.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.658+0000 7f1474ff9700 1 -- 172.21.15.22:0/2327225068 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f148002b030 con 0x561d388bf6f0 2023-09-20T21:50:08.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.658+0000 7f14a7160b80 10 client.25771 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:08.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.658+0000 7f14a7160b80 10 client.25771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:50:08.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.658+0000 7f14a7160b80 20 client.25771 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:08.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.659+0000 7f14a7160b80 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561d38929f50 0x561d3892c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:08.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.659+0000 7f14a7160b80 4 client.25771 mds_command: new command op to 24578 tid=0[{"prefix": "damage ls"}] 2023-09-20T21:50:08.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.659+0000 7f14a7160b80 1 -- 172.21.15.22:0/2327225068 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561d38809e60 con 0x561d38929f50 2023-09-20T21:50:08.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.659+0000 7f1490d9a700 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561d38929f50 0x561d3892c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:08.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.660+0000 7f1474ff9700 10 client.25771 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:08.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.660+0000 7f1490d9a700 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561d38929f50 0x561d3892c370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32da0a33464004d9 server_cookie=a5d3b33643e926e5 in_seq=0 out_seq=0 2023-09-20T21:50:08.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f1474ff9700 1 -- 172.21.15.22:0/2327225068 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561d38809e60 con 0x561d38929f50 2023-09-20T21:50:08.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f1474ff9700 10 client.25771 handle_command_reply: tid=0 2023-09-20T21:50:08.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f14a7160b80 1 client.25771 shutdown 2023-09-20T21:50:08.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f14a7160b80 1 -- 172.21.15.22:0/2327225068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388bf6f0 msgr2=0x561d388cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f14a7160b80 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388bf6f0 0x561d388cfe30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1480020000 tx=0x7f1480014930 comp rx=0 tx=0).stop 2023-09-20T21:50:08.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f14a7160b80 1 -- 172.21.15.22:0/2327225068 shutdown_connections 2023-09-20T21:50:08.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f14a7160b80 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561d38929f50 0x561d3892c370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f14a7160b80 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d388c0be0 0x561d388ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f14a7160b80 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d388bf6f0 0x561d388cfe30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f14a7160b80 1 --2- 172.21.15.22:0/2327225068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561d388c0050 0x561d388d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f14a7160b80 1 -- 172.21.15.22:0/2327225068 >> 172.21.15.22:0/2327225068 conn(0x561d387ec000 msgr2=0x561d388c70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f1474ff9700 0 client.25771 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:08.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.661+0000 7f14a7160b80 1 -- 172.21.15.22:0/2327225068 shutdown_connections 2023-09-20T21:50:08.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.662+0000 7f14a7160b80 1 -- 172.21.15.22:0/2327225068 wait complete. 2023-09-20T21:50:08.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.662+0000 7f14a7160b80 20 client.25771 trim_cache size 0 max 16384 2023-09-20T21:50:08.669 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2023-09-20T21:50:08.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.663+0000 7f14937ff700 1 -- 172.21.15.22:0/1997369479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f147c0a1450 msgr2=0x7f147c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.663+0000 7f14937ff700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f147c0a1450 0x7f147c0a3910 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f148c070750 tx=0x7f1478006cd0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.663+0000 7f14937ff700 1 -- 172.21.15.22:0/1997369479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c104980 msgr2=0x7f148c19c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:08.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.663+0000 7f14937ff700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c104980 0x7f148c19c390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f148400f690 tx=0x7f148400f770 comp rx=0 tx=0).stop 2023-09-20T21:50:08.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.664+0000 7f14937ff700 1 -- 172.21.15.22:0/1997369479 shutdown_connections 2023-09-20T21:50:08.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.664+0000 7f14937ff700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f148c104980 0x7f148c19c390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.664+0000 7f14937ff700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f147c0a1450 0x7f147c0a3910 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.664+0000 7f14937ff700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f148c0ff640 0x7f148c06f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.664+0000 7f14937ff700 1 --2- 172.21.15.22:0/1997369479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f148c101fb0 0x7f148c06fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:08.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.664+0000 7f14937ff700 1 -- 172.21.15.22:0/1997369479 >> 172.21.15.22:0/1997369479 conn(0x7f148c0fb240 msgr2=0x7f148c100310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:08.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.664+0000 7f14937ff700 1 -- 172.21.15.22:0/1997369479 shutdown_connections 2023-09-20T21:50:08.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:08.664+0000 7f14937ff700 1 -- 172.21.15.22:0/1997369479 wait complete. 2023-09-20T21:50:09.696 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-09-20T21:50:09.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-09-20T21:50:10.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.099+0000 7f747dfff700 1 -- 172.21.15.22:0/2250517873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7478108f90 msgr2=0x7f747810f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.099+0000 7f747dfff700 1 --2- 172.21.15.22:0/2250517873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7478108f90 0x7f747810f840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f747000b0d0 tx=0x7f747000b3e0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.099+0000 7f747dfff700 1 -- 172.21.15.22:0/2250517873 shutdown_connections 2023-09-20T21:50:10.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.099+0000 7f747dfff700 1 --2- 172.21.15.22:0/2250517873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7478108f90 0x7f747810f840 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.099+0000 7f747dfff700 1 --2- 172.21.15.22:0/2250517873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7478107400 0x7f7478107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.099+0000 7f747dfff700 1 --2- 172.21.15.22:0/2250517873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74781085f0 0x7f7478108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.099+0000 7f747dfff700 1 -- 172.21.15.22:0/2250517873 >> 172.21.15.22:0/2250517873 conn(0x7f74780763c0 msgr2=0x7f74780787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.100+0000 7f747dfff700 1 -- 172.21.15.22:0/2250517873 shutdown_connections 2023-09-20T21:50:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.100+0000 7f747dfff700 1 -- 172.21.15.22:0/2250517873 wait complete. 2023-09-20T21:50:10.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.101+0000 7f747dfff700 1 Processor -- start 2023-09-20T21:50:10.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.101+0000 7f747dfff700 1 -- start start 2023-09-20T21:50:10.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.101+0000 7f747dfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7478107400 0x7f747807f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.101+0000 7f747dfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74781085f0 0x7f747807f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.101+0000 7f747dfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7478108f90 0x7f7478080a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.101+0000 7f747dfff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7478080fd0 con 0x7f74781085f0 2023-09-20T21:50:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.101+0000 7f747dfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7478081140 con 0x7f7478107400 2023-09-20T21:50:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.101+0000 7f747dfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7478081300 con 0x7f7478108f90 2023-09-20T21:50:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7477fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7478108f90 0x7f7478080a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7477fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7478108f90 0x7f7478080a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:55334/0 (socket says 172.21.15.22:55334) 2023-09-20T21:50:10.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7477fff700 1 -- 172.21.15.22:0/2720926008 learned_addr learned my addr 172.21.15.22:0/2720926008 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7476ffd700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74781085f0 0x7f747807f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7477fff700 1 -- 172.21.15.22:0/2720926008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7478107400 msgr2=0x7f747807f480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-20T21:50:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7477fff700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7478107400 0x7f747807f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7477fff700 1 -- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74781085f0 msgr2=0x7f747807f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7477fff700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74781085f0 0x7f747807f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7477fff700 1 -- 172.21.15.22:0/2720926008 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7470009d00 con 0x7f7478108f90 2023-09-20T21:50:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7476ffd700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74781085f0 0x7f747807f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:50:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.102+0000 7f7477fff700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7478108f90 0x7f7478080a00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f747000bcd0 tx=0x7f747000bdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.103+0000 7f7474ff9700 1 -- 172.21.15.22:0/2720926008 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747000e040 con 0x7f7478108f90 2023-09-20T21:50:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.103+0000 7f7474ff9700 1 -- 172.21.15.22:0/2720926008 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7470009450 con 0x7f7478108f90 2023-09-20T21:50:10.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.103+0000 7f7474ff9700 1 -- 172.21.15.22:0/2720926008 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747001dbb0 con 0x7f7478108f90 2023-09-20T21:50:10.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.103+0000 7f747dfff700 1 -- 172.21.15.22:0/2720926008 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7478081520 con 0x7f7478108f90 2023-09-20T21:50:10.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.103+0000 7f747dfff700 1 -- 172.21.15.22:0/2720926008 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7478081a00 con 0x7f7478108f90 2023-09-20T21:50:10.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.108+0000 7f7474ff9700 1 -- 172.21.15.22:0/2720926008 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f747001b870 con 0x7f7478108f90 2023-09-20T21:50:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.108+0000 7f74919d9b80 1 Processor -- start 2023-09-20T21:50:10.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.109+0000 7f74919d9b80 1 -- start start 2023-09-20T21:50:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.109+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 0x55ce7c0a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.109+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.109+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.109+0000 7f74919d9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7c018d20 con 0x55ce7c0a50d0 2023-09-20T21:50:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.109+0000 7f74919d9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7bffccf0 con 0x55ce7c0a5c60 2023-09-20T21:50:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.109+0000 7f74919d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7bffa610 con 0x55ce7c0a4770 2023-09-20T21:50:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.109+0000 7f74777fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f74777fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40962/0 (socket says 172.21.15.22:40962) 2023-09-20T21:50:10.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f74777fe700 1 -- 172.21.15.22:0/1909655590 learned_addr learned my addr 172.21.15.22:0/1909655590 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f7476ffd700 1 --2- 172.21.15.22:0/1909655590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f7476ffd700 1 --2- 172.21.15.22:0/1909655590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 0x55ce7c0a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f74777fe700 1 -- 172.21.15.22:0/1909655590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 msgr2=0x55ce7c0a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f74777fe700 1 --2- 172.21.15.22:0/1909655590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 0x55ce7c0a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f74777fe700 1 -- 172.21.15.22:0/1909655590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 msgr2=0x55ce7c0a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f74777fe700 1 --2- 172.21.15.22:0/1909655590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f74777fe700 1 -- 172.21.15.22:0/1909655590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce7be20e60 con 0x55ce7c0a50d0 2023-09-20T21:50:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f7476ffd700 1 --2- 172.21.15.22:0/1909655590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 0x55ce7c0a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:50:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.110+0000 7f74777fe700 1 --2- 172.21.15.22:0/1909655590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0a5660 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7468009c10 tx=0x7f7468009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d55a74e6f6039fe server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.111+0000 7f7474ff9700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f74600a13d0 0x7f74600a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.111+0000 7f745dffb700 1 -- 172.21.15.22:0/1909655590 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746800ebb0 con 0x55ce7c0a50d0 2023-09-20T21:50:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.111+0000 7f745dffb700 1 -- 172.21.15.22:0/1909655590 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7468004cf0 con 0x55ce7c0a50d0 2023-09-20T21:50:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.111+0000 7f745dffb700 1 -- 172.21.15.22:0/1909655590 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74680056f0 con 0x55ce7c0a50d0 2023-09-20T21:50:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.111+0000 7f7474ff9700 1 -- 172.21.15.22:0/2720926008 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f74700ce410 con 0x7f7478108f90 2023-09-20T21:50:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.112+0000 7f7476ffd700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f74600a13d0 0x7f74600a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.112+0000 7f74919d9b80 1 -- 172.21.15.22:0/1909655590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a50d0 msgr2=0x55ce7c0a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.112+0000 7f74919d9b80 1 --2- 172.21.15.22:0/1909655590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0a5660 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7468009c10 tx=0x7f7468009f20 comp rx=0 tx=0).stop 2023-09-20T21:50:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.112+0000 7f74919d9b80 1 -- 172.21.15.22:0/1909655590 shutdown_connections 2023-09-20T21:50:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.112+0000 7f74919d9b80 1 --2- 172.21.15.22:0/1909655590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.112+0000 7f74919d9b80 1 --2- 172.21.15.22:0/1909655590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 0x55ce7c0a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.112+0000 7f74919d9b80 1 --2- 172.21.15.22:0/1909655590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0a5660 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.112+0000 7f74919d9b80 1 -- 172.21.15.22:0/1909655590 >> 172.21.15.22:0/1909655590 conn(0x55ce7bfd1080 msgr2=0x55ce7c09f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.112+0000 7f74919d9b80 1 -- 172.21.15.22:0/1909655590 shutdown_connections 2023-09-20T21:50:10.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.113+0000 7f74919d9b80 1 -- 172.21.15.22:0/1909655590 wait complete. 2023-09-20T21:50:10.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.113+0000 7f7476ffd700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f74600a13d0 0x7f74600a3890 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55ce7c0a0130 tx=0x7f746c008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.113+0000 7f74919d9b80 1 -- start start 2023-09-20T21:50:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.113+0000 7f74919d9b80 1 Processor -- start 2023-09-20T21:50:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 0x55ce7c0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f74919d9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7bff5f00 con 0x55ce7c0a4770 2023-09-20T21:50:10.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f74919d9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7bf927c0 con 0x55ce7c0a5c60 2023-09-20T21:50:10.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f74919d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7bf92460 con 0x55ce7c0a50d0 2023-09-20T21:50:10.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f74919d9b80 10 client.0 _ll_register_callbacks cb 0x55ce7bedcb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:10.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f74777fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 0x55ce7c0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f74777fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 0x55ce7c0ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:40978/0 (socket says 172.21.15.22:40978) 2023-09-20T21:50:10.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f74777fe700 1 -- 172.21.15.22:0/3833305054 learned_addr learned my addr 172.21.15.22:0/3833305054 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f7477fff700 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.114+0000 7f7476ffd700 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.115+0000 7f7477fff700 1 -- 172.21.15.22:0/3833305054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 msgr2=0x55ce7c0ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.115+0000 7f7477fff700 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.115+0000 7f7477fff700 1 -- 172.21.15.22:0/3833305054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 msgr2=0x55ce7c0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.115+0000 7f7477fff700 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 0x55ce7c0ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.115+0000 7f7477fff700 1 -- 172.21.15.22:0/3833305054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74680098f0 con 0x55ce7c0a5c60 2023-09-20T21:50:10.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.115+0000 7f74777fe700 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 0x55ce7c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:50:10.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.115+0000 7f7477fff700 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0b57b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f747000bed0 tx=0x7f747009b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c66de10195f6387 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.115+0000 7f7456ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:50:10.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.115+0000 7f7456ffd700 1 -- 172.21.15.22:0/3833305054 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747009bb40 con 0x55ce7c0a5c60 2023-09-20T21:50:10.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.115+0000 7f7456ffd700 1 -- 172.21.15.22:0/3833305054 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f747009b440 con 0x55ce7c0a5c60 2023-09-20T21:50:10.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.116+0000 7f74919d9b80 1 -- 172.21.15.22:0/3833305054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce7bf66c30 con 0x55ce7c0a5c60 2023-09-20T21:50:10.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.116+0000 7f7456ffd700 1 -- 172.21.15.22:0/3833305054 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747009f270 con 0x55ce7c0a5c60 2023-09-20T21:50:10.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.116+0000 7f74919d9b80 10 client.25793 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:10.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.116+0000 7f74919d9b80 1 -- 172.21.15.22:0/3833305054 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ce7be09ce0 con 0x55ce7c0a5c60 2023-09-20T21:50:10.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.116+0000 7f7456ffd700 1 -- 172.21.15.22:0/3833305054 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce7bf66c30 con 0x55ce7c0a5c60 2023-09-20T21:50:10.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.117+0000 7f7456ffd700 1 -- 172.21.15.22:0/3833305054 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f747009a930 con 0x55ce7c0a5c60 2023-09-20T21:50:10.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.117+0000 7f74919d9b80 10 client.25793 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:10.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.117+0000 7f74919d9b80 10 client.25793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:50:10.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.117+0000 7f74919d9b80 20 client.25793 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:10.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.117+0000 7f74919d9b80 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ce7c0cad40 0x55ce7c0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.117+0000 7f74919d9b80 4 client.25793 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:50:10.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.117+0000 7f74919d9b80 1 -- 172.21.15.22:0/3833305054 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce7c0730a0 con 0x55ce7c0cad40 2023-09-20T21:50:10.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.117+0000 7f74777fe700 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ce7c0cad40 0x55ce7c0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.118+0000 7f74777fe700 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ce7c0cad40 0x55ce7c0cd160 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc1f78a87c141e9 server_cookie=a8a09f28efe2b703 in_seq=0 out_seq=0 2023-09-20T21:50:10.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.118+0000 7f7456ffd700 10 client.25793 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:10.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.118+0000 7f7456ffd700 1 -- 172.21.15.22:0/3833305054 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ce7c0730a0 con 0x55ce7c0cad40 2023-09-20T21:50:10.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.118+0000 7f7456ffd700 10 client.25793 handle_command_reply: tid=0 2023-09-20T21:50:10.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.118+0000 7f74919d9b80 1 client.25793 shutdown 2023-09-20T21:50:10.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 1 -- 172.21.15.22:0/3833305054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 msgr2=0x55ce7c0b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0b57b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f747000bed0 tx=0x7f747009b000 comp rx=0 tx=0).stop 2023-09-20T21:50:10.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 1 -- 172.21.15.22:0/3833305054 shutdown_connections 2023-09-20T21:50:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ce7c0cad40 0x55ce7c0cd160 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0b57b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 0x55ce7c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 1 --2- 172.21.15.22:0/3833305054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 1 -- 172.21.15.22:0/3833305054 >> 172.21.15.22:0/3833305054 conn(0x55ce7bfd1080 msgr2=0x55ce7c09e900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f7456ffd700 0 client.25793 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 1 -- 172.21.15.22:0/3833305054 shutdown_connections 2023-09-20T21:50:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 1 -- 172.21.15.22:0/3833305054 wait complete. 2023-09-20T21:50:10.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.119+0000 7f74919d9b80 20 client.25793 trim_cache size 0 max 16384 2023-09-20T21:50:10.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.140+0000 7f74919d9b80 1 Processor -- start 2023-09-20T21:50:10.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.140+0000 7f74919d9b80 1 -- start start 2023-09-20T21:50:10.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.140+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 0x55ce7c0a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.140+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.140+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0a9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.140+0000 7f74919d9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7be769a0 con 0x55ce7c0a4770 2023-09-20T21:50:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.140+0000 7f74919d9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7c099ca0 con 0x55ce7c0a5c60 2023-09-20T21:50:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.140+0000 7f74919d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7c0adda0 con 0x55ce7c0a50d0 2023-09-20T21:50:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.141+0000 7f7477fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.141+0000 7f7476ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.141+0000 7f7477fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0a9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43132/0 (socket says 172.21.15.22:43132) 2023-09-20T21:50:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.141+0000 7f7477fff700 1 -- 172.21.15.22:0/188679428 learned_addr learned my addr 172.21.15.22:0/188679428 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.141+0000 7f7476ffd700 1 -- 172.21.15.22:0/188679428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 msgr2=0x55ce7c0a9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.141+0000 7f7476ffd700 1 --2- 172.21.15.22:0/188679428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0a9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.141+0000 7f7476ffd700 1 -- 172.21.15.22:0/188679428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 msgr2=0x55ce7c0a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.141+0000 7f7476ffd700 1 --2- 172.21.15.22:0/188679428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 0x55ce7c0a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.141+0000 7f7476ffd700 1 -- 172.21.15.22:0/188679428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce7c09b630 con 0x55ce7c0a50d0 2023-09-20T21:50:10.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.141+0000 7f7477fff700 1 --2- 172.21.15.22:0/188679428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:50:10.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.142+0000 7f7476ffd700 1 --2- 172.21.15.22:0/188679428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f746c00f4c0 tx=0x7f746c004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f31d8b6464fd2f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.142+0000 7f745e7fc700 1 -- 172.21.15.22:0/188679428 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c014560 con 0x55ce7c0a50d0 2023-09-20T21:50:10.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.142+0000 7f745e7fc700 1 -- 172.21.15.22:0/188679428 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f746c014b80 con 0x55ce7c0a50d0 2023-09-20T21:50:10.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.142+0000 7f745e7fc700 1 -- 172.21.15.22:0/188679428 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c021d50 con 0x55ce7c0a50d0 2023-09-20T21:50:10.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.143+0000 7f74919d9b80 1 -- 172.21.15.22:0/188679428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 msgr2=0x55ce7c0b5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.143+0000 7f74919d9b80 1 --2- 172.21.15.22:0/188679428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f746c00f4c0 tx=0x7f746c004790 comp rx=0 tx=0).stop 2023-09-20T21:50:10.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.143+0000 7f74919d9b80 1 -- 172.21.15.22:0/188679428 shutdown_connections 2023-09-20T21:50:10.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.143+0000 7f74919d9b80 1 --2- 172.21.15.22:0/188679428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.143+0000 7f74919d9b80 1 --2- 172.21.15.22:0/188679428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a4770 0x55ce7c0a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.143+0000 7f74919d9b80 1 --2- 172.21.15.22:0/188679428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.143+0000 7f74919d9b80 1 -- 172.21.15.22:0/188679428 >> 172.21.15.22:0/188679428 conn(0x55ce7bfd1080 msgr2=0x55ce7c0ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.144+0000 7f74919d9b80 1 -- 172.21.15.22:0/188679428 shutdown_connections 2023-09-20T21:50:10.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.144+0000 7f74919d9b80 1 -- 172.21.15.22:0/188679428 wait complete. 2023-09-20T21:50:10.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.145+0000 7f74919d9b80 1 -- start start 2023-09-20T21:50:10.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.145+0000 7f74919d9b80 1 Processor -- start 2023-09-20T21:50:10.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.146+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 0x55ce7c0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.146+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.146+0000 7f74919d9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.146+0000 7f74919d9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7c0b3db0 con 0x55ce7c0a5c60 2023-09-20T21:50:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.146+0000 7f74919d9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7c0b3f20 con 0x55ce7c0a50d0 2023-09-20T21:50:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.146+0000 7f74919d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7c0b40e0 con 0x55ce7c0a4770 2023-09-20T21:50:10.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.146+0000 7f74919d9b80 10 client.0 _ll_register_callbacks cb 0x55ce7bedcb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:10.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.146+0000 7f7476ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.146+0000 7f7476ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:43134/0 (socket says 172.21.15.22:43134) 2023-09-20T21:50:10.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.146+0000 7f7476ffd700 1 -- 172.21.15.22:0/349278311 learned_addr learned my addr 172.21.15.22:0/349278311 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:10.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.147+0000 7f74777fe700 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 0x55ce7c0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.147+0000 7f7476ffd700 1 -- 172.21.15.22:0/349278311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 msgr2=0x55ce7c0b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.147+0000 7f7476ffd700 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 0x55ce7c0b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.147+0000 7f7476ffd700 1 -- 172.21.15.22:0/349278311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a5c60 msgr2=0x55ce7c0b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.147+0000 7f7476ffd700 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.147+0000 7f7476ffd700 1 -- 172.21.15.22:0/349278311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce7be09ce0 con 0x55ce7c0a50d0 2023-09-20T21:50:10.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.147+0000 7f7476ffd700 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c910 tx=0x7f746c021c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdae91a8a902659c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.148+0000 7f7456ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:50:10.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.148+0000 7f7456ffd700 1 -- 172.21.15.22:0/349278311 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c021480 con 0x55ce7c0a50d0 2023-09-20T21:50:10.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.148+0000 7f7456ffd700 1 -- 172.21.15.22:0/349278311 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f746c01a040 con 0x55ce7c0a50d0 2023-09-20T21:50:10.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.148+0000 7f74919d9b80 1 -- 172.21.15.22:0/349278311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7468004cf0 con 0x55ce7c0a50d0 2023-09-20T21:50:10.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.148+0000 7f7456ffd700 1 -- 172.21.15.22:0/349278311 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f746c014770 con 0x55ce7c0a50d0 2023-09-20T21:50:10.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.148+0000 7f74919d9b80 10 client.25799 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:10.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.148+0000 7f74919d9b80 1 -- 172.21.15.22:0/349278311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f746c00e040 con 0x55ce7c0a50d0 2023-09-20T21:50:10.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.149+0000 7f7456ffd700 1 -- 172.21.15.22:0/349278311 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7468004cf0 con 0x55ce7c0a50d0 2023-09-20T21:50:10.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.149+0000 7f7456ffd700 1 -- 172.21.15.22:0/349278311 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f746c032030 con 0x55ce7c0a50d0 2023-09-20T21:50:10.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.149+0000 7f74919d9b80 10 client.25799 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:10.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.149+0000 7f74919d9b80 10 client.25799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:50:10.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.149+0000 7f74919d9b80 20 client.25799 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:10.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.149+0000 7f74919d9b80 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ce7c1200c0 0x55ce7c1224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.149+0000 7f74919d9b80 4 client.25799 mds_command: new command op to 24578 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-09-20T21:50:10.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.149+0000 7f74919d9b80 1 -- 172.21.15.22:0/349278311 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ce7bfeeee0 con 0x55ce7c1200c0 2023-09-20T21:50:10.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.149+0000 7f74777fe700 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ce7c1200c0 0x55ce7c1224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.150+0000 7f74777fe700 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ce7c1200c0 0x55ce7c1224e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d9f20805ddc717 server_cookie=43ae99dee936a1d6 in_seq=0 out_seq=0 2023-09-20T21:50:10.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.150+0000 7f7456ffd700 10 client.25799 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:10.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.152+0000 7f7456ffd700 1 -- 172.21.15.22:0/349278311 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ce7bfeeee0 con 0x55ce7c1200c0 2023-09-20T21:50:10.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.152+0000 7f7456ffd700 10 client.25799 handle_command_reply: tid=0 2023-09-20T21:50:10.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.152+0000 7f74919d9b80 1 client.25799 shutdown 2023-09-20T21:50:10.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f74919d9b80 1 -- 172.21.15.22:0/349278311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a50d0 msgr2=0x55ce7c0b5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f74919d9b80 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c910 tx=0x7f746c021c80 comp rx=0 tx=0).stop 2023-09-20T21:50:10.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f74919d9b80 1 -- 172.21.15.22:0/349278311 shutdown_connections 2023-09-20T21:50:10.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f74919d9b80 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ce7c1200c0 0x55ce7c1224e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f74919d9b80 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce7c0a5c60 0x55ce7c0b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f7456ffd700 0 client.25799 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:10.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f74919d9b80 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce7c0a4770 0x55ce7c0b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f74919d9b80 1 --2- 172.21.15.22:0/349278311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ce7c0a50d0 0x55ce7c0b5610 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f74919d9b80 1 -- 172.21.15.22:0/349278311 >> 172.21.15.22:0/349278311 conn(0x55ce7bfd1080 msgr2=0x55ce7c0a8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f74919d9b80 1 -- 172.21.15.22:0/349278311 shutdown_connections 2023-09-20T21:50:10.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.153+0000 7f74919d9b80 1 -- 172.21.15.22:0/349278311 wait complete. 2023-09-20T21:50:10.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.154+0000 7f74919d9b80 20 client.25799 trim_cache size 0 max 16384 2023-09-20T21:50:10.162 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:50:10.162 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2023-09-20T21:50:10.162 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:50:10.163 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2023-09-20T21:50:10.163 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:50:10.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.156+0000 7f747dfff700 1 -- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f74600a13d0 msgr2=0x7f74600a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.156+0000 7f747dfff700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f74600a13d0 0x7f74600a3890 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55ce7c0a0130 tx=0x7f746c008040 comp rx=0 tx=0).stop 2023-09-20T21:50:10.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.156+0000 7f747dfff700 1 -- 172.21.15.22:0/2720926008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7478108f90 msgr2=0x7f7478080a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.156+0000 7f747dfff700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7478108f90 0x7f7478080a00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f747000bcd0 tx=0x7f747000bdb0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.156+0000 7f747dfff700 1 -- 172.21.15.22:0/2720926008 shutdown_connections 2023-09-20T21:50:10.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.156+0000 7f747dfff700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7478108f90 0x7f7478080a00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.156+0000 7f747dfff700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f74600a13d0 0x7f74600a3890 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.156+0000 7f747dfff700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7478107400 0x7f747807f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.156+0000 7f747dfff700 1 --2- 172.21.15.22:0/2720926008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74781085f0 0x7f747807f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.156+0000 7f747dfff700 1 -- 172.21.15.22:0/2720926008 >> 172.21.15.22:0/2720926008 conn(0x7f74780763c0 msgr2=0x7f74780785d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.157+0000 7f747dfff700 1 -- 172.21.15.22:0/2720926008 shutdown_connections 2023-09-20T21:50:10.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.158+0000 7f747dfff700 1 -- 172.21.15.22:0/2720926008 wait complete. 2023-09-20T21:50:10.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:50:10.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:10 smithi022 ceph-mon[97651]: pgmap v384: 129 pgs: 129 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 803 op/s 2023-09-20T21:50:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.546+0000 7fee7a1ff700 1 -- 172.21.15.22:0/1395320443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee74101fe0 msgr2=0x7fee74102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.546+0000 7fee7a1ff700 1 --2- 172.21.15.22:0/1395320443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee74101fe0 0x7fee74102440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fee680098f0 tx=0x7fee68009c00 comp rx=0 tx=0).stop 2023-09-20T21:50:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.546+0000 7fee7a1ff700 1 -- 172.21.15.22:0/1395320443 shutdown_connections 2023-09-20T21:50:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.546+0000 7fee7a1ff700 1 --2- 172.21.15.22:0/1395320443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee74102980 0x7fee74109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.546+0000 7fee7a1ff700 1 --2- 172.21.15.22:0/1395320443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee74100df0 0x7fee74101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.546+0000 7fee7a1ff700 1 --2- 172.21.15.22:0/1395320443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee74101fe0 0x7fee74102440 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.546+0000 7fee7a1ff700 1 -- 172.21.15.22:0/1395320443 >> 172.21.15.22:0/1395320443 conn(0x7fee740fc360 msgr2=0x7fee740fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.546+0000 7fee7a1ff700 1 -- 172.21.15.22:0/1395320443 shutdown_connections 2023-09-20T21:50:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.546+0000 7fee7a1ff700 1 -- 172.21.15.22:0/1395320443 wait complete. 2023-09-20T21:50:10.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.547+0000 7fee7a1ff700 1 Processor -- start 2023-09-20T21:50:10.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee7a1ff700 1 -- start start 2023-09-20T21:50:10.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee7a1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee74100df0 0x7fee74196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee7a1ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee74101fe0 0x7fee74197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee7a1ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee74102980 0x7fee7419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee7a1ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7419e6c0 con 0x7fee74101fe0 2023-09-20T21:50:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee7a1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7419e800 con 0x7fee74100df0 2023-09-20T21:50:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee7a1ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7419e990 con 0x7fee74102980 2023-09-20T21:50:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee72ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee74101fe0 0x7fee74197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee72ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee74101fe0 0x7fee74197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41040/0 (socket says 172.21.15.22:41040) 2023-09-20T21:50:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee72ffd700 1 -- 172.21.15.22:0/831335706 learned_addr learned my addr 172.21.15.22:0/831335706 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee73fff700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee74102980 0x7fee7419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.548+0000 7fee737fe700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee74100df0 0x7fee74196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee72ffd700 1 -- 172.21.15.22:0/831335706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee74102980 msgr2=0x7fee7419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee72ffd700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee74102980 0x7fee7419e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee72ffd700 1 -- 172.21.15.22:0/831335706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee74100df0 msgr2=0x7fee74196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee72ffd700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee74100df0 0x7fee74196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee72ffd700 1 -- 172.21.15.22:0/831335706 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68009520 con 0x7fee74101fe0 2023-09-20T21:50:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee72ffd700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee74101fe0 0x7fee74197080 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fee68009fd0 tx=0x7fee6800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee70ff9700 1 -- 172.21.15.22:0/831335706 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6801c070 con 0x7fee74101fe0 2023-09-20T21:50:10.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee70ff9700 1 -- 172.21.15.22:0/831335706 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee68005040 con 0x7fee74101fe0 2023-09-20T21:50:10.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee70ff9700 1 -- 172.21.15.22:0/831335706 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68004a20 con 0x7fee74101fe0 2023-09-20T21:50:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee7a1ff700 1 -- 172.21.15.22:0/831335706 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7419ec10 con 0x7fee74101fe0 2023-09-20T21:50:10.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.549+0000 7fee7a1ff700 1 -- 172.21.15.22:0/831335706 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7419efe0 con 0x7fee74101fe0 2023-09-20T21:50:10.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.553+0000 7fee70ff9700 1 -- 172.21.15.22:0/831335706 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fee68020070 con 0x7fee74101fe0 2023-09-20T21:50:10.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.554+0000 7fee8dc10b80 1 Processor -- start 2023-09-20T21:50:10.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.554+0000 7fee8dc10b80 1 -- start start 2023-09-20T21:50:10.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.554+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.554+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.554+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0192b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.554+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e019224ca0 con 0x55e0192b4b00 2023-09-20T21:50:10.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.554+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e019208c70 con 0x55e0192b5ff0 2023-09-20T21:50:10.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.554+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e019206590 con 0x55e0192b5460 2023-09-20T21:50:10.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.554+0000 7fee70ff9700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fee5c0a1300 0x7fee5c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee737fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee737fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:55398/0 (socket says 172.21.15.22:55398) 2023-09-20T21:50:10.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee737fe700 1 -- 172.21.15.22:0/126165965 learned_addr learned my addr 172.21.15.22:0/126165965 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:10.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee70ff9700 1 -- 172.21.15.22:0/831335706 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fee680cdcc0 con 0x7fee74101fe0 2023-09-20T21:50:10.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee72ffd700 1 --2- 172.21.15.22:0/126165965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0192b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee73fff700 1 --2- 172.21.15.22:0/126165965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee737fe700 1 -- 172.21.15.22:0/126165965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 msgr2=0x55e0192b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee737fe700 1 --2- 172.21.15.22:0/126165965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0192b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee737fe700 1 -- 172.21.15.22:0/126165965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 msgr2=0x55e0192b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee737fe700 1 --2- 172.21.15.22:0/126165965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee73fff700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fee5c0a1300 0x7fee5c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee737fe700 1 -- 172.21.15.22:0/126165965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e01902cde0 con 0x55e0192b5460 2023-09-20T21:50:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.555+0000 7fee737fe700 1 --2- 172.21.15.22:0/126165965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b59f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fee64009d30 tx=0x7fee64011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287504de1351aef8 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee59ffb700 1 -- 172.21.15.22:0/126165965 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64004220 con 0x55e0192b5460 2023-09-20T21:50:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee59ffb700 1 -- 172.21.15.22:0/126165965 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee64004cf0 con 0x55e0192b5460 2023-09-20T21:50:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee59ffb700 1 -- 172.21.15.22:0/126165965 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee640054e0 con 0x55e0192b5460 2023-09-20T21:50:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee73fff700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fee5c0a1300 0x7fee5c0a37c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009ec0 tx=0x7fee6c00b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee8dc10b80 1 -- 172.21.15.22:0/126165965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 msgr2=0x55e0192b59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/126165965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b59f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fee64009d30 tx=0x7fee64011720 comp rx=0 tx=0).stop 2023-09-20T21:50:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee8dc10b80 1 -- 172.21.15.22:0/126165965 shutdown_connections 2023-09-20T21:50:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/126165965 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0192b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/126165965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/126165965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b59f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee8dc10b80 1 -- 172.21.15.22:0/126165965 >> 172.21.15.22:0/126165965 conn(0x55e0191dd000 msgr2=0x55e0192ab120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.556+0000 7fee8dc10b80 1 -- 172.21.15.22:0/126165965 shutdown_connections 2023-09-20T21:50:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.557+0000 7fee8dc10b80 1 -- 172.21.15.22:0/126165965 wait complete. 2023-09-20T21:50:10.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.557+0000 7fee8dc10b80 1 -- start start 2023-09-20T21:50:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.557+0000 7fee8dc10b80 1 Processor -- start 2023-09-20T21:50:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0192b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e019201e80 con 0x55e0192b4b00 2023-09-20T21:50:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01919e740 con 0x55e0192b5ff0 2023-09-20T21:50:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e01919e3e0 con 0x55e0192b5460 2023-09-20T21:50:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee8dc10b80 10 client.0 _ll_register_callbacks cb 0x55e0190e8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee72ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee72ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:55400/0 (socket says 172.21.15.22:55400) 2023-09-20T21:50:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee72ffd700 1 -- 172.21.15.22:0/2362748166 learned_addr learned my addr 172.21.15.22:0/2362748166 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee737fe700 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee73fff700 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0192b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee72ffd700 1 -- 172.21.15.22:0/2362748166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 msgr2=0x55e0192b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee72ffd700 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0192b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee72ffd700 1 -- 172.21.15.22:0/2362748166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 msgr2=0x55e0192b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee72ffd700 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.558+0000 7fee72ffd700 1 -- 172.21.15.22:0/2362748166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee64009960 con 0x55e0192b5460 2023-09-20T21:50:10.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.559+0000 7fee72ffd700 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b2030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee680cb3d0 tx=0x7fee6809a700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31c2a1407ac7547c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.559+0000 7fee52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:50:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.559+0000 7fee52ffd700 1 -- 172.21.15.22:0/2362748166 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6809eb20 con 0x55e0192b5460 2023-09-20T21:50:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.559+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2362748166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e019172bb0 con 0x55e0192b5460 2023-09-20T21:50:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.559+0000 7fee52ffd700 1 -- 172.21.15.22:0/2362748166 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fee680a2cd0 con 0x55e0192b5460 2023-09-20T21:50:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.559+0000 7fee52ffd700 1 -- 172.21.15.22:0/2362748166 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee6809af50 con 0x55e0192b5460 2023-09-20T21:50:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.559+0000 7fee8dc10b80 10 client.25798 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.559+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2362748166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e019015c60 con 0x55e0192b5460 2023-09-20T21:50:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.559+0000 7fee52ffd700 1 -- 172.21.15.22:0/2362748166 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e019172bb0 con 0x55e0192b5460 2023-09-20T21:50:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.559+0000 7fee52ffd700 1 -- 172.21.15.22:0/2362748166 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fee680a2840 con 0x55e0192b5460 2023-09-20T21:50:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.560+0000 7fee8dc10b80 10 client.25798 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.560+0000 7fee8dc10b80 10 client.25798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:50:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.560+0000 7fee8dc10b80 20 client.25798 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.560+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e0192c5c60 0x55e0192c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.560+0000 7fee8dc10b80 4 client.25798 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:50:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.560+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2362748166 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e01927f020 con 0x55e0192c5c60 2023-09-20T21:50:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.560+0000 7fee737fe700 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e0192c5c60 0x55e0192c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.560+0000 7fee737fe700 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e0192c5c60 0x55e0192c8080 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a471c6984185e5ce server_cookie=68b12b1c6c47bf5f in_seq=0 out_seq=0 2023-09-20T21:50:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.560+0000 7fee52ffd700 10 client.25798 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.561+0000 7fee52ffd700 1 -- 172.21.15.22:0/2362748166 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e01927f020 con 0x55e0192c5c60 2023-09-20T21:50:10.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.561+0000 7fee52ffd700 10 client.25798 handle_command_reply: tid=0 2023-09-20T21:50:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.561+0000 7fee8dc10b80 1 client.25798 shutdown 2023-09-20T21:50:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.561+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2362748166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 msgr2=0x55e0192b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.561+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b2030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee680cb3d0 tx=0x7fee6809a700 comp rx=0 tx=0).stop 2023-09-20T21:50:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.561+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2362748166 shutdown_connections 2023-09-20T21:50:10.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.561+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e0192c5c60 0x55e0192c8080 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.561+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0192b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.561+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.562+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/2362748166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e0192b2030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.562+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2362748166 >> 172.21.15.22:0/2362748166 conn(0x55e0191dd000 msgr2=0x55e0192aa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.562+0000 7fee52ffd700 0 client.25798 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:10.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.562+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2362748166 shutdown_connections 2023-09-20T21:50:10.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.562+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2362748166 wait complete. 2023-09-20T21:50:10.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.562+0000 7fee8dc10b80 20 client.25798 trim_cache size 0 max 16384 2023-09-20T21:50:10.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.579+0000 7fee8dc10b80 1 Processor -- start 2023-09-20T21:50:10.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee8dc10b80 1 -- start start 2023-09-20T21:50:10.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5460 0x55e0192bac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5ff0 0x55e0192c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e019082920 con 0x55e0192b4b00 2023-09-20T21:50:10.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0192b5e50 con 0x55e0192b5460 2023-09-20T21:50:10.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0192acc40 con 0x55e0192b5ff0 2023-09-20T21:50:10.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee737fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee737fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41060/0 (socket says 172.21.15.22:41060) 2023-09-20T21:50:10.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee737fe700 1 -- 172.21.15.22:0/2222548813 learned_addr learned my addr 172.21.15.22:0/2222548813 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:10.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee72ffd700 1 --2- 172.21.15.22:0/2222548813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5460 0x55e0192bac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.580+0000 7fee73fff700 1 --2- 172.21.15.22:0/2222548813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5ff0 0x55e0192c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.581+0000 7fee72ffd700 1 -- 172.21.15.22:0/2222548813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5ff0 msgr2=0x55e0192c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.581+0000 7fee72ffd700 1 --2- 172.21.15.22:0/2222548813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5ff0 0x55e0192c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.581+0000 7fee72ffd700 1 -- 172.21.15.22:0/2222548813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 msgr2=0x55e0192b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.581+0000 7fee72ffd700 1 --2- 172.21.15.22:0/2222548813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.581+0000 7fee72ffd700 1 -- 172.21.15.22:0/2222548813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e019053250 con 0x55e0192b5460 2023-09-20T21:50:10.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.581+0000 7fee737fe700 1 --2- 172.21.15.22:0/2222548813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:50:10.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.581+0000 7fee72ffd700 1 --2- 172.21.15.22:0/2222548813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5460 0x55e0192bac30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fee6800c010 tx=0x7fee680ba050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cdff9e5f5a5b70 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.581+0000 7fee5a7fc700 1 -- 172.21.15.22:0/2222548813 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680ba2b0 con 0x55e0192b5460 2023-09-20T21:50:10.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.581+0000 7fee5a7fc700 1 -- 172.21.15.22:0/2222548813 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee680a2cd0 con 0x55e0192b5460 2023-09-20T21:50:10.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.581+0000 7fee5a7fc700 1 -- 172.21.15.22:0/2222548813 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6809b1b0 con 0x55e0192b5460 2023-09-20T21:50:10.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2222548813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5460 msgr2=0x55e0192bac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/2222548813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5460 0x55e0192bac30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fee6800c010 tx=0x7fee680ba050 comp rx=0 tx=0).stop 2023-09-20T21:50:10.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2222548813 shutdown_connections 2023-09-20T21:50:10.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/2222548813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5ff0 0x55e0192c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/2222548813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e0192b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/2222548813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5460 0x55e0192bac30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2222548813 >> 172.21.15.22:0/2222548813 conn(0x55e0191dd000 msgr2=0x55e0192b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2222548813 shutdown_connections 2023-09-20T21:50:10.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 -- 172.21.15.22:0/2222548813 wait complete. 2023-09-20T21:50:10.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 -- start start 2023-09-20T21:50:10.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.582+0000 7fee8dc10b80 1 Processor -- start 2023-09-20T21:50:10.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e019313a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e019315fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee8dc10b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0193184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e019318b10 con 0x55e0192b4b00 2023-09-20T21:50:10.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e019318c80 con 0x55e0192b5ff0 2023-09-20T21:50:10.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee8dc10b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e019318e40 con 0x55e0192b5460 2023-09-20T21:50:10.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee8dc10b80 10 client.0 _ll_register_callbacks cb 0x55e0190e8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:10.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee737fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e019313a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee73fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0193184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee737fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e019313a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41072/0 (socket says 172.21.15.22:41072) 2023-09-20T21:50:10.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee737fe700 1 -- 172.21.15.22:0/1692289098 learned_addr learned my addr 172.21.15.22:0/1692289098 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:10.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee72ffd700 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e019315fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee73fff700 1 -- 172.21.15.22:0/1692289098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 msgr2=0x55e019315fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.583+0000 7fee73fff700 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e019315fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee73fff700 1 -- 172.21.15.22:0/1692289098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 msgr2=0x55e019313a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:10.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee73fff700 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e019313a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee73fff700 1 -- 172.21.15.22:0/1692289098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e01902cde0 con 0x55e0192b5ff0 2023-09-20T21:50:10.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee737fe700 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e019313a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:50:10.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee73fff700 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0193184f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55e0192ac000 tx=0x7fee6c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3e381246adcce5b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:10.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:50:10.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee52ffd700 1 -- 172.21.15.22:0/1692289098 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00e030 con 0x55e0192b5ff0 2023-09-20T21:50:10.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee8dc10b80 1 -- 172.21.15.22:0/1692289098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee64004cf0 con 0x55e0192b5ff0 2023-09-20T21:50:10.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee52ffd700 1 -- 172.21.15.22:0/1692289098 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fee6c004db0 con 0x55e0192b5ff0 2023-09-20T21:50:10.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee52ffd700 1 -- 172.21.15.22:0/1692289098 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee6c029430 con 0x55e0192b5ff0 2023-09-20T21:50:10.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee8dc10b80 10 client.25817 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:10.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.584+0000 7fee8dc10b80 1 -- 172.21.15.22:0/1692289098 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e019015c60 con 0x55e0192b5ff0 2023-09-20T21:50:10.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.585+0000 7fee52ffd700 1 -- 172.21.15.22:0/1692289098 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee64004cf0 con 0x55e0192b5ff0 2023-09-20T21:50:10.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.585+0000 7fee52ffd700 1 -- 172.21.15.22:0/1692289098 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fee6c02f030 con 0x55e0192b5ff0 2023-09-20T21:50:10.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.585+0000 7fee8dc10b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:10.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.585+0000 7fee8dc10b80 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:50:10.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.585+0000 7fee8dc10b80 20 client.25817 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:10.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.585+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e01931aa30 0x55e01931ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:10.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.585+0000 7fee8dc10b80 4 client.25817 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:50:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.585+0000 7fee8dc10b80 1 -- 172.21.15.22:0/1692289098 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e0191fae60 con 0x55e01931aa30 2023-09-20T21:50:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.585+0000 7fee737fe700 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e01931aa30 0x55e01931ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee737fe700 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e01931aa30 0x55e01931ce50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb067cf601d561da server_cookie=c480d78abb8d8796 in_seq=0 out_seq=0 2023-09-20T21:50:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee52ffd700 10 client.25817 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee52ffd700 1 -- 172.21.15.22:0/1692289098 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e0191fae60 con 0x55e01931aa30 2023-09-20T21:50:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee52ffd700 10 client.25817 handle_command_reply: tid=0 2023-09-20T21:50:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 client.25817 shutdown 2023-09-20T21:50:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 -- 172.21.15.22:0/1692289098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 msgr2=0x55e0193184f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0193184f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55e0192ac000 tx=0x7fee6c004ad0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 -- 172.21.15.22:0/1692289098 shutdown_connections 2023-09-20T21:50:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e01931aa30 0x55e01931ce50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e0192b5ff0 0x55e0193184f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e0192b4b00 0x55e019313a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee52ffd700 0 client.25817 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 --2- 172.21.15.22:0/1692289098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e0192b5460 0x55e019315fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 -- 172.21.15.22:0/1692289098 >> 172.21.15.22:0/1692289098 conn(0x55e0191dd000 msgr2=0x55e0192c7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 -- 172.21.15.22:0/1692289098 shutdown_connections 2023-09-20T21:50:10.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 1 -- 172.21.15.22:0/1692289098 wait complete. 2023-09-20T21:50:10.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.586+0000 7fee8dc10b80 20 client.25817 trim_cache size 0 max 16384 2023-09-20T21:50:10.613 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:50:10.614 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:50:10.614 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:50:10.614 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:50:10.614 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:50:10.614 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:50:10.615 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:50:10.615 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:50:10.615 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:50:10.615 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:50:10.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 -- 172.21.15.22:0/831335706 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fee5c0a1300 msgr2=0x7fee5c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fee5c0a1300 0x7fee5c0a37c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009ec0 tx=0x7fee6c00b040 comp rx=0 tx=0).stop 2023-09-20T21:50:10.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 -- 172.21.15.22:0/831335706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee74101fe0 msgr2=0x7fee74197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:10.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee74101fe0 0x7fee74197080 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fee68009fd0 tx=0x7fee6800b560 comp rx=0 tx=0).stop 2023-09-20T21:50:10.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 -- 172.21.15.22:0/831335706 shutdown_connections 2023-09-20T21:50:10.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee74102980 0x7fee7419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fee5c0a1300 0x7fee5c0a37c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee74100df0 0x7fee74196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 --2- 172.21.15.22:0/831335706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee74101fe0 0x7fee74197080 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:10.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 -- 172.21.15.22:0/831335706 >> 172.21.15.22:0/831335706 conn(0x7fee740fc360 msgr2=0x7fee74105540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:10.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 -- 172.21.15.22:0/831335706 shutdown_connections 2023-09-20T21:50:10.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:10.587+0000 7fee7a1ff700 1 -- 172.21.15.22:0/831335706 wait complete. 2023-09-20T21:50:10.619 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:50:10.620 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:10 smithi089 ceph-mon[134090]: pgmap v384: 129 pgs: 129 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 803 op/s 2023-09-20T21:50:10.620 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:10 smithi123 ceph-mon[134433]: pgmap v384: 129 pgs: 129 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 MiB/s wr, 803 op/s 2023-09-20T21:50:11.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub queued for path: / 2023-09-20T21:50:11.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:50:11.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:50:11.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:50:11.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:50:11.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:50:11.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:50:11.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:50:11.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: active paths [/] 2023-09-20T21:50:11.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.597 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:11 smithi123 ceph-mon[134433]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub queued for path: / 2023-09-20T21:50:11.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:50:11.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:50:11.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:50:11.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:50:11.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:50:11.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:50:11.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:50:11.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: active paths [/] 2023-09-20T21:50:11.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:11 smithi089 ceph-mon[134090]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub queued for path: / 2023-09-20T21:50:11.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:50:11.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:50:11.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:50:11.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:50:11.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:50:11.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:50:11.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:50:11.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: active paths [/] 2023-09-20T21:50:11.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:11.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:11 smithi022 ceph-mon[97651]: scrub summary: idle+waiting paths [/] 2023-09-20T21:50:12.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:12 smithi123 ceph-mon[134433]: pgmap v385: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 801 op/s 2023-09-20T21:50:12.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:12 smithi089 ceph-mon[134090]: pgmap v385: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 801 op/s 2023-09-20T21:50:12.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:12 smithi022 ceph-mon[97651]: pgmap v385: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 801 op/s 2023-09-20T21:50:13.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 50.0 s, 1062.9 tps, lat 97.293 ms stddev 137.495 2023-09-20T21:50:14.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:13 smithi022 ceph-mon[97651]: pgmap v386: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 17 MiB/s wr, 732 op/s 2023-09-20T21:50:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:13 smithi123 ceph-mon[134433]: pgmap v386: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 17 MiB/s wr, 732 op/s 2023-09-20T21:50:14.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:13 smithi089 ceph-mon[134090]: pgmap v386: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 17 MiB/s wr, 732 op/s 2023-09-20T21:50:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:16 smithi123 ceph-mon[134433]: pgmap v387: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 20 MiB/s wr, 852 op/s 2023-09-20T21:50:16.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:16 smithi089 ceph-mon[134090]: pgmap v387: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 20 MiB/s wr, 852 op/s 2023-09-20T21:50:16.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:16 smithi022 ceph-mon[97651]: pgmap v387: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 20 MiB/s wr, 852 op/s 2023-09-20T21:50:18.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 55.0 s, 1311.1 tps, lat 76.379 ms stddev 95.219 2023-09-20T21:50:18.598 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:18 smithi123 ceph-mon[134433]: pgmap v388: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 25 MiB/s wr, 891 op/s 2023-09-20T21:50:18.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:18 smithi089 ceph-mon[134090]: pgmap v388: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 25 MiB/s wr, 891 op/s 2023-09-20T21:50:18.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:18 smithi022 ceph-mon[97651]: pgmap v388: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 25 MiB/s wr, 891 op/s 2023-09-20T21:50:20.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:20 smithi123 ceph-mon[134433]: pgmap v389: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 25 MiB/s wr, 864 op/s 2023-09-20T21:50:20.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:20 smithi089 ceph-mon[134090]: pgmap v389: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 25 MiB/s wr, 864 op/s 2023-09-20T21:50:20.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:20 smithi022 ceph-mon[97651]: pgmap v389: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 25 MiB/s wr, 864 op/s 2023-09-20T21:50:22.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:22 smithi123 ceph-mon[134433]: pgmap v390: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 26 MiB/s wr, 871 op/s 2023-09-20T21:50:22.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:22 smithi089 ceph-mon[134090]: pgmap v390: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 26 MiB/s wr, 871 op/s 2023-09-20T21:50:22.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:22 smithi022 ceph-mon[97651]: pgmap v390: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 26 MiB/s wr, 871 op/s 2023-09-20T21:50:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 60.0 s, 1327.3 tps, lat 75.062 ms stddev 85.234 2023-09-20T21:50:23.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:23 smithi022 ceph-mon[97651]: pgmap v391: 129 pgs: 129 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 27 MiB/s wr, 819 op/s 2023-09-20T21:50:24.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:23 smithi123 ceph-mon[134433]: pgmap v391: 129 pgs: 129 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 27 MiB/s wr, 819 op/s 2023-09-20T21:50:24.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:23 smithi089 ceph-mon[134090]: pgmap v391: 129 pgs: 129 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 27 MiB/s wr, 819 op/s 2023-09-20T21:50:26.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:26 smithi022 ceph-mon[97651]: pgmap v392: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 27 MiB/s wr, 892 op/s 2023-09-20T21:50:26.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:26 smithi123 ceph-mon[134433]: pgmap v392: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 27 MiB/s wr, 892 op/s 2023-09-20T21:50:26.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:26 smithi089 ceph-mon[134090]: pgmap v392: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 27 MiB/s wr, 892 op/s 2023-09-20T21:50:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 65.0 s, 826.4 tps, lat 113.044 ms stddev 247.265 2023-09-20T21:50:28.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:28 smithi123 ceph-mon[134433]: pgmap v393: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 28 MiB/s wr, 865 op/s 2023-09-20T21:50:28.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:28 smithi089 ceph-mon[134090]: pgmap v393: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 28 MiB/s wr, 865 op/s 2023-09-20T21:50:28.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:28 smithi022 ceph-mon[97651]: pgmap v393: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 28 MiB/s wr, 865 op/s 2023-09-20T21:50:30.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:30 smithi123 ceph-mon[134433]: pgmap v394: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 33 MiB/s wr, 1.41k op/s 2023-09-20T21:50:30.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:30 smithi089 ceph-mon[134090]: pgmap v394: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 33 MiB/s wr, 1.41k op/s 2023-09-20T21:50:30.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:30 smithi022 ceph-mon[97651]: pgmap v394: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 33 MiB/s wr, 1.41k op/s 2023-09-20T21:50:31.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:31 smithi022 ceph-mon[97651]: pgmap v395: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2023-09-20T21:50:31.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:31 smithi123 ceph-mon[134433]: pgmap v395: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2023-09-20T21:50:31.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:31 smithi089 ceph-mon[134090]: pgmap v395: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 37 MiB/s wr, 1.83k op/s 2023-09-20T21:50:33.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 70.0 s, 1566.6 tps, lat 68.793 ms stddev 112.359 2023-09-20T21:50:33.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:33 smithi022 ceph-mon[97651]: pgmap v396: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 33 MiB/s wr, 1.80k op/s 2023-09-20T21:50:34.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:33 smithi123 ceph-mon[134433]: pgmap v396: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 33 MiB/s wr, 1.80k op/s 2023-09-20T21:50:34.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:33 smithi089 ceph-mon[134090]: pgmap v396: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 33 MiB/s wr, 1.80k op/s 2023-09-20T21:50:36.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:36 smithi123 ceph-mon[134433]: pgmap v397: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 33 MiB/s wr, 1.86k op/s 2023-09-20T21:50:36.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:36 smithi089 ceph-mon[134090]: pgmap v397: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 33 MiB/s wr, 1.86k op/s 2023-09-20T21:50:36.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:36 smithi022 ceph-mon[97651]: pgmap v397: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 33 MiB/s wr, 1.86k op/s 2023-09-20T21:50:38.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 75.0 s, 1583.1 tps, lat 63.474 ms stddev 74.397 2023-09-20T21:50:38.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:38 smithi123 ceph-mon[134433]: pgmap v398: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 MiB/s wr, 1.80k op/s 2023-09-20T21:50:38.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:38 smithi089 ceph-mon[134090]: pgmap v398: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 MiB/s wr, 1.80k op/s 2023-09-20T21:50:38.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:38 smithi022 ceph-mon[97651]: pgmap v398: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 MiB/s wr, 1.80k op/s 2023-09-20T21:50:40.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:40 smithi123 ceph-mon[134433]: pgmap v399: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 MiB/s wr, 1.79k op/s 2023-09-20T21:50:40.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:40 smithi089 ceph-mon[134090]: pgmap v399: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 MiB/s wr, 1.79k op/s 2023-09-20T21:50:40.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:50:40.737 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:40 smithi022 ceph-mon[97651]: pgmap v399: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 MiB/s wr, 1.79k op/s 2023-09-20T21:50:41.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.008+0000 7f5efebff700 1 -- 172.21.15.22:0/3857654419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef8104180 msgr2=0x7f5ef8106570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:41.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.008+0000 7f5efebff700 1 --2- 172.21.15.22:0/3857654419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef8104180 0x7f5ef8106570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000b0a0 tx=0x7f5ef000b3b0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.008+0000 7f5efebff700 1 -- 172.21.15.22:0/3857654419 shutdown_connections 2023-09-20T21:50:41.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.008+0000 7f5efebff700 1 --2- 172.21.15.22:0/3857654419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ef8106ab0 0x7f5ef8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.008+0000 7f5efebff700 1 --2- 172.21.15.22:0/3857654419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef8101850 0x7f5ef8103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.008+0000 7f5efebff700 1 --2- 172.21.15.22:0/3857654419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef8104180 0x7f5ef8106570 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.008+0000 7f5efebff700 1 -- 172.21.15.22:0/3857654419 >> 172.21.15.22:0/3857654419 conn(0x7f5ef80fb240 msgr2=0x7f5ef80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.008+0000 7f5efebff700 1 -- 172.21.15.22:0/3857654419 shutdown_connections 2023-09-20T21:50:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.009+0000 7f5efebff700 1 -- 172.21.15.22:0/3857654419 wait complete. 2023-09-20T21:50:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.010+0000 7f5efebff700 1 Processor -- start 2023-09-20T21:50:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.010+0000 7f5efebff700 1 -- start start 2023-09-20T21:50:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.011+0000 7f5efebff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef8101850 0x7f5ef8196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.011+0000 7f5efebff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ef8104180 0x7f5ef8197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.011+0000 7f5efebff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef8106ab0 0x7f5ef819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.011+0000 7f5efebff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef819e790 con 0x7f5ef8104180 2023-09-20T21:50:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.011+0000 7f5efebff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef806a830 con 0x7f5ef8106ab0 2023-09-20T21:50:41.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.011+0000 7f5efebff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef806a9f0 con 0x7f5ef8101850 2023-09-20T21:50:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.011+0000 7f5efd19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef8106ab0 0x7f5ef819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.011+0000 7f5efd19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef8106ab0 0x7f5ef819e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37630/0 (socket says 172.21.15.22:37630) 2023-09-20T21:50:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.011+0000 7f5efd19c700 1 -- 172.21.15.22:0/2497883028 learned_addr learned my addr 172.21.15.22:0/2497883028 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.011+0000 7f5efc99b700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef8101850 0x7f5ef8196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.012+0000 7f5efc99b700 1 -- 172.21.15.22:0/2497883028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef8106ab0 msgr2=0x7f5ef819e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.012+0000 7f5efc99b700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef8106ab0 0x7f5ef819e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.012+0000 7f5efc99b700 1 -- 172.21.15.22:0/2497883028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ef8104180 msgr2=0x7f5ef8197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.012+0000 7f5efc99b700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ef8104180 0x7f5ef8197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.012+0000 7f5efc99b700 1 -- 172.21.15.22:0/2497883028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef0009d00 con 0x7f5ef8101850 2023-09-20T21:50:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.012+0000 7f5efc99b700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef8101850 0x7f5ef8196c50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009c50 tx=0x7f5eec009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.013+0000 7f5ef5ffb700 1 -- 172.21.15.22:0/2497883028 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec015070 con 0x7f5ef8101850 2023-09-20T21:50:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.013+0000 7f5ef5ffb700 1 -- 172.21.15.22:0/2497883028 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eec019410 con 0x7f5ef8101850 2023-09-20T21:50:41.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.013+0000 7f5efebff700 1 -- 172.21.15.22:0/2497883028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef80ff6a0 con 0x7f5ef8101850 2023-09-20T21:50:41.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.013+0000 7f5ef5ffb700 1 -- 172.21.15.22:0/2497883028 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec018610 con 0x7f5ef8101850 2023-09-20T21:50:41.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.013+0000 7f5efebff700 1 -- 172.21.15.22:0/2497883028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef80ffbb0 con 0x7f5ef8101850 2023-09-20T21:50:41.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.022+0000 7f5f124eeb80 1 Processor -- start 2023-09-20T21:50:41.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.022+0000 7f5f124eeb80 1 -- start start 2023-09-20T21:50:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.023+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e196f0 0x55c2f0e19b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.023+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1a050 0x55c2f0e1a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.023+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e1b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.023+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0d8dca0 con 0x55c2f0e1abe0 2023-09-20T21:50:41.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.023+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0d71c70 con 0x55c2f0e1a050 2023-09-20T21:50:41.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.023+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0d6f590 con 0x55c2f0e196f0 2023-09-20T21:50:41.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.024+0000 7f5ef7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e1b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.025+0000 7f5efd19c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e196f0 0x55c2f0e19b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.027+0000 7f5efd19c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e196f0 0x55c2f0e19b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:59784/0 (socket says 172.21.15.22:59784) 2023-09-20T21:50:41.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.027+0000 7f5efd19c700 1 -- 172.21.15.22:0/3911266812 learned_addr learned my addr 172.21.15.22:0/3911266812 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:41.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.026+0000 7f5ef5ffb700 1 -- 172.21.15.22:0/2497883028 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5eec019880 con 0x7f5ef8101850 2023-09-20T21:50:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.027+0000 7f5efd19c700 1 -- 172.21.15.22:0/3911266812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1a050 msgr2=0x55c2f0e1a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.027+0000 7f5efc99b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1a050 0x55c2f0e1a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.028+0000 7f5efd19c700 1 --2- 172.21.15.22:0/3911266812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1a050 0x55c2f0e1a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.028+0000 7f5efd19c700 1 -- 172.21.15.22:0/3911266812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e1abe0 msgr2=0x55c2f0e1b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.028+0000 7f5efd19c700 1 --2- 172.21.15.22:0/3911266812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e1b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.028+0000 7f5efd19c700 1 -- 172.21.15.22:0/3911266812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2f0b95de0 con 0x55c2f0e196f0 2023-09-20T21:50:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.028+0000 7f5ef7fff700 1 --2- 172.21.15.22:0/3911266812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e1b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:50:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.028+0000 7f5efc99b700 1 --2- 172.21.15.22:0/3911266812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1a050 0x55c2f0e1a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:50:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.028+0000 7f5efd19c700 1 --2- 172.21.15.22:0/3911266812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e196f0 0x55c2f0e19b10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009e20 tx=0x7f5ee8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99671ebe1c8cc777 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.028+0000 7f5edeffd700 1 -- 172.21.15.22:0/3911266812 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8019630 con 0x55c2f0e196f0 2023-09-20T21:50:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.028+0000 7f5edeffd700 1 -- 172.21.15.22:0/3911266812 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee8019c50 con 0x55c2f0e196f0 2023-09-20T21:50:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.028+0000 7f5edeffd700 1 -- 172.21.15.22:0/3911266812 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee800fc80 con 0x55c2f0e196f0 2023-09-20T21:50:41.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.029+0000 7f5ef5ffb700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5ee00a1480 0x7f5ee00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.029+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3911266812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e196f0 msgr2=0x55c2f0e19b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.029+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/3911266812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e196f0 0x55c2f0e19b10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009e20 tx=0x7f5ee8011720 comp rx=0 tx=0).stop 2023-09-20T21:50:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.029+0000 7f5efc99b700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5ee00a1480 0x7f5ee00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.029+0000 7f5ef5ffb700 1 -- 172.21.15.22:0/2497883028 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f5eec0d0620 con 0x7f5ef8101850 2023-09-20T21:50:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.033+0000 7f5efc99b700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5ee00a1480 0x7f5ee00a3940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55c2f0e19490 tx=0x7f5eec012040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.033+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3911266812 shutdown_connections 2023-09-20T21:50:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.033+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/3911266812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e1b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.033+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/3911266812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e196f0 0x55c2f0e19b10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.033+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/3911266812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1a050 0x55c2f0e1a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.033+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3911266812 >> 172.21.15.22:0/3911266812 conn(0x55c2f0d46000 msgr2=0x55c2f0e141d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:41.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.033+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3911266812 shutdown_connections 2023-09-20T21:50:41.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.033+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3911266812 wait complete. 2023-09-20T21:50:41.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.034+0000 7f5f124eeb80 1 -- start start 2023-09-20T21:50:41.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.035+0000 7f5f124eeb80 1 Processor -- start 2023-09-20T21:50:41.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.035+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.035+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 0x55c2f0e2bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.035+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e32f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.035+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0d6ae80 con 0x55c2f0e196f0 2023-09-20T21:50:41.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.035+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0d07740 con 0x55c2f0e1abe0 2023-09-20T21:50:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.035+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0d073e0 con 0x55c2f0e1a050 2023-09-20T21:50:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.035+0000 7f5f124eeb80 10 client.0 _ll_register_callbacks cb 0x55c2f0c51a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.036+0000 7f5ef7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 0x55c2f0e2bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.036+0000 7f5ef7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 0x55c2f0e2bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:59786/0 (socket says 172.21.15.22:59786) 2023-09-20T21:50:41.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.036+0000 7f5ef7fff700 1 -- 172.21.15.22:0/2921312007 learned_addr learned my addr 172.21.15.22:0/2921312007 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.036+0000 7f5efd19c700 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e32f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.037+0000 7f5efd19c700 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.037+0000 7f5ef7fff700 1 -- 172.21.15.22:0/2921312007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 msgr2=0x55c2f0e32f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:41.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.037+0000 7f5ef7fff700 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e32f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.037+0000 7f5ef7fff700 1 -- 172.21.15.22:0/2921312007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 msgr2=0x55c2f0e2ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:41.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.037+0000 7f5ef7fff700 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e2ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.037+0000 7f5ef7fff700 1 -- 172.21.15.22:0/2921312007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef8109b10 con 0x55c2f0e1a050 2023-09-20T21:50:41.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.038+0000 7f5ef7fff700 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 0x55c2f0e2bf50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00077e0 tx=0x7f5ef0007810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49397c2215249989 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:41.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.038+0000 7f5ed7fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:50:41.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.038+0000 7f5ed7fff700 1 -- 172.21.15.22:0/2921312007 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef000e040 con 0x55c2f0e1a050 2023-09-20T21:50:41.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.038+0000 7f5ed7fff700 1 -- 172.21.15.22:0/2921312007 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f5ef0003a00 con 0x55c2f0e1a050 2023-09-20T21:50:41.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.038+0000 7f5f124eeb80 1 -- 172.21.15.22:0/2921312007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2f0cdbbb0 con 0x55c2f0e1a050 2023-09-20T21:50:41.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.038+0000 7f5ed7fff700 1 -- 172.21.15.22:0/2921312007 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ef0003bc0 con 0x55c2f0e1a050 2023-09-20T21:50:41.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.039+0000 7f5f124eeb80 10 client.25816 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:41.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.039+0000 7f5f124eeb80 1 -- 172.21.15.22:0/2921312007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f5ee8009b00 con 0x55c2f0e1a050 2023-09-20T21:50:41.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.039+0000 7f5ed7fff700 1 -- 172.21.15.22:0/2921312007 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2f0cdbbb0 con 0x55c2f0e1a050 2023-09-20T21:50:41.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.039+0000 7f5ed7fff700 1 -- 172.21.15.22:0/2921312007 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5ef0019070 con 0x55c2f0e1a050 2023-09-20T21:50:41.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.039+0000 7f5f124eeb80 10 client.25816 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:41.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.039+0000 7f5f124eeb80 10 client.25816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:50:41.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.039+0000 7f5f124eeb80 20 client.25816 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:41.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.039+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2f0e37820 0x55c2f0e39c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.039+0000 7f5f124eeb80 4 client.25816 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:50:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.039+0000 7f5f124eeb80 1 -- 172.21.15.22:0/2921312007 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2f0de8020 con 0x55c2f0e37820 2023-09-20T21:50:41.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.040+0000 7f5efd19c700 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2f0e37820 0x55c2f0e39c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.042+0000 7f5ed7fff700 10 client.25816 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:41.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.042+0000 7f5efd19c700 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2f0e37820 0x55c2f0e39c40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae4014db2961771 server_cookie=de1a3dd7a13edcc0 in_seq=0 out_seq=0 2023-09-20T21:50:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.043+0000 7f5ed7fff700 1 -- 172.21.15.22:0/2921312007 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c2f0de8020 con 0x55c2f0e37820 2023-09-20T21:50:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.043+0000 7f5ed7fff700 10 client.25816 handle_command_reply: tid=0 2023-09-20T21:50:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.043+0000 7f5f124eeb80 1 client.25816 shutdown 2023-09-20T21:50:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 1 -- 172.21.15.22:0/2921312007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 msgr2=0x55c2f0e2bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:41.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 0x55c2f0e2bf50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00077e0 tx=0x7f5ef0007810 comp rx=0 tx=0).stop 2023-09-20T21:50:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 1 -- 172.21.15.22:0/2921312007 shutdown_connections 2023-09-20T21:50:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2f0e37820 0x55c2f0e39c40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e32f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5ed7fff700 0 client.25816 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:41.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e2ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/2921312007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 0x55c2f0e2bf50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 1 -- 172.21.15.22:0/2921312007 >> 172.21.15.22:0/2921312007 conn(0x55c2f0d46000 msgr2=0x55c2f0e13880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:41.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 1 -- 172.21.15.22:0/2921312007 shutdown_connections 2023-09-20T21:50:41.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 1 -- 172.21.15.22:0/2921312007 wait complete. 2023-09-20T21:50:41.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.044+0000 7f5f124eeb80 20 client.25816 trim_cache size 0 max 16384 2023-09-20T21:50:41.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.080+0000 7f5f124eeb80 1 Processor -- start 2023-09-20T21:50:41.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.084+0000 7f5f124eeb80 1 -- start start 2023-09-20T21:50:41.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.084+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e1afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.084+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1a050 0x55c2f0e32f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.084+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e33880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.084+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0beb920 con 0x55c2f0e196f0 2023-09-20T21:50:41.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.084+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0e1aa40 con 0x55c2f0e1a050 2023-09-20T21:50:41.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.084+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0e37360 con 0x55c2f0e1abe0 2023-09-20T21:50:41.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.091+0000 7f5efd19c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e33880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.091+0000 7f5efd19c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e33880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:59788/0 (socket says 172.21.15.22:59788) 2023-09-20T21:50:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.091+0000 7f5efd19c700 1 -- 172.21.15.22:0/982088525 learned_addr learned my addr 172.21.15.22:0/982088525 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.093+0000 7f5efd19c700 1 --2- 172.21.15.22:0/982088525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e1afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.093+0000 7f5efd19c700 1 -- 172.21.15.22:0/982088525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1a050 msgr2=0x55c2f0e32f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:50:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.093+0000 7f5efd19c700 1 --2- 172.21.15.22:0/982088525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1a050 0x55c2f0e32f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.093+0000 7f5efd19c700 1 -- 172.21.15.22:0/982088525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 msgr2=0x55c2f0e1afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.093+0000 7f5efd19c700 1 --2- 172.21.15.22:0/982088525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e1afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.093+0000 7f5efd19c700 1 -- 172.21.15.22:0/982088525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2f0b7ec60 con 0x55c2f0e1abe0 2023-09-20T21:50:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.096+0000 7f5efd19c700 1 --2- 172.21.15.22:0/982088525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e33880 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8011a70 tx=0x7f5ee8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5418be77f89fd38c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.100+0000 7f5edf7fe700 1 -- 172.21.15.22:0/982088525 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8033070 con 0x55c2f0e1abe0 2023-09-20T21:50:41.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.100+0000 7f5edf7fe700 1 -- 172.21.15.22:0/982088525 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee800f930 con 0x55c2f0e1abe0 2023-09-20T21:50:41.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.100+0000 7f5edf7fe700 1 -- 172.21.15.22:0/982088525 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80197b0 con 0x55c2f0e1abe0 2023-09-20T21:50:41.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.101+0000 7f5f124eeb80 1 -- 172.21.15.22:0/982088525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1abe0 msgr2=0x55c2f0e33880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:41.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.101+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/982088525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e33880 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8011a70 tx=0x7f5ee8000bc0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.102+0000 7f5f124eeb80 1 -- 172.21.15.22:0/982088525 shutdown_connections 2023-09-20T21:50:41.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.102+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/982088525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e33880 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.102+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/982088525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e1afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.102+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/982088525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1a050 0x55c2f0e32f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.102+0000 7f5f124eeb80 1 -- 172.21.15.22:0/982088525 >> 172.21.15.22:0/982088525 conn(0x55c2f0d46000 msgr2=0x55c2f0e102e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:41.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.104+0000 7f5f124eeb80 1 -- 172.21.15.22:0/982088525 shutdown_connections 2023-09-20T21:50:41.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.104+0000 7f5f124eeb80 1 -- 172.21.15.22:0/982088525 wait complete. 2023-09-20T21:50:41.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.104+0000 7f5f124eeb80 1 -- start start 2023-09-20T21:50:41.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.105+0000 7f5f124eeb80 1 Processor -- start 2023-09-20T21:50:41.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.105+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e30bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.105+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 0x55c2f0e31110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.105+0000 7f5f124eeb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e32120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.105+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0e32660 con 0x55c2f0e196f0 2023-09-20T21:50:41.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.105+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0e327d0 con 0x55c2f0e1abe0 2023-09-20T21:50:41.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.105+0000 7f5f124eeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2f0e32990 con 0x55c2f0e1a050 2023-09-20T21:50:41.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.105+0000 7f5f124eeb80 10 client.0 _ll_register_callbacks cb 0x55c2f0c51a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:50:41.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.109+0000 7f5efd19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e32120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.109+0000 7f5efd19c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e32120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37656/0 (socket says 172.21.15.22:37656) 2023-09-20T21:50:41.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.109+0000 7f5efd19c700 1 -- 172.21.15.22:0/3866794974 learned_addr learned my addr 172.21.15.22:0/3866794974 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:50:41.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.109+0000 7f5efd19c700 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e30bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.109+0000 7f5ef7fff700 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 0x55c2f0e31110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.109+0000 7f5efd19c700 1 -- 172.21.15.22:0/3866794974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 msgr2=0x55c2f0e31110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:41.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.109+0000 7f5efd19c700 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 0x55c2f0e31110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.110+0000 7f5efd19c700 1 -- 172.21.15.22:0/3866794974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 msgr2=0x55c2f0e30bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:50:41.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.110+0000 7f5efd19c700 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e30bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.110+0000 7f5efd19c700 1 -- 172.21.15.22:0/3866794974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2f0b95de0 con 0x55c2f0e1abe0 2023-09-20T21:50:41.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.110+0000 7f5efd19c700 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e32120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800f900 tx=0x7f5ee80092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ce9dd9f283ede7 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:50:41.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.110+0000 7f5ed7fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:50:41.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.110+0000 7f5ed7fff700 1 -- 172.21.15.22:0/3866794974 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8033040 con 0x55c2f0e1abe0 2023-09-20T21:50:41.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.111+0000 7f5ed7fff700 1 -- 172.21.15.22:0/3866794974 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f5ee800faa0 con 0x55c2f0e1abe0 2023-09-20T21:50:41.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.112+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3866794974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ee8019c50 con 0x55c2f0e1abe0 2023-09-20T21:50:41.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.114+0000 7f5ed7fff700 1 -- 172.21.15.22:0/3866794974 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee8019aa0 con 0x55c2f0e1abe0 2023-09-20T21:50:41.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.114+0000 7f5f124eeb80 10 client.25829 fetch_fsmap learned FSMap version 30 2023-09-20T21:50:41.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.114+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3866794974 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c2f0bbc250 con 0x55c2f0e1abe0 2023-09-20T21:50:41.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.114+0000 7f5ed7fff700 1 -- 172.21.15.22:0/3866794974 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee8019c50 con 0x55c2f0e1abe0 2023-09-20T21:50:41.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.115+0000 7f5ed7fff700 1 -- 172.21.15.22:0/3866794974 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5ee8004710 con 0x55c2f0e1abe0 2023-09-20T21:50:41.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.115+0000 7f5f124eeb80 10 client.25829 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:50:41.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.115+0000 7f5f124eeb80 10 client.25829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:50:41.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.115+0000 7f5f124eeb80 20 client.25829 populate_metadata read hostname 'smithi022' 2023-09-20T21:50:41.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.115+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2f0e8c1b0 0x55c2f0e8e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:50:41.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.115+0000 7f5f124eeb80 4 client.25829 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:50:41.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.115+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3866794974 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c2f0d63e60 con 0x55c2f0e8c1b0 2023-09-20T21:50:41.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.115+0000 7f5ef7fff700 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2f0e8c1b0 0x55c2f0e8e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:50:41.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.116+0000 7f5ef7fff700 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2f0e8c1b0 0x55c2f0e8e5d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26329aaeefc8ee11 server_cookie=e69c50252635d861 in_seq=0 out_seq=0 2023-09-20T21:50:41.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.116+0000 7f5ed7fff700 10 client.25829 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:41.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.116+0000 7f5ed7fff700 1 -- 172.21.15.22:0/3866794974 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c2f0d63e60 con 0x55c2f0e8c1b0 2023-09-20T21:50:41.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.116+0000 7f5ed7fff700 10 client.25829 handle_command_reply: tid=0 2023-09-20T21:50:41.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.116+0000 7f5f124eeb80 1 client.25829 shutdown 2023-09-20T21:50:41.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.118+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3866794974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 msgr2=0x55c2f0e32120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:41.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.118+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e32120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800f900 tx=0x7f5ee80092d0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.119+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3866794974 shutdown_connections 2023-09-20T21:50:41.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.119+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2f0e8c1b0 0x55c2f0e8e5d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.119+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2f0e1abe0 0x55c2f0e32120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.119+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2f0e196f0 0x55c2f0e30bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.119+0000 7f5f124eeb80 1 --2- 172.21.15.22:0/3866794974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2f0e1a050 0x55c2f0e31110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.119+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3866794974 >> 172.21.15.22:0/3866794974 conn(0x55c2f0d46000 msgr2=0x55c2f0e33880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:41.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.119+0000 7f5ed7fff700 0 client.25829 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:50:41.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.119+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3866794974 shutdown_connections 2023-09-20T21:50:41.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.120+0000 7f5f124eeb80 1 -- 172.21.15.22:0/3866794974 wait complete. 2023-09-20T21:50:41.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.120+0000 7f5f124eeb80 20 client.25829 trim_cache size 0 max 16384 2023-09-20T21:50:41.123 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:50:41.123 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:50:41.124 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:50:41.124 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:50:41.124 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:50:41.124 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:50:41.124 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:50:41.125 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:50:41.125 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:50:41.125 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:50:41.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.121+0000 7f5efebff700 1 -- 172.21.15.22:0/2497883028 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5ee00a1480 msgr2=0x7f5ee00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:41.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.121+0000 7f5efebff700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5ee00a1480 0x7f5ee00a3940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55c2f0e19490 tx=0x7f5eec012040 comp rx=0 tx=0).stop 2023-09-20T21:50:41.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.121+0000 7f5efebff700 1 -- 172.21.15.22:0/2497883028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef8101850 msgr2=0x7f5ef8196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:50:41.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.121+0000 7f5efebff700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef8101850 0x7f5ef8196c50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009c50 tx=0x7f5eec009f60 comp rx=0 tx=0).stop 2023-09-20T21:50:41.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.122+0000 7f5efebff700 1 -- 172.21.15.22:0/2497883028 shutdown_connections 2023-09-20T21:50:41.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.122+0000 7f5efebff700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5ee00a1480 0x7f5ee00a3940 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55c2f0e19490 tx=0x7f5eec012040 comp rx=0 tx=0).stop 2023-09-20T21:50:41.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.122+0000 7f5efebff700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ef8106ab0 0x7f5ef819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.122+0000 7f5efebff700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ef8101850 0x7f5ef8196c50 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009c50 tx=0x7f5eec009f60 comp rx=0 tx=0).stop 2023-09-20T21:50:41.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.122+0000 7f5efebff700 1 --2- 172.21.15.22:0/2497883028 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ef8104180 0x7f5ef8197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:50:41.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.122+0000 7f5efebff700 1 -- 172.21.15.22:0/2497883028 >> 172.21.15.22:0/2497883028 conn(0x7f5ef80fb240 msgr2=0x7f5ef81024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:50:41.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.123+0000 7f5efebff700 1 -- 172.21.15.22:0/2497883028 shutdown_connections 2023-09-20T21:50:41.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:50:41.123+0000 7f5efebff700 1 -- 172.21.15.22:0/2497883028 wait complete. 2023-09-20T21:50:41.154 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:50:42.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:42 smithi123 ceph-mon[134433]: pgmap v400: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.20k op/s 2023-09-20T21:50:42.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:42 smithi089 ceph-mon[134090]: pgmap v400: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.20k op/s 2023-09-20T21:50:42.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:42 smithi022 ceph-mon[97651]: pgmap v400: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.20k op/s 2023-09-20T21:50:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 80.0 s, 1728.7 tps, lat 57.817 ms stddev 61.483 2023-09-20T21:50:43.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:43 smithi022 ceph-mon[97651]: pgmap v401: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 697 op/s 2023-09-20T21:50:44.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:43 smithi123 ceph-mon[134433]: pgmap v401: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 697 op/s 2023-09-20T21:50:44.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:43 smithi089 ceph-mon[134090]: pgmap v401: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 697 op/s 2023-09-20T21:50:46.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:46 smithi123 ceph-mon[134433]: pgmap v402: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 666 op/s 2023-09-20T21:50:46.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:46 smithi089 ceph-mon[134090]: pgmap v402: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 666 op/s 2023-09-20T21:50:46.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:46 smithi022 ceph-mon[97651]: pgmap v402: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 14 MiB/s wr, 666 op/s 2023-09-20T21:50:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 85.0 s, 1650.4 tps, lat 60.253 ms stddev 65.500 2023-09-20T21:50:48.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:48 smithi123 ceph-mon[134433]: pgmap v403: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 641 op/s 2023-09-20T21:50:48.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:48 smithi089 ceph-mon[134090]: pgmap v403: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 641 op/s 2023-09-20T21:50:48.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:48 smithi022 ceph-mon[97651]: pgmap v403: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 641 op/s 2023-09-20T21:50:50.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:50 smithi123 ceph-mon[134433]: pgmap v404: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 13 MiB/s wr, 690 op/s 2023-09-20T21:50:50.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:50 smithi089 ceph-mon[134090]: pgmap v404: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 13 MiB/s wr, 690 op/s 2023-09-20T21:50:50.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:50 smithi022 ceph-mon[97651]: pgmap v404: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 13 MiB/s wr, 690 op/s 2023-09-20T21:50:52.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:52 smithi123 ceph-mon[134433]: pgmap v405: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 678 op/s 2023-09-20T21:50:52.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:52 smithi089 ceph-mon[134090]: pgmap v405: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 678 op/s 2023-09-20T21:50:52.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:52 smithi022 ceph-mon[97651]: pgmap v405: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 12 MiB/s wr, 678 op/s 2023-09-20T21:50:53.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 90.0 s, 1583.8 tps, lat 63.000 ms stddev 64.783 2023-09-20T21:50:53.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:53 smithi022 ceph-mon[97651]: pgmap v406: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 MiB/s wr, 635 op/s 2023-09-20T21:50:54.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:53 smithi123 ceph-mon[134433]: pgmap v406: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 MiB/s wr, 635 op/s 2023-09-20T21:50:54.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:53 smithi089 ceph-mon[134090]: pgmap v406: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 MiB/s wr, 635 op/s 2023-09-20T21:50:56.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:56 smithi123 ceph-mon[134433]: pgmap v407: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 MiB/s wr, 721 op/s 2023-09-20T21:50:56.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:56 smithi089 ceph-mon[134090]: pgmap v407: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 MiB/s wr, 721 op/s 2023-09-20T21:50:56.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:56 smithi022 ceph-mon[97651]: pgmap v407: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 MiB/s wr, 721 op/s 2023-09-20T21:50:58.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 95.0 s, 1423.0 tps, lat 70.546 ms stddev 78.910 2023-09-20T21:50:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:50:58 smithi123 ceph-mon[134433]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 688 op/s 2023-09-20T21:50:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:50:58 smithi089 ceph-mon[134090]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 688 op/s 2023-09-20T21:50:58.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:50:58 smithi022 ceph-mon[97651]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 688 op/s 2023-09-20T21:51:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:00 smithi123 ceph-mon[134433]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 MiB/s wr, 708 op/s 2023-09-20T21:51:00.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:00 smithi089 ceph-mon[134090]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 MiB/s wr, 708 op/s 2023-09-20T21:51:00.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:00 smithi022 ceph-mon[97651]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 MiB/s wr, 708 op/s 2023-09-20T21:51:02.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:02 smithi022 ceph-mon[97651]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 711 op/s 2023-09-20T21:51:02.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:02 smithi123 ceph-mon[134433]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 711 op/s 2023-09-20T21:51:02.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:02 smithi089 ceph-mon[134090]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 711 op/s 2023-09-20T21:51:03.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 100.0 s, 1298.1 tps, lat 76.210 ms stddev 86.659 2023-09-20T21:51:03.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:03 smithi022 ceph-mon[97651]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.1 MiB/s wr, 645 op/s 2023-09-20T21:51:04.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:03 smithi123 ceph-mon[134433]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.1 MiB/s wr, 645 op/s 2023-09-20T21:51:04.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:03 smithi089 ceph-mon[134090]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.1 MiB/s wr, 645 op/s 2023-09-20T21:51:06.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:06 smithi123 ceph-mon[134433]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 740 op/s 2023-09-20T21:51:06.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:06 smithi089 ceph-mon[134090]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 740 op/s 2023-09-20T21:51:06.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:06 smithi022 ceph-mon[97651]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 740 op/s 2023-09-20T21:51:08.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 105.0 s, 957.3 tps, lat 104.440 ms stddev 118.291 2023-09-20T21:51:08.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:08 smithi022 ceph-mon[97651]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 728 op/s 2023-09-20T21:51:08.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:51:08.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:08 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:08 smithi123 ceph-mon[134433]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 728 op/s 2023-09-20T21:51:08.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:51:08.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:08 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:08 smithi089 ceph-mon[134090]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 728 op/s 2023-09-20T21:51:08.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:51:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:08.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:08 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:09.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:51:09.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:51:09.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:09 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:09.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:51:09.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:51:09.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:09 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:09.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:51:09.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:51:09.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:09 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:51:10.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:10 smithi022 ceph-mon[97651]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 807 op/s 2023-09-20T21:51:10.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:10 smithi123 ceph-mon[134433]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 807 op/s 2023-09-20T21:51:10.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:10 smithi089 ceph-mon[134090]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 807 op/s 2023-09-20T21:51:11.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:51:11.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.473+0000 7f68bbdff700 1 -- 172.21.15.22:0/3935147318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68b40769e0 msgr2=0x7f68b4076e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:11.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.473+0000 7f68bbdff700 1 --2- 172.21.15.22:0/3935147318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68b40769e0 0x7f68b4076e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68b00098f0 tx=0x7f68b0009c00 comp rx=0 tx=0).stop 2023-09-20T21:51:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.476+0000 7f68bbdff700 1 -- 172.21.15.22:0/3935147318 shutdown_connections 2023-09-20T21:51:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.476+0000 7f68bbdff700 1 --2- 172.21.15.22:0/3935147318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b40773a0 0x7f68b4077c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.476+0000 7f68bbdff700 1 --2- 172.21.15.22:0/3935147318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68b40757a0 0x7f68b4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.476+0000 7f68bbdff700 1 --2- 172.21.15.22:0/3935147318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68b40769e0 0x7f68b4076e60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.476+0000 7f68bbdff700 1 -- 172.21.15.22:0/3935147318 >> 172.21.15.22:0/3935147318 conn(0x7f68b40fe590 msgr2=0x7f68b41009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:11.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.477+0000 7f68bbdff700 1 -- 172.21.15.22:0/3935147318 shutdown_connections 2023-09-20T21:51:11.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.477+0000 7f68bbdff700 1 -- 172.21.15.22:0/3935147318 wait complete. 2023-09-20T21:51:11.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.478+0000 7f68bbdff700 1 Processor -- start 2023-09-20T21:51:11.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.479+0000 7f68bbdff700 1 -- start start 2023-09-20T21:51:11.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.479+0000 7f68bbdff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68b40757a0 0x7f68b410e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.479+0000 7f68bbdff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b40769e0 0x7f68b410ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.479+0000 7f68bbdff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68b40773a0 0x7f68b410fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.479+0000 7f68bbdff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4110420 con 0x7f68b40757a0 2023-09-20T21:51:11.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.479+0000 7f68bbdff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4110560 con 0x7f68b40769e0 2023-09-20T21:51:11.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.479+0000 7f68bbdff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b41106f0 con 0x7f68b40773a0 2023-09-20T21:51:11.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.480+0000 7f68ba39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68b40773a0 0x7f68b410fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.480+0000 7f68ba39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68b40773a0 0x7f68b410fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:56308/0 (socket says 172.21.15.22:56308) 2023-09-20T21:51:11.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.480+0000 7f68ba39c700 1 -- 172.21.15.22:0/3893545964 learned_addr learned my addr 172.21.15.22:0/3893545964 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:51:11.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.482+0000 7f68ba39c700 1 -- 172.21.15.22:0/3893545964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b40769e0 msgr2=0x7f68b410ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:11.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.482+0000 7f68ba39c700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b40769e0 0x7f68b410ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.482+0000 7f68ba39c700 1 -- 172.21.15.22:0/3893545964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68b40757a0 msgr2=0x7f68b410e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:11.484 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.482+0000 7f68ba39c700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68b40757a0 0x7f68b410e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.482+0000 7f68ba39c700 1 -- 172.21.15.22:0/3893545964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b0009520 con 0x7f68b40773a0 2023-09-20T21:51:11.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.482+0000 7f68ba39c700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68b40773a0 0x7f68b410fe50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68a400e970 tx=0x7f68a400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:11.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.490+0000 7f68aaffd700 1 -- 172.21.15.22:0/3893545964 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a400cbd0 con 0x7f68b40773a0 2023-09-20T21:51:11.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.490+0000 7f68aaffd700 1 -- 172.21.15.22:0/3893545964 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68a400ee50 con 0x7f68b40773a0 2023-09-20T21:51:11.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.490+0000 7f68aaffd700 1 -- 172.21.15.22:0/3893545964 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4005610 con 0x7f68b40773a0 2023-09-20T21:51:11.491 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:11 smithi022 ceph-mon[97651]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 14 MiB/s wr, 825 op/s 2023-09-20T21:51:11.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.494+0000 7f68bbdff700 1 -- 172.21.15.22:0/3893545964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b41109d0 con 0x7f68b40773a0 2023-09-20T21:51:11.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.494+0000 7f68bbdff700 1 -- 172.21.15.22:0/3893545964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b4110f10 con 0x7f68b40773a0 2023-09-20T21:51:11.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.504+0000 7f68cf792b80 1 Processor -- start 2023-09-20T21:51:11.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.506+0000 7f68cf792b80 1 -- start start 2023-09-20T21:51:11.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.506+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e8db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.506+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8e050 0x555a50e8e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.506+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8ebe0 0x555a50e8f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.506+0000 7f68cf792b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50e01ca0 con 0x555a50e8e050 2023-09-20T21:51:11.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.506+0000 7f68cf792b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50de5c70 con 0x555a50e8ebe0 2023-09-20T21:51:11.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.506+0000 7f68cf792b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50de3590 con 0x555a50e8d6f0 2023-09-20T21:51:11.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.507+0000 7f68b939a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e8db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.507+0000 7f68b939a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e8db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:56324/0 (socket says 172.21.15.22:56324) 2023-09-20T21:51:11.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.507+0000 7f68b939a700 1 -- 172.21.15.22:0/3747071947 learned_addr learned my addr 172.21.15.22:0/3747071947 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:51:11.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.507+0000 7f68aaffd700 1 -- 172.21.15.22:0/3893545964 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f68a4004750 con 0x7f68b40773a0 2023-09-20T21:51:11.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.507+0000 7f68b939a700 1 --2- 172.21.15.22:0/3747071947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8ebe0 0x555a50e8f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.508+0000 7f68b939a700 1 -- 172.21.15.22:0/3747071947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8ebe0 msgr2=0x555a50e8f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:11.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.508+0000 7f68b939a700 1 --2- 172.21.15.22:0/3747071947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8ebe0 0x555a50e8f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.508+0000 7f68b939a700 1 -- 172.21.15.22:0/3747071947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8e050 msgr2=0x555a50e8e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:11.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.508+0000 7f68b939a700 1 --2- 172.21.15.22:0/3747071947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8e050 0x555a50e8e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.508+0000 7f68b939a700 1 -- 172.21.15.22:0/3747071947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a50c09de0 con 0x555a50e8d6f0 2023-09-20T21:51:11.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.508+0000 7f68b939a700 1 --2- 172.21.15.22:0/3747071947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e8db10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c010 tx=0x7f68b0009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63738725dd8eef98 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:11.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.509+0000 7f689ffff700 1 -- 172.21.15.22:0/3747071947 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b000f700 con 0x555a50e8d6f0 2023-09-20T21:51:11.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.509+0000 7f689ffff700 1 -- 172.21.15.22:0/3747071947 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68b000f860 con 0x555a50e8d6f0 2023-09-20T21:51:11.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.509+0000 7f689ffff700 1 -- 172.21.15.22:0/3747071947 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0003680 con 0x555a50e8d6f0 2023-09-20T21:51:11.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.510+0000 7f68aaffd700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68a00a1500 0x7f68a00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.510+0000 7f68aaffd700 1 -- 172.21.15.22:0/3893545964 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f68a4014070 con 0x7f68b40773a0 2023-09-20T21:51:11.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.511+0000 7f68cf792b80 1 -- 172.21.15.22:0/3747071947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 msgr2=0x555a50e8db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:11.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.511+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3747071947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e8db10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f68b000c010 tx=0x7f68b0009400 comp rx=0 tx=0).stop 2023-09-20T21:51:11.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.512+0000 7f68cf792b80 1 -- 172.21.15.22:0/3747071947 shutdown_connections 2023-09-20T21:51:11.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.512+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3747071947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8ebe0 0x555a50e8f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.512+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3747071947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e8db10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.512+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3747071947 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8e050 0x555a50e8e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.512+0000 7f68cf792b80 1 -- 172.21.15.22:0/3747071947 >> 172.21.15.22:0/3747071947 conn(0x555a50dba000 msgr2=0x555a50e881d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:11.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.513+0000 7f68cf792b80 1 -- 172.21.15.22:0/3747071947 shutdown_connections 2023-09-20T21:51:11.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.513+0000 7f68cf792b80 1 -- 172.21.15.22:0/3747071947 wait complete. 2023-09-20T21:51:11.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.514+0000 7f68cf792b80 1 -- start start 2023-09-20T21:51:11.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.515+0000 7f68cf792b80 1 Processor -- start 2023-09-20T21:51:11.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.515+0000 7f68b9b9b700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68a00a1500 0x7f68a00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.515+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e97240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.515+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8e050 0x555a50e97780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.516+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8ebe0 0x555a50e9e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.516+0000 7f68cf792b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50ddee80 con 0x555a50e8e050 2023-09-20T21:51:11.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.516+0000 7f68cf792b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50d7b740 con 0x555a50e8ebe0 2023-09-20T21:51:11.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.516+0000 7f68cf792b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50d7b3e0 con 0x555a50e8d6f0 2023-09-20T21:51:11.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.516+0000 7f68cf792b80 10 client.0 _ll_register_callbacks cb 0x555a50cc5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:51:11.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.516+0000 7f68b9b9b700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68a00a1500 0x7f68a00a39c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x555a50e89150 tx=0x7f68ac0091d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:11.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.516+0000 7f68b9b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e97240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.516+0000 7f68b9b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e97240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:56338/0 (socket says 172.21.15.22:56338) 2023-09-20T21:51:11.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.516+0000 7f68b9b9b700 1 -- 172.21.15.22:0/3246282781 learned_addr learned my addr 172.21.15.22:0/3246282781 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:51:11.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.516+0000 7f68ba39c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8ebe0 0x555a50e9e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.517+0000 7f68b9b9b700 1 -- 172.21.15.22:0/3246282781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8ebe0 msgr2=0x555a50e9e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:11.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.517+0000 7f68b9b9b700 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8ebe0 0x555a50e9e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.517+0000 7f68b9b9b700 1 -- 172.21.15.22:0/3246282781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8e050 msgr2=0x555a50e97780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:11.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.517+0000 7f68b939a700 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8e050 0x555a50e97780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.517+0000 7f68b9b9b700 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8e050 0x555a50e97780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.517+0000 7f68b9b9b700 1 -- 172.21.15.22:0/3246282781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b0042040 con 0x555a50e8d6f0 2023-09-20T21:51:11.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.517+0000 7f68b939a700 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8e050 0x555a50e97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:51:11.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.517+0000 7f68b9b9b700 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e97240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e410 tx=0x7f68ac004c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a3f1d6482499b2f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:11.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.518+0000 7f689d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:51:11.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.518+0000 7f689d7fa700 1 -- 172.21.15.22:0/3246282781 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac01b650 con 0x555a50e8d6f0 2023-09-20T21:51:11.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.518+0000 7f68cf792b80 1 -- 172.21.15.22:0/3246282781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a50d4fbb0 con 0x555a50e8d6f0 2023-09-20T21:51:11.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.518+0000 7f689d7fa700 1 -- 172.21.15.22:0/3246282781 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f68ac01b7b0 con 0x555a50e8d6f0 2023-09-20T21:51:11.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.518+0000 7f689d7fa700 1 -- 172.21.15.22:0/3246282781 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68ac01b970 con 0x555a50e8d6f0 2023-09-20T21:51:11.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.519+0000 7f68cf792b80 10 client.25834 fetch_fsmap learned FSMap version 30 2023-09-20T21:51:11.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.519+0000 7f68cf792b80 1 -- 172.21.15.22:0/3246282781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555a50bf2c60 con 0x555a50e8d6f0 2023-09-20T21:51:11.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.519+0000 7f689d7fa700 1 -- 172.21.15.22:0/3246282781 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a50d4fbb0 con 0x555a50e8d6f0 2023-09-20T21:51:11.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.519+0000 7f689d7fa700 1 -- 172.21.15.22:0/3246282781 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f68ac023c50 con 0x555a50e8d6f0 2023-09-20T21:51:11.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.519+0000 7f68cf792b80 10 client.25834 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:51:11.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.519+0000 7f68cf792b80 10 client.25834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:51:11.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.520+0000 7f68cf792b80 20 client.25834 populate_metadata read hostname 'smithi022' 2023-09-20T21:51:11.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.520+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x555a50ea30f0 0x555a50ea5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.520+0000 7f68cf792b80 4 client.25834 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:51:11.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.520+0000 7f68cf792b80 1 -- 172.21.15.22:0/3246282781 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a50e5c020 con 0x555a50ea30f0 2023-09-20T21:51:11.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.520+0000 7f68b939a700 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x555a50ea30f0 0x555a50ea5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.521+0000 7f68b939a700 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x555a50ea30f0 0x555a50ea5510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5479cbbabe76f24e server_cookie=6cfe29081078ac5b in_seq=0 out_seq=0 2023-09-20T21:51:11.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.522+0000 7f689d7fa700 10 client.25834 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:51:11.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.523+0000 7f689d7fa700 1 -- 172.21.15.22:0/3246282781 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555a50e5c020 con 0x555a50ea30f0 2023-09-20T21:51:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.523+0000 7f689d7fa700 10 client.25834 handle_command_reply: tid=0 2023-09-20T21:51:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.523+0000 7f68cf792b80 1 client.25834 shutdown 2023-09-20T21:51:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.524+0000 7f68cf792b80 1 -- 172.21.15.22:0/3246282781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 msgr2=0x555a50e97240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.524+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e97240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e410 tx=0x7f68ac004c50 comp rx=0 tx=0).stop 2023-09-20T21:51:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.525+0000 7f68cf792b80 1 -- 172.21.15.22:0/3246282781 shutdown_connections 2023-09-20T21:51:11.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.525+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x555a50ea30f0 0x555a50ea5510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.525+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8ebe0 0x555a50e9e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.525+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8d6f0 0x555a50e97240 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e410 tx=0x7f68ac004c50 comp rx=0 tx=0).stop 2023-09-20T21:51:11.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.525+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3246282781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8e050 0x555a50e97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.525+0000 7f68cf792b80 1 -- 172.21.15.22:0/3246282781 >> 172.21.15.22:0/3246282781 conn(0x555a50dba000 msgr2=0x555a50e881a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:11.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.525+0000 7f689d7fa700 0 client.25834 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:51:11.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.525+0000 7f68cf792b80 1 -- 172.21.15.22:0/3246282781 shutdown_connections 2023-09-20T21:51:11.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.526+0000 7f68cf792b80 1 -- 172.21.15.22:0/3246282781 wait complete. 2023-09-20T21:51:11.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.526+0000 7f68cf792b80 20 client.25834 trim_cache size 0 max 16384 2023-09-20T21:51:11.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.558+0000 7f68cf792b80 1 Processor -- start 2023-09-20T21:51:11.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.558+0000 7f68cf792b80 1 -- start start 2023-09-20T21:51:11.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.559+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 0x555a50e880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.559+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 0x555a50e9e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.559+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 0x555a50e9f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.559+0000 7f68cf792b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50c5f920 con 0x555a50e8ebe0 2023-09-20T21:51:11.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.559+0000 7f68cf792b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50e95790 con 0x555a50e8e050 2023-09-20T21:51:11.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.559+0000 7f68cf792b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50e88cc0 con 0x555a50e8f210 2023-09-20T21:51:11.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.559+0000 7f68b9b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 0x555a50e880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68b9b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 0x555a50e880e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:58036/0 (socket says 172.21.15.22:58036) 2023-09-20T21:51:11.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68b9b9b700 1 -- 172.21.15.22:0/204481212 learned_addr learned my addr 172.21.15.22:0/204481212 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:51:11.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68ba39c700 1 --2- 172.21.15.22:0/204481212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 0x555a50e9f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68b939a700 1 --2- 172.21.15.22:0/204481212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 0x555a50e9e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68ba39c700 1 -- 172.21.15.22:0/204481212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 msgr2=0x555a50e880e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:11.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68ba39c700 1 --2- 172.21.15.22:0/204481212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 0x555a50e880e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68ba39c700 1 -- 172.21.15.22:0/204481212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 msgr2=0x555a50e9e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:11.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68ba39c700 1 --2- 172.21.15.22:0/204481212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 0x555a50e9e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68ba39c700 1 -- 172.21.15.22:0/204481212 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a50c30250 con 0x555a50e8f210 2023-09-20T21:51:11.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68b939a700 1 --2- 172.21.15.22:0/204481212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 0x555a50e9e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:51:11.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.560+0000 7f68ba39c700 1 --2- 172.21.15.22:0/204481212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 0x555a50e9f0b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f68a400c5b0 tx=0x7f68a40050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54f802c9791f4880 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:11.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.561+0000 7f68a8bff700 1 -- 172.21.15.22:0/204481212 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a40b6070 con 0x555a50e8f210 2023-09-20T21:51:11.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.561+0000 7f68a8bff700 1 -- 172.21.15.22:0/204481212 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68a409a160 con 0x555a50e8f210 2023-09-20T21:51:11.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.561+0000 7f68a8bff700 1 -- 172.21.15.22:0/204481212 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4098590 con 0x555a50e8f210 2023-09-20T21:51:11.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.562+0000 7f68cf792b80 1 -- 172.21.15.22:0/204481212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 msgr2=0x555a50e9f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:11.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.562+0000 7f68cf792b80 1 --2- 172.21.15.22:0/204481212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 0x555a50e9f0b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f68a400c5b0 tx=0x7f68a40050b0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.562+0000 7f68cf792b80 1 -- 172.21.15.22:0/204481212 shutdown_connections 2023-09-20T21:51:11.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.562+0000 7f68cf792b80 1 --2- 172.21.15.22:0/204481212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 0x555a50e9f0b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.562+0000 7f68cf792b80 1 --2- 172.21.15.22:0/204481212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 0x555a50e880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.562+0000 7f68cf792b80 1 --2- 172.21.15.22:0/204481212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 0x555a50e9e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.562+0000 7f68cf792b80 1 -- 172.21.15.22:0/204481212 >> 172.21.15.22:0/204481212 conn(0x555a50dba000 msgr2=0x555a50e87880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:11.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.562+0000 7f68cf792b80 1 -- 172.21.15.22:0/204481212 shutdown_connections 2023-09-20T21:51:11.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.562+0000 7f68cf792b80 1 -- 172.21.15.22:0/204481212 wait complete. 2023-09-20T21:51:11.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.563+0000 7f68cf792b80 1 -- start start 2023-09-20T21:51:11.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.563+0000 7f68cf792b80 1 Processor -- start 2023-09-20T21:51:11.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.563+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 0x555a50e9e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.563+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 0x555a50e9e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.563+0000 7f68cf792b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 0x555a50e9c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.563+0000 7f68cf792b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50e9cdd0 con 0x555a50e8ebe0 2023-09-20T21:51:11.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.563+0000 7f68cf792b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50e9cf40 con 0x555a50e8e050 2023-09-20T21:51:11.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.563+0000 7f68cf792b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a50e9d100 con 0x555a50e8f210 2023-09-20T21:51:11.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.563+0000 7f68cf792b80 10 client.0 _ll_register_callbacks cb 0x555a50cc5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:51:11.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.564+0000 7f68ba39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 0x555a50e9c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.564+0000 7f68ba39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 0x555a50e9c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:56366/0 (socket says 172.21.15.22:56366) 2023-09-20T21:51:11.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.564+0000 7f68ba39c700 1 -- 172.21.15.22:0/3285720780 learned_addr learned my addr 172.21.15.22:0/3285720780 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:51:11.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.564+0000 7f68b9b9b700 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 0x555a50e9e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.564+0000 7f68b939a700 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 0x555a50e9e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.564+0000 7f68b9b9b700 1 -- 172.21.15.22:0/3285720780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 msgr2=0x555a50e9c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:11.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.564+0000 7f68b9b9b700 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 0x555a50e9c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.564+0000 7f68b9b9b700 1 -- 172.21.15.22:0/3285720780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 msgr2=0x555a50e9e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:11.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.564+0000 7f68b9b9b700 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 0x555a50e9e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.564+0000 7f68b9b9b700 1 -- 172.21.15.22:0/3285720780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b4110f10 con 0x555a50e8e050 2023-09-20T21:51:11.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.565+0000 7f68b9b9b700 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 0x555a50e9e0e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00a690 tx=0x7f68ac01a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=935a56b6527ed6f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:11.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.565+0000 7f689d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:51:11.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.565+0000 7f689d7fa700 1 -- 172.21.15.22:0/3285720780 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac01ae30 con 0x555a50e8e050 2023-09-20T21:51:11.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.565+0000 7f689d7fa700 1 -- 172.21.15.22:0/3285720780 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f68ac036070 con 0x555a50e8e050 2023-09-20T21:51:11.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.565+0000 7f68cf792b80 1 -- 172.21.15.22:0/3285720780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68b000f860 con 0x555a50e8e050 2023-09-20T21:51:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f689d7fa700 1 -- 172.21.15.22:0/3285720780 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68ac03f480 con 0x555a50e8e050 2023-09-20T21:51:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f68cf792b80 10 client.25844 fetch_fsmap learned FSMap version 30 2023-09-20T21:51:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f68cf792b80 1 -- 172.21.15.22:0/3285720780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555a50c082b0 con 0x555a50e8e050 2023-09-20T21:51:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f689d7fa700 1 -- 172.21.15.22:0/3285720780 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68b000f860 con 0x555a50e8e050 2023-09-20T21:51:11.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f689d7fa700 1 -- 172.21.15.22:0/3285720780 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f68ac03bd10 con 0x555a50e8e050 2023-09-20T21:51:11.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f68cf792b80 10 client.25844 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:51:11.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f68cf792b80 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:51:11.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f68cf792b80 20 client.25844 populate_metadata read hostname 'smithi022' 2023-09-20T21:51:11.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x555a50ef8300 0x555a50efa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:11.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f68cf792b80 4 client.25844 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:51:11.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.566+0000 7f68cf792b80 1 -- 172.21.15.22:0/3285720780 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555a50dd7e60 con 0x555a50ef8300 2023-09-20T21:51:11.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.567+0000 7f68b939a700 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x555a50ef8300 0x555a50efa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:11.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.571+0000 7f689d7fa700 10 client.25844 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:51:11.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.571+0000 7f68b939a700 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x555a50ef8300 0x555a50efa720 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34104035dd8eea50 server_cookie=a77b342b7552100f in_seq=0 out_seq=0 2023-09-20T21:51:11.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.572+0000 7f689d7fa700 1 -- 172.21.15.22:0/3285720780 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555a50dd7e60 con 0x555a50ef8300 2023-09-20T21:51:11.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.572+0000 7f689d7fa700 10 client.25844 handle_command_reply: tid=0 2023-09-20T21:51:11.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.572+0000 7f68cf792b80 1 client.25844 shutdown 2023-09-20T21:51:11.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.573+0000 7f68cf792b80 1 -- 172.21.15.22:0/3285720780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 msgr2=0x555a50e9e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:11.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.573+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 0x555a50e9e0e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00a690 tx=0x7f68ac01a570 comp rx=0 tx=0).stop 2023-09-20T21:51:11.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.573+0000 7f68cf792b80 1 -- 172.21.15.22:0/3285720780 shutdown_connections 2023-09-20T21:51:11.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.573+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x555a50ef8300 0x555a50efa720 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.573+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a50e8f210 0x555a50e9c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.573+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a50e8e050 0x555a50e9e0e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.573+0000 7f68cf792b80 1 --2- 172.21.15.22:0/3285720780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a50e8ebe0 0x555a50e9e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.573+0000 7f68cf792b80 1 -- 172.21.15.22:0/3285720780 >> 172.21.15.22:0/3285720780 conn(0x555a50dba000 msgr2=0x555a50e88030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:11.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.573+0000 7f689d7fa700 0 client.25844 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:51:11.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.574+0000 7f68cf792b80 1 -- 172.21.15.22:0/3285720780 shutdown_connections 2023-09-20T21:51:11.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.574+0000 7f68cf792b80 1 -- 172.21.15.22:0/3285720780 wait complete. 2023-09-20T21:51:11.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.578+0000 7f68cf792b80 20 client.25844 trim_cache size 0 max 16384 2023-09-20T21:51:11.580 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:51:11.580 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:51:11.580 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:51:11.581 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:51:11.581 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:51:11.581 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:51:11.581 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:51:11.581 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:51:11.582 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:51:11.582 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:51:11.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.580+0000 7f68bbdff700 1 -- 172.21.15.22:0/3893545964 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68a00a1500 msgr2=0x7f68a00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:11.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.580+0000 7f68bbdff700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68a00a1500 0x7f68a00a39c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x555a50e89150 tx=0x7f68ac0091d0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.580+0000 7f68bbdff700 1 -- 172.21.15.22:0/3893545964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68b40773a0 msgr2=0x7f68b410fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:11.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.580+0000 7f68bbdff700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68b40773a0 0x7f68b410fe50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68a400e970 tx=0x7f68a400ed30 comp rx=0 tx=0).stop 2023-09-20T21:51:11.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.583+0000 7f68bbdff700 1 -- 172.21.15.22:0/3893545964 shutdown_connections 2023-09-20T21:51:11.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.583+0000 7f68bbdff700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68b40773a0 0x7f68b410fe50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.583+0000 7f68bbdff700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68b40757a0 0x7f68b410e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.583+0000 7f68bbdff700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f68a00a1500 0x7f68a00a39c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.583+0000 7f68bbdff700 1 --2- 172.21.15.22:0/3893545964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b40769e0 0x7f68b410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:11.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.583+0000 7f68bbdff700 1 -- 172.21.15.22:0/3893545964 >> 172.21.15.22:0/3893545964 conn(0x7f68b40fe590 msgr2=0x7f68b4100170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:11.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.584+0000 7f68bbdff700 1 -- 172.21.15.22:0/3893545964 shutdown_connections 2023-09-20T21:51:11.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:11.585+0000 7f68bbdff700 1 -- 172.21.15.22:0/3893545964 wait complete. 2023-09-20T21:51:11.617 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:51:11.848 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:11 smithi123 ceph-mon[134433]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 14 MiB/s wr, 825 op/s 2023-09-20T21:51:11.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:11 smithi089 ceph-mon[134090]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 14 MiB/s wr, 825 op/s 2023-09-20T21:51:13.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 110.0 s, 980.6 tps, lat 101.827 ms stddev 143.532 2023-09-20T21:51:13.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:13 smithi022 ceph-mon[97651]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 MiB/s wr, 730 op/s 2023-09-20T21:51:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:13 smithi123 ceph-mon[134433]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 MiB/s wr, 730 op/s 2023-09-20T21:51:14.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:13 smithi089 ceph-mon[134090]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 13 MiB/s wr, 730 op/s 2023-09-20T21:51:16.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:16 smithi123 ceph-mon[134433]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 805 op/s 2023-09-20T21:51:16.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:16 smithi089 ceph-mon[134090]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 805 op/s 2023-09-20T21:51:16.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:16 smithi022 ceph-mon[97651]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 805 op/s 2023-09-20T21:51:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 115.0 s, 1279.4 tps, lat 78.454 ms stddev 77.223 2023-09-20T21:51:18.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:18 smithi123 ceph-mon[134433]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 742 op/s 2023-09-20T21:51:18.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:18 smithi089 ceph-mon[134090]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 742 op/s 2023-09-20T21:51:18.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:18 smithi022 ceph-mon[97651]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 742 op/s 2023-09-20T21:51:20.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:20 smithi022 ceph-mon[97651]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 762 op/s 2023-09-20T21:51:20.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:20 smithi123 ceph-mon[134433]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 762 op/s 2023-09-20T21:51:20.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:20 smithi089 ceph-mon[134090]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 762 op/s 2023-09-20T21:51:22.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:22 smithi022 ceph-mon[97651]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 705 op/s 2023-09-20T21:51:22.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:22 smithi123 ceph-mon[134433]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 705 op/s 2023-09-20T21:51:22.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:22 smithi089 ceph-mon[134090]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 705 op/s 2023-09-20T21:51:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 120.0 s, 1202.8 tps, lat 83.518 ms stddev 94.947 2023-09-20T21:51:23.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:23 smithi022 ceph-mon[97651]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 637 op/s 2023-09-20T21:51:24.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:23 smithi123 ceph-mon[134433]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 637 op/s 2023-09-20T21:51:24.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:23 smithi089 ceph-mon[134090]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 637 op/s 2023-09-20T21:51:26.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:26 smithi022 ceph-mon[97651]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 713 op/s 2023-09-20T21:51:26.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:26 smithi123 ceph-mon[134433]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 713 op/s 2023-09-20T21:51:26.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:26 smithi089 ceph-mon[134090]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 713 op/s 2023-09-20T21:51:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 125.0 s, 1160.3 tps, lat 84.553 ms stddev 91.532 2023-09-20T21:51:28.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:28 smithi022 ceph-mon[97651]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 708 op/s 2023-09-20T21:51:28.848 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:28 smithi123 ceph-mon[134433]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 708 op/s 2023-09-20T21:51:28.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:28 smithi089 ceph-mon[134090]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 708 op/s 2023-09-20T21:51:30.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:30 smithi022 ceph-mon[97651]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 725 op/s 2023-09-20T21:51:30.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:30 smithi123 ceph-mon[134433]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 725 op/s 2023-09-20T21:51:30.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:30 smithi089 ceph-mon[134090]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 14 MiB/s wr, 725 op/s 2023-09-20T21:51:32.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:32 smithi022 ceph-mon[97651]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 1.47k op/s 2023-09-20T21:51:32.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:32 smithi123 ceph-mon[134433]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 1.47k op/s 2023-09-20T21:51:32.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:32 smithi089 ceph-mon[134090]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 1.47k op/s 2023-09-20T21:51:33.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 130.0 s, 792.1 tps, lat 128.819 ms stddev 345.115 2023-09-20T21:51:34.003 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:33 smithi022 ceph-mon[97651]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2023-09-20T21:51:34.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:33 smithi123 ceph-mon[134433]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2023-09-20T21:51:34.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:33 smithi089 ceph-mon[134090]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2023-09-20T21:51:36.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:36 smithi123 ceph-mon[134433]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 1.75k op/s 2023-09-20T21:51:36.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:36 smithi089 ceph-mon[134090]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 1.75k op/s 2023-09-20T21:51:36.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:36 smithi022 ceph-mon[97651]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 1.75k op/s 2023-09-20T21:51:38.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 135.0 s, 1205.8 tps, lat 82.969 ms stddev 99.490 2023-09-20T21:51:38.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:38 smithi089 ceph-mon[134090]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2023-09-20T21:51:38.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:38 smithi022 ceph-mon[97651]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2023-09-20T21:51:38.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:38 smithi123 ceph-mon[134433]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 1.76k op/s 2023-09-20T21:51:40.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:40 smithi022 ceph-mon[97651]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 1.68k op/s 2023-09-20T21:51:40.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:40 smithi123 ceph-mon[134433]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 1.68k op/s 2023-09-20T21:51:40.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:40 smithi089 ceph-mon[134090]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 1.68k op/s 2023-09-20T21:51:41.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:51:41.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.969+0000 7f0625f9d700 1 -- 172.21.15.22:0/2249844928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0620100c50 msgr2=0x7f0620101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:41.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.969+0000 7f0625f9d700 1 --2- 172.21.15.22:0/2249844928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0620100c50 0x7f0620101070 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f06140099c0 tx=0x7f0614009cd0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.969+0000 7f0625f9d700 1 -- 172.21.15.22:0/2249844928 shutdown_connections 2023-09-20T21:51:41.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.969+0000 7f0625f9d700 1 --2- 172.21.15.22:0/2249844928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06201027e0 0x7f0620109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.969+0000 7f0625f9d700 1 --2- 172.21.15.22:0/2249844928 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0620100c50 0x7f0620101070 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.969+0000 7f0625f9d700 1 --2- 172.21.15.22:0/2249844928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0620101e40 0x7f06201022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.969+0000 7f0625f9d700 1 -- 172.21.15.22:0/2249844928 >> 172.21.15.22:0/2249844928 conn(0x7f06200fc1c0 msgr2=0x7f06200fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:41.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.969+0000 7f0625f9d700 1 -- 172.21.15.22:0/2249844928 shutdown_connections 2023-09-20T21:51:41.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.969+0000 7f0625f9d700 1 -- 172.21.15.22:0/2249844928 wait complete. 2023-09-20T21:51:41.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.970+0000 7f0625f9d700 1 Processor -- start 2023-09-20T21:51:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.970+0000 7f0625f9d700 1 -- start start 2023-09-20T21:51:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.970+0000 7f0625f9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0620100c50 0x7f06201968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.970+0000 7f0625f9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0620101e40 0x7f0620196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.970+0000 7f0625f9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06201027e0 0x7f062019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:41.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.970+0000 7f0625f9d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062019e420 con 0x7f06201027e0 2023-09-20T21:51:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.970+0000 7f0625f9d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062019e560 con 0x7f0620100c50 2023-09-20T21:51:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.970+0000 7f0625f9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062019e6f0 con 0x7f0620101e40 2023-09-20T21:51:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f0624f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0620100c50 0x7f06201968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f0624f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0620100c50 0x7f06201968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:40336/0 (socket says 172.21.15.22:40336) 2023-09-20T21:51:41.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f0624f9b700 1 -- 172.21.15.22:0/952377581 learned_addr learned my addr 172.21.15.22:0/952377581 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:51:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f062579c700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06201027e0 0x7f062019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f061ffff700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0620101e40 0x7f0620196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f0624f9b700 1 -- 172.21.15.22:0/952377581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0620101e40 msgr2=0x7f0620196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:41.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f0624f9b700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0620101e40 0x7f0620196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f0624f9b700 1 -- 172.21.15.22:0/952377581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06201027e0 msgr2=0x7f062019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f0624f9b700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06201027e0 0x7f062019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f0624f9b700 1 -- 172.21.15.22:0/952377581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06140095f0 con 0x7f0620100c50 2023-09-20T21:51:41.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f0624f9b700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0620100c50 0x7f06201968a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f06140098e0 tx=0x7f061400f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f061dffb700 1 -- 172.21.15.22:0/952377581 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061401d070 con 0x7f0620100c50 2023-09-20T21:51:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.971+0000 7f061dffb700 1 -- 172.21.15.22:0/952377581 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0614005510 con 0x7f0620100c50 2023-09-20T21:51:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.972+0000 7f0625f9d700 1 -- 172.21.15.22:0/952377581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062019e970 con 0x7f0620100c50 2023-09-20T21:51:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.972+0000 7f061dffb700 1 -- 172.21.15.22:0/952377581 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614004d00 con 0x7f0620100c50 2023-09-20T21:51:41.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.972+0000 7f0625f9d700 1 -- 172.21.15.22:0/952377581 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062019ed90 con 0x7f0620100c50 2023-09-20T21:51:41.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.975+0000 7f061dffb700 1 -- 172.21.15.22:0/952377581 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f061400fc30 con 0x7f0620100c50 2023-09-20T21:51:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.976+0000 7f063ab66b80 1 Processor -- start 2023-09-20T21:51:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.976+0000 7f061dffb700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06100a13a0 0x7f06100a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.976+0000 7f063ab66b80 1 -- start start 2023-09-20T21:51:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.976+0000 7f061dffb700 1 -- 172.21.15.22:0/952377581 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f06140cdf90 con 0x7f0620100c50 2023-09-20T21:51:41.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabd8f0 0x55d2caabdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f062579c700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06100a13a0 0x7f06100a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabe250 0x55d2caabe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caabf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f063ab66b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2caa31ca0 con 0x55d2caabd8f0 2023-09-20T21:51:41.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f063ab66b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2caa15c70 con 0x55d2caabe250 2023-09-20T21:51:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f063ab66b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2caa13590 con 0x55d2caabed60 2023-09-20T21:51:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f0624f9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabd8f0 0x55d2caabdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f0624f9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabd8f0 0x55d2caabdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50926/0 (socket says 172.21.15.22:50926) 2023-09-20T21:51:41.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f0624f9b700 1 -- 172.21.15.22:0/3160789069 learned_addr learned my addr 172.21.15.22:0/3160789069 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:51:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f061ffff700 1 --2- 172.21.15.22:0/3160789069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabe250 0x55d2caabe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f062579c700 1 --2- 172.21.15.22:0/3160789069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caabf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f0624f9b700 1 -- 172.21.15.22:0/3160789069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 msgr2=0x55d2caabf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f0624f9b700 1 --2- 172.21.15.22:0/3160789069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caabf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f0624f9b700 1 -- 172.21.15.22:0/3160789069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabe250 msgr2=0x55d2caabe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f0624f9b700 1 --2- 172.21.15.22:0/3160789069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabe250 0x55d2caabe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f0624f9b700 1 -- 172.21.15.22:0/3160789069 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2ca839de0 con 0x55d2caabd8f0 2023-09-20T21:51:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.977+0000 7f062579c700 1 --2- 172.21.15.22:0/3160789069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caabf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:51:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.978+0000 7f062579c700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06100a13a0 0x7f06100a3860 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0618009550 tx=0x7f06180077e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:41.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.978+0000 7f0624f9b700 1 --2- 172.21.15.22:0/3160789069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabd8f0 0x55d2caabdd10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f061409af50 tx=0x7f061409a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc127fb854ef8da server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.978+0000 7f060affd700 1 -- 172.21.15.22:0/3160789069 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061409b6c0 con 0x55d2caabd8f0 2023-09-20T21:51:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.978+0000 7f060affd700 1 -- 172.21.15.22:0/3160789069 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f061409afc0 con 0x55d2caabd8f0 2023-09-20T21:51:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.978+0000 7f060affd700 1 -- 172.21.15.22:0/3160789069 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061409a710 con 0x55d2caabd8f0 2023-09-20T21:51:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.978+0000 7f063ab66b80 1 -- 172.21.15.22:0/3160789069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabd8f0 msgr2=0x55d2caabdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:41.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.978+0000 7f063ab66b80 1 --2- 172.21.15.22:0/3160789069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabd8f0 0x55d2caabdd10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f061409af50 tx=0x7f061409a9f0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.979+0000 7f063ab66b80 1 -- 172.21.15.22:0/3160789069 shutdown_connections 2023-09-20T21:51:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.979+0000 7f063ab66b80 1 --2- 172.21.15.22:0/3160789069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caabf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.979+0000 7f063ab66b80 1 --2- 172.21.15.22:0/3160789069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabd8f0 0x55d2caabdd10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.979+0000 7f063ab66b80 1 --2- 172.21.15.22:0/3160789069 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabe250 0x55d2caabe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.979+0000 7f063ab66b80 1 -- 172.21.15.22:0/3160789069 >> 172.21.15.22:0/3160789069 conn(0x55d2ca9ea000 msgr2=0x55d2caab83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:41.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.979+0000 7f063ab66b80 1 -- 172.21.15.22:0/3160789069 shutdown_connections 2023-09-20T21:51:41.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.979+0000 7f063ab66b80 1 -- 172.21.15.22:0/3160789069 wait complete. 2023-09-20T21:51:41.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.979+0000 7f063ab66b80 1 -- start start 2023-09-20T21:51:41.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.979+0000 7f063ab66b80 1 Processor -- start 2023-09-20T21:51:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caac6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 0x55d2caac7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 0x55d2caace570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:41.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f063ab66b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2caa0ee80 con 0x55d2caabed60 2023-09-20T21:51:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f063ab66b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ca9ab740 con 0x55d2caabd8f0 2023-09-20T21:51:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f063ab66b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ca9ab3e0 con 0x55d2caabe250 2023-09-20T21:51:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f063ab66b80 10 client.0 _ll_register_callbacks cb 0x55d2ca8f5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:51:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f062579c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 0x55d2caace570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:41.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f0624f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caac6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f0624f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caac6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:40366/0 (socket says 172.21.15.22:40366) 2023-09-20T21:51:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f0624f9b700 1 -- 172.21.15.22:0/3163829057 learned_addr learned my addr 172.21.15.22:0/3163829057 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:51:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.980+0000 7f061ffff700 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 0x55d2caac7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f0624f9b700 1 -- 172.21.15.22:0/3163829057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 msgr2=0x55d2caac7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:41.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f0624f9b700 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 0x55d2caac7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f0624f9b700 1 -- 172.21.15.22:0/3163829057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 msgr2=0x55d2caace570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f0624f9b700 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 0x55d2caace570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f0624f9b700 1 -- 172.21.15.22:0/3163829057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062019ed90 con 0x55d2caabd8f0 2023-09-20T21:51:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f062579c700 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 0x55d2caace570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:51:41.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f0624f9b700 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caac6ff0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06140995b0 tx=0x7f0614009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47fc57066f7acbd1 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:41.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f05fffff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:51:41.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f05fffff700 1 -- 172.21.15.22:0/3163829057 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614095dd0 con 0x55d2caabd8f0 2023-09-20T21:51:41.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f05fffff700 1 -- 172.21.15.22:0/3163829057 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f06140a2fa0 con 0x55d2caabd8f0 2023-09-20T21:51:41.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f063ab66b80 1 -- 172.21.15.22:0/3163829057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2ca97fbb0 con 0x55d2caabd8f0 2023-09-20T21:51:41.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f05fffff700 1 -- 172.21.15.22:0/3163829057 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06140b32e0 con 0x55d2caabd8f0 2023-09-20T21:51:41.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f063ab66b80 10 client.25856 fetch_fsmap learned FSMap version 30 2023-09-20T21:51:41.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f063ab66b80 1 -- 172.21.15.22:0/3163829057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d2ca822c60 con 0x55d2caabd8f0 2023-09-20T21:51:41.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.981+0000 7f05fffff700 1 -- 172.21.15.22:0/3163829057 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2ca97fbb0 con 0x55d2caabd8f0 2023-09-20T21:51:42.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.982+0000 7f05fffff700 1 -- 172.21.15.22:0/3163829057 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f061409fb30 con 0x55d2caabd8f0 2023-09-20T21:51:42.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.982+0000 7f063ab66b80 10 client.25856 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:51:42.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.982+0000 7f063ab66b80 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:51:42.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.982+0000 7f063ab66b80 20 client.25856 populate_metadata read hostname 'smithi022' 2023-09-20T21:51:42.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.982+0000 7f063ab66b80 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d2caad2880 0x55d2caad4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:42.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.982+0000 7f063ab66b80 4 client.25856 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:51:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.982+0000 7f063ab66b80 1 -- 172.21.15.22:0/3163829057 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2caa8c020 con 0x55d2caad2880 2023-09-20T21:51:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.982+0000 7f061ffff700 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d2caad2880 0x55d2caad4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.982+0000 7f061ffff700 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d2caad2880 0x55d2caad4ca0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c07c86a7624e7 server_cookie=1b0061b5801c7c50 in_seq=0 out_seq=0 2023-09-20T21:51:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.982+0000 7f05fffff700 10 client.25856 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:51:42.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.983+0000 7f05fffff700 1 -- 172.21.15.22:0/3163829057 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d2caa8c020 con 0x55d2caad2880 2023-09-20T21:51:42.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.983+0000 7f05fffff700 10 client.25856 handle_command_reply: tid=0 2023-09-20T21:51:42.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.983+0000 7f063ab66b80 1 client.25856 shutdown 2023-09-20T21:51:42.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.983+0000 7f063ab66b80 1 -- 172.21.15.22:0/3163829057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 msgr2=0x55d2caac6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:42.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.983+0000 7f063ab66b80 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caac6ff0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06140995b0 tx=0x7f0614009cd0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.983+0000 7f063ab66b80 1 -- 172.21.15.22:0/3163829057 shutdown_connections 2023-09-20T21:51:42.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.983+0000 7f063ab66b80 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d2caad2880 0x55d2caad4ca0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.983+0000 7f063ab66b80 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 0x55d2caace570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.983+0000 7f05fffff700 0 client.25856 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:51:42.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.983+0000 7f063ab66b80 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caac6ff0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.984+0000 7f063ab66b80 1 --2- 172.21.15.22:0/3163829057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 0x55d2caac7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.984+0000 7f063ab66b80 1 -- 172.21.15.22:0/3163829057 >> 172.21.15.22:0/3163829057 conn(0x55d2ca9ea000 msgr2=0x55d2caab7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:42.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.984+0000 7f063ab66b80 1 -- 172.21.15.22:0/3163829057 shutdown_connections 2023-09-20T21:51:42.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.984+0000 7f063ab66b80 1 -- 172.21.15.22:0/3163829057 wait complete. 2023-09-20T21:51:42.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:41.984+0000 7f063ab66b80 20 client.25856 trim_cache size 0 max 16384 2023-09-20T21:51:42.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.009+0000 7f063ab66b80 1 Processor -- start 2023-09-20T21:51:42.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.018+0000 7f063ab66b80 1 -- start start 2023-09-20T21:51:42.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.018+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caacee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:42.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.018+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabe250 0x55d2caace500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:42.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.018+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caac4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:42.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.019+0000 7f063ab66b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ca88f920 con 0x55d2caabe250 2023-09-20T21:51:42.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.019+0000 7f063ab66b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2caaba1c0 con 0x55d2caabd8f0 2023-09-20T21:51:42.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.019+0000 7f063ab66b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2caa8c020 con 0x55d2caabed60 2023-09-20T21:51:42.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.020+0000 7f062579c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caac4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:42.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.020+0000 7f062579c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caac4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:44772/0 (socket says 172.21.15.22:44772) 2023-09-20T21:51:42.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.020+0000 7f062579c700 1 -- 172.21.15.22:0/2017884243 learned_addr learned my addr 172.21.15.22:0/2017884243 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:51:42.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.020+0000 7f061ffff700 1 --2- 172.21.15.22:0/2017884243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabe250 0x55d2caace500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:42.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.021+0000 7f062579c700 1 -- 172.21.15.22:0/2017884243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 msgr2=0x55d2caacee60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:51:42.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.021+0000 7f062579c700 1 --2- 172.21.15.22:0/2017884243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caacee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.021+0000 7f062579c700 1 -- 172.21.15.22:0/2017884243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabe250 msgr2=0x55d2caace500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:42.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.021+0000 7f062579c700 1 --2- 172.21.15.22:0/2017884243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabe250 0x55d2caace500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.021+0000 7f062579c700 1 -- 172.21.15.22:0/2017884243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2ca860250 con 0x55d2caabed60 2023-09-20T21:51:42.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.022+0000 7f062579c700 1 --2- 172.21.15.22:0/2017884243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caac4f10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f061801a7b0 tx=0x7f0618009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79941159e9b1a29 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:42.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.031+0000 7f060b7fe700 1 -- 172.21.15.22:0/2017884243 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061801f070 con 0x55d2caabed60 2023-09-20T21:51:42.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.031+0000 7f060b7fe700 1 -- 172.21.15.22:0/2017884243 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0618047030 con 0x55d2caabed60 2023-09-20T21:51:42.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.031+0000 7f060b7fe700 1 -- 172.21.15.22:0/2017884243 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0618016660 con 0x55d2caabed60 2023-09-20T21:51:42.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.032+0000 7f063ab66b80 1 -- 172.21.15.22:0/2017884243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 msgr2=0x55d2caac4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:42.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.032+0000 7f063ab66b80 1 --2- 172.21.15.22:0/2017884243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caac4f10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f061801a7b0 tx=0x7f0618009040 comp rx=0 tx=0).stop 2023-09-20T21:51:42.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.039+0000 7f063ab66b80 1 -- 172.21.15.22:0/2017884243 shutdown_connections 2023-09-20T21:51:42.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.039+0000 7f063ab66b80 1 --2- 172.21.15.22:0/2017884243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabed60 0x55d2caac4f10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.039+0000 7f063ab66b80 1 --2- 172.21.15.22:0/2017884243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caacee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.039+0000 7f063ab66b80 1 --2- 172.21.15.22:0/2017884243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabe250 0x55d2caace500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.039+0000 7f063ab66b80 1 -- 172.21.15.22:0/2017884243 >> 172.21.15.22:0/2017884243 conn(0x55d2ca9ea000 msgr2=0x55d2caab4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.057+0000 7f063ab66b80 1 -- 172.21.15.22:0/2017884243 shutdown_connections 2023-09-20T21:51:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.057+0000 7f063ab66b80 1 -- 172.21.15.22:0/2017884243 wait complete. 2023-09-20T21:51:42.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.058+0000 7f063ab66b80 1 -- start start 2023-09-20T21:51:42.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.058+0000 7f063ab66b80 1 Processor -- start 2023-09-20T21:51:42.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.059+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caaca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:42.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.059+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 0x55d2caacac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:42.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.059+0000 7f063ab66b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 0x55d2caacd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:42.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.059+0000 7f063ab66b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2caacdea0 con 0x55d2caabed60 2023-09-20T21:51:42.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.059+0000 7f063ab66b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2caace010 con 0x55d2caabd8f0 2023-09-20T21:51:42.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.059+0000 7f063ab66b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2caace180 con 0x55d2caabe250 2023-09-20T21:51:42.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.059+0000 7f063ab66b80 10 client.0 _ll_register_callbacks cb 0x55d2ca8f5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:51:42.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.064+0000 7f0624f9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caaca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:42.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.064+0000 7f061ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 0x55d2caacac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:42.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.064+0000 7f061ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 0x55d2caacac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:44782/0 (socket says 172.21.15.22:44782) 2023-09-20T21:51:42.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.064+0000 7f061ffff700 1 -- 172.21.15.22:0/537334536 learned_addr learned my addr 172.21.15.22:0/537334536 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:51:42.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.071+0000 7f062579c700 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 0x55d2caacd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:42.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.072+0000 7f061ffff700 1 -- 172.21.15.22:0/537334536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 msgr2=0x55d2caaca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:42.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.072+0000 7f061ffff700 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caaca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.072+0000 7f061ffff700 1 -- 172.21.15.22:0/537334536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 msgr2=0x55d2caacd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:51:42.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.072+0000 7f061ffff700 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 0x55d2caacd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.072+0000 7f061ffff700 1 -- 172.21.15.22:0/537334536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061801a430 con 0x55d2caabe250 2023-09-20T21:51:42.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.072+0000 7f061ffff700 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 0x55d2caacac60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55d2caad55b0 tx=0x7f060c020d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d4639aac5e85ba server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:51:42.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.077+0000 7f05fffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:51:42.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.077+0000 7f05fffff700 1 -- 172.21.15.22:0/537334536 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c017040 con 0x55d2caabe250 2023-09-20T21:51:42.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.077+0000 7f05fffff700 1 -- 172.21.15.22:0/537334536 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f060c004540 con 0x55d2caabe250 2023-09-20T21:51:42.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.081+0000 7f063ab66b80 1 -- 172.21.15.22:0/537334536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f061409afc0 con 0x55d2caabe250 2023-09-20T21:51:42.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.082+0000 7f05fffff700 1 -- 172.21.15.22:0/537334536 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f060c003880 con 0x55d2caabe250 2023-09-20T21:51:42.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.082+0000 7f063ab66b80 10 client.25855 fetch_fsmap learned FSMap version 30 2023-09-20T21:51:42.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.082+0000 7f063ab66b80 1 -- 172.21.15.22:0/537334536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d2ca8382b0 con 0x55d2caabe250 2023-09-20T21:51:42.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.083+0000 7f05fffff700 1 -- 172.21.15.22:0/537334536 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f061409afc0 con 0x55d2caabe250 2023-09-20T21:51:42.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.083+0000 7f05fffff700 1 -- 172.21.15.22:0/537334536 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f060c012bc0 con 0x55d2caabe250 2023-09-20T21:51:42.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.083+0000 7f063ab66b80 10 client.25855 fetch_fsmap finished waiting for FSMap version 30 2023-09-20T21:51:42.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.083+0000 7f063ab66b80 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:51:42.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.083+0000 7f063ab66b80 20 client.25855 populate_metadata read hostname 'smithi022' 2023-09-20T21:51:42.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.083+0000 7f063ab66b80 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d2caad2880 0x55d2cab251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:51:42.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.083+0000 7f063ab66b80 4 client.25855 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:51:42.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.083+0000 7f063ab66b80 1 -- 172.21.15.22:0/537334536 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d2caa07e60 con 0x55d2caad2880 2023-09-20T21:51:42.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.084+0000 7f0624f9b700 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d2caad2880 0x55d2cab251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:51:42.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.085+0000 7f0624f9b700 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d2caad2880 0x55d2cab251f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cd78973c1a4872e server_cookie=527a238dd7814c3f in_seq=0 out_seq=0 2023-09-20T21:51:42.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.085+0000 7f05fffff700 10 client.25855 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:51:42.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.085+0000 7f05fffff700 1 -- 172.21.15.22:0/537334536 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d2caa07e60 con 0x55d2caad2880 2023-09-20T21:51:42.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.085+0000 7f05fffff700 10 client.25855 handle_command_reply: tid=0 2023-09-20T21:51:42.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.085+0000 7f063ab66b80 1 client.25855 shutdown 2023-09-20T21:51:42.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.085+0000 7f063ab66b80 1 -- 172.21.15.22:0/537334536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 msgr2=0x55d2caacac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:42.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.085+0000 7f063ab66b80 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 0x55d2caacac60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55d2caad55b0 tx=0x7f060c020d40 comp rx=0 tx=0).stop 2023-09-20T21:51:42.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.089+0000 7f063ab66b80 1 -- 172.21.15.22:0/537334536 shutdown_connections 2023-09-20T21:51:42.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.089+0000 7f063ab66b80 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d2caad2880 0x55d2cab251f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.089+0000 7f063ab66b80 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2caabed60 0x55d2caacd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.089+0000 7f063ab66b80 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d2caabd8f0 0x55d2caaca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.089+0000 7f063ab66b80 1 --2- 172.21.15.22:0/537334536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2caabe250 0x55d2caacac60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.089+0000 7f063ab66b80 1 -- 172.21.15.22:0/537334536 >> 172.21.15.22:0/537334536 conn(0x55d2ca9ea000 msgr2=0x55d2caacee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:42.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.089+0000 7f05fffff700 0 client.25855 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:51:42.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.089+0000 7f063ab66b80 1 -- 172.21.15.22:0/537334536 shutdown_connections 2023-09-20T21:51:42.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.089+0000 7f063ab66b80 1 -- 172.21.15.22:0/537334536 wait complete. 2023-09-20T21:51:42.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.091+0000 7f063ab66b80 20 client.25855 trim_cache size 0 max 16384 2023-09-20T21:51:42.096 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:51:42.097 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:51:42.097 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:51:42.097 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:51:42.097 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:51:42.098 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:51:42.098 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:51:42.098 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:51:42.098 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:51:42.098 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:51:42.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.092+0000 7f0625f9d700 1 -- 172.21.15.22:0/952377581 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06100a13a0 msgr2=0x7f06100a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:42.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.092+0000 7f0625f9d700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06100a13a0 0x7f06100a3860 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0618009550 tx=0x7f06180077e0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.092+0000 7f0625f9d700 1 -- 172.21.15.22:0/952377581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0620100c50 msgr2=0x7f06201968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:51:42.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.092+0000 7f0625f9d700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0620100c50 0x7f06201968a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f06140098e0 tx=0x7f061400f740 comp rx=0 tx=0).stop 2023-09-20T21:51:42.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.093+0000 7f0625f9d700 1 -- 172.21.15.22:0/952377581 shutdown_connections 2023-09-20T21:51:42.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.093+0000 7f0625f9d700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06201027e0 0x7f062019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.093+0000 7f0625f9d700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f06100a13a0 0x7f06100a3860 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.093+0000 7f0625f9d700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0620100c50 0x7f06201968a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.093+0000 7f0625f9d700 1 --2- 172.21.15.22:0/952377581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0620101e40 0x7f0620196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:51:42.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.093+0000 7f0625f9d700 1 -- 172.21.15.22:0/952377581 >> 172.21.15.22:0/952377581 conn(0x7f06200fc1c0 msgr2=0x7f06200fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:51:42.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.093+0000 7f0625f9d700 1 -- 172.21.15.22:0/952377581 shutdown_connections 2023-09-20T21:51:42.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:51:42.093+0000 7f0625f9d700 1 -- 172.21.15.22:0/952377581 wait complete. 2023-09-20T21:51:42.127 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:51:42.443 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:42 smithi022 ceph-mon[97651]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.74k op/s 2023-09-20T21:51:42.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:42 smithi123 ceph-mon[134433]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.74k op/s 2023-09-20T21:51:42.865 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:42 smithi089 ceph-mon[134090]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.74k op/s 2023-09-20T21:51:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 140.0 s, 1298.8 tps, lat 76.280 ms stddev 89.208 2023-09-20T21:51:43.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:43 smithi022 ceph-mon[97651]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 910 op/s 2023-09-20T21:51:44.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:43 smithi123 ceph-mon[134433]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 910 op/s 2023-09-20T21:51:44.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:43 smithi089 ceph-mon[134090]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 910 op/s 2023-09-20T21:51:46.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:46 smithi089 ceph-mon[134090]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 715 op/s 2023-09-20T21:51:46.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:46 smithi022 ceph-mon[97651]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 715 op/s 2023-09-20T21:51:46.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:46 smithi123 ceph-mon[134433]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 715 op/s 2023-09-20T21:51:48.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 145.0 s, 1226.8 tps, lat 80.855 ms stddev 83.649 2023-09-20T21:51:48.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:48 smithi022 ceph-mon[97651]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 636 op/s 2023-09-20T21:51:48.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:48 smithi123 ceph-mon[134433]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 636 op/s 2023-09-20T21:51:48.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:48 smithi089 ceph-mon[134090]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 636 op/s 2023-09-20T21:51:50.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:50 smithi022 ceph-mon[97651]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 9.2 MiB/s wr, 559 op/s 2023-09-20T21:51:50.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:50 smithi123 ceph-mon[134433]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 9.2 MiB/s wr, 559 op/s 2023-09-20T21:51:50.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:50 smithi089 ceph-mon[134090]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 9.2 MiB/s wr, 559 op/s 2023-09-20T21:51:52.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:52 smithi022 ceph-mon[97651]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 707 op/s 2023-09-20T21:51:52.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:52 smithi123 ceph-mon[134433]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 707 op/s 2023-09-20T21:51:52.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:52 smithi089 ceph-mon[134090]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 707 op/s 2023-09-20T21:51:53.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 150.0 s, 1205.5 tps, lat 82.381 ms stddev 85.896 2023-09-20T21:51:53.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:53 smithi022 ceph-mon[97651]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.0 MiB/s wr, 612 op/s 2023-09-20T21:51:54.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:53 smithi123 ceph-mon[134433]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.0 MiB/s wr, 612 op/s 2023-09-20T21:51:54.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:53 smithi089 ceph-mon[134090]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.0 MiB/s wr, 612 op/s 2023-09-20T21:51:56.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:56 smithi123 ceph-mon[134433]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.5 MiB/s wr, 654 op/s 2023-09-20T21:51:56.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:56 smithi089 ceph-mon[134090]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.5 MiB/s wr, 654 op/s 2023-09-20T21:51:56.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:56 smithi022 ceph-mon[97651]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.5 MiB/s wr, 654 op/s 2023-09-20T21:51:58.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 155.0 s, 1280.3 tps, lat 78.833 ms stddev 88.032 2023-09-20T21:51:58.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:51:58 smithi123 ceph-mon[134433]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 693 op/s 2023-09-20T21:51:58.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:51:58 smithi089 ceph-mon[134090]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 693 op/s 2023-09-20T21:51:58.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:51:58 smithi022 ceph-mon[97651]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 693 op/s 2023-09-20T21:52:00.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:00 smithi022 ceph-mon[97651]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 618 op/s 2023-09-20T21:52:00.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:00 smithi123 ceph-mon[134433]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 618 op/s 2023-09-20T21:52:00.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:00 smithi089 ceph-mon[134090]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 618 op/s 2023-09-20T21:52:02.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:02 smithi022 ceph-mon[97651]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 777 op/s 2023-09-20T21:52:02.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:02 smithi123 ceph-mon[134433]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 777 op/s 2023-09-20T21:52:02.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:02 smithi089 ceph-mon[134090]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 777 op/s 2023-09-20T21:52:03.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 160.0 s, 1269.1 tps, lat 80.354 ms stddev 90.260 2023-09-20T21:52:03.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:03 smithi022 ceph-mon[97651]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.9 MiB/s wr, 640 op/s 2023-09-20T21:52:04.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:03 smithi123 ceph-mon[134433]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.9 MiB/s wr, 640 op/s 2023-09-20T21:52:04.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:03 smithi089 ceph-mon[134090]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 8.9 MiB/s wr, 640 op/s 2023-09-20T21:52:06.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:06 smithi123 ceph-mon[134433]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 9.0 MiB/s wr, 653 op/s 2023-09-20T21:52:06.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:06 smithi089 ceph-mon[134090]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 9.0 MiB/s wr, 653 op/s 2023-09-20T21:52:06.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:06 smithi022 ceph-mon[97651]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 9.0 MiB/s wr, 653 op/s 2023-09-20T21:52:08.572 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 165.0 s, 1029.4 tps, lat 94.012 ms stddev 124.769 2023-09-20T21:52:08.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:08 smithi022 ceph-mon[97651]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.3 MiB/s wr, 686 op/s 2023-09-20T21:52:08.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:08 smithi123 ceph-mon[134433]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.3 MiB/s wr, 686 op/s 2023-09-20T21:52:08.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:08 smithi089 ceph-mon[134090]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.3 MiB/s wr, 686 op/s 2023-09-20T21:52:10.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:10 smithi022 ceph-mon[97651]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.3 MiB/s wr, 587 op/s 2023-09-20T21:52:10.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:10 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:52:10.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:10 smithi123 ceph-mon[134433]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.3 MiB/s wr, 587 op/s 2023-09-20T21:52:10.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:10 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:52:10.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:10 smithi089 ceph-mon[134090]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.3 MiB/s wr, 587 op/s 2023-09-20T21:52:10.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:10 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:52:11.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:11 smithi022 ceph-mon[97651]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:52:11.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:52:11.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:52:11.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:11 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:52:11.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:11 smithi123 ceph-mon[134433]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:52:11.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:52:11.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:52:11.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:11 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:52:11.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:11 smithi089 ceph-mon[134090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:52:11.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:52:11.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:52:11.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:11 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:52:12.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:52:12.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.540+0000 7fbb4dfff700 1 -- 172.21.15.22:0/3144192531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb4810d120 msgr2=0x7fbb4810d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:12.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.540+0000 7fbb4dfff700 1 --2- 172.21.15.22:0/3144192531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb4810d120 0x7fbb4810d5a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbb3801a9a0 tx=0x7fbb3801acb0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.542+0000 7fbb4dfff700 1 -- 172.21.15.22:0/3144192531 shutdown_connections 2023-09-20T21:52:12.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.542+0000 7fbb4dfff700 1 --2- 172.21.15.22:0/3144192531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb4810d120 0x7fbb4810d5a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.542+0000 7fbb4dfff700 1 --2- 172.21.15.22:0/3144192531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb48107de0 0x7fbb4810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.542+0000 7fbb4dfff700 1 --2- 172.21.15.22:0/3144192531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb4810a750 0x7fbb4810cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.542+0000 7fbb4dfff700 1 -- 172.21.15.22:0/3144192531 >> 172.21.15.22:0/3144192531 conn(0x7fbb4806db90 msgr2=0x7fbb4806ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:12.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.554+0000 7fbb4dfff700 1 -- 172.21.15.22:0/3144192531 shutdown_connections 2023-09-20T21:52:12.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.559+0000 7fbb4dfff700 1 -- 172.21.15.22:0/3144192531 wait complete. 2023-09-20T21:52:12.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.561+0000 7fbb4dfff700 1 Processor -- start 2023-09-20T21:52:12.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.564+0000 7fbb4dfff700 1 -- start start 2023-09-20T21:52:12.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.565+0000 7fbb4dfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb48107de0 0x7fbb4819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.565+0000 7fbb4dfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4810a750 0x7fbb4819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.565+0000 7fbb4dfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb4810d120 0x7fbb481a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.565+0000 7fbb4dfff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481a4bb0 con 0x7fbb4810d120 2023-09-20T21:52:12.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.565+0000 7fbb4dfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481a4cf0 con 0x7fbb4810a750 2023-09-20T21:52:12.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.565+0000 7fbb4dfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481a4e80 con 0x7fbb48107de0 2023-09-20T21:52:12.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.567+0000 7fbb46ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4810a750 0x7fbb4819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.567+0000 7fbb46ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4810a750 0x7fbb4819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:49738/0 (socket says 172.21.15.22:49738) 2023-09-20T21:52:12.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.567+0000 7fbb46ffd700 1 -- 172.21.15.22:0/2448635479 learned_addr learned my addr 172.21.15.22:0/2448635479 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:52:12.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.567+0000 7fbb46ffd700 1 -- 172.21.15.22:0/2448635479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb48107de0 msgr2=0x7fbb4819d030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:52:12.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.567+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb48107de0 0x7fbb4819d030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.567+0000 7fbb46ffd700 1 -- 172.21.15.22:0/2448635479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb4810d120 msgr2=0x7fbb481a45e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:52:12.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.567+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb4810d120 0x7fbb481a45e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.567+0000 7fbb46ffd700 1 -- 172.21.15.22:0/2448635479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3801a600 con 0x7fbb4810a750 2023-09-20T21:52:12.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.572+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4810a750 0x7fbb4819d570 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c920 tx=0x7fbb3c00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:12.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.575+0000 7fbb44ff9700 1 -- 172.21.15.22:0/2448635479 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c007b00 con 0x7fbb4810a750 2023-09-20T21:52:12.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.575+0000 7fbb44ff9700 1 -- 172.21.15.22:0/2448635479 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb3c004cf0 con 0x7fbb4810a750 2023-09-20T21:52:12.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.575+0000 7fbb44ff9700 1 -- 172.21.15.22:0/2448635479 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0054f0 con 0x7fbb4810a750 2023-09-20T21:52:12.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.575+0000 7fbb4dfff700 1 -- 172.21.15.22:0/2448635479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb481a5160 con 0x7fbb4810a750 2023-09-20T21:52:12.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.575+0000 7fbb4dfff700 1 -- 172.21.15.22:0/2448635479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb481a5620 con 0x7fbb4810a750 2023-09-20T21:52:12.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.583+0000 7fbb618d8b80 1 Processor -- start 2023-09-20T21:52:12.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.585+0000 7fbb44ff9700 1 -- 172.21.15.22:0/2448635479 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fbb3c004750 con 0x7fbb4810a750 2023-09-20T21:52:12.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.586+0000 7fbb618d8b80 1 -- start start 2023-09-20T21:52:12.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.586+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686d6f0 0x557d5686db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.586+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686e050 0x557d5686e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.586+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686ebe0 0x557d5686f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.586+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d567e1ca0 con 0x557d5686d6f0 2023-09-20T21:52:12.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.586+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d567c5c70 con 0x557d5686ebe0 2023-09-20T21:52:12.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.586+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d567c3590 con 0x557d5686e050 2023-09-20T21:52:12.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.587+0000 7fbb477fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686e050 0x557d5686e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.587+0000 7fbb477fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686e050 0x557d5686e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:51798/0 (socket says 172.21.15.22:51798) 2023-09-20T21:52:12.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.587+0000 7fbb477fe700 1 -- 172.21.15.22:0/3182694239 learned_addr learned my addr 172.21.15.22:0/3182694239 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:52:12.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.587+0000 7fbb477fe700 1 -- 172.21.15.22:0/3182694239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686ebe0 msgr2=0x557d5686f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:12.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.588+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686ebe0 0x557d5686f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.588+0000 7fbb47fff700 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686d6f0 0x557d5686db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.588+0000 7fbb477fe700 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686ebe0 0x557d5686f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.588+0000 7fbb477fe700 1 -- 172.21.15.22:0/3182694239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686d6f0 msgr2=0x557d5686db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:12.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.588+0000 7fbb477fe700 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686d6f0 0x557d5686db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.588+0000 7fbb477fe700 1 -- 172.21.15.22:0/3182694239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d565e9de0 con 0x557d5686e050 2023-09-20T21:52:12.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.588+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686ebe0 0x557d5686f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:52:12.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.588+0000 7fbb47fff700 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686d6f0 0x557d5686db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:52:12.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.588+0000 7fbb477fe700 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686e050 0x557d5686e5e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009c60 tx=0x7fbb40009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238b6d3a9f332a2b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:12.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.589+0000 7fbb2dffb700 1 -- 172.21.15.22:0/3182694239 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40004bb0 con 0x557d5686e050 2023-09-20T21:52:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.589+0000 7fbb2dffb700 1 -- 172.21.15.22:0/3182694239 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb4000f040 con 0x557d5686e050 2023-09-20T21:52:12.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.589+0000 7fbb2dffb700 1 -- 172.21.15.22:0/3182694239 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40013650 con 0x557d5686e050 2023-09-20T21:52:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.590+0000 7fbb44ff9700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb300a13d0 0x7fbb300a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.590+0000 7fbb44ff9700 1 -- 172.21.15.22:0/2448635479 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fbb3c0ccb80 con 0x7fbb4810a750 2023-09-20T21:52:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.592+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3182694239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686e050 msgr2=0x557d5686e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.592+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686e050 0x557d5686e5e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009c60 tx=0x7fbb40009f70 comp rx=0 tx=0).stop 2023-09-20T21:52:12.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.592+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3182694239 shutdown_connections 2023-09-20T21:52:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.592+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686ebe0 0x557d5686f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.592+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686d6f0 0x557d5686db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.592+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/3182694239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686e050 0x557d5686e5e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.592+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3182694239 >> 172.21.15.22:0/3182694239 conn(0x557d5679a000 msgr2=0x557d568681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:12.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.592+0000 7fbb47fff700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb300a13d0 0x7fbb300a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.592+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3182694239 shutdown_connections 2023-09-20T21:52:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.593+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3182694239 wait complete. 2023-09-20T21:52:12.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.594+0000 7fbb618d8b80 1 -- start start 2023-09-20T21:52:12.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.594+0000 7fbb618d8b80 1 Processor -- start 2023-09-20T21:52:12.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.595+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 0x557d568771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.595+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686e050 0x557d56877730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.595+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686ebe0 0x557d5687e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.595+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d567bee80 con 0x557d5686ebe0 2023-09-20T21:52:12.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.595+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5675b740 con 0x557d5686e050 2023-09-20T21:52:12.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.595+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5675b3e0 con 0x557d5686d6f0 2023-09-20T21:52:12.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.595+0000 7fbb618d8b80 10 client.0 _ll_register_callbacks cb 0x557d566a5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:52:12.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.596+0000 7fbb47fff700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb300a13d0 0x7fbb300a3890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbb38004820 tx=0x7fbb38005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:12.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.596+0000 7fbb47fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686ebe0 0x557d5687e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.596+0000 7fbb47fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686ebe0 0x557d5687e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55184/0 (socket says 172.21.15.22:55184) 2023-09-20T21:52:12.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.596+0000 7fbb47fff700 1 -- 172.21.15.22:0/4000823261 learned_addr learned my addr 172.21.15.22:0/4000823261 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:52:12.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.596+0000 7fbb477fe700 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 0x557d568771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.597+0000 7fbb47fff700 1 -- 172.21.15.22:0/4000823261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 msgr2=0x557d568771f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:12.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.597+0000 7fbb47fff700 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 0x557d568771f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.597+0000 7fbb47fff700 1 -- 172.21.15.22:0/4000823261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686e050 msgr2=0x557d56877730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.597+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686e050 0x557d56877730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.597+0000 7fbb47fff700 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686e050 0x557d56877730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.597+0000 7fbb47fff700 1 -- 172.21.15.22:0/4000823261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb400098c0 con 0x557d5686ebe0 2023-09-20T21:52:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.597+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686e050 0x557d56877730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:52:12.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.597+0000 7fbb47fff700 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686ebe0 0x557d5687e770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbb3801a9a0 tx=0x7fbb3801a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62cd46599a9247d server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.597+0000 7fbb26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:52:12.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.598+0000 7fbb26ffd700 1 -- 172.21.15.22:0/4000823261 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3802e070 con 0x557d5686ebe0 2023-09-20T21:52:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.598+0000 7fbb26ffd700 1 -- 172.21.15.22:0/4000823261 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fbb38003fb0 con 0x557d5686ebe0 2023-09-20T21:52:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.598+0000 7fbb618d8b80 1 -- 172.21.15.22:0/4000823261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d5672fbb0 con 0x557d5686ebe0 2023-09-20T21:52:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.598+0000 7fbb26ffd700 1 -- 172.21.15.22:0/4000823261 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb38004190 con 0x557d5686ebe0 2023-09-20T21:52:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.598+0000 7fbb618d8b80 10 client.15747 fetch_fsmap learned FSMap version 31 2023-09-20T21:52:12.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.598+0000 7fbb618d8b80 1 -- 172.21.15.22:0/4000823261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557d565d2c60 con 0x557d5686ebe0 2023-09-20T21:52:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.599+0000 7fbb26ffd700 1 -- 172.21.15.22:0/4000823261 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb38047070 con 0x557d5686ebe0 2023-09-20T21:52:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.599+0000 7fbb26ffd700 1 -- 172.21.15.22:0/4000823261 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 31) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fbb38034410 con 0x557d5686ebe0 2023-09-20T21:52:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.599+0000 7fbb618d8b80 10 client.15747 fetch_fsmap finished waiting for FSMap version 31 2023-09-20T21:52:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.599+0000 7fbb618d8b80 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:52:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.599+0000 7fbb618d8b80 20 client.15747 populate_metadata read hostname 'smithi022' 2023-09-20T21:52:12.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.599+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557d56883010 0x557d56885430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.600+0000 7fbb618d8b80 4 client.15747 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:52:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.600+0000 7fbb618d8b80 1 -- 172.21.15.22:0/4000823261 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d5683c020 con 0x557d56883010 2023-09-20T21:52:12.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.600+0000 7fbb477fe700 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557d56883010 0x557d56885430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.601+0000 7fbb477fe700 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557d56883010 0x557d56885430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ef7a4af8cdcc0d4 server_cookie=63aed9b849f5c5ae in_seq=0 out_seq=0 2023-09-20T21:52:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.602+0000 7fbb26ffd700 10 client.15747 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:52:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.603+0000 7fbb26ffd700 1 -- 172.21.15.22:0/4000823261 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557d5683c020 con 0x557d56883010 2023-09-20T21:52:12.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.603+0000 7fbb26ffd700 10 client.15747 handle_command_reply: tid=0 2023-09-20T21:52:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.603+0000 7fbb618d8b80 1 client.15747 shutdown 2023-09-20T21:52:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.604+0000 7fbb618d8b80 1 -- 172.21.15.22:0/4000823261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686ebe0 msgr2=0x557d5687e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.604+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686ebe0 0x557d5687e770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbb3801a9a0 tx=0x7fbb3801a320 comp rx=0 tx=0).stop 2023-09-20T21:52:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.604+0000 7fbb618d8b80 1 -- 172.21.15.22:0/4000823261 shutdown_connections 2023-09-20T21:52:12.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.604+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557d56883010 0x557d56885430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.604+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686ebe0 0x557d5687e770 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.604+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 0x557d568771f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.604+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/4000823261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686e050 0x557d56877730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.604+0000 7fbb618d8b80 1 -- 172.21.15.22:0/4000823261 >> 172.21.15.22:0/4000823261 conn(0x557d5679a000 msgr2=0x557d56867880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:12.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.604+0000 7fbb26ffd700 0 client.15747 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:52:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.605+0000 7fbb618d8b80 1 -- 172.21.15.22:0/4000823261 shutdown_connections 2023-09-20T21:52:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.605+0000 7fbb618d8b80 1 -- 172.21.15.22:0/4000823261 wait complete. 2023-09-20T21:52:12.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.605+0000 7fbb618d8b80 20 client.15747 trim_cache size 0 max 16384 2023-09-20T21:52:12.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.635+0000 7fbb618d8b80 1 Processor -- start 2023-09-20T21:52:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb618d8b80 1 -- start start 2023-09-20T21:52:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5679a000 0x557d56885020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 0x557d5687e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 0x557d56874ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5663f920 con 0x557d5686e050 2023-09-20T21:52:12.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d568766a0 con 0x557d5679a000 2023-09-20T21:52:12.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d56880350 con 0x557d5686d6f0 2023-09-20T21:52:12.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb477fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5679a000 0x557d56885020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb477fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5679a000 0x557d56885020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:49764/0 (socket says 172.21.15.22:49764) 2023-09-20T21:52:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb477fe700 1 -- 172.21.15.22:0/323984913 learned_addr learned my addr 172.21.15.22:0/323984913 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:52:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.637+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/323984913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 0x557d5687e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.638+0000 7fbb47fff700 1 --2- 172.21.15.22:0/323984913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 0x557d56874ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.638+0000 7fbb46ffd700 1 -- 172.21.15.22:0/323984913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5679a000 msgr2=0x557d56885020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.638+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/323984913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5679a000 0x557d56885020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.638+0000 7fbb46ffd700 1 -- 172.21.15.22:0/323984913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 msgr2=0x557d56874ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.638+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/323984913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 0x557d56874ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.638+0000 7fbb46ffd700 1 -- 172.21.15.22:0/323984913 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d56610250 con 0x557d5686d6f0 2023-09-20T21:52:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.638+0000 7fbb477fe700 1 --2- 172.21.15.22:0/323984913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5679a000 0x557d56885020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:52:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.639+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/323984913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 0x557d5687e730 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0ca340 tx=0x7fbb3c0997c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b012fcd29cdf8e25 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.639+0000 7fbb2e7fc700 1 -- 172.21.15.22:0/323984913 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0d1040 con 0x557d5686d6f0 2023-09-20T21:52:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.639+0000 7fbb2e7fc700 1 -- 172.21.15.22:0/323984913 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb3c007c60 con 0x557d5686d6f0 2023-09-20T21:52:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.639+0000 7fbb2e7fc700 1 -- 172.21.15.22:0/323984913 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c005740 con 0x557d5686d6f0 2023-09-20T21:52:12.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.640+0000 7fbb618d8b80 1 -- 172.21.15.22:0/323984913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 msgr2=0x557d5687e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:12.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.640+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/323984913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 0x557d5687e730 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0ca340 tx=0x7fbb3c0997c0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.641+0000 7fbb618d8b80 1 -- 172.21.15.22:0/323984913 shutdown_connections 2023-09-20T21:52:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.641+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/323984913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 0x557d56874ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.641+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/323984913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5679a000 0x557d56885020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.641+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/323984913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5686d6f0 0x557d5687e730 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.641+0000 7fbb618d8b80 1 -- 172.21.15.22:0/323984913 >> 172.21.15.22:0/323984913 conn(0x557d56868500 msgr2=0x557d56867880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.641+0000 7fbb618d8b80 1 -- 172.21.15.22:0/323984913 shutdown_connections 2023-09-20T21:52:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.641+0000 7fbb618d8b80 1 -- 172.21.15.22:0/323984913 wait complete. 2023-09-20T21:52:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.642+0000 7fbb618d8b80 1 -- start start 2023-09-20T21:52:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.642+0000 7fbb618d8b80 1 Processor -- start 2023-09-20T21:52:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.643+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5679a000 0x557d5687e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.643+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686d6f0 0x557d5687cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.643+0000 7fbb618d8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 0x557d5687d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.643+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5687d620 con 0x557d5686e050 2023-09-20T21:52:12.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.643+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5687d790 con 0x557d5686d6f0 2023-09-20T21:52:12.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.643+0000 7fbb618d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5687d950 con 0x557d5679a000 2023-09-20T21:52:12.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.643+0000 7fbb618d8b80 10 client.0 _ll_register_callbacks cb 0x557d566a5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:52:12.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.643+0000 7fbb477fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5679a000 0x557d5687e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.644+0000 7fbb477fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5679a000 0x557d5687e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:51834/0 (socket says 172.21.15.22:51834) 2023-09-20T21:52:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.644+0000 7fbb477fe700 1 -- 172.21.15.22:0/3443469998 learned_addr learned my addr 172.21.15.22:0/3443469998 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:52:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.644+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686d6f0 0x557d5687cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.644+0000 7fbb477fe700 1 -- 172.21.15.22:0/3443469998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686d6f0 msgr2=0x557d5687cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.644+0000 7fbb477fe700 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686d6f0 0x557d5687cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.644+0000 7fbb477fe700 1 -- 172.21.15.22:0/3443469998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 msgr2=0x557d5687d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:12.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.644+0000 7fbb47fff700 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 0x557d5687d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.644+0000 7fbb477fe700 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 0x557d5687d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.644+0000 7fbb477fe700 1 -- 172.21.15.22:0/3443469998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d565e9de0 con 0x557d5679a000 2023-09-20T21:52:12.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.644+0000 7fbb47fff700 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 0x557d5687d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:52:12.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.649+0000 7fbb477fe700 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5679a000 0x557d5687e4f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x557d5686d490 tx=0x7fbb40005860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebdceb1d3a2f9174 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:12.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.649+0000 7fbb26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:52:12.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.655+0000 7fbb26ffd700 1 -- 172.21.15.22:0/3443469998 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40024d20 con 0x557d5679a000 2023-09-20T21:52:12.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.655+0000 7fbb26ffd700 1 -- 172.21.15.22:0/3443469998 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fbb40024150 con 0x557d5679a000 2023-09-20T21:52:12.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.657+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3443469998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb4000f040 con 0x557d5679a000 2023-09-20T21:52:12.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.674+0000 7fbb26ffd700 1 -- 172.21.15.22:0/3443469998 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb40024e80 con 0x557d5679a000 2023-09-20T21:52:12.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.678+0000 7fbb618d8b80 10 client.25867 fetch_fsmap learned FSMap version 31 2023-09-20T21:52:12.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.678+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3443469998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fbb481a5620 con 0x557d5679a000 2023-09-20T21:52:12.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.681+0000 7fbb26ffd700 1 -- 172.21.15.22:0/3443469998 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb4000f040 con 0x557d5679a000 2023-09-20T21:52:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.681+0000 7fbb26ffd700 1 -- 172.21.15.22:0/3443469998 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fbb4002d440 con 0x557d5679a000 2023-09-20T21:52:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.681+0000 7fbb618d8b80 10 client.25867 fetch_fsmap finished waiting for FSMap version 31 2023-09-20T21:52:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.681+0000 7fbb618d8b80 10 client.25867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:52:12.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.681+0000 7fbb618d8b80 20 client.25867 populate_metadata read hostname 'smithi022' 2023-09-20T21:52:12.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.681+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557d5686ebe0 0x557d568da670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:12.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.681+0000 7fbb618d8b80 4 client.25867 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:52:12.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.681+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3443469998 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557d567b7e60 con 0x557d5686ebe0 2023-09-20T21:52:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.682+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557d5686ebe0 0x557d568da670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.682+0000 7fbb46ffd700 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557d5686ebe0 0x557d568da670 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f458d65697600547 server_cookie=d9238af37464341b in_seq=0 out_seq=0 2023-09-20T21:52:12.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.682+0000 7fbb26ffd700 10 client.25867 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:52:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.683+0000 7fbb26ffd700 1 -- 172.21.15.22:0/3443469998 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557d567b7e60 con 0x557d5686ebe0 2023-09-20T21:52:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.683+0000 7fbb26ffd700 10 client.25867 handle_command_reply: tid=0 2023-09-20T21:52:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.683+0000 7fbb618d8b80 1 client.25867 shutdown 2023-09-20T21:52:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.684+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3443469998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5679a000 msgr2=0x557d5687e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:12.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.684+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5679a000 0x557d5687e4f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x557d5686d490 tx=0x7fbb40005860 comp rx=0 tx=0).stop 2023-09-20T21:52:12.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.684+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3443469998 shutdown_connections 2023-09-20T21:52:12.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.684+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557d5686ebe0 0x557d568da670 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.684+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d5686e050 0x557d5687d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.684+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d5679a000 0x557d5687e4f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.684+0000 7fbb618d8b80 1 --2- 172.21.15.22:0/3443469998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557d5686d6f0 0x557d5687cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.684+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3443469998 >> 172.21.15.22:0/3443469998 conn(0x557d56868500 msgr2=0x557d56885020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:12.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.684+0000 7fbb26ffd700 0 client.25867 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:52:12.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.686+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3443469998 shutdown_connections 2023-09-20T21:52:12.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.688+0000 7fbb618d8b80 1 -- 172.21.15.22:0/3443469998 wait complete. 2023-09-20T21:52:12.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.689+0000 7fbb618d8b80 20 client.25867 trim_cache size 0 max 16384 2023-09-20T21:52:12.690 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:52:12.690 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:52:12.690 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:52:12.690 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:52:12.690 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:52:12.691 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:52:12.691 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:52:12.691 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:52:12.691 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:52:12.691 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:52:12.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.690+0000 7fbb4dfff700 1 -- 172.21.15.22:0/2448635479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb300a13d0 msgr2=0x7fbb300a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:12.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.690+0000 7fbb4dfff700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb300a13d0 0x7fbb300a3890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbb38004820 tx=0x7fbb38005fd0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.690+0000 7fbb4dfff700 1 -- 172.21.15.22:0/2448635479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4810a750 msgr2=0x7fbb4819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.690+0000 7fbb4dfff700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4810a750 0x7fbb4819d570 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c920 tx=0x7fbb3c00cce0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.691+0000 7fbb4dfff700 1 -- 172.21.15.22:0/2448635479 shutdown_connections 2023-09-20T21:52:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.691+0000 7fbb4dfff700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb4810d120 0x7fbb481a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.691+0000 7fbb4dfff700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb48107de0 0x7fbb4819d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.691+0000 7fbb4dfff700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbb300a13d0 0x7fbb300a3890 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.691+0000 7fbb4dfff700 1 --2- 172.21.15.22:0/2448635479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb4810a750 0x7fbb4819d570 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:12.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.691+0000 7fbb4dfff700 1 -- 172.21.15.22:0/2448635479 >> 172.21.15.22:0/2448635479 conn(0x7fbb4806db90 msgr2=0x7fbb4806ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.691+0000 7fbb4dfff700 1 -- 172.21.15.22:0/2448635479 shutdown_connections 2023-09-20T21:52:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:12.691+0000 7fbb4dfff700 1 -- 172.21.15.22:0/2448635479 wait complete. 2023-09-20T21:52:12.732 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:52:12.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:12 smithi123 ceph-mon[134433]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 MiB/s wr, 724 op/s 2023-09-20T21:52:12.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:12 smithi089 ceph-mon[134090]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 MiB/s wr, 724 op/s 2023-09-20T21:52:13.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:12 smithi022 ceph-mon[97651]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 9.5 MiB/s wr, 724 op/s 2023-09-20T21:52:13.570 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 170.0 s, 799.1 tps, lat 127.872 ms stddev 148.690 2023-09-20T21:52:13.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:13 smithi022 ceph-mon[97651]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.0 MiB/s wr, 550 op/s 2023-09-20T21:52:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:13 smithi123 ceph-mon[134433]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.0 MiB/s wr, 550 op/s 2023-09-20T21:52:14.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:13 smithi089 ceph-mon[134090]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.0 MiB/s wr, 550 op/s 2023-09-20T21:52:16.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:16 smithi022 ceph-mon[97651]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2023-09-20T21:52:16.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:16 smithi123 ceph-mon[134433]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2023-09-20T21:52:16.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:16 smithi089 ceph-mon[134090]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 606 op/s 2023-09-20T21:52:18.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 175.0 s, 1105.6 tps, lat 90.793 ms stddev 102.239 2023-09-20T21:52:18.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:18 smithi022 ceph-mon[97651]: pgmap v448: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.4 MiB/s wr, 690 op/s 2023-09-20T21:52:18.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:18 smithi123 ceph-mon[134433]: pgmap v448: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.4 MiB/s wr, 690 op/s 2023-09-20T21:52:18.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:18 smithi089 ceph-mon[134090]: pgmap v448: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.4 MiB/s wr, 690 op/s 2023-09-20T21:52:20.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:20 smithi022 ceph-mon[97651]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 615 op/s 2023-09-20T21:52:20.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:20 smithi123 ceph-mon[134433]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 615 op/s 2023-09-20T21:52:20.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:20 smithi089 ceph-mon[134090]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 615 op/s 2023-09-20T21:52:21.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:21 smithi022 ceph-mon[97651]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:52:21.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:21 smithi123 ceph-mon[134433]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:52:21.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:21 smithi089 ceph-mon[134090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T21:52:22.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:22 smithi022 ceph-mon[97651]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 766 op/s 2023-09-20T21:52:22.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:22 smithi123 ceph-mon[134433]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 766 op/s 2023-09-20T21:52:22.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:22 smithi089 ceph-mon[134090]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.1 MiB/s wr, 766 op/s 2023-09-20T21:52:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 180.0 s, 972.4 tps, lat 102.672 ms stddev 118.004 2023-09-20T21:52:23.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:23 smithi022 ceph-mon[97651]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.3 MiB/s wr, 646 op/s 2023-09-20T21:52:24.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:23 smithi123 ceph-mon[134433]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.3 MiB/s wr, 646 op/s 2023-09-20T21:52:24.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:23 smithi089 ceph-mon[134090]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.3 MiB/s wr, 646 op/s 2023-09-20T21:52:26.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:26 smithi022 ceph-mon[97651]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 695 op/s 2023-09-20T21:52:26.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:26 smithi123 ceph-mon[134433]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 695 op/s 2023-09-20T21:52:26.866 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:26 smithi089 ceph-mon[134090]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 695 op/s 2023-09-20T21:52:28.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 185.0 s, 721.6 tps, lat 136.007 ms stddev 133.697 2023-09-20T21:52:28.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:28 smithi022 ceph-mon[97651]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 727 op/s 2023-09-20T21:52:28.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:28 smithi123 ceph-mon[134433]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 727 op/s 2023-09-20T21:52:28.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:28 smithi089 ceph-mon[134090]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 MiB/s wr, 727 op/s 2023-09-20T21:52:30.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:30 smithi022 ceph-mon[97651]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.5 MiB/s wr, 605 op/s 2023-09-20T21:52:30.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:30 smithi123 ceph-mon[134433]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.5 MiB/s wr, 605 op/s 2023-09-20T21:52:30.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:30 smithi089 ceph-mon[134090]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.5 MiB/s wr, 605 op/s 2023-09-20T21:52:32.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:32 smithi022 ceph-mon[97651]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 771 op/s 2023-09-20T21:52:32.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:32 smithi123 ceph-mon[134433]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 771 op/s 2023-09-20T21:52:32.865 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:32 smithi089 ceph-mon[134090]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 771 op/s 2023-09-20T21:52:33.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 190.0 s, 961.2 tps, lat 105.086 ms stddev 120.319 2023-09-20T21:52:34.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:33 smithi123 ceph-mon[134433]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.2 MiB/s wr, 642 op/s 2023-09-20T21:52:34.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:33 smithi089 ceph-mon[134090]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.2 MiB/s wr, 642 op/s 2023-09-20T21:52:34.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:33 smithi022 ceph-mon[97651]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.2 MiB/s wr, 642 op/s 2023-09-20T21:52:36.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:36 smithi089 ceph-mon[134090]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.9 MiB/s wr, 694 op/s 2023-09-20T21:52:36.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:36 smithi022 ceph-mon[97651]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.9 MiB/s wr, 694 op/s 2023-09-20T21:52:36.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:36 smithi123 ceph-mon[134433]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.9 MiB/s wr, 694 op/s 2023-09-20T21:52:38.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 195.0 s, 838.2 tps, lat 119.507 ms stddev 153.197 2023-09-20T21:52:38.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:38 smithi022 ceph-mon[97651]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 721 op/s 2023-09-20T21:52:38.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:38 smithi123 ceph-mon[134433]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 721 op/s 2023-09-20T21:52:38.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:38 smithi089 ceph-mon[134090]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 721 op/s 2023-09-20T21:52:40.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:40 smithi022 ceph-mon[97651]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 MiB/s wr, 653 op/s 2023-09-20T21:52:40.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:40 smithi123 ceph-mon[134433]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 MiB/s wr, 653 op/s 2023-09-20T21:52:40.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:40 smithi089 ceph-mon[134090]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 MiB/s wr, 653 op/s 2023-09-20T21:52:42.734 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:52:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:42 smithi022 ceph-mon[97651]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 13 MiB/s wr, 776 op/s 2023-09-20T21:52:42.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:42 smithi123 ceph-mon[134433]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 13 MiB/s wr, 776 op/s 2023-09-20T21:52:42.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:42 smithi089 ceph-mon[134090]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 13 MiB/s wr, 776 op/s 2023-09-20T21:52:43.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.200+0000 7f0f741ff700 1 -- 172.21.15.22:0/1094367370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c10d120 msgr2=0x7f0f6c10d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:43.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.200+0000 7f0f741ff700 1 --2- 172.21.15.22:0/1094367370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c10d120 0x7f0f6c10d5a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400b0a0 tx=0x7f0f6400b3b0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.204+0000 7f0f741ff700 1 -- 172.21.15.22:0/1094367370 shutdown_connections 2023-09-20T21:52:43.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.204+0000 7f0f741ff700 1 --2- 172.21.15.22:0/1094367370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c10d120 0x7f0f6c10d5a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.204+0000 7f0f741ff700 1 --2- 172.21.15.22:0/1094367370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f6c107de0 0x7f0f6c10a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.204+0000 7f0f741ff700 1 --2- 172.21.15.22:0/1094367370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6c10a750 0x7f0f6c10cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.204+0000 7f0f741ff700 1 -- 172.21.15.22:0/1094367370 >> 172.21.15.22:0/1094367370 conn(0x7f0f6c06db90 msgr2=0x7f0f6c06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:43.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.210+0000 7f0f741ff700 1 -- 172.21.15.22:0/1094367370 shutdown_connections 2023-09-20T21:52:43.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.211+0000 7f0f741ff700 1 -- 172.21.15.22:0/1094367370 wait complete. 2023-09-20T21:52:43.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.212+0000 7f0f741ff700 1 Processor -- start 2023-09-20T21:52:43.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.213+0000 7f0f741ff700 1 -- start start 2023-09-20T21:52:43.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.213+0000 7f0f741ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6c107de0 0x7f0f6c1a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.213+0000 7f0f741ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c10a750 0x7f0f6c1a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.213+0000 7f0f741ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f6c10d120 0x7f0f6c1acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.213+0000 7f0f741ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1ad540 con 0x7f0f6c10a750 2023-09-20T21:52:43.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.213+0000 7f0f741ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1ad680 con 0x7f0f6c107de0 2023-09-20T21:52:43.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.213+0000 7f0f741ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1ad810 con 0x7f0f6c10d120 2023-09-20T21:52:43.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.214+0000 7f0f7279c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f6c10d120 0x7f0f6c1acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.214+0000 7f0f7279c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f6c10d120 0x7f0f6c1acf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:46504/0 (socket says 172.21.15.22:46504) 2023-09-20T21:52:43.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.214+0000 7f0f7279c700 1 -- 172.21.15.22:0/3790491286 learned_addr learned my addr 172.21.15.22:0/3790491286 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:52:43.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.215+0000 7f0f7179a700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c10a750 0x7f0f6c1a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.215+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6c107de0 0x7f0f6c1a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.215+0000 7f0f7279c700 1 -- 172.21.15.22:0/3790491286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6c107de0 msgr2=0x7f0f6c1a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.215+0000 7f0f7279c700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6c107de0 0x7f0f6c1a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.215+0000 7f0f7279c700 1 -- 172.21.15.22:0/3790491286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c10a750 msgr2=0x7f0f6c1a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.215+0000 7f0f7279c700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c10a750 0x7f0f6c1a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.215+0000 7f0f7279c700 1 -- 172.21.15.22:0/3790491286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f64009d00 con 0x7f0f6c10d120 2023-09-20T21:52:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.216+0000 7f0f7279c700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f6c10d120 0x7f0f6c1acf70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400b780 tx=0x7f0f64009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:43.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.217+0000 7f0f6affd700 1 -- 172.21.15.22:0/3790491286 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400e040 con 0x7f0f6c10d120 2023-09-20T21:52:43.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.217+0000 7f0f6affd700 1 -- 172.21.15.22:0/3790491286 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f64003e10 con 0x7f0f6c10d120 2023-09-20T21:52:43.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.217+0000 7f0f6affd700 1 -- 172.21.15.22:0/3790491286 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6403d6d0 con 0x7f0f6c10d120 2023-09-20T21:52:43.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.217+0000 7f0f741ff700 1 -- 172.21.15.22:0/3790491286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c073010 con 0x7f0f6c10d120 2023-09-20T21:52:43.221 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.217+0000 7f0f741ff700 1 -- 172.21.15.22:0/3790491286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c0734d0 con 0x7f0f6c10d120 2023-09-20T21:52:43.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.226+0000 7f0f87a8ab80 1 Processor -- start 2023-09-20T21:52:43.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.228+0000 7f0f6affd700 1 -- 172.21.15.22:0/3790491286 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f0f6403f480 con 0x7f0f6c10d120 2023-09-20T21:52:43.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.228+0000 7f0f87a8ab80 1 -- start start 2023-09-20T21:52:43.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.229+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955d6f0 0x55e34955db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.229+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955e050 0x55e34955e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.229+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955ebe0 0x55e34955f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.229+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3494d1ca0 con 0x55e34955ebe0 2023-09-20T21:52:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.229+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3494b5c70 con 0x55e34955d6f0 2023-09-20T21:52:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.229+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3494b3590 con 0x55e34955e050 2023-09-20T21:52:43.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.229+0000 7f0f71f9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955e050 0x55e34955e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.229+0000 7f0f71f9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955e050 0x55e34955e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:46512/0 (socket says 172.21.15.22:46512) 2023-09-20T21:52:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.229+0000 7f0f7179a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955d6f0 0x55e34955db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.230+0000 7f0f71f9b700 1 -- 172.21.15.22:0/3625063907 learned_addr learned my addr 172.21.15.22:0/3625063907 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:52:43.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.230+0000 7f0f71f9b700 1 -- 172.21.15.22:0/3625063907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955d6f0 msgr2=0x55e34955db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.230+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/3625063907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955d6f0 0x55e34955db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.230+0000 7f0f71f9b700 1 -- 172.21.15.22:0/3625063907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955ebe0 msgr2=0x55e34955f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.230+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/3625063907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955ebe0 0x55e34955f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.230+0000 7f0f71f9b700 1 -- 172.21.15.22:0/3625063907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3492d9de0 con 0x55e34955e050 2023-09-20T21:52:43.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.231+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/3625063907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955e050 0x55e34955e5e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009d10 tx=0x7f0f5c0116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9544926abcc17896 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.231+0000 7f0f63fff700 1 -- 172.21.15.22:0/3625063907 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0097e0 con 0x55e34955e050 2023-09-20T21:52:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.231+0000 7f0f6affd700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f580a1500 0x7f0f580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.231+0000 7f0f63fff700 1 -- 172.21.15.22:0/3625063907 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f5c011ed0 con 0x55e34955e050 2023-09-20T21:52:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.231+0000 7f0f63fff700 1 -- 172.21.15.22:0/3625063907 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c00f8e0 con 0x55e34955e050 2023-09-20T21:52:43.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.232+0000 7f0f6affd700 1 -- 172.21.15.22:0/3790491286 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f0f640f00a0 con 0x7f0f6c10d120 2023-09-20T21:52:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.232+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/3625063907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955e050 msgr2=0x55e34955e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.232+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/3625063907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955e050 0x55e34955e5e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009d10 tx=0x7f0f5c0116c0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.233+0000 7f0f7179a700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f580a1500 0x7f0f580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.233+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/3625063907 shutdown_connections 2023-09-20T21:52:43.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.233+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/3625063907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955ebe0 0x55e34955f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.233+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/3625063907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955d6f0 0x55e34955db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.233+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/3625063907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955e050 0x55e34955e5e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.233+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/3625063907 >> 172.21.15.22:0/3625063907 conn(0x55e34948a000 msgr2=0x55e3495581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.233+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/3625063907 shutdown_connections 2023-09-20T21:52:43.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.234+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/3625063907 wait complete. 2023-09-20T21:52:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.235+0000 7f0f87a8ab80 1 -- start start 2023-09-20T21:52:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.235+0000 7f0f87a8ab80 1 Processor -- start 2023-09-20T21:52:43.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.235+0000 7f0f7179a700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f580a1500 0x7f0f580a39c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55e3495590b0 tx=0x7f0f54004750 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.236+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955d6f0 0x55e3495671f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.236+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955e050 0x55e349567730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.236+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 0x55e34956e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.236+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3494aee80 con 0x55e34955d6f0 2023-09-20T21:52:43.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.236+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34944b740 con 0x55e34955e050 2023-09-20T21:52:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.236+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34944b3e0 con 0x55e34955ebe0 2023-09-20T21:52:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.236+0000 7f0f87a8ab80 10 client.0 _ll_register_callbacks cb 0x55e349395a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:52:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f7179a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955e050 0x55e349567730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f7179a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955e050 0x55e349567730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:52046/0 (socket says 172.21.15.22:52046) 2023-09-20T21:52:43.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f7179a700 1 -- 172.21.15.22:0/2490018038 learned_addr learned my addr 172.21.15.22:0/2490018038 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:52:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f7279c700 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 0x55e34956e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955d6f0 0x55e3495671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f7179a700 1 -- 172.21.15.22:0/2490018038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 msgr2=0x55e34956e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f7179a700 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 0x55e34956e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f7179a700 1 -- 172.21.15.22:0/2490018038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955d6f0 msgr2=0x55e3495671f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f7179a700 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955d6f0 0x55e3495671f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f7179a700 1 -- 172.21.15.22:0/2490018038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c009940 con 0x55e34955e050 2023-09-20T21:52:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955d6f0 0x55e3495671f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:52:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f7179a700 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955e050 0x55e349567730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0f54004120 tx=0x7f0f54009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=216dbcabd144f1a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:43.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.237+0000 7f0f617fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:52:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.238+0000 7f0f617fa700 1 -- 172.21.15.22:0/2490018038 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54016070 con 0x55e34955e050 2023-09-20T21:52:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.238+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/2490018038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e34941fbb0 con 0x55e34955e050 2023-09-20T21:52:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.238+0000 7f0f617fa700 1 -- 172.21.15.22:0/2490018038 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f0f54003b10 con 0x55e34955e050 2023-09-20T21:52:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.238+0000 7f0f617fa700 1 -- 172.21.15.22:0/2490018038 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f54003cd0 con 0x55e34955e050 2023-09-20T21:52:43.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.238+0000 7f0f87a8ab80 10 client.25883 fetch_fsmap learned FSMap version 32 2023-09-20T21:52:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.238+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/2490018038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e3492c2c60 con 0x55e34955e050 2023-09-20T21:52:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.239+0000 7f0f617fa700 1 -- 172.21.15.22:0/2490018038 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e34941fbb0 con 0x55e34955e050 2023-09-20T21:52:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.239+0000 7f0f617fa700 1 -- 172.21.15.22:0/2490018038 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0f54029520 con 0x55e34955e050 2023-09-20T21:52:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.239+0000 7f0f87a8ab80 10 client.25883 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:52:43.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.239+0000 7f0f87a8ab80 10 client.25883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:52:43.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.239+0000 7f0f87a8ab80 20 client.25883 populate_metadata read hostname 'smithi022' 2023-09-20T21:52:43.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.240+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e3495730a0 0x55e3495754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.240+0000 7f0f87a8ab80 4 client.25883 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:52:43.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.240+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/2490018038 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e34952c020 con 0x55e3495730a0 2023-09-20T21:52:43.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.240+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e3495730a0 0x55e3495754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.241+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e3495730a0 0x55e3495754e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7497d117641daf0 server_cookie=1c3580e8086e59a6 in_seq=0 out_seq=0 2023-09-20T21:52:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.241+0000 7f0f617fa700 10 client.25883 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:52:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.242+0000 7f0f617fa700 1 -- 172.21.15.22:0/2490018038 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e34952c020 con 0x55e3495730a0 2023-09-20T21:52:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.242+0000 7f0f617fa700 10 client.25883 handle_command_reply: tid=0 2023-09-20T21:52:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.242+0000 7f0f87a8ab80 1 client.25883 shutdown 2023-09-20T21:52:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.243+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/2490018038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955e050 msgr2=0x55e349567730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.243+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955e050 0x55e349567730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0f54004120 tx=0x7f0f54009260 comp rx=0 tx=0).stop 2023-09-20T21:52:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.244+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/2490018038 shutdown_connections 2023-09-20T21:52:43.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.244+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e3495730a0 0x55e3495754e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.244+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 0x55e34956e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.244+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955d6f0 0x55e3495671f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.244+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/2490018038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955e050 0x55e349567730 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.244+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/2490018038 >> 172.21.15.22:0/2490018038 conn(0x55e34948a000 msgr2=0x55e349557880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:43.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.244+0000 7f0f617fa700 0 client.25883 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:52:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.244+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/2490018038 shutdown_connections 2023-09-20T21:52:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.245+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/2490018038 wait complete. 2023-09-20T21:52:43.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.246+0000 7f0f87a8ab80 20 client.25883 trim_cache size 0 max 16384 2023-09-20T21:52:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.277+0000 7f0f87a8ab80 1 Processor -- start 2023-09-20T21:52:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.277+0000 7f0f87a8ab80 1 -- start start 2023-09-20T21:52:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.278+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955d6f0 0x55e34955efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.278+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 0x55e34956e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.278+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955ebe0 0x55e3495750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.278+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34932f920 con 0x55e34955e050 2023-09-20T21:52:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.278+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e349566d90 con 0x55e34955ebe0 2023-09-20T21:52:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.278+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34955ea90 con 0x55e34955d6f0 2023-09-20T21:52:43.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.278+0000 7f0f71f9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955d6f0 0x55e34955efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.279+0000 7f0f71f9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955d6f0 0x55e34955efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:46534/0 (socket says 172.21.15.22:46534) 2023-09-20T21:52:43.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.279+0000 7f0f71f9b700 1 -- 172.21.15.22:0/1733385717 learned_addr learned my addr 172.21.15.22:0/1733385717 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:52:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.279+0000 7f0f7179a700 1 --2- 172.21.15.22:0/1733385717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 0x55e34956e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.280+0000 7f0f71f9b700 1 -- 172.21.15.22:0/1733385717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955ebe0 msgr2=0x55e3495750b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:52:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.280+0000 7f0f7279c700 1 --2- 172.21.15.22:0/1733385717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955ebe0 0x55e3495750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.280+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/1733385717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955ebe0 0x55e3495750b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.280+0000 7f0f71f9b700 1 -- 172.21.15.22:0/1733385717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 msgr2=0x55e34956e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.280+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/1733385717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 0x55e34956e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.280+0000 7f0f71f9b700 1 -- 172.21.15.22:0/1733385717 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e349300250 con 0x55e34955d6f0 2023-09-20T21:52:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.280+0000 7f0f7279c700 1 --2- 172.21.15.22:0/1733385717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955ebe0 0x55e3495750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:52:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.281+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/1733385717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955d6f0 0x55e34955efc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c01c040 tx=0x7f0f5c005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1dc618b2fa7c1b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:43.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.281+0000 7f0f68ff9700 1 -- 172.21.15.22:0/1733385717 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c028b50 con 0x55e34955d6f0 2023-09-20T21:52:43.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.282+0000 7f0f68ff9700 1 -- 172.21.15.22:0/1733385717 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f5c0285b0 con 0x55e34955d6f0 2023-09-20T21:52:43.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.282+0000 7f0f68ff9700 1 -- 172.21.15.22:0/1733385717 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c003680 con 0x55e34955d6f0 2023-09-20T21:52:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.282+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/1733385717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955d6f0 msgr2=0x55e34955efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.282+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/1733385717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955d6f0 0x55e34955efc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c01c040 tx=0x7f0f5c005960 comp rx=0 tx=0).stop 2023-09-20T21:52:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.283+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/1733385717 shutdown_connections 2023-09-20T21:52:43.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.283+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/1733385717 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955ebe0 0x55e3495750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.283+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/1733385717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955d6f0 0x55e34955efc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.283+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/1733385717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 0x55e34956e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.283+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/1733385717 >> 172.21.15.22:0/1733385717 conn(0x55e34948a000 msgr2=0x55e349565030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:43.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.283+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/1733385717 shutdown_connections 2023-09-20T21:52:43.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.283+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/1733385717 wait complete. 2023-09-20T21:52:43.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.283+0000 7f0f87a8ab80 1 -- start start 2023-09-20T21:52:43.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.284+0000 7f0f87a8ab80 1 Processor -- start 2023-09-20T21:52:43.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.285+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955d6f0 0x55e34956e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.285+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 0x55e34956e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.285+0000 7f0f87a8ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 0x55e34956c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.285+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34956cf30 con 0x55e34955e050 2023-09-20T21:52:43.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.285+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34956d0a0 con 0x55e34955d6f0 2023-09-20T21:52:43.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.285+0000 7f0f87a8ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34956d260 con 0x55e34955ebe0 2023-09-20T21:52:43.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.285+0000 7f0f87a8ab80 10 client.0 _ll_register_callbacks cb 0x55e349395a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:52:43.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.286+0000 7f0f7179a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 0x55e34956e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.286+0000 7f0f7179a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 0x55e34956e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44032/0 (socket says 172.21.15.22:44032) 2023-09-20T21:52:43.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.286+0000 7f0f7179a700 1 -- 172.21.15.22:0/4112941682 learned_addr learned my addr 172.21.15.22:0/4112941682 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:52:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.288+0000 7f0f7279c700 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 0x55e34956c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.289+0000 7f0f7279c700 1 -- 172.21.15.22:0/4112941682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955d6f0 msgr2=0x55e34956e1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:52:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.289+0000 7f0f7279c700 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955d6f0 0x55e34956e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.289+0000 7f0f7279c700 1 -- 172.21.15.22:0/4112941682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 msgr2=0x55e34956e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:52:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.289+0000 7f0f7279c700 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 0x55e34956e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.289+0000 7f0f7279c700 1 -- 172.21.15.22:0/4112941682 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3492d9de0 con 0x55e34955ebe0 2023-09-20T21:52:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.289+0000 7f0f7279c700 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 0x55e34956c820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f640bce60 tx=0x7f0f6400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2378184cb54f162 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:52:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.289+0000 7f0f617fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:52:43.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.289+0000 7f0f617fa700 1 -- 172.21.15.22:0/4112941682 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f640bd860 con 0x55e34955ebe0 2023-09-20T21:52:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.289+0000 7f0f617fa700 1 -- 172.21.15.22:0/4112941682 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f0f640bd160 con 0x55e34955ebe0 2023-09-20T21:52:43.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.290+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/4112941682 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3494a7e60 con 0x55e34955ebe0 2023-09-20T21:52:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.290+0000 7f0f617fa700 1 -- 172.21.15.22:0/4112941682 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f640c0f90 con 0x55e34955ebe0 2023-09-20T21:52:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.290+0000 7f0f87a8ab80 10 client.25885 fetch_fsmap learned FSMap version 32 2023-09-20T21:52:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.290+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/4112941682 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e3492d82b0 con 0x55e34955ebe0 2023-09-20T21:52:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.291+0000 7f0f617fa700 1 -- 172.21.15.22:0/4112941682 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3494a7e60 con 0x55e34955ebe0 2023-09-20T21:52:43.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.291+0000 7f0f617fa700 1 -- 172.21.15.22:0/4112941682 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0f640d5220 con 0x55e34955ebe0 2023-09-20T21:52:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.291+0000 7f0f87a8ab80 10 client.25885 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:52:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.291+0000 7f0f87a8ab80 10 client.25885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:52:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.291+0000 7f0f87a8ab80 20 client.25885 populate_metadata read hostname 'smithi022' 2023-09-20T21:52:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.291+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e3495c7fa0 0x55e3495ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:52:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.291+0000 7f0f87a8ab80 4 client.25885 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:52:43.293 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.291+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/4112941682 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e3494a7b80 con 0x55e3495c7fa0 2023-09-20T21:52:43.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.292+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e3495c7fa0 0x55e3495ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:52:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.293+0000 7f0f617fa700 10 client.25885 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:52:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.294+0000 7f0f71f9b700 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e3495c7fa0 0x55e3495ca3c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a24c0f85d605465 server_cookie=fd98af866ffa73e in_seq=0 out_seq=0 2023-09-20T21:52:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.294+0000 7f0f617fa700 1 -- 172.21.15.22:0/4112941682 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e3494a7b80 con 0x55e3495c7fa0 2023-09-20T21:52:43.295 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.294+0000 7f0f617fa700 10 client.25885 handle_command_reply: tid=0 2023-09-20T21:52:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.294+0000 7f0f87a8ab80 1 client.25885 shutdown 2023-09-20T21:52:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.294+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/4112941682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 msgr2=0x55e34956c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.294+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 0x55e34956c820 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f640bce60 tx=0x7f0f6400b3b0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.295+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/4112941682 shutdown_connections 2023-09-20T21:52:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.295+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e3495c7fa0 0x55e3495ca3c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.295+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e34955ebe0 0x55e34956c820 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.295+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e34955d6f0 0x55e34956e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.295+0000 7f0f87a8ab80 1 --2- 172.21.15.22:0/4112941682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e34955e050 0x55e34956e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.295+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/4112941682 >> 172.21.15.22:0/4112941682 conn(0x55e34948a000 msgr2=0x55e3495750b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:43.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.295+0000 7f0f617fa700 0 client.25885 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:52:43.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.296+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/4112941682 shutdown_connections 2023-09-20T21:52:43.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.297+0000 7f0f87a8ab80 1 -- 172.21.15.22:0/4112941682 wait complete. 2023-09-20T21:52:43.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.298+0000 7f0f87a8ab80 20 client.25885 trim_cache size 0 max 16384 2023-09-20T21:52:43.299 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:52:43.299 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:52:43.299 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:52:43.299 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:52:43.299 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:52:43.300 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:52:43.300 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:52:43.300 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:52:43.300 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:52:43.300 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:52:43.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.299+0000 7f0f741ff700 1 -- 172.21.15.22:0/3790491286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f580a1500 msgr2=0x7f0f580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:43.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.299+0000 7f0f741ff700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f580a1500 0x7f0f580a39c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55e3495590b0 tx=0x7f0f54004750 comp rx=0 tx=0).stop 2023-09-20T21:52:43.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.299+0000 7f0f741ff700 1 -- 172.21.15.22:0/3790491286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f6c10d120 msgr2=0x7f0f6c1acf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:52:43.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.299+0000 7f0f741ff700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f6c10d120 0x7f0f6c1acf70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400b780 tx=0x7f0f64009560 comp rx=0 tx=0).stop 2023-09-20T21:52:43.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.299+0000 7f0f741ff700 1 -- 172.21.15.22:0/3790491286 shutdown_connections 2023-09-20T21:52:43.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.299+0000 7f0f741ff700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f6c10d120 0x7f0f6c1acf70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.299+0000 7f0f741ff700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0f580a1500 0x7f0f580a39c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.299+0000 7f0f741ff700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f6c107de0 0x7f0f6c1a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.299+0000 7f0f741ff700 1 --2- 172.21.15.22:0/3790491286 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f6c10a750 0x7f0f6c1a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:52:43.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.300+0000 7f0f741ff700 1 -- 172.21.15.22:0/3790491286 >> 172.21.15.22:0/3790491286 conn(0x7f0f6c06db90 msgr2=0x7f0f6c10b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:52:43.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.300+0000 7f0f741ff700 1 -- 172.21.15.22:0/3790491286 shutdown_connections 2023-09-20T21:52:43.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:52:43.300+0000 7f0f741ff700 1 -- 172.21.15.22:0/3790491286 wait complete. 2023-09-20T21:52:43.347 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:52:43.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 200.0 s, 870.3 tps, lat 115.508 ms stddev 124.100 2023-09-20T21:52:44.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:43 smithi123 ceph-mon[134433]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.7 MiB/s wr, 605 op/s 2023-09-20T21:52:44.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:43 smithi089 ceph-mon[134090]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.7 MiB/s wr, 605 op/s 2023-09-20T21:52:44.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:43 smithi022 ceph-mon[97651]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9.7 MiB/s wr, 605 op/s 2023-09-20T21:52:46.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:46 smithi022 ceph-mon[97651]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 11 MiB/s wr, 641 op/s 2023-09-20T21:52:46.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:46 smithi123 ceph-mon[134433]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 11 MiB/s wr, 641 op/s 2023-09-20T21:52:46.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:46 smithi089 ceph-mon[134090]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 11 MiB/s wr, 641 op/s 2023-09-20T21:52:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 205.0 s, 933.7 tps, lat 107.661 ms stddev 122.631 2023-09-20T21:52:48.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:48 smithi022 ceph-mon[97651]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 640 op/s 2023-09-20T21:52:48.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:48 smithi123 ceph-mon[134433]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 640 op/s 2023-09-20T21:52:48.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:48 smithi089 ceph-mon[134090]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 640 op/s 2023-09-20T21:52:50.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:50 smithi022 ceph-mon[97651]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 MiB/s wr, 592 op/s 2023-09-20T21:52:50.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:50 smithi123 ceph-mon[134433]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 MiB/s wr, 592 op/s 2023-09-20T21:52:50.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:50 smithi089 ceph-mon[134090]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 MiB/s wr, 592 op/s 2023-09-20T21:52:52.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:52 smithi022 ceph-mon[97651]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 703 op/s 2023-09-20T21:52:52.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:52 smithi123 ceph-mon[134433]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 703 op/s 2023-09-20T21:52:52.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:52 smithi089 ceph-mon[134090]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 703 op/s 2023-09-20T21:52:53.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 210.0 s, 1015.0 tps, lat 98.049 ms stddev 108.508 2023-09-20T21:52:54.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:53 smithi123 ceph-mon[134433]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.8 MiB/s wr, 588 op/s 2023-09-20T21:52:54.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:53 smithi089 ceph-mon[134090]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.8 MiB/s wr, 588 op/s 2023-09-20T21:52:54.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:53 smithi022 ceph-mon[97651]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.8 MiB/s wr, 588 op/s 2023-09-20T21:52:56.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:56 smithi022 ceph-mon[97651]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 660 op/s 2023-09-20T21:52:56.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:56 smithi123 ceph-mon[134433]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 660 op/s 2023-09-20T21:52:56.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:56 smithi089 ceph-mon[134090]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 MiB/s wr, 660 op/s 2023-09-20T21:52:58.574 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 215.0 s, 801.4 tps, lat 123.510 ms stddev 134.365 2023-09-20T21:52:58.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:52:58 smithi022 ceph-mon[97651]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 686 op/s 2023-09-20T21:52:58.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:52:58 smithi123 ceph-mon[134433]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 686 op/s 2023-09-20T21:52:58.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:52:58 smithi089 ceph-mon[134090]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 686 op/s 2023-09-20T21:53:00.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:00 smithi022 ceph-mon[97651]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 MiB/s wr, 654 op/s 2023-09-20T21:53:00.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:00 smithi123 ceph-mon[134433]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 MiB/s wr, 654 op/s 2023-09-20T21:53:00.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:00 smithi089 ceph-mon[134090]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 MiB/s wr, 654 op/s 2023-09-20T21:53:02.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:02 smithi022 ceph-mon[97651]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 16 MiB/s wr, 1.06k op/s 2023-09-20T21:53:02.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:02 smithi123 ceph-mon[134433]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 16 MiB/s wr, 1.06k op/s 2023-09-20T21:53:02.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:02 smithi089 ceph-mon[134090]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 16 MiB/s wr, 1.06k op/s 2023-09-20T21:53:03.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 220.0 s, 551.7 tps, lat 184.132 ms stddev 390.162 2023-09-20T21:53:04.101 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:03 smithi123 ceph-mon[134433]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-09-20T21:53:04.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:03 smithi089 ceph-mon[134090]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-09-20T21:53:04.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:03 smithi022 ceph-mon[97651]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-09-20T21:53:06.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:06 smithi022 ceph-mon[97651]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-09-20T21:53:06.842 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:06 smithi123 ceph-mon[134433]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-09-20T21:53:06.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:06 smithi089 ceph-mon[134090]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 17 MiB/s wr, 1.33k op/s 2023-09-20T21:53:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 225.0 s, 967.2 tps, lat 99.403 ms stddev 128.829 2023-09-20T21:53:08.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:08 smithi022 ceph-mon[97651]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 1.45k op/s 2023-09-20T21:53:08.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:08 smithi123 ceph-mon[134433]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 1.45k op/s 2023-09-20T21:53:08.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:08 smithi089 ceph-mon[134090]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 1.45k op/s 2023-09-20T21:53:10.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:10 smithi123 ceph-mon[134433]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2023-09-20T21:53:10.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:10 smithi089 ceph-mon[134090]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2023-09-20T21:53:10.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:10 smithi022 ceph-mon[97651]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2023-09-20T21:53:12.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:12 smithi123 ceph-mon[134433]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2023-09-20T21:53:12.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:12 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:53:12.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:12 smithi089 ceph-mon[134090]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2023-09-20T21:53:12.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:12 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:53:12.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:12 smithi022 ceph-mon[97651]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2023-09-20T21:53:12.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:12 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:53:13.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:53:13.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 230.0 s, 874.8 tps, lat 115.648 ms stddev 144.563 2023-09-20T21:53:13.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.778+0000 7fa988bff700 1 -- 172.21.15.22:0/2657810235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa984107de0 msgr2=0x7fa98410a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:13.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.778+0000 7fa988bff700 1 --2- 172.21.15.22:0/2657810235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa984107de0 0x7fa98410a210 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b0a0 tx=0x7fa97c00b3b0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.782+0000 7fa988bff700 1 -- 172.21.15.22:0/2657810235 shutdown_connections 2023-09-20T21:53:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.782+0000 7fa988bff700 1 --2- 172.21.15.22:0/2657810235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98410d120 0x7fa98410d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.782+0000 7fa988bff700 1 --2- 172.21.15.22:0/2657810235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa984107de0 0x7fa98410a210 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.782+0000 7fa988bff700 1 --2- 172.21.15.22:0/2657810235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa98410a750 0x7fa98410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.782+0000 7fa988bff700 1 -- 172.21.15.22:0/2657810235 >> 172.21.15.22:0/2657810235 conn(0x7fa98406db90 msgr2=0x7fa98406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:13.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.790+0000 7fa988bff700 1 -- 172.21.15.22:0/2657810235 shutdown_connections 2023-09-20T21:53:13.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.794+0000 7fa988bff700 1 -- 172.21.15.22:0/2657810235 wait complete. 2023-09-20T21:53:13.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.796+0000 7fa988bff700 1 Processor -- start 2023-09-20T21:53:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.797+0000 7fa988bff700 1 -- start start 2023-09-20T21:53:13.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.797+0000 7fa988bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa984107de0 0x7fa9841a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.797+0000 7fa988bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa98410a750 0x7fa9841a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.797+0000 7fa988bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98410d120 0x7fa9841acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.798+0000 7fa988bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9841ad4f0 con 0x7fa984107de0 2023-09-20T21:53:13.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.798+0000 7fa988bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9841ad630 con 0x7fa98410a750 2023-09-20T21:53:13.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.798+0000 7fa988bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9841ad7c0 con 0x7fa98410d120 2023-09-20T21:53:13.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.802+0000 7fa981d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa98410a750 0x7fa9841a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.802+0000 7fa981d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa98410a750 0x7fa9841a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:58414/0 (socket says 172.21.15.22:58414) 2023-09-20T21:53:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.802+0000 7fa981d9b700 1 -- 172.21.15.22:0/1419904867 learned_addr learned my addr 172.21.15.22:0/1419904867 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:53:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.804+0000 7fa981d9b700 1 -- 172.21.15.22:0/1419904867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98410d120 msgr2=0x7fa9841acf20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-20T21:53:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.804+0000 7fa981d9b700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98410d120 0x7fa9841acf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.804+0000 7fa981d9b700 1 -- 172.21.15.22:0/1419904867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa984107de0 msgr2=0x7fa9841a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.804+0000 7fa981d9b700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa984107de0 0x7fa9841a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.804+0000 7fa981d9b700 1 -- 172.21.15.22:0/1419904867 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c009d00 con 0x7fa98410a750 2023-09-20T21:53:13.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.808+0000 7fa981d9b700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa98410a750 0x7fa9841a5eb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa97400d950 tx=0x7fa97400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:13.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.809+0000 7fa9737fe700 1 -- 172.21.15.22:0/1419904867 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974009b40 con 0x7fa98410a750 2023-09-20T21:53:13.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.809+0000 7fa9737fe700 1 -- 172.21.15.22:0/1419904867 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa974009ca0 con 0x7fa98410a750 2023-09-20T21:53:13.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.809+0000 7fa9737fe700 1 -- 172.21.15.22:0/1419904867 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97400f620 con 0x7fa98410a750 2023-09-20T21:53:13.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.813+0000 7fa988bff700 1 -- 172.21.15.22:0/1419904867 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98410f570 con 0x7fa98410a750 2023-09-20T21:53:13.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.813+0000 7fa988bff700 1 -- 172.21.15.22:0/1419904867 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98410fa80 con 0x7fa98410a750 2023-09-20T21:53:13.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.820+0000 7fa99c4d3b80 1 Processor -- start 2023-09-20T21:53:13.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.822+0000 7fa9737fe700 1 -- 172.21.15.22:0/1419904867 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa974010460 con 0x7fa98410a750 2023-09-20T21:53:13.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.823+0000 7fa99c4d3b80 1 -- start start 2023-09-20T21:53:13.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.823+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729e6f0 0x55dc9729eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.823+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729f050 0x55dc9729f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.823+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729fbe0 0x55dc972a0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.823+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc97212ca0 con 0x55dc9729e6f0 2023-09-20T21:53:13.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.823+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc971f6c70 con 0x55dc9729fbe0 2023-09-20T21:53:13.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.823+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc971f4590 con 0x55dc9729f050 2023-09-20T21:53:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.824+0000 7fa98259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729f050 0x55dc9729f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.825+0000 7fa98259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729f050 0x55dc9729f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:52372/0 (socket says 172.21.15.22:52372) 2023-09-20T21:53:13.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.825+0000 7fa98259c700 1 -- 172.21.15.22:0/2936826426 learned_addr learned my addr 172.21.15.22:0/2936826426 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:53:13.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.826+0000 7fa98259c700 1 -- 172.21.15.22:0/2936826426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729fbe0 msgr2=0x55dc972a0060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:53:13.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.826+0000 7fa98259c700 1 --2- 172.21.15.22:0/2936826426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729fbe0 0x55dc972a0060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.826+0000 7fa98259c700 1 -- 172.21.15.22:0/2936826426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729e6f0 msgr2=0x55dc9729eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:53:13.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.826+0000 7fa98259c700 1 --2- 172.21.15.22:0/2936826426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729e6f0 0x55dc9729eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.826+0000 7fa98259c700 1 -- 172.21.15.22:0/2936826426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc9701ade0 con 0x55dc9729f050 2023-09-20T21:53:13.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.826+0000 7fa98259c700 1 --2- 172.21.15.22:0/2936826426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729f050 0x55dc9729f5e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b070 tx=0x7fa97c008e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=145e55ada91ce332 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:13.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.826+0000 7fa970ff9700 1 -- 172.21.15.22:0/2936826426 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00e040 con 0x55dc9729f050 2023-09-20T21:53:13.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.826+0000 7fa970ff9700 1 -- 172.21.15.22:0/2936826426 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa97c009390 con 0x55dc9729f050 2023-09-20T21:53:13.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.826+0000 7fa970ff9700 1 -- 172.21.15.22:0/2936826426 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0052d0 con 0x55dc9729f050 2023-09-20T21:53:13.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.827+0000 7fa9737fe700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa96c0a1500 0x7fa96c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.827+0000 7fa9737fe700 1 -- 172.21.15.22:0/1419904867 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa9740ccd00 con 0x7fa98410a750 2023-09-20T21:53:13.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.827+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2936826426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729f050 msgr2=0x55dc9729f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:13.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.827+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/2936826426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729f050 0x55dc9729f5e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b070 tx=0x7fa97c008e50 comp rx=0 tx=0).stop 2023-09-20T21:53:13.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.827+0000 7fa982d9d700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa96c0a1500 0x7fa96c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.829+0000 7fa982d9d700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa96c0a1500 0x7fa96c0a39c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55dc9729a0b0 tx=0x7fa978006cd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.830+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2936826426 shutdown_connections 2023-09-20T21:53:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.830+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/2936826426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729fbe0 0x55dc972a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.830+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/2936826426 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729e6f0 0x55dc9729eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.830+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/2936826426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729f050 0x55dc9729f5e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.830+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2936826426 >> 172.21.15.22:0/2936826426 conn(0x55dc971cb000 msgr2=0x55dc972991d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.830+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2936826426 shutdown_connections 2023-09-20T21:53:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.831+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2936826426 wait complete. 2023-09-20T21:53:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.831+0000 7fa99c4d3b80 1 -- start start 2023-09-20T21:53:13.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.832+0000 7fa99c4d3b80 1 Processor -- start 2023-09-20T21:53:13.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.833+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.833+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729f050 0x55dc972a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.833+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729fbe0 0x55dc972af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.833+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc971efe80 con 0x55dc9729f050 2023-09-20T21:53:13.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.833+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc9718c740 con 0x55dc9729fbe0 2023-09-20T21:53:13.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.833+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc9718c3e0 con 0x55dc9729e6f0 2023-09-20T21:53:13.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.833+0000 7fa99c4d3b80 10 client.0 _ll_register_callbacks cb 0x55dc970d6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:53:13.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.834+0000 7fa98259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.834+0000 7fa98259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:52388/0 (socket says 172.21.15.22:52388) 2023-09-20T21:53:13.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.834+0000 7fa98259c700 1 -- 172.21.15.22:0/1153111465 learned_addr learned my addr 172.21.15.22:0/1153111465 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:53:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.834+0000 7fa982d9d700 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729fbe0 0x55dc972af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.834+0000 7fa981d9b700 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729f050 0x55dc972a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.834+0000 7fa98259c700 1 -- 172.21.15.22:0/1153111465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729fbe0 msgr2=0x55dc972af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:53:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.834+0000 7fa98259c700 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729fbe0 0x55dc972af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.834+0000 7fa98259c700 1 -- 172.21.15.22:0/1153111465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729f050 msgr2=0x55dc972a86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:53:13.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.834+0000 7fa98259c700 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729f050 0x55dc972a86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.834+0000 7fa98259c700 1 -- 172.21.15.22:0/1153111465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98410e120 con 0x55dc9729e6f0 2023-09-20T21:53:13.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.835+0000 7fa98259c700 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a81b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b830 tx=0x7fa97c003ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd32d5e3d428e2e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:13.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.835+0000 7fa969ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:53:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.841+0000 7fa969ffb700 1 -- 172.21.15.22:0/1153111465 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00e040 con 0x55dc9729e6f0 2023-09-20T21:53:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.841+0000 7fa969ffb700 1 -- 172.21.15.22:0/1153111465 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa97c0054f0 con 0x55dc9729e6f0 2023-09-20T21:53:13.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.841+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/1153111465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc97160bb0 con 0x55dc9729e6f0 2023-09-20T21:53:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.841+0000 7fa969ffb700 1 -- 172.21.15.22:0/1153111465 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa97c0056b0 con 0x55dc9729e6f0 2023-09-20T21:53:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.841+0000 7fa99c4d3b80 10 client.25891 fetch_fsmap learned FSMap version 32 2023-09-20T21:53:13.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.841+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/1153111465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dc97003c60 con 0x55dc9729e6f0 2023-09-20T21:53:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.842+0000 7fa969ffb700 1 -- 172.21.15.22:0/1153111465 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc97160bb0 con 0x55dc9729e6f0 2023-09-20T21:53:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.842+0000 7fa969ffb700 1 -- 172.21.15.22:0/1153111465 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa97c02c070 con 0x55dc9729e6f0 2023-09-20T21:53:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.842+0000 7fa99c4d3b80 10 client.25891 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:53:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.842+0000 7fa99c4d3b80 10 client.25891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:53:13.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.842+0000 7fa99c4d3b80 20 client.25891 populate_metadata read hostname 'smithi022' 2023-09-20T21:53:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.842+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55dc972b3b70 0x55dc972b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.842+0000 7fa99c4d3b80 4 client.25891 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:53:13.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.842+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/1153111465 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc9726d020 con 0x55dc972b3b70 2023-09-20T21:53:13.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.843+0000 7fa981d9b700 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55dc972b3b70 0x55dc972b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.843+0000 7fa969ffb700 10 client.25891 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:53:13.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.843+0000 7fa981d9b700 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55dc972b3b70 0x55dc972b5f60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf06165ff561811 server_cookie=983f42c661774897 in_seq=0 out_seq=0 2023-09-20T21:53:13.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.844+0000 7fa969ffb700 1 -- 172.21.15.22:0/1153111465 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dc9726d020 con 0x55dc972b3b70 2023-09-20T21:53:13.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.844+0000 7fa969ffb700 10 client.25891 handle_command_reply: tid=0 2023-09-20T21:53:13.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.844+0000 7fa99c4d3b80 1 client.25891 shutdown 2023-09-20T21:53:13.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/1153111465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 msgr2=0x55dc972a81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:13.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a81b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b830 tx=0x7fa97c003ca0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/1153111465 shutdown_connections 2023-09-20T21:53:13.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55dc972b3b70 0x55dc972b5f60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729fbe0 0x55dc972af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a81b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa969ffb700 0 client.25891 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:53:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/1153111465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729f050 0x55dc972a86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/1153111465 >> 172.21.15.22:0/1153111465 conn(0x55dc971cb000 msgr2=0x55dc97298880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:13.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/1153111465 shutdown_connections 2023-09-20T21:53:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/1153111465 wait complete. 2023-09-20T21:53:13.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.845+0000 7fa99c4d3b80 20 client.25891 trim_cache size 0 max 16384 2023-09-20T21:53:13.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.876+0000 7fa99c4d3b80 1 Processor -- start 2023-09-20T21:53:13.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.876+0000 7fa99c4d3b80 1 -- start start 2023-09-20T21:53:13.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.876+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.876+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729f050 0x55dc972af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.876+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 0x55dc97298880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.876+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc97070920 con 0x55dc9729fbe0 2023-09-20T21:53:13.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.876+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc9726d020 con 0x55dc9729f050 2023-09-20T21:53:13.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.876+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc97160bb0 con 0x55dc9729e6f0 2023-09-20T21:53:13.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.877+0000 7fa98259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.877+0000 7fa98259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:52402/0 (socket says 172.21.15.22:52402) 2023-09-20T21:53:13.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.877+0000 7fa98259c700 1 -- 172.21.15.22:0/462229956 learned_addr learned my addr 172.21.15.22:0/462229956 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:53:13.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.877+0000 7fa982d9d700 1 --2- 172.21.15.22:0/462229956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 0x55dc97298880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.877+0000 7fa981d9b700 1 --2- 172.21.15.22:0/462229956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729f050 0x55dc972af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.878+0000 7fa981d9b700 1 -- 172.21.15.22:0/462229956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 msgr2=0x55dc972a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:53:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.878+0000 7fa981d9b700 1 --2- 172.21.15.22:0/462229956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.878+0000 7fa981d9b700 1 -- 172.21.15.22:0/462229956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 msgr2=0x55dc97298880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:53:13.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.878+0000 7fa981d9b700 1 --2- 172.21.15.22:0/462229956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 0x55dc97298880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.878+0000 7fa981d9b700 1 -- 172.21.15.22:0/462229956 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc97041250 con 0x55dc9729f050 2023-09-20T21:53:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.878+0000 7fa98259c700 1 --2- 172.21.15.22:0/462229956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:53:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.878+0000 7fa981d9b700 1 --2- 172.21.15.22:0/462229956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729f050 0x55dc972af6f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa974009520 tx=0x7fa9740b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551a36f903b6acde server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.879+0000 7fa9717fa700 1 -- 172.21.15.22:0/462229956 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97409dad0 con 0x55dc9729f050 2023-09-20T21:53:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.879+0000 7fa9717fa700 1 -- 172.21.15.22:0/462229956 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9740a1d10 con 0x55dc9729f050 2023-09-20T21:53:13.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.879+0000 7fa9717fa700 1 -- 172.21.15.22:0/462229956 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97409a160 con 0x55dc9729f050 2023-09-20T21:53:13.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.880+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/462229956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729f050 msgr2=0x55dc972af6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:13.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.880+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/462229956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729f050 0x55dc972af6f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa974009520 tx=0x7fa9740b9040 comp rx=0 tx=0).stop 2023-09-20T21:53:13.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.880+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/462229956 shutdown_connections 2023-09-20T21:53:13.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.880+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/462229956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 0x55dc97298880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.880+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/462229956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.880+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/462229956 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729f050 0x55dc972af6f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.880+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/462229956 >> 172.21.15.22:0/462229956 conn(0x55dc971cb000 msgr2=0x55dc972991a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:13.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.880+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/462229956 shutdown_connections 2023-09-20T21:53:13.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.880+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/462229956 wait complete. 2023-09-20T21:53:13.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.881+0000 7fa99c4d3b80 1 -- start start 2023-09-20T21:53:13.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.881+0000 7fa99c4d3b80 1 Processor -- start 2023-09-20T21:53:13.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.882+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.882+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729f050 0x55dc972af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.882+0000 7fa99c4d3b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 0x55dc972ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.882+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc972adf20 con 0x55dc9729fbe0 2023-09-20T21:53:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.882+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc972ae090 con 0x55dc9729f050 2023-09-20T21:53:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.882+0000 7fa99c4d3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc972ae250 con 0x55dc9729e6f0 2023-09-20T21:53:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.882+0000 7fa99c4d3b80 10 client.0 _ll_register_callbacks cb 0x55dc970d6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:53:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.883+0000 7fa982d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 0x55dc972ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.883+0000 7fa982d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 0x55dc972ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:32804/0 (socket says 172.21.15.22:32804) 2023-09-20T21:53:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.883+0000 7fa982d9d700 1 -- 172.21.15.22:0/2739164387 learned_addr learned my addr 172.21.15.22:0/2739164387 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:53:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.884+0000 7fa982d9d700 1 -- 172.21.15.22:0/2739164387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 msgr2=0x55dc972af160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:53:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.884+0000 7fa982d9d700 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972af160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.884+0000 7fa982d9d700 1 -- 172.21.15.22:0/2739164387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729f050 msgr2=0x55dc972af6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:53:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.884+0000 7fa982d9d700 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729f050 0x55dc972af6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.884+0000 7fa982d9d700 1 -- 172.21.15.22:0/2739164387 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98410fa80 con 0x55dc9729fbe0 2023-09-20T21:53:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.884+0000 7fa982d9d700 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 0x55dc972ad840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa978010650 tx=0x7fa978004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c991e7ddef419b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.884+0000 7fa969ffb700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:53:13.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.885+0000 7fa969ffb700 1 -- 172.21.15.22:0/2739164387 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97801a040 con 0x55dc9729fbe0 2023-09-20T21:53:13.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.885+0000 7fa969ffb700 1 -- 172.21.15.22:0/2739164387 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa978021650 con 0x55dc9729fbe0 2023-09-20T21:53:13.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.885+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2739164387 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa97c009390 con 0x55dc9729fbe0 2023-09-20T21:53:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.886+0000 7fa969ffb700 1 -- 172.21.15.22:0/2739164387 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9780152e0 con 0x55dc9729fbe0 2023-09-20T21:53:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.886+0000 7fa99c4d3b80 10 client.15777 fetch_fsmap learned FSMap version 32 2023-09-20T21:53:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.886+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2739164387 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dc970192b0 con 0x55dc9729fbe0 2023-09-20T21:53:13.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.886+0000 7fa969ffb700 1 -- 172.21.15.22:0/2739164387 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa97c009390 con 0x55dc9729fbe0 2023-09-20T21:53:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.886+0000 7fa969ffb700 1 -- 172.21.15.22:0/2739164387 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa978013940 con 0x55dc9729fbe0 2023-09-20T21:53:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.887+0000 7fa99c4d3b80 10 client.15777 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:53:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.887+0000 7fa99c4d3b80 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:53:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.887+0000 7fa99c4d3b80 20 client.15777 populate_metadata read hostname 'smithi022' 2023-09-20T21:53:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.887+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55dc97308f90 0x55dc9730b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.887+0000 7fa99c4d3b80 4 client.15777 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:53:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.887+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2739164387 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dc971e8e60 con 0x55dc97308f90 2023-09-20T21:53:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.887+0000 7fa98259c700 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55dc97308f90 0x55dc9730b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.887+0000 7fa969ffb700 10 client.15777 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:53:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa98259c700 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55dc97308f90 0x55dc9730b3b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6b72af72c2ca49 server_cookie=ffef3f536893e14c in_seq=0 out_seq=0 2023-09-20T21:53:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa969ffb700 1 -- 172.21.15.22:0/2739164387 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55dc971e8e60 con 0x55dc97308f90 2023-09-20T21:53:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa969ffb700 10 client.15777 handle_command_reply: tid=0 2023-09-20T21:53:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa99c4d3b80 1 client.15777 shutdown 2023-09-20T21:53:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2739164387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 msgr2=0x55dc972ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 0x55dc972ad840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa978010650 tx=0x7fa978004870 comp rx=0 tx=0).stop 2023-09-20T21:53:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2739164387 shutdown_connections 2023-09-20T21:53:13.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55dc97308f90 0x55dc9730b3b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dc9729fbe0 0x55dc972ad840 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc9729e6f0 0x55dc972af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa99c4d3b80 1 --2- 172.21.15.22:0/2739164387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dc9729f050 0x55dc972af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2739164387 >> 172.21.15.22:0/2739164387 conn(0x55dc971cb000 msgr2=0x55dc972a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:13.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.888+0000 7fa969ffb700 0 client.15777 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:53:13.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.889+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2739164387 shutdown_connections 2023-09-20T21:53:13.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.889+0000 7fa99c4d3b80 1 -- 172.21.15.22:0/2739164387 wait complete. 2023-09-20T21:53:13.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.889+0000 7fa99c4d3b80 20 client.15777 trim_cache size 0 max 16384 2023-09-20T21:53:13.897 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:53:13.897 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:53:13.897 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:53:13.897 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:53:13.897 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:53:13.898 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:53:13.898 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:53:13.898 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:53:13.898 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:53:13.898 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:53:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 -- 172.21.15.22:0/1419904867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa96c0a1500 msgr2=0x7fa96c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa96c0a1500 0x7fa96c0a39c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55dc9729a0b0 tx=0x7fa978006cd0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 -- 172.21.15.22:0/1419904867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa98410a750 msgr2=0x7fa9841a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa98410a750 0x7fa9841a5eb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa97400d950 tx=0x7fa97400dd10 comp rx=0 tx=0).stop 2023-09-20T21:53:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 -- 172.21.15.22:0/1419904867 shutdown_connections 2023-09-20T21:53:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98410d120 0x7fa9841acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa984107de0 0x7fa9841a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa96c0a1500 0x7fa96c0a39c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 --2- 172.21.15.22:0/1419904867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa98410a750 0x7fa9841a5eb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 -- 172.21.15.22:0/1419904867 >> 172.21.15.22:0/1419904867 conn(0x7fa98406db90 msgr2=0x7fa98410b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 -- 172.21.15.22:0/1419904867 shutdown_connections 2023-09-20T21:53:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:13.890+0000 7fa988bff700 1 -- 172.21.15.22:0/1419904867 wait complete. 2023-09-20T21:53:13.920 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:53:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:13 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:53:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:13 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:53:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:13 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:53:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:13 smithi123 ceph-mon[134433]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-09-20T21:53:14.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:13 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:53:14.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:13 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:53:14.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:13 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:53:14.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:13 smithi089 ceph-mon[134090]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-09-20T21:53:14.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:13 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:53:14.254 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:13 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:53:14.254 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:13 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:53:14.254 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:13 smithi022 ceph-mon[97651]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-09-20T21:53:16.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:16 smithi022 ceph-mon[97651]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 778 op/s 2023-09-20T21:53:16.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:16 smithi123 ceph-mon[134433]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 778 op/s 2023-09-20T21:53:16.867 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:16 smithi089 ceph-mon[134090]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 11 MiB/s wr, 778 op/s 2023-09-20T21:53:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 235.0 s, 802.3 tps, lat 126.108 ms stddev 154.760 2023-09-20T21:53:18.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:18 smithi123 ceph-mon[134433]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 718 op/s 2023-09-20T21:53:18.867 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:18 smithi089 ceph-mon[134090]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 718 op/s 2023-09-20T21:53:18.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:18 smithi022 ceph-mon[97651]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 718 op/s 2023-09-20T21:53:20.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:20 smithi123 ceph-mon[134433]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 554 op/s 2023-09-20T21:53:20.867 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:20 smithi089 ceph-mon[134090]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 554 op/s 2023-09-20T21:53:20.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:20 smithi022 ceph-mon[97651]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 554 op/s 2023-09-20T21:53:21.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:21 smithi089 ceph-mon[134090]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 614 op/s 2023-09-20T21:53:22.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:21 smithi022 ceph-mon[97651]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 614 op/s 2023-09-20T21:53:22.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:21 smithi123 ceph-mon[134433]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 614 op/s 2023-09-20T21:53:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 240.0 s, 826.0 tps, lat 121.620 ms stddev 130.416 2023-09-20T21:53:24.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:23 smithi022 ceph-mon[97651]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.8 MiB/s wr, 551 op/s 2023-09-20T21:53:24.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:23 smithi123 ceph-mon[134433]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.8 MiB/s wr, 551 op/s 2023-09-20T21:53:24.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:23 smithi089 ceph-mon[134090]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.8 MiB/s wr, 551 op/s 2023-09-20T21:53:26.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:26 smithi022 ceph-mon[97651]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.6 MiB/s wr, 533 op/s 2023-09-20T21:53:26.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:26 smithi123 ceph-mon[134433]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.6 MiB/s wr, 533 op/s 2023-09-20T21:53:26.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:26 smithi089 ceph-mon[134090]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.6 MiB/s wr, 533 op/s 2023-09-20T21:53:28.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 245.0 s, 847.1 tps, lat 116.449 ms stddev 126.000 2023-09-20T21:53:28.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:28 smithi123 ceph-mon[134433]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 MiB/s wr, 676 op/s 2023-09-20T21:53:28.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:28 smithi089 ceph-mon[134090]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 MiB/s wr, 676 op/s 2023-09-20T21:53:28.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:28 smithi022 ceph-mon[97651]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 MiB/s wr, 676 op/s 2023-09-20T21:53:30.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:30 smithi123 ceph-mon[134433]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.0 MiB/s wr, 596 op/s 2023-09-20T21:53:30.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:30 smithi089 ceph-mon[134090]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.0 MiB/s wr, 596 op/s 2023-09-20T21:53:30.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:30 smithi022 ceph-mon[97651]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.0 MiB/s wr, 596 op/s 2023-09-20T21:53:32.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:31 smithi022 ceph-mon[97651]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 647 op/s 2023-09-20T21:53:32.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:31 smithi123 ceph-mon[134433]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 647 op/s 2023-09-20T21:53:32.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:31 smithi089 ceph-mon[134090]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 647 op/s 2023-09-20T21:53:33.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 250.0 s, 1040.5 tps, lat 97.534 ms stddev 102.576 2023-09-20T21:53:34.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:33 smithi022 ceph-mon[97651]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 598 op/s 2023-09-20T21:53:34.342 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:33 smithi123 ceph-mon[134433]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 598 op/s 2023-09-20T21:53:34.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:33 smithi089 ceph-mon[134090]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 598 op/s 2023-09-20T21:53:36.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:36 smithi022 ceph-mon[97651]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.3 MiB/s wr, 544 op/s 2023-09-20T21:53:36.850 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:36 smithi123 ceph-mon[134433]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.3 MiB/s wr, 544 op/s 2023-09-20T21:53:36.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:36 smithi089 ceph-mon[134090]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.3 MiB/s wr, 544 op/s 2023-09-20T21:53:38.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 255.0 s, 886.8 tps, lat 110.829 ms stddev 120.077 2023-09-20T21:53:38.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:38 smithi123 ceph-mon[134433]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 659 op/s 2023-09-20T21:53:38.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:38 smithi089 ceph-mon[134090]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 659 op/s 2023-09-20T21:53:38.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:38 smithi022 ceph-mon[97651]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 659 op/s 2023-09-20T21:53:40.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:40 smithi123 ceph-mon[134433]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.0 MiB/s wr, 553 op/s 2023-09-20T21:53:40.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:40 smithi089 ceph-mon[134090]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.0 MiB/s wr, 553 op/s 2023-09-20T21:53:40.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:40 smithi022 ceph-mon[97651]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.0 MiB/s wr, 553 op/s 2023-09-20T21:53:43.003 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:42 smithi022 ceph-mon[97651]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 MiB/s wr, 604 op/s 2023-09-20T21:53:43.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:42 smithi123 ceph-mon[134433]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 MiB/s wr, 604 op/s 2023-09-20T21:53:43.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:42 smithi089 ceph-mon[134090]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 MiB/s wr, 604 op/s 2023-09-20T21:53:43.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 260.0 s, 838.8 tps, lat 119.689 ms stddev 142.035 2023-09-20T21:53:43.921 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:53:44.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:43 smithi123 ceph-mon[134433]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 MiB/s wr, 551 op/s 2023-09-20T21:53:44.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:43 smithi089 ceph-mon[134090]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 MiB/s wr, 551 op/s 2023-09-20T21:53:44.125 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:43 smithi022 ceph-mon[97651]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.9 MiB/s wr, 551 op/s 2023-09-20T21:53:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.426+0000 7f40b75ff700 1 -- 172.21.15.22:0/921754653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b010a750 msgr2=0x7f40b010cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:44.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.426+0000 7f40b75ff700 1 --2- 172.21.15.22:0/921754653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b010a750 0x7f40b010cbe0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009990 tx=0x7f40ac009ca0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.426+0000 7f40b75ff700 1 -- 172.21.15.22:0/921754653 shutdown_connections 2023-09-20T21:53:44.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.426+0000 7f40b75ff700 1 --2- 172.21.15.22:0/921754653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b010d120 0x7f40b010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.426+0000 7f40b75ff700 1 --2- 172.21.15.22:0/921754653 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0107de0 0x7f40b010a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.426+0000 7f40b75ff700 1 --2- 172.21.15.22:0/921754653 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b010a750 0x7f40b010cbe0 secure :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009990 tx=0x7f40ac009ca0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.426+0000 7f40b75ff700 1 -- 172.21.15.22:0/921754653 >> 172.21.15.22:0/921754653 conn(0x7f40b006db90 msgr2=0x7f40b006ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:44.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.455+0000 7f40b75ff700 1 -- 172.21.15.22:0/921754653 shutdown_connections 2023-09-20T21:53:44.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.463+0000 7f40b75ff700 1 -- 172.21.15.22:0/921754653 wait complete. 2023-09-20T21:53:44.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.464+0000 7f40b75ff700 1 Processor -- start 2023-09-20T21:53:44.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.464+0000 7f40b75ff700 1 -- start start 2023-09-20T21:53:44.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.465+0000 7f40b75ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0107de0 0x7f40b0117600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.465+0000 7f40b75ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b010d120 0x7f40b0117b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.465+0000 7f40b75ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b0118b80 0x7f40b0119000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.465+0000 7f40b75ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b0119540 con 0x7f40b010d120 2023-09-20T21:53:44.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.465+0000 7f40b75ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b01196b0 con 0x7f40b0107de0 2023-09-20T21:53:44.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.465+0000 7f40b75ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b0119870 con 0x7f40b0118b80 2023-09-20T21:53:44.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.471+0000 7f40b5b9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b0118b80 0x7f40b0119000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.471+0000 7f40b4b9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b010d120 0x7f40b0117b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.471+0000 7f40b4b9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b010d120 0x7f40b0117b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46406/0 (socket says 172.21.15.22:46406) 2023-09-20T21:53:44.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.471+0000 7f40b4b9a700 1 -- 172.21.15.22:0/2978101282 learned_addr learned my addr 172.21.15.22:0/2978101282 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:53:44.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.472+0000 7f40b539b700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0107de0 0x7f40b0117600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.472+0000 7f40b4b9a700 1 -- 172.21.15.22:0/2978101282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b0118b80 msgr2=0x7f40b0119000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:44.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.472+0000 7f40b4b9a700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b0118b80 0x7f40b0119000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.472+0000 7f40b4b9a700 1 -- 172.21.15.22:0/2978101282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0107de0 msgr2=0x7f40b0117600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:44.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.472+0000 7f40b4b9a700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0107de0 0x7f40b0117600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.472+0000 7f40b4b9a700 1 -- 172.21.15.22:0/2978101282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac0095f0 con 0x7f40b010d120 2023-09-20T21:53:44.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.473+0000 7f40b4b9a700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b010d120 0x7f40b0117b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f40ac000c00 tx=0x7f40ac00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:44.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.473+0000 7f40a67fc700 1 -- 172.21.15.22:0/2978101282 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac01d070 con 0x7f40b010d120 2023-09-20T21:53:44.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.474+0000 7f40a67fc700 1 -- 172.21.15.22:0/2978101282 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ac021410 con 0x7f40b010d120 2023-09-20T21:53:44.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.474+0000 7f40a67fc700 1 -- 172.21.15.22:0/2978101282 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac017650 con 0x7f40b010d120 2023-09-20T21:53:44.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.478+0000 7f40b75ff700 1 -- 172.21.15.22:0/2978101282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b0119af0 con 0x7f40b010d120 2023-09-20T21:53:44.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.478+0000 7f40b75ff700 1 -- 172.21.15.22:0/2978101282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b01b6d00 con 0x7f40b010d120 2023-09-20T21:53:44.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.488+0000 7f40a67fc700 1 -- 172.21.15.22:0/2978101282 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f40ac021580 con 0x7f40b010d120 2023-09-20T21:53:44.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.489+0000 7f40cafa2b80 1 Processor -- start 2023-09-20T21:53:44.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.491+0000 7f40a67fc700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1500 0x7f409c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.491+0000 7f40a67fc700 1 -- 172.21.15.22:0/2978101282 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f40ac0cdb10 con 0x7f40b010d120 2023-09-20T21:53:44.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.493+0000 7f40b5b9c700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1500 0x7f409c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.494+0000 7f40b5b9c700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1500 0x7f409c0a39c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f40a8003f10 tx=0x7f40a8009100 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:44.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.498+0000 7f40cafa2b80 1 -- start start 2023-09-20T21:53:44.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.499+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939c6f0 0x55a3c939cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.499+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939d050 0x55a3c939d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.499+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939dbe0 0x55a3c939e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.499+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c9310ca0 con 0x55a3c939d050 2023-09-20T21:53:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.499+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c92f4c70 con 0x55a3c939dbe0 2023-09-20T21:53:44.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.499+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c92f2590 con 0x55a3c939c6f0 2023-09-20T21:53:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.503+0000 7f40b539b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939c6f0 0x55a3c939cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.503+0000 7f40b539b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939c6f0 0x55a3c939cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:34626/0 (socket says 172.21.15.22:34626) 2023-09-20T21:53:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.503+0000 7f40b539b700 1 -- 172.21.15.22:0/1380125034 learned_addr learned my addr 172.21.15.22:0/1380125034 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:53:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.503+0000 7f40b4b9a700 1 --2- 172.21.15.22:0/1380125034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939d050 0x55a3c939d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.503+0000 7f40b539b700 1 -- 172.21.15.22:0/1380125034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939dbe0 msgr2=0x55a3c939e060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:53:44.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.503+0000 7f40b539b700 1 --2- 172.21.15.22:0/1380125034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939dbe0 0x55a3c939e060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.503+0000 7f40b539b700 1 -- 172.21.15.22:0/1380125034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939d050 msgr2=0x55a3c939d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:53:44.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.503+0000 7f40b539b700 1 --2- 172.21.15.22:0/1380125034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939d050 0x55a3c939d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.503+0000 7f40b539b700 1 -- 172.21.15.22:0/1380125034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3c9118de0 con 0x55a3c939c6f0 2023-09-20T21:53:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.512+0000 7f40b539b700 1 --2- 172.21.15.22:0/1380125034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939c6f0 0x55a3c939cb10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f40a0007810 tx=0x7f40a0007b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=969beae5d247fe84 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:44.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.513+0000 7f409b7fe700 1 -- 172.21.15.22:0/1380125034 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a00125f0 con 0x55a3c939c6f0 2023-09-20T21:53:44.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.513+0000 7f409b7fe700 1 -- 172.21.15.22:0/1380125034 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40a0012c10 con 0x55a3c939c6f0 2023-09-20T21:53:44.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.513+0000 7f409b7fe700 1 -- 172.21.15.22:0/1380125034 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0013e40 con 0x55a3c939c6f0 2023-09-20T21:53:44.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.514+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1380125034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939c6f0 msgr2=0x55a3c939cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:44.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.514+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/1380125034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939c6f0 0x55a3c939cb10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f40a0007810 tx=0x7f40a0007b20 comp rx=0 tx=0).stop 2023-09-20T21:53:44.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.515+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1380125034 shutdown_connections 2023-09-20T21:53:44.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.515+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/1380125034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939dbe0 0x55a3c939e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.515+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/1380125034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939c6f0 0x55a3c939cb10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.515+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/1380125034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939d050 0x55a3c939d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.515+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1380125034 >> 172.21.15.22:0/1380125034 conn(0x55a3c92c9000 msgr2=0x55a3c93971d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:44.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.515+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1380125034 shutdown_connections 2023-09-20T21:53:44.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.515+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1380125034 wait complete. 2023-09-20T21:53:44.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.516+0000 7f40cafa2b80 1 -- start start 2023-09-20T21:53:44.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.516+0000 7f40cafa2b80 1 Processor -- start 2023-09-20T21:53:44.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.519+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939c6f0 0x55a3c93a6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.519+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939d050 0x55a3c93a6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.519+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 0x55a3c93ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.519+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c92ede80 con 0x55a3c939d050 2023-09-20T21:53:44.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.519+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c928a740 con 0x55a3c939c6f0 2023-09-20T21:53:44.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.519+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c928a3e0 con 0x55a3c939dbe0 2023-09-20T21:53:44.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.519+0000 7f40cafa2b80 10 client.0 _ll_register_callbacks cb 0x55a3c91d4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:53:44.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.520+0000 7f40b5b9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 0x55a3c93ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.520+0000 7f40b5b9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 0x55a3c93ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:34634/0 (socket says 172.21.15.22:34634) 2023-09-20T21:53:44.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.520+0000 7f40b5b9c700 1 -- 172.21.15.22:0/3593475977 learned_addr learned my addr 172.21.15.22:0/3593475977 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:53:44.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.521+0000 7f40b5b9c700 1 -- 172.21.15.22:0/3593475977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939c6f0 msgr2=0x55a3c93a6250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:53:44.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.521+0000 7f40b5b9c700 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939c6f0 0x55a3c93a6250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.521+0000 7f40b5b9c700 1 -- 172.21.15.22:0/3593475977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939d050 msgr2=0x55a3c93a6790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:53:44.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.521+0000 7f40b5b9c700 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939d050 0x55a3c93a6790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.521+0000 7f40b5b9c700 1 -- 172.21.15.22:0/3593475977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a000e040 con 0x55a3c939dbe0 2023-09-20T21:53:44.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.521+0000 7f40b5b9c700 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 0x55a3c93ad7d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007880 tx=0x7f40a8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=580d40c0bbf8dbb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:44.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.526+0000 7f4098ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:53:44.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.526+0000 7f4098ff9700 1 -- 172.21.15.22:0/3593475977 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8015950 con 0x55a3c939dbe0 2023-09-20T21:53:44.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.526+0000 7f4098ff9700 1 -- 172.21.15.22:0/3593475977 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f40a8015ab0 con 0x55a3c939dbe0 2023-09-20T21:53:44.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.526+0000 7f40cafa2b80 1 -- 172.21.15.22:0/3593475977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3c925ebb0 con 0x55a3c939dbe0 2023-09-20T21:53:44.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.527+0000 7f4098ff9700 1 -- 172.21.15.22:0/3593475977 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40a8015c70 con 0x55a3c939dbe0 2023-09-20T21:53:44.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.527+0000 7f40cafa2b80 10 client.25906 fetch_fsmap learned FSMap version 32 2023-09-20T21:53:44.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.527+0000 7f40cafa2b80 1 -- 172.21.15.22:0/3593475977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a3c9101c60 con 0x55a3c939dbe0 2023-09-20T21:53:44.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.528+0000 7f4098ff9700 1 -- 172.21.15.22:0/3593475977 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3c925ebb0 con 0x55a3c939dbe0 2023-09-20T21:53:44.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.528+0000 7f4098ff9700 1 -- 172.21.15.22:0/3593475977 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f40a80163f0 con 0x55a3c939dbe0 2023-09-20T21:53:44.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.528+0000 7f40cafa2b80 10 client.25906 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:53:44.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.528+0000 7f40cafa2b80 10 client.25906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:53:44.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.528+0000 7f40cafa2b80 20 client.25906 populate_metadata read hostname 'smithi022' 2023-09-20T21:53:44.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.528+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a3c93b20f0 0x55a3c93b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.528+0000 7f40cafa2b80 4 client.25906 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:53:44.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.528+0000 7f40cafa2b80 1 -- 172.21.15.22:0/3593475977 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3c936b020 con 0x55a3c93b20f0 2023-09-20T21:53:44.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.529+0000 7f40b539b700 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a3c93b20f0 0x55a3c93b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.532+0000 7f40b539b700 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a3c93b20f0 0x55a3c93b4510 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dab6906124ab1185 server_cookie=1a5e2f9c51973570 in_seq=0 out_seq=0 2023-09-20T21:53:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.532+0000 7f4098ff9700 10 client.25906 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:53:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.533+0000 7f4098ff9700 1 -- 172.21.15.22:0/3593475977 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a3c936b020 con 0x55a3c93b20f0 2023-09-20T21:53:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.533+0000 7f4098ff9700 10 client.25906 handle_command_reply: tid=0 2023-09-20T21:53:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.533+0000 7f40cafa2b80 1 client.25906 shutdown 2023-09-20T21:53:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.535+0000 7f40cafa2b80 1 -- 172.21.15.22:0/3593475977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 msgr2=0x55a3c93ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.535+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 0x55a3c93ad7d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007880 tx=0x7f40a8004b40 comp rx=0 tx=0).stop 2023-09-20T21:53:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.538+0000 7f40cafa2b80 1 -- 172.21.15.22:0/3593475977 shutdown_connections 2023-09-20T21:53:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.538+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a3c93b20f0 0x55a3c93b4510 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.538+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 0x55a3c93ad7d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.538+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939c6f0 0x55a3c93a6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.538+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/3593475977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939d050 0x55a3c93a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.538+0000 7f40cafa2b80 1 -- 172.21.15.22:0/3593475977 >> 172.21.15.22:0/3593475977 conn(0x55a3c92c9000 msgr2=0x55a3c9396880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.538+0000 7f4098ff9700 0 client.25906 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:53:44.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.538+0000 7f40cafa2b80 1 -- 172.21.15.22:0/3593475977 shutdown_connections 2023-09-20T21:53:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.538+0000 7f40cafa2b80 1 -- 172.21.15.22:0/3593475977 wait complete. 2023-09-20T21:53:44.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.540+0000 7f40cafa2b80 20 client.25906 trim_cache size 0 max 16384 2023-09-20T21:53:44.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.574+0000 7f40cafa2b80 1 Processor -- start 2023-09-20T21:53:44.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.574+0000 7f40cafa2b80 1 -- start start 2023-09-20T21:53:44.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.574+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939c6f0 0x55a3c93a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.574+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939d050 0x55a3c93ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.574+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939dbe0 0x55a3c93a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.574+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c916e920 con 0x55a3c939dbe0 2023-09-20T21:53:44.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.574+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c93a5700 con 0x55a3c939c6f0 2023-09-20T21:53:44.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.574+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c939e490 con 0x55a3c939d050 2023-09-20T21:53:44.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.575+0000 7f40b4b9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939d050 0x55a3c93ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.580+0000 7f40b539b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939c6f0 0x55a3c93a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.580+0000 7f40b4b9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939d050 0x55a3c93ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:34650/0 (socket says 172.21.15.22:34650) 2023-09-20T21:53:44.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.580+0000 7f40b4b9a700 1 -- 172.21.15.22:0/2291616265 learned_addr learned my addr 172.21.15.22:0/2291616265 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:53:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.589+0000 7f40b4b9a700 1 -- 172.21.15.22:0/2291616265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939c6f0 msgr2=0x55a3c93a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:53:44.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.589+0000 7f40b4b9a700 1 --2- 172.21.15.22:0/2291616265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939c6f0 0x55a3c93a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.589+0000 7f40b4b9a700 1 -- 172.21.15.22:0/2291616265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939dbe0 msgr2=0x55a3c93a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:53:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.589+0000 7f40b4b9a700 1 --2- 172.21.15.22:0/2291616265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939dbe0 0x55a3c93a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.589+0000 7f40b4b9a700 1 -- 172.21.15.22:0/2291616265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3c913f250 con 0x55a3c939d050 2023-09-20T21:53:44.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.603+0000 7f40b4b9a700 1 --2- 172.21.15.22:0/2291616265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939d050 0x55a3c93ad790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f40ac021830 tx=0x7f40ac021c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b384a72c45e6ddb2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:44.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.608+0000 7f409bfff700 1 -- 172.21.15.22:0/2291616265 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac09a4a0 con 0x55a3c939d050 2023-09-20T21:53:44.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.608+0000 7f409bfff700 1 -- 172.21.15.22:0/2291616265 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ac09b240 con 0x55a3c939d050 2023-09-20T21:53:44.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.608+0000 7f409bfff700 1 -- 172.21.15.22:0/2291616265 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0cb450 con 0x55a3c939d050 2023-09-20T21:53:44.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.609+0000 7f40cafa2b80 1 -- 172.21.15.22:0/2291616265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939d050 msgr2=0x55a3c93ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:44.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.609+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/2291616265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939d050 0x55a3c93ad790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f40ac021830 tx=0x7f40ac021c20 comp rx=0 tx=0).stop 2023-09-20T21:53:44.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.612+0000 7f40cafa2b80 1 -- 172.21.15.22:0/2291616265 shutdown_connections 2023-09-20T21:53:44.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.612+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/2291616265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939dbe0 0x55a3c93a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.612+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/2291616265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939c6f0 0x55a3c93a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.612+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/2291616265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939d050 0x55a3c93ad790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.612+0000 7f40cafa2b80 1 -- 172.21.15.22:0/2291616265 >> 172.21.15.22:0/2291616265 conn(0x55a3c92c9000 msgr2=0x55a3c93b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:44.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.624+0000 7f40cafa2b80 1 -- 172.21.15.22:0/2291616265 shutdown_connections 2023-09-20T21:53:44.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.625+0000 7f40cafa2b80 1 -- 172.21.15.22:0/2291616265 wait complete. 2023-09-20T21:53:44.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.625+0000 7f40cafa2b80 1 -- start start 2023-09-20T21:53:44.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.626+0000 7f40cafa2b80 1 Processor -- start 2023-09-20T21:53:44.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.626+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939c6f0 0x55a3c93ab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.626+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939d050 0x55a3c93ab990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.626+0000 7f40cafa2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 0x55a3c93ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.626+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c93acee0 con 0x55a3c939c6f0 2023-09-20T21:53:44.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.626+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c93ad050 con 0x55a3c939d050 2023-09-20T21:53:44.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.626+0000 7f40cafa2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3c93ad210 con 0x55a3c939dbe0 2023-09-20T21:53:44.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.627+0000 7f40cafa2b80 10 client.0 _ll_register_callbacks cb 0x55a3c91d4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:53:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.627+0000 7f40b4b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939d050 0x55a3c93ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.628+0000 7f40b4b9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939d050 0x55a3c93ab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:50226/0 (socket says 172.21.15.22:50226) 2023-09-20T21:53:44.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.628+0000 7f40b4b9a700 1 -- 172.21.15.22:0/1565669594 learned_addr learned my addr 172.21.15.22:0/1565669594 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:53:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.628+0000 7f40b5b9c700 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 0x55a3c93ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.628+0000 7f40b4b9a700 1 -- 172.21.15.22:0/1565669594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 msgr2=0x55a3c93ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:53:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.628+0000 7f40b4b9a700 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 0x55a3c93ac9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.628+0000 7f40b4b9a700 1 -- 172.21.15.22:0/1565669594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939c6f0 msgr2=0x55a3c93ab450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:53:44.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.628+0000 7f40b4b9a700 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939c6f0 0x55a3c93ab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.628+0000 7f40b4b9a700 1 -- 172.21.15.22:0/1565669594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b01b6d00 con 0x55a3c939d050 2023-09-20T21:53:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.629+0000 7f40b4b9a700 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939d050 0x55a3c93ab990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0095c0 tx=0x7f40ac0cb1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe067020b95772b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:53:44.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.631+0000 7f4098ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:53:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.631+0000 7f4098ff9700 1 -- 172.21.15.22:0/1565669594 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac09a4a0 con 0x55a3c939d050 2023-09-20T21:53:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.631+0000 7f4098ff9700 1 -- 172.21.15.22:0/1565669594 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f40ac09e970 con 0x55a3c939d050 2023-09-20T21:53:44.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.631+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1565669594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40a0012c10 con 0x55a3c939d050 2023-09-20T21:53:44.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.634+0000 7f4098ff9700 1 -- 172.21.15.22:0/1565669594 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40ac09ab40 con 0x55a3c939d050 2023-09-20T21:53:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.638+0000 7f40cafa2b80 10 client.25904 fetch_fsmap learned FSMap version 32 2023-09-20T21:53:44.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.638+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1565669594 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a3c91172b0 con 0x55a3c939d050 2023-09-20T21:53:44.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.652+0000 7f4098ff9700 1 -- 172.21.15.22:0/1565669594 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40a0012c10 con 0x55a3c939d050 2023-09-20T21:53:44.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.652+0000 7f4098ff9700 1 -- 172.21.15.22:0/1565669594 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f40ac09bb50 con 0x55a3c939d050 2023-09-20T21:53:44.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.653+0000 7f40cafa2b80 10 client.25904 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:53:44.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.653+0000 7f40cafa2b80 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:53:44.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.653+0000 7f40cafa2b80 20 client.25904 populate_metadata read hostname 'smithi022' 2023-09-20T21:53:44.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.653+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a3c9406910 0x55a3c9408d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:53:44.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.653+0000 7f40cafa2b80 4 client.25904 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:53:44.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.653+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1565669594 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a3c92e6e60 con 0x55a3c9406910 2023-09-20T21:53:44.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.654+0000 7f40b539b700 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a3c9406910 0x55a3c9408d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:53:44.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.656+0000 7f40b539b700 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a3c9406910 0x55a3c9408d00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e41309afab795750 server_cookie=1efb520d0fa301d6 in_seq=0 out_seq=0 2023-09-20T21:53:44.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.660+0000 7f4098ff9700 10 client.25904 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:53:44.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.660+0000 7f4098ff9700 1 -- 172.21.15.22:0/1565669594 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a3c92e6e60 con 0x55a3c9406910 2023-09-20T21:53:44.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.660+0000 7f4098ff9700 10 client.25904 handle_command_reply: tid=0 2023-09-20T21:53:44.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.660+0000 7f40cafa2b80 1 client.25904 shutdown 2023-09-20T21:53:44.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.661+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1565669594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939d050 msgr2=0x55a3c93ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:44.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.661+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939d050 0x55a3c93ab990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0095c0 tx=0x7f40ac0cb1c0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.662+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1565669594 shutdown_connections 2023-09-20T21:53:44.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.662+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a3c9406910 0x55a3c9408d00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.662+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c939dbe0 0x55a3c93ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.662+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a3c939c6f0 0x55a3c93ab450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.662+0000 7f40cafa2b80 1 --2- 172.21.15.22:0/1565669594 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a3c939d050 0x55a3c93ab990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.662+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1565669594 >> 172.21.15.22:0/1565669594 conn(0x55a3c92c9000 msgr2=0x55a3c93a4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:44.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.662+0000 7f4098ff9700 0 client.25904 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:53:44.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.662+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1565669594 shutdown_connections 2023-09-20T21:53:44.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.662+0000 7f40cafa2b80 1 -- 172.21.15.22:0/1565669594 wait complete. 2023-09-20T21:53:44.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.663+0000 7f40cafa2b80 20 client.25904 trim_cache size 0 max 16384 2023-09-20T21:53:44.665 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:53:44.665 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:53:44.665 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:53:44.665 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:53:44.665 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:53:44.666 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:53:44.666 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:53:44.666 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:53:44.666 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:53:44.666 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:53:44.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.666+0000 7f40b75ff700 1 -- 172.21.15.22:0/2978101282 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1500 msgr2=0x7f409c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:44.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.666+0000 7f40b75ff700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1500 0x7f409c0a39c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f40a8003f10 tx=0x7f40a8009100 comp rx=0 tx=0).stop 2023-09-20T21:53:44.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.666+0000 7f40b75ff700 1 -- 172.21.15.22:0/2978101282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b010d120 msgr2=0x7f40b0117b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:53:44.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.666+0000 7f40b75ff700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b010d120 0x7f40b0117b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f40ac000c00 tx=0x7f40ac00f690 comp rx=0 tx=0).stop 2023-09-20T21:53:44.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.666+0000 7f40b75ff700 1 -- 172.21.15.22:0/2978101282 shutdown_connections 2023-09-20T21:53:44.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.666+0000 7f40b75ff700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b0118b80 0x7f40b0119000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.666+0000 7f40b75ff700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f409c0a1500 0x7f409c0a39c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.666+0000 7f40b75ff700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40b0107de0 0x7f40b0117600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.666+0000 7f40b75ff700 1 --2- 172.21.15.22:0/2978101282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40b010d120 0x7f40b0117b40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:53:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.666+0000 7f40b75ff700 1 -- 172.21.15.22:0/2978101282 >> 172.21.15.22:0/2978101282 conn(0x7f40b006db90 msgr2=0x7f40b006ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:53:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.667+0000 7f40b75ff700 1 -- 172.21.15.22:0/2978101282 shutdown_connections 2023-09-20T21:53:44.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:53:44.667+0000 7f40b75ff700 1 -- 172.21.15.22:0/2978101282 wait complete. 2023-09-20T21:53:44.698 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:53:46.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:46 smithi022 ceph-mon[97651]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.7 MiB/s wr, 520 op/s 2023-09-20T21:53:46.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:46 smithi123 ceph-mon[134433]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.7 MiB/s wr, 520 op/s 2023-09-20T21:53:46.870 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:46 smithi089 ceph-mon[134090]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.7 MiB/s wr, 520 op/s 2023-09-20T21:53:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 265.0 s, 685.9 tps, lat 148.119 ms stddev 176.923 2023-09-20T21:53:48.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:48 smithi123 ceph-mon[134433]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.6 MiB/s wr, 630 op/s 2023-09-20T21:53:48.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:48 smithi089 ceph-mon[134090]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.6 MiB/s wr, 630 op/s 2023-09-20T21:53:48.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:48 smithi022 ceph-mon[97651]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.6 MiB/s wr, 630 op/s 2023-09-20T21:53:50.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:50 smithi123 ceph-mon[134433]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 MiB/s wr, 538 op/s 2023-09-20T21:53:50.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:50 smithi089 ceph-mon[134090]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 MiB/s wr, 538 op/s 2023-09-20T21:53:50.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:50 smithi022 ceph-mon[97651]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 MiB/s wr, 538 op/s 2023-09-20T21:53:52.013 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:51 smithi022 ceph-mon[97651]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 606 op/s 2023-09-20T21:53:52.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:51 smithi123 ceph-mon[134433]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 606 op/s 2023-09-20T21:53:52.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:51 smithi089 ceph-mon[134090]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 606 op/s 2023-09-20T21:53:53.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 270.0 s, 870.4 tps, lat 113.202 ms stddev 148.422 2023-09-20T21:53:54.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:53 smithi022 ceph-mon[97651]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 MiB/s wr, 570 op/s 2023-09-20T21:53:54.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:53 smithi123 ceph-mon[134433]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 MiB/s wr, 570 op/s 2023-09-20T21:53:54.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:53 smithi089 ceph-mon[134090]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 MiB/s wr, 570 op/s 2023-09-20T21:53:56.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:56 smithi123 ceph-mon[134433]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 MiB/s wr, 537 op/s 2023-09-20T21:53:56.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:56 smithi089 ceph-mon[134090]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 MiB/s wr, 537 op/s 2023-09-20T21:53:56.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:56 smithi022 ceph-mon[97651]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 MiB/s wr, 537 op/s 2023-09-20T21:53:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 275.0 s, 811.7 tps, lat 122.856 ms stddev 130.970 2023-09-20T21:53:58.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:53:58 smithi123 ceph-mon[134433]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 MiB/s wr, 667 op/s 2023-09-20T21:53:58.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:53:58 smithi089 ceph-mon[134090]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 MiB/s wr, 667 op/s 2023-09-20T21:53:58.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:53:58 smithi022 ceph-mon[97651]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 MiB/s wr, 667 op/s 2023-09-20T21:54:00.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:00 smithi022 ceph-mon[97651]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.0 MiB/s wr, 603 op/s 2023-09-20T21:54:01.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:00 smithi123 ceph-mon[134433]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.0 MiB/s wr, 603 op/s 2023-09-20T21:54:01.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:00 smithi089 ceph-mon[134090]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.0 MiB/s wr, 603 op/s 2023-09-20T21:54:02.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:01 smithi022 ceph-mon[97651]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 652 op/s 2023-09-20T21:54:02.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:01 smithi123 ceph-mon[134433]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 652 op/s 2023-09-20T21:54:02.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:01 smithi089 ceph-mon[134090]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 652 op/s 2023-09-20T21:54:03.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 280.0 s, 714.4 tps, lat 141.240 ms stddev 145.692 2023-09-20T21:54:04.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:03 smithi022 ceph-mon[97651]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 MiB/s wr, 570 op/s 2023-09-20T21:54:04.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:03 smithi123 ceph-mon[134433]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 MiB/s wr, 570 op/s 2023-09-20T21:54:04.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:03 smithi089 ceph-mon[134090]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 MiB/s wr, 570 op/s 2023-09-20T21:54:06.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:06 smithi123 ceph-mon[134433]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.8 MiB/s wr, 529 op/s 2023-09-20T21:54:06.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:06 smithi089 ceph-mon[134090]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.8 MiB/s wr, 529 op/s 2023-09-20T21:54:06.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:06 smithi022 ceph-mon[97651]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.8 MiB/s wr, 529 op/s 2023-09-20T21:54:08.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 285.0 s, 806.8 tps, lat 122.802 ms stddev 141.946 2023-09-20T21:54:08.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:08 smithi123 ceph-mon[134433]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 MiB/s wr, 636 op/s 2023-09-20T21:54:08.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:08 smithi089 ceph-mon[134090]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 MiB/s wr, 636 op/s 2023-09-20T21:54:08.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:08 smithi022 ceph-mon[97651]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 MiB/s wr, 636 op/s 2023-09-20T21:54:09.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:09 smithi022 ceph-mon[97651]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.0 MiB/s wr, 542 op/s 2023-09-20T21:54:10.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:09 smithi123 ceph-mon[134433]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.0 MiB/s wr, 542 op/s 2023-09-20T21:54:10.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:09 smithi089 ceph-mon[134090]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.0 MiB/s wr, 542 op/s 2023-09-20T21:54:12.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:12 smithi123 ceph-mon[134433]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 587 op/s 2023-09-20T21:54:12.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:12 smithi089 ceph-mon[134090]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 587 op/s 2023-09-20T21:54:12.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:12 smithi022 ceph-mon[97651]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 587 op/s 2023-09-20T21:54:13.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 290.0 s, 725.7 tps, lat 136.598 ms stddev 187.954 2023-09-20T21:54:14.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:13 smithi022 ceph-mon[97651]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.3 MiB/s wr, 501 op/s 2023-09-20T21:54:14.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:13 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:54:14.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:13 smithi123 ceph-mon[134433]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.3 MiB/s wr, 501 op/s 2023-09-20T21:54:14.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:13 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:54:14.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:13 smithi089 ceph-mon[134090]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.3 MiB/s wr, 501 op/s 2023-09-20T21:54:14.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:13 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:54:14.699 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:54:15.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.219+0000 7fe0c35ff700 1 -- 172.21.15.22:0/1181182029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0bc074e20 msgr2=0x7fe0bc073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:15.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.219+0000 7fe0c35ff700 1 --2- 172.21.15.22:0/1181182029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0bc074e20 0x7fe0bc073270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009990 tx=0x7fe0ac009ca0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.228+0000 7fe0c35ff700 1 -- 172.21.15.22:0/1181182029 shutdown_connections 2023-09-20T21:54:15.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.228+0000 7fe0c35ff700 1 --2- 172.21.15.22:0/1181182029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0bc10d3c0 0x7fe0bc10f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.228+0000 7fe0c35ff700 1 --2- 172.21.15.22:0/1181182029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0bc074e20 0x7fe0bc073270 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.228+0000 7fe0c35ff700 1 --2- 172.21.15.22:0/1181182029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0bc073840 0x7fe0bc073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.228+0000 7fe0c35ff700 1 -- 172.21.15.22:0/1181182029 >> 172.21.15.22:0/1181182029 conn(0x7fe0bc0fc360 msgr2=0x7fe0bc0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.230+0000 7fe0c35ff700 1 -- 172.21.15.22:0/1181182029 shutdown_connections 2023-09-20T21:54:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.230+0000 7fe0c35ff700 1 -- 172.21.15.22:0/1181182029 wait complete. 2023-09-20T21:54:15.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.232+0000 7fe0c35ff700 1 Processor -- start 2023-09-20T21:54:15.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.232+0000 7fe0c35ff700 1 -- start start 2023-09-20T21:54:15.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.232+0000 7fe0c35ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0bc073840 0x7fe0bc06f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.233+0000 7fe0c35ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0bc074e20 0x7fe0bc06fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.233+0000 7fe0c35ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0bc10d3c0 0x7fe0bc1ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.233+0000 7fe0c35ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc072cc0 con 0x7fe0bc073840 2023-09-20T21:54:15.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.233+0000 7fe0c35ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc072e00 con 0x7fe0bc10d3c0 2023-09-20T21:54:15.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.233+0000 7fe0c35ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc104bb0 con 0x7fe0bc074e20 2023-09-20T21:54:15.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.233+0000 7fe0c0b9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0bc074e20 0x7fe0bc06fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.233+0000 7fe0c0b9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0bc074e20 0x7fe0bc06fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43436/0 (socket says 172.21.15.22:43436) 2023-09-20T21:54:15.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.233+0000 7fe0c0b9a700 1 -- 172.21.15.22:0/32327301 learned_addr learned my addr 172.21.15.22:0/32327301 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:54:15.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.234+0000 7fe0c1b9c700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0bc10d3c0 0x7fe0bc1ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.234+0000 7fe0c139b700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0bc073840 0x7fe0bc06f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.234+0000 7fe0c0b9a700 1 -- 172.21.15.22:0/32327301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0bc10d3c0 msgr2=0x7fe0bc1ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:15.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.234+0000 7fe0c0b9a700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0bc10d3c0 0x7fe0bc1ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.234+0000 7fe0c0b9a700 1 -- 172.21.15.22:0/32327301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0bc073840 msgr2=0x7fe0bc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:15.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.234+0000 7fe0c0b9a700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0bc073840 0x7fe0bc06f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.234+0000 7fe0c0b9a700 1 -- 172.21.15.22:0/32327301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac0095f0 con 0x7fe0bc074e20 2023-09-20T21:54:15.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.235+0000 7fe0c0b9a700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0bc074e20 0x7fe0bc06fbe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400b9c0 tx=0x7fe0b400bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:15.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.236+0000 7fe0b27fc700 1 -- 172.21.15.22:0/32327301 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400c950 con 0x7fe0bc074e20 2023-09-20T21:54:15.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.236+0000 7fe0b27fc700 1 -- 172.21.15.22:0/32327301 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0b4014440 con 0x7fe0bc074e20 2023-09-20T21:54:15.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.236+0000 7fe0b27fc700 1 -- 172.21.15.22:0/32327301 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40125a0 con 0x7fe0bc074e20 2023-09-20T21:54:15.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.236+0000 7fe0c35ff700 1 -- 172.21.15.22:0/32327301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc104e90 con 0x7fe0bc074e20 2023-09-20T21:54:15.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.236+0000 7fe0c35ff700 1 -- 172.21.15.22:0/32327301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc105350 con 0x7fe0bc074e20 2023-09-20T21:54:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.242+0000 7fe0d6ff0b80 1 Processor -- start 2023-09-20T21:54:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.244+0000 7fe0d6ff0b80 1 -- start start 2023-09-20T21:54:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.244+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cb96f0 0x557225cb9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.244+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cba050 0x557225cba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.244+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cbabe0 0x557225cbb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.244+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225c2dca0 con 0x557225cb96f0 2023-09-20T21:54:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.244+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225c11c70 con 0x557225cbabe0 2023-09-20T21:54:15.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.244+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225c0f590 con 0x557225cba050 2023-09-20T21:54:15.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.247+0000 7fe0c139b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cba050 0x557225cba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.247+0000 7fe0c139b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cba050 0x557225cba5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43444/0 (socket says 172.21.15.22:43444) 2023-09-20T21:54:15.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.247+0000 7fe0c139b700 1 -- 172.21.15.22:0/298045077 learned_addr learned my addr 172.21.15.22:0/298045077 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:54:15.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.248+0000 7fe0c139b700 1 -- 172.21.15.22:0/298045077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cbabe0 msgr2=0x557225cbb060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:54:15.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.248+0000 7fe0c139b700 1 --2- 172.21.15.22:0/298045077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cbabe0 0x557225cbb060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.248+0000 7fe0c139b700 1 -- 172.21.15.22:0/298045077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cb96f0 msgr2=0x557225cb9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:15.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.248+0000 7fe0c139b700 1 --2- 172.21.15.22:0/298045077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cb96f0 0x557225cb9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.248+0000 7fe0c139b700 1 -- 172.21.15.22:0/298045077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557225a35de0 con 0x557225cba050 2023-09-20T21:54:15.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.250+0000 7fe0c139b700 1 --2- 172.21.15.22:0/298045077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cba050 0x557225cba5e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac005cc0 tx=0x7fe0ac009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=708743ca3ecbf3b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:15.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.251+0000 7fe0a77fe700 1 -- 172.21.15.22:0/298045077 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00b9d0 con 0x557225cba050 2023-09-20T21:54:15.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.251+0000 7fe0a77fe700 1 -- 172.21.15.22:0/298045077 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0ac01e040 con 0x557225cba050 2023-09-20T21:54:15.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.251+0000 7fe0a77fe700 1 -- 172.21.15.22:0/298045077 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00f460 con 0x557225cba050 2023-09-20T21:54:15.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.252+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/298045077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cba050 msgr2=0x557225cba5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:15.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.252+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/298045077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cba050 0x557225cba5e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac005cc0 tx=0x7fe0ac009ee0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.253+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/298045077 shutdown_connections 2023-09-20T21:54:15.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.253+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/298045077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cbabe0 0x557225cbb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.253+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/298045077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cb96f0 0x557225cb9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.253+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/298045077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cba050 0x557225cba5e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.253+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/298045077 >> 172.21.15.22:0/298045077 conn(0x557225be6000 msgr2=0x557225cb41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:15.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.262+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/298045077 shutdown_connections 2023-09-20T21:54:15.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.264+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/298045077 wait complete. 2023-09-20T21:54:15.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.265+0000 7fe0d6ff0b80 1 -- start start 2023-09-20T21:54:15.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.265+0000 7fe0b27fc700 1 -- 172.21.15.22:0/32327301 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe0b40148b0 con 0x7fe0bc074e20 2023-09-20T21:54:15.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.265+0000 7fe0d6ff0b80 1 Processor -- start 2023-09-20T21:54:15.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.266+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cc3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.266+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.266+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cc88f0 0x557225ccace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.266+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225c0ae80 con 0x557225cc88f0 2023-09-20T21:54:15.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.266+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225ba7740 con 0x557225cba050 2023-09-20T21:54:15.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.266+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225ba73e0 con 0x557225cb96f0 2023-09-20T21:54:15.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.266+0000 7fe0d6ff0b80 10 client.0 _ll_register_callbacks cb 0x557225af1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:54:15.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.270+0000 7fe0c1b9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cc3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.270+0000 7fe0c139b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.271+0000 7fe0c139b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:58556/0 (socket says 172.21.15.22:58556) 2023-09-20T21:54:15.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.271+0000 7fe0c139b700 1 -- 172.21.15.22:0/3510664588 learned_addr learned my addr 172.21.15.22:0/3510664588 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:54:15.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.275+0000 7fe0c139b700 1 -- 172.21.15.22:0/3510664588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 msgr2=0x557225cc3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:15.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.275+0000 7fe0c139b700 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cc3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.275+0000 7fe0c139b700 1 -- 172.21.15.22:0/3510664588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cc88f0 msgr2=0x557225ccace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:15.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.275+0000 7fe0c139b700 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cc88f0 0x557225ccace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.275+0000 7fe0c139b700 1 -- 172.21.15.22:0/3510664588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc110160 con 0x557225cba050 2023-09-20T21:54:15.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.280+0000 7fe0b27fc700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe0a80a1410 0x7fe0a80a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.280+0000 7fe0b27fc700 1 -- 172.21.15.22:0/32327301 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe0b4006e40 con 0x7fe0bc074e20 2023-09-20T21:54:15.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.291+0000 7fe0c139b700 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc38c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009990 tx=0x7fe0ac00bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34db27a8eca5ab5e server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:15.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.292+0000 7fe0a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:54:15.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.292+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/3510664588 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0050c0 con 0x557225cba050 2023-09-20T21:54:15.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.292+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/3510664588 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe0ac005220 con 0x557225cba050 2023-09-20T21:54:15.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.292+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3510664588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557225b7bbb0 con 0x557225cba050 2023-09-20T21:54:15.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.293+0000 7fe0c1b9c700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe0a80a1410 0x7fe0a80a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.300+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/3510664588 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0ac02c480 con 0x557225cba050 2023-09-20T21:54:15.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.305+0000 7fe0d6ff0b80 10 client.25913 fetch_fsmap learned FSMap version 32 2023-09-20T21:54:15.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.305+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3510664588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557225a1ec60 con 0x557225cba050 2023-09-20T21:54:15.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.314+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/3510664588 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557225b7bbb0 con 0x557225cba050 2023-09-20T21:54:15.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.314+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/3510664588 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe0ac034070 con 0x557225cba050 2023-09-20T21:54:15.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.317+0000 7fe0d6ff0b80 10 client.25913 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:54:15.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.317+0000 7fe0d6ff0b80 10 client.25913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:54:15.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.317+0000 7fe0d6ff0b80 20 client.25913 populate_metadata read hostname 'smithi022' 2023-09-20T21:54:15.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.317+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557225ccf0c0 0x557225cd14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.317+0000 7fe0d6ff0b80 4 client.25913 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:54:15.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.317+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3510664588 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557225c88020 con 0x557225ccf0c0 2023-09-20T21:54:15.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.326+0000 7fe0c0b9a700 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557225ccf0c0 0x557225cd14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.345+0000 7fe0c1b9c700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe0a80a1410 0x7fe0a80a38d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80048a0 tx=0x7fe0b8008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:15.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.365+0000 7fe0c0b9a700 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557225ccf0c0 0x557225cd14b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4bb2382e147c7fa server_cookie=5d94e88b400c016a in_seq=0 out_seq=0 2023-09-20T21:54:15.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.372+0000 7fe0a4ff9700 10 client.25913 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:54:15.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.372+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/3510664588 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557225c88020 con 0x557225ccf0c0 2023-09-20T21:54:15.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.372+0000 7fe0a4ff9700 10 client.25913 handle_command_reply: tid=0 2023-09-20T21:54:15.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.373+0000 7fe0d6ff0b80 1 client.25913 shutdown 2023-09-20T21:54:15.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.379+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3510664588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 msgr2=0x557225cc38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:15.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.379+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc38c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009990 tx=0x7fe0ac00bc20 comp rx=0 tx=0).stop 2023-09-20T21:54:15.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.379+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3510664588 shutdown_connections 2023-09-20T21:54:15.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.379+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557225ccf0c0 0x557225cd14b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.379+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cc88f0 0x557225ccace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.379+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cc3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.379+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/3510664588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc38c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.379+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3510664588 >> 172.21.15.22:0/3510664588 conn(0x557225be6000 msgr2=0x557225cb3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:15.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.381+0000 7fe0a4ff9700 0 client.25913 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:54:15.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.383+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3510664588 shutdown_connections 2023-09-20T21:54:15.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.390+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3510664588 wait complete. 2023-09-20T21:54:15.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.400+0000 7fe0d6ff0b80 20 client.25913 trim_cache size 0 max 16384 2023-09-20T21:54:15.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.446+0000 7fe0d6ff0b80 1 Processor -- start 2023-09-20T21:54:15.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.447+0000 7fe0d6ff0b80 1 -- start start 2023-09-20T21:54:15.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.448+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cdec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.448+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.448+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cc88f0 0x557225cbdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.448+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225a8b920 con 0x557225cc88f0 2023-09-20T21:54:15.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.448+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225b7bbb0 con 0x557225cba050 2023-09-20T21:54:15.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.448+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225cb5b90 con 0x557225cb96f0 2023-09-20T21:54:15.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.449+0000 7fe0c139b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cdec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.449+0000 7fe0c139b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cdec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:43464/0 (socket says 172.21.15.22:43464) 2023-09-20T21:54:15.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.449+0000 7fe0c139b700 1 -- 172.21.15.22:0/3389546024 learned_addr learned my addr 172.21.15.22:0/3389546024 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:54:15.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.450+0000 7fe0c0b9a700 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.450+0000 7fe0c1b9c700 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cc88f0 0x557225cbdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.450+0000 7fe0c139b700 1 -- 172.21.15.22:0/3389546024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 msgr2=0x557225cc1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:15.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.450+0000 7fe0c139b700 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.450+0000 7fe0c139b700 1 -- 172.21.15.22:0/3389546024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cc88f0 msgr2=0x557225cbdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:15.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.450+0000 7fe0c139b700 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cc88f0 0x557225cbdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.450+0000 7fe0c139b700 1 -- 172.21.15.22:0/3389546024 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557225a5c250 con 0x557225cb96f0 2023-09-20T21:54:15.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.451+0000 7fe0c1b9c700 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cc88f0 0x557225cbdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:54:15.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.451+0000 7fe0c0b9a700 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:54:15.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.451+0000 7fe0c139b700 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cdec60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac003790 tx=0x7fe0ac005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ca2780eca9e55b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:15.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.452+0000 7fe0a7fff700 1 -- 172.21.15.22:0/3389546024 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac02ae20 con 0x557225cb96f0 2023-09-20T21:54:15.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.452+0000 7fe0a7fff700 1 -- 172.21.15.22:0/3389546024 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0ac005940 con 0x557225cb96f0 2023-09-20T21:54:15.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.452+0000 7fe0a7fff700 1 -- 172.21.15.22:0/3389546024 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac023e40 con 0x557225cb96f0 2023-09-20T21:54:15.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.453+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3389546024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 msgr2=0x557225cdec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:15.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.453+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cdec60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac003790 tx=0x7fe0ac005e80 comp rx=0 tx=0).stop 2023-09-20T21:54:15.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.453+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3389546024 shutdown_connections 2023-09-20T21:54:15.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.453+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cc88f0 0x557225cbdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.453+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cdec60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.453+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/3389546024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cba050 0x557225cc1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.453+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3389546024 >> 172.21.15.22:0/3389546024 conn(0x557225be6000 msgr2=0x557225cd10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:15.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.453+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3389546024 shutdown_connections 2023-09-20T21:54:15.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.453+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/3389546024 wait complete. 2023-09-20T21:54:15.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.454+0000 7fe0d6ff0b80 1 -- start start 2023-09-20T21:54:15.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.454+0000 7fe0d6ff0b80 1 Processor -- start 2023-09-20T21:54:15.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.455+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cde310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.455+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cba050 0x557225cde850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.455+0000 7fe0d6ff0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cc88f0 0x557225d21a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.455+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225d21fb0 con 0x557225cba050 2023-09-20T21:54:15.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.455+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225d220f0 con 0x557225cc88f0 2023-09-20T21:54:15.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.455+0000 7fe0d6ff0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x557225d222b0 con 0x557225cb96f0 2023-09-20T21:54:15.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.455+0000 7fe0d6ff0b80 10 client.0 _ll_register_callbacks cb 0x557225af1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:54:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.455+0000 7fe0c1b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cc88f0 0x557225d21a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.455+0000 7fe0c1b9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cc88f0 0x557225d21a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:58584/0 (socket says 172.21.15.22:58584) 2023-09-20T21:54:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.455+0000 7fe0c1b9c700 1 -- 172.21.15.22:0/1008734644 learned_addr learned my addr 172.21.15.22:0/1008734644 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:54:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.456+0000 7fe0c139b700 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cde310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.456+0000 7fe0c0b9a700 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cba050 0x557225cde850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.456+0000 7fe0c1b9c700 1 -- 172.21.15.22:0/1008734644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 msgr2=0x557225cde310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.456+0000 7fe0c1b9c700 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cde310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.456+0000 7fe0c1b9c700 1 -- 172.21.15.22:0/1008734644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cba050 msgr2=0x557225cde850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.456+0000 7fe0c1b9c700 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cba050 0x557225cde850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.456+0000 7fe0c1b9c700 1 -- 172.21.15.22:0/1008734644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b40cb040 con 0x557225cc88f0 2023-09-20T21:54:15.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.456+0000 7fe0c139b700 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cde310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:54:15.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.456+0000 7fe0c1b9c700 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cc88f0 0x557225d21a70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8010fe0 tx=0x7fe0b8009110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f66bc1fe611adc server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:15.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.457+0000 7fe0a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:54:15.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.457+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/1008734644 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8015e60 con 0x557225cc88f0 2023-09-20T21:54:15.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.457+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/1008734644 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe0b8016420 con 0x557225cc88f0 2023-09-20T21:54:15.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.457+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/1008734644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0ac01e040 con 0x557225cc88f0 2023-09-20T21:54:15.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.458+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/1008734644 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0b8005a50 con 0x557225cc88f0 2023-09-20T21:54:15.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.458+0000 7fe0d6ff0b80 10 client.25919 fetch_fsmap learned FSMap version 32 2023-09-20T21:54:15.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.458+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/1008734644 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557225a1ec60 con 0x557225cc88f0 2023-09-20T21:54:15.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.458+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/1008734644 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0ac01e040 con 0x557225cc88f0 2023-09-20T21:54:15.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.458+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/1008734644 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe0b8006020 con 0x557225cc88f0 2023-09-20T21:54:15.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.459+0000 7fe0d6ff0b80 10 client.25919 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:54:15.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.459+0000 7fe0d6ff0b80 10 client.25919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:54:15.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.459+0000 7fe0d6ff0b80 20 client.25919 populate_metadata read hostname 'smithi022' 2023-09-20T21:54:15.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.459+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557225d23d10 0x557225d26130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:15.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.459+0000 7fe0d6ff0b80 4 client.25919 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:54:15.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.459+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/1008734644 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557225c03e60 con 0x557225d23d10 2023-09-20T21:54:15.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.459+0000 7fe0c139b700 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557225d23d10 0x557225d26130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:15.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.460+0000 7fe0c139b700 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557225d23d10 0x557225d26130 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c88db6c67d81c716 server_cookie=b354a99a294599a7 in_seq=0 out_seq=0 2023-09-20T21:54:15.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.460+0000 7fe0a4ff9700 10 client.25919 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:54:15.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.460+0000 7fe0a4ff9700 1 -- 172.21.15.22:0/1008734644 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557225c03e60 con 0x557225d23d10 2023-09-20T21:54:15.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.460+0000 7fe0a4ff9700 10 client.25919 handle_command_reply: tid=0 2023-09-20T21:54:15.467 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:54:15.467 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:54:15.467 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:54:15.467 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:54:15.468 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:54:15.468 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:54:15.468 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:54:15.468 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:54:15.468 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:54:15.469 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:54:15.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.460+0000 7fe0d6ff0b80 1 client.25919 shutdown 2023-09-20T21:54:15.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.460+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/1008734644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cc88f0 msgr2=0x557225d21a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:15.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.460+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cc88f0 0x557225d21a70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8010fe0 tx=0x7fe0b8009110 comp rx=0 tx=0).stop 2023-09-20T21:54:15.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.461+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/1008734644 shutdown_connections 2023-09-20T21:54:15.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.461+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x557225d23d10 0x557225d26130 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.461+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557225cc88f0 0x557225d21a70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.461+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557225cb96f0 0x557225cde310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.461+0000 7fe0d6ff0b80 1 --2- 172.21.15.22:0/1008734644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557225cba050 0x557225cde850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.461+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/1008734644 >> 172.21.15.22:0/1008734644 conn(0x557225be6000 msgr2=0x557225cbdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:15.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.461+0000 7fe0a4ff9700 0 client.25919 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:54:15.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.461+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/1008734644 shutdown_connections 2023-09-20T21:54:15.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.461+0000 7fe0d6ff0b80 1 -- 172.21.15.22:0/1008734644 wait complete. 2023-09-20T21:54:15.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.462+0000 7fe0d6ff0b80 20 client.25919 trim_cache size 0 max 16384 2023-09-20T21:54:15.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.463+0000 7fe0c35ff700 1 -- 172.21.15.22:0/32327301 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe0a80a1410 msgr2=0x7fe0a80a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:15.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.463+0000 7fe0c35ff700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe0a80a1410 0x7fe0a80a38d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80048a0 tx=0x7fe0b8008040 comp rx=0 tx=0).stop 2023-09-20T21:54:15.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.463+0000 7fe0c35ff700 1 -- 172.21.15.22:0/32327301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0bc074e20 msgr2=0x7fe0bc06fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:15.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.463+0000 7fe0c35ff700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0bc074e20 0x7fe0bc06fbe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400b9c0 tx=0x7fe0b400bcd0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.463+0000 7fe0c35ff700 1 -- 172.21.15.22:0/32327301 shutdown_connections 2023-09-20T21:54:15.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.463+0000 7fe0c35ff700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe0a80a1410 0x7fe0a80a38d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.463+0000 7fe0c35ff700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe0bc10d3c0 0x7fe0bc1ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.463+0000 7fe0c35ff700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0bc073840 0x7fe0bc06f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.463+0000 7fe0c35ff700 1 --2- 172.21.15.22:0/32327301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0bc074e20 0x7fe0bc06fbe0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:15.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.463+0000 7fe0c35ff700 1 -- 172.21.15.22:0/32327301 >> 172.21.15.22:0/32327301 conn(0x7fe0bc0fc360 msgr2=0x7fe0bc0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:15.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.471+0000 7fe0c35ff700 1 -- 172.21.15.22:0/32327301 shutdown_connections 2023-09-20T21:54:15.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:15.472+0000 7fe0c35ff700 1 -- 172.21.15.22:0/32327301 wait complete. 2023-09-20T21:54:15.518 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:54:15.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:15 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:54:15.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:15 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:54:15.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:15 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:54:15.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:15 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:54:15.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:15 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:54:15.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:15 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:54:15.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:15 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:54:15.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:15 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:54:15.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:15 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:54:16.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:16 smithi089 ceph-mon[134090]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 499 op/s 2023-09-20T21:54:16.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:16 smithi022 ceph-mon[97651]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 499 op/s 2023-09-20T21:54:16.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:16 smithi123 ceph-mon[134433]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 499 op/s 2023-09-20T21:54:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 295.0 s, 588.0 tps, lat 172.313 ms stddev 181.032 2023-09-20T21:54:18.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:18 smithi123 ceph-mon[134433]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2023-09-20T21:54:18.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:18 smithi089 ceph-mon[134090]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2023-09-20T21:54:18.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:18 smithi022 ceph-mon[97651]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 MiB/s wr, 621 op/s 2023-09-20T21:54:20.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:20 smithi123 ceph-mon[134433]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 537 op/s 2023-09-20T21:54:20.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:20 smithi089 ceph-mon[134090]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 537 op/s 2023-09-20T21:54:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:20 smithi022 ceph-mon[97651]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 537 op/s 2023-09-20T21:54:22.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:22 smithi123 ceph-mon[134433]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 594 op/s 2023-09-20T21:54:22.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:22 smithi089 ceph-mon[134090]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 594 op/s 2023-09-20T21:54:22.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:22 smithi022 ceph-mon[97651]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 594 op/s 2023-09-20T21:54:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 300.0 s, 550.8 tps, lat 174.273 ms stddev 176.777 2023-09-20T21:54:24.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:23 smithi022 ceph-mon[97651]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 535 op/s 2023-09-20T21:54:24.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:23 smithi123 ceph-mon[134433]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 535 op/s 2023-09-20T21:54:24.365 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:23 smithi089 ceph-mon[134090]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 535 op/s 2023-09-20T21:54:26.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:26 smithi123 ceph-mon[134433]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 555 op/s 2023-09-20T21:54:26.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:26 smithi089 ceph-mon[134090]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 555 op/s 2023-09-20T21:54:27.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:26 smithi022 ceph-mon[97651]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 555 op/s 2023-09-20T21:54:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 305.0 s, 505.2 tps, lat 207.038 ms stddev 229.230 2023-09-20T21:54:28.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:28 smithi123 ceph-mon[134433]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 646 op/s 2023-09-20T21:54:28.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:28 smithi089 ceph-mon[134090]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 646 op/s 2023-09-20T21:54:29.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:28 smithi022 ceph-mon[97651]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 646 op/s 2023-09-20T21:54:30.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:30 smithi123 ceph-mon[134433]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 598 op/s 2023-09-20T21:54:30.865 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:30 smithi089 ceph-mon[134090]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 598 op/s 2023-09-20T21:54:30.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:30 smithi022 ceph-mon[97651]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 598 op/s 2023-09-20T21:54:32.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:32 smithi123 ceph-mon[134433]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 685 op/s 2023-09-20T21:54:32.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:32 smithi089 ceph-mon[134090]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 685 op/s 2023-09-20T21:54:33.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:32 smithi022 ceph-mon[97651]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 685 op/s 2023-09-20T21:54:33.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 310.0 s, 559.7 tps, lat 176.347 ms stddev 201.537 2023-09-20T21:54:34.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:33 smithi022 ceph-mon[97651]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 587 op/s 2023-09-20T21:54:34.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:33 smithi123 ceph-mon[134433]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 587 op/s 2023-09-20T21:54:34.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:33 smithi089 ceph-mon[134090]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 587 op/s 2023-09-20T21:54:36.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:36 smithi123 ceph-mon[134433]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 596 op/s 2023-09-20T21:54:36.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:36 smithi089 ceph-mon[134090]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 596 op/s 2023-09-20T21:54:36.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:36 smithi022 ceph-mon[97651]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 596 op/s 2023-09-20T21:54:38.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 315.0 s, 716.3 tps, lat 141.771 ms stddev 160.817 2023-09-20T21:54:38.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:38 smithi123 ceph-mon[134433]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.9 MiB/s wr, 667 op/s 2023-09-20T21:54:38.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:38 smithi089 ceph-mon[134090]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.9 MiB/s wr, 667 op/s 2023-09-20T21:54:38.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:38 smithi022 ceph-mon[97651]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.9 MiB/s wr, 667 op/s 2023-09-20T21:54:40.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:40 smithi123 ceph-mon[134433]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 623 op/s 2023-09-20T21:54:40.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:40 smithi089 ceph-mon[134090]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 623 op/s 2023-09-20T21:54:40.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:40 smithi022 ceph-mon[97651]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 623 op/s 2023-09-20T21:54:42.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:42 smithi089 ceph-mon[134090]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 636 op/s 2023-09-20T21:54:42.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:42 smithi022 ceph-mon[97651]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 636 op/s 2023-09-20T21:54:43.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:42 smithi123 ceph-mon[134433]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 636 op/s 2023-09-20T21:54:43.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 320.0 s, 507.7 tps, lat 192.533 ms stddev 215.711 2023-09-20T21:54:44.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:43 smithi022 ceph-mon[97651]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.5 MiB/s wr, 521 op/s 2023-09-20T21:54:44.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:43 smithi123 ceph-mon[134433]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.5 MiB/s wr, 521 op/s 2023-09-20T21:54:44.368 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:43 smithi089 ceph-mon[134090]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.5 MiB/s wr, 521 op/s 2023-09-20T21:54:45.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:54:45.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.976+0000 7f37c39ff700 1 -- 172.21.15.22:0/1567144812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc0ff6e0 msgr2=0x7f37bc0ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:45.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.976+0000 7f37c39ff700 1 --2- 172.21.15.22:0/1567144812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc0ff6e0 0x7f37bc0ffb00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009990 tx=0x7f37ac009ca0 comp rx=0 tx=0).stop 2023-09-20T21:54:45.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.977+0000 7f37c39ff700 1 -- 172.21.15.22:0/1567144812 shutdown_connections 2023-09-20T21:54:45.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.977+0000 7f37c39ff700 1 --2- 172.21.15.22:0/1567144812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc100a00 0x7f37bc100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:45.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.977+0000 7f37c39ff700 1 --2- 172.21.15.22:0/1567144812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc0ff6e0 0x7f37bc0ffb00 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009990 tx=0x7f37ac009ca0 comp rx=0 tx=0).stop 2023-09-20T21:54:45.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.977+0000 7f37c39ff700 1 --2- 172.21.15.22:0/1567144812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37bc100040 0x7f37bc1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:45.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.977+0000 7f37c39ff700 1 -- 172.21.15.22:0/1567144812 >> 172.21.15.22:0/1567144812 conn(0x7f37bc0fb240 msgr2=0x7f37bc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:45.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.978+0000 7f37c39ff700 1 -- 172.21.15.22:0/1567144812 shutdown_connections 2023-09-20T21:54:45.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.978+0000 7f37c39ff700 1 -- 172.21.15.22:0/1567144812 wait complete. 2023-09-20T21:54:45.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.980+0000 7f37c39ff700 1 Processor -- start 2023-09-20T21:54:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.980+0000 7f37c39ff700 1 -- start start 2023-09-20T21:54:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.980+0000 7f37c39ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc100040 0x7f37bc105b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.980+0000 7f37c39ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc100a00 0x7f37bc106080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.981+0000 7f37c39ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37bc10b0e0 0x7f37bc10b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.981+0000 7f37c39ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc106660 con 0x7f37bc10b0e0 2023-09-20T21:54:45.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.981+0000 7f37c39ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc1067a0 con 0x7f37bc100040 2023-09-20T21:54:45.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.981+0000 7f37c39ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc1ade60 con 0x7f37bc100a00 2023-09-20T21:54:45.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.981+0000 7f37c1f9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37bc10b0e0 0x7f37bc10b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:45.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.981+0000 7f37c1f9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37bc10b0e0 0x7f37bc10b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47736/0 (socket says 172.21.15.22:47736) 2023-09-20T21:54:45.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.981+0000 7f37c1f9c700 1 -- 172.21.15.22:0/3554755410 learned_addr learned my addr 172.21.15.22:0/3554755410 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:54:45.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.981+0000 7f37c0f9a700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc100a00 0x7f37bc106080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:45.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.982+0000 7f37c1f9c700 1 -- 172.21.15.22:0/3554755410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc100a00 msgr2=0x7f37bc106080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:45.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.982+0000 7f37c1f9c700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc100a00 0x7f37bc106080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:45.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.982+0000 7f37c1f9c700 1 -- 172.21.15.22:0/3554755410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc100040 msgr2=0x7f37bc105b40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-20T21:54:45.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.982+0000 7f37c1f9c700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc100040 0x7f37bc105b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:45.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.982+0000 7f37c1f9c700 1 -- 172.21.15.22:0/3554755410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0095f0 con 0x7f37bc10b0e0 2023-09-20T21:54:45.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.982+0000 7f37c1f9c700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37bc10b0e0 0x7f37bc10b560 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f37b400b780 tx=0x7f37b400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:45.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.983+0000 7f37b27fc700 1 -- 172.21.15.22:0/3554755410 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4010860 con 0x7f37bc10b0e0 2023-09-20T21:54:45.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.983+0000 7f37b27fc700 1 -- 172.21.15.22:0/3554755410 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b4010e80 con 0x7f37bc10b0e0 2023-09-20T21:54:45.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.983+0000 7f37c39ff700 1 -- 172.21.15.22:0/3554755410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc1ae140 con 0x7f37bc10b0e0 2023-09-20T21:54:45.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.983+0000 7f37c39ff700 1 -- 172.21.15.22:0/3554755410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc1ae6d0 con 0x7f37bc10b0e0 2023-09-20T21:54:45.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.984+0000 7f37b27fc700 1 -- 172.21.15.22:0/3554755410 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4007370 con 0x7f37bc10b0e0 2023-09-20T21:54:45.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.990+0000 7f37b27fc700 1 -- 172.21.15.22:0/3554755410 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f37b40109c0 con 0x7f37bc10b0e0 2023-09-20T21:54:45.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.991+0000 7f37d740fb80 1 Processor -- start 2023-09-20T21:54:45.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.991+0000 7f37d740fb80 1 -- start start 2023-09-20T21:54:45.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.992+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66517b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:45.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.992+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 0x556d665185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:45.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.992+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66519060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:45.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.992+0000 7f37d740fb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6648bca0 con 0x556d66518050 2023-09-20T21:54:45.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.992+0000 7f37d740fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6646fc70 con 0x556d665176f0 2023-09-20T21:54:45.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.992+0000 7f37d740fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d6646d590 con 0x556d66518be0 2023-09-20T21:54:45.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.992+0000 7f37c0f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66517b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:45.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.992+0000 7f37c0f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66517b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:53506/0 (socket says 172.21.15.22:53506) 2023-09-20T21:54:45.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.992+0000 7f37c0f9a700 1 -- 172.21.15.22:0/253555324 learned_addr learned my addr 172.21.15.22:0/253555324 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:54:45.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.992+0000 7f37c0f9a700 1 --2- 172.21.15.22:0/253555324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66519060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.993+0000 7f37c0f9a700 1 -- 172.21.15.22:0/253555324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 msgr2=0x556d66519060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.993+0000 7f37c0f9a700 1 --2- 172.21.15.22:0/253555324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66519060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.993+0000 7f37c0f9a700 1 -- 172.21.15.22:0/253555324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 msgr2=0x556d665185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.993+0000 7f37c0f9a700 1 --2- 172.21.15.22:0/253555324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 0x556d665185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.993+0000 7f37c0f9a700 1 -- 172.21.15.22:0/253555324 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d66293de0 con 0x556d665176f0 2023-09-20T21:54:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.994+0000 7f37c0f9a700 1 --2- 172.21.15.22:0/253555324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66517b10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37b8017700 tx=0x7f37b80095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b19492105bbc675 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.994+0000 7f37b27fc700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f37a80a1480 0x7f37a80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.994+0000 7f37b27fc700 1 -- 172.21.15.22:0/3554755410 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f37b40ccc40 con 0x7f37bc10b0e0 2023-09-20T21:54:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.995+0000 7f37a77fe700 1 -- 172.21.15.22:0/253555324 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8017d20 con 0x556d665176f0 2023-09-20T21:54:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.995+0000 7f37a77fe700 1 -- 172.21.15.22:0/253555324 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b8017e80 con 0x556d665176f0 2023-09-20T21:54:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.995+0000 7f37a77fe700 1 -- 172.21.15.22:0/253555324 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b80135c0 con 0x556d665176f0 2023-09-20T21:54:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.996+0000 7f37d740fb80 1 -- 172.21.15.22:0/253555324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 msgr2=0x556d66517b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.996+0000 7f37d740fb80 1 --2- 172.21.15.22:0/253555324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66517b10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37b8017700 tx=0x7f37b80095b0 comp rx=0 tx=0).stop 2023-09-20T21:54:45.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.996+0000 7f37c179b700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f37a80a1480 0x7f37a80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:45.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.998+0000 7f37d740fb80 1 -- 172.21.15.22:0/253555324 shutdown_connections 2023-09-20T21:54:45.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.998+0000 7f37d740fb80 1 --2- 172.21.15.22:0/253555324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66519060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:45.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.998+0000 7f37d740fb80 1 --2- 172.21.15.22:0/253555324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66517b10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.998+0000 7f37d740fb80 1 --2- 172.21.15.22:0/253555324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 0x556d665185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.998+0000 7f37d740fb80 1 -- 172.21.15.22:0/253555324 >> 172.21.15.22:0/253555324 conn(0x556d66444000 msgr2=0x556d665121d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:46.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:45.999+0000 7f37d740fb80 1 -- 172.21.15.22:0/253555324 shutdown_connections 2023-09-20T21:54:46.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.000+0000 7f37c179b700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f37a80a1480 0x7f37a80a3940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x556d66513010 tx=0x7f37ac005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:46.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.000+0000 7f37d740fb80 1 -- 172.21.15.22:0/253555324 wait complete. 2023-09-20T21:54:46.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.000+0000 7f37d740fb80 1 -- start start 2023-09-20T21:54:46.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.001+0000 7f37d740fb80 1 Processor -- start 2023-09-20T21:54:46.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.002+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66521200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.002+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 0x556d66521740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.002+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66528780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.002+0000 7f37d740fb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66468e80 con 0x556d66518050 2023-09-20T21:54:46.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.002+0000 7f37d740fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66405740 con 0x556d665176f0 2023-09-20T21:54:46.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.002+0000 7f37d740fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d664053e0 con 0x556d66518be0 2023-09-20T21:54:46.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.002+0000 7f37d740fb80 10 client.0 _ll_register_callbacks cb 0x556d6634fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:54:46.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.002+0000 7f37c179b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66521200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:46.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.003+0000 7f37c179b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66521200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:53518/0 (socket says 172.21.15.22:53518) 2023-09-20T21:54:46.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.003+0000 7f37c179b700 1 -- 172.21.15.22:0/3188665635 learned_addr learned my addr 172.21.15.22:0/3188665635 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:54:46.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.003+0000 7f37c179b700 1 -- 172.21.15.22:0/3188665635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 msgr2=0x556d66528780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:54:46.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.003+0000 7f37c179b700 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66528780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.003+0000 7f37c179b700 1 -- 172.21.15.22:0/3188665635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 msgr2=0x556d66521740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:46.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.003+0000 7f37c179b700 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 0x556d66521740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.003+0000 7f37c179b700 1 -- 172.21.15.22:0/3188665635 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b80173e0 con 0x556d665176f0 2023-09-20T21:54:46.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.004+0000 7f37c179b700 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66521200 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f37ac005e50 tx=0x7f37ac009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=187bd1fb703ebff7 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:46.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.004+0000 7f37a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:54:46.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.005+0000 7f37a4ff9700 1 -- 172.21.15.22:0/3188665635 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac029070 con 0x556d665176f0 2023-09-20T21:54:46.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.005+0000 7f37a4ff9700 1 -- 172.21.15.22:0/3188665635 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f37ac0056c0 con 0x556d665176f0 2023-09-20T21:54:46.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.005+0000 7f37d740fb80 1 -- 172.21.15.22:0/3188665635 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d664e6020 con 0x556d665176f0 2023-09-20T21:54:46.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.005+0000 7f37a4ff9700 1 -- 172.21.15.22:0/3188665635 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37ac01d070 con 0x556d665176f0 2023-09-20T21:54:46.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.011+0000 7f37d740fb80 10 client.25928 fetch_fsmap learned FSMap version 32 2023-09-20T21:54:46.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.011+0000 7f37d740fb80 1 -- 172.21.15.22:0/3188665635 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556d6627cc60 con 0x556d665176f0 2023-09-20T21:54:46.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.022+0000 7f37a4ff9700 1 -- 172.21.15.22:0/3188665635 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d664e6020 con 0x556d665176f0 2023-09-20T21:54:46.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.022+0000 7f37a4ff9700 1 -- 172.21.15.22:0/3188665635 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f37ac00f460 con 0x556d665176f0 2023-09-20T21:54:46.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.022+0000 7f37d740fb80 10 client.25928 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:54:46.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.022+0000 7f37d740fb80 10 client.25928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:54:46.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.022+0000 7f37d740fb80 20 client.25928 populate_metadata read hostname 'smithi022' 2023-09-20T21:54:46.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.022+0000 7f37d740fb80 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x556d6652d030 0x556d6652f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.022+0000 7f37d740fb80 4 client.25928 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:54:46.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.022+0000 7f37d740fb80 1 -- 172.21.15.22:0/3188665635 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d66461e60 con 0x556d6652d030 2023-09-20T21:54:46.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.037+0000 7f37c0f9a700 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x556d6652d030 0x556d6652f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:46.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.059+0000 7f37c0f9a700 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x556d6652d030 0x556d6652f470 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9879a9a2552ddb9 server_cookie=2c2cc358ace771b1 in_seq=0 out_seq=0 2023-09-20T21:54:46.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.060+0000 7f37a4ff9700 10 client.25928 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:54:46.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.071+0000 7f37a4ff9700 1 -- 172.21.15.22:0/3188665635 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556d66461e60 con 0x556d6652d030 2023-09-20T21:54:46.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.071+0000 7f37a4ff9700 10 client.25928 handle_command_reply: tid=0 2023-09-20T21:54:46.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.071+0000 7f37d740fb80 1 client.25928 shutdown 2023-09-20T21:54:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.076+0000 7f37d740fb80 1 -- 172.21.15.22:0/3188665635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 msgr2=0x556d66521200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:46.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.076+0000 7f37d740fb80 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66521200 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f37ac005e50 tx=0x7f37ac009310 comp rx=0 tx=0).stop 2023-09-20T21:54:46.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.090+0000 7f37d740fb80 1 -- 172.21.15.22:0/3188665635 shutdown_connections 2023-09-20T21:54:46.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.090+0000 7f37d740fb80 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x556d6652d030 0x556d6652f470 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.090+0000 7f37d740fb80 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66528780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.090+0000 7f37d740fb80 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d66521200 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.090+0000 7f37d740fb80 1 --2- 172.21.15.22:0/3188665635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 0x556d66521740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.090+0000 7f37d740fb80 1 -- 172.21.15.22:0/3188665635 >> 172.21.15.22:0/3188665635 conn(0x556d66444000 msgr2=0x556d66511880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:46.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.092+0000 7f37a4ff9700 0 client.25928 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:54:46.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.114+0000 7f37d740fb80 1 -- 172.21.15.22:0/3188665635 shutdown_connections 2023-09-20T21:54:46.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.121+0000 7f37d740fb80 1 -- 172.21.15.22:0/3188665635 wait complete. 2023-09-20T21:54:46.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.132+0000 7f37d740fb80 20 client.25928 trim_cache size 0 max 16384 2023-09-20T21:54:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.173+0000 7f37d740fb80 1 Processor -- start 2023-09-20T21:54:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.173+0000 7f37d740fb80 1 -- start start 2023-09-20T21:54:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.173+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d6651fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.173+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 0x556d66528710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.173+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d6651bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.173+0000 7f37d740fb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d662e9920 con 0x556d66518050 2023-09-20T21:54:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.173+0000 7f37d740fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d664e6050 con 0x556d665176f0 2023-09-20T21:54:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.173+0000 7f37d740fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d665223f0 con 0x556d66518be0 2023-09-20T21:54:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.174+0000 7f37c1f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d6651bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.174+0000 7f37c1f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d6651bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:35930/0 (socket says 172.21.15.22:35930) 2023-09-20T21:54:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.174+0000 7f37c1f9c700 1 -- 172.21.15.22:0/991198240 learned_addr learned my addr 172.21.15.22:0/991198240 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:54:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.174+0000 7f37c0f9a700 1 --2- 172.21.15.22:0/991198240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 0x556d66528710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.174+0000 7f37c179b700 1 --2- 172.21.15.22:0/991198240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d6651fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:46.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.175+0000 7f37c1f9c700 1 -- 172.21.15.22:0/991198240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 msgr2=0x556d6651fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:46.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.175+0000 7f37c1f9c700 1 --2- 172.21.15.22:0/991198240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d6651fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.175+0000 7f37c1f9c700 1 -- 172.21.15.22:0/991198240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 msgr2=0x556d66528710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:46.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.175+0000 7f37c1f9c700 1 --2- 172.21.15.22:0/991198240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 0x556d66528710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.175+0000 7f37c1f9c700 1 -- 172.21.15.22:0/991198240 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d662ba250 con 0x556d66518be0 2023-09-20T21:54:46.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.176+0000 7f37c1f9c700 1 --2- 172.21.15.22:0/991198240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d6651bc00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37b40960e0 tx=0x7f37b4099610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be46417bf3d43235 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:46.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.176+0000 7f37a7fff700 1 -- 172.21.15.22:0/991198240 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4099be0 con 0x556d66518be0 2023-09-20T21:54:46.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.176+0000 7f37a7fff700 1 -- 172.21.15.22:0/991198240 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b409da10 con 0x556d66518be0 2023-09-20T21:54:46.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.176+0000 7f37a7fff700 1 -- 172.21.15.22:0/991198240 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4099330 con 0x556d66518be0 2023-09-20T21:54:46.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.177+0000 7f37d740fb80 1 -- 172.21.15.22:0/991198240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 msgr2=0x556d6651bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:46.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.177+0000 7f37d740fb80 1 --2- 172.21.15.22:0/991198240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d6651bc00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37b40960e0 tx=0x7f37b4099610 comp rx=0 tx=0).stop 2023-09-20T21:54:46.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.178+0000 7f37d740fb80 1 -- 172.21.15.22:0/991198240 shutdown_connections 2023-09-20T21:54:46.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.178+0000 7f37d740fb80 1 --2- 172.21.15.22:0/991198240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d6651bc00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.178+0000 7f37d740fb80 1 --2- 172.21.15.22:0/991198240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d665176f0 0x556d6651fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.178+0000 7f37d740fb80 1 --2- 172.21.15.22:0/991198240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d66518050 0x556d66528710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.178+0000 7f37d740fb80 1 -- 172.21.15.22:0/991198240 >> 172.21.15.22:0/991198240 conn(0x556d66444000 msgr2=0x556d6652f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:46.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.178+0000 7f37d740fb80 1 -- 172.21.15.22:0/991198240 shutdown_connections 2023-09-20T21:54:46.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.178+0000 7f37d740fb80 1 -- 172.21.15.22:0/991198240 wait complete. 2023-09-20T21:54:46.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.179+0000 7f37d740fb80 1 -- start start 2023-09-20T21:54:46.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.179+0000 7f37d740fb80 1 Processor -- start 2023-09-20T21:54:46.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.180+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d665176f0 0x556d665245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.180+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d66518050 0x556d66524b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.180+0000 7f37d740fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66527b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.180+0000 7f37d740fb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d66528080 con 0x556d665176f0 2023-09-20T21:54:46.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.180+0000 7f37d740fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d665281f0 con 0x556d66518050 2023-09-20T21:54:46.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.180+0000 7f37d740fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d665283b0 con 0x556d66518be0 2023-09-20T21:54:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.180+0000 7f37d740fb80 10 client.0 _ll_register_callbacks cb 0x556d6634fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:54:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.180+0000 7f37c0f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d66518050 0x556d66524b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.180+0000 7f37c0f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d66518050 0x556d66524b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:53538/0 (socket says 172.21.15.22:53538) 2023-09-20T21:54:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.180+0000 7f37c0f9a700 1 -- 172.21.15.22:0/344177288 learned_addr learned my addr 172.21.15.22:0/344177288 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:54:46.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.181+0000 7f37c1f9c700 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66527b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.181+0000 7f37c1f9c700 1 -- 172.21.15.22:0/344177288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d66518050 msgr2=0x556d66524b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.181+0000 7f37c1f9c700 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d66518050 0x556d66524b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.181+0000 7f37c1f9c700 1 -- 172.21.15.22:0/344177288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d665176f0 msgr2=0x556d665245e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:54:46.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.181+0000 7f37c1f9c700 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d665176f0 0x556d665245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.181+0000 7f37c1f9c700 1 -- 172.21.15.22:0/344177288 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc1ae6d0 con 0x556d66518be0 2023-09-20T21:54:46.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.182+0000 7f37c1f9c700 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66527b40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f37b40099e0 tx=0x7f37b40a1d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d53bad6f2d0b8f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:54:46.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.182+0000 7f37a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:54:46.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.182+0000 7f37a4ff9700 1 -- 172.21.15.22:0/344177288 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b40a1ea0 con 0x556d66518be0 2023-09-20T21:54:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.182+0000 7f37a4ff9700 1 -- 172.21.15.22:0/344177288 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f37b40a17c0 con 0x556d66518be0 2023-09-20T21:54:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.182+0000 7f37d740fb80 1 -- 172.21.15.22:0/344177288 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d663d9bb0 con 0x556d66518be0 2023-09-20T21:54:46.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.183+0000 7f37a4ff9700 1 -- 172.21.15.22:0/344177288 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b40a1980 con 0x556d66518be0 2023-09-20T21:54:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.184+0000 7f37d740fb80 10 client.25936 fetch_fsmap learned FSMap version 32 2023-09-20T21:54:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.184+0000 7f37d740fb80 1 -- 172.21.15.22:0/344177288 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556d662922b0 con 0x556d66518be0 2023-09-20T21:54:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.184+0000 7f37a4ff9700 1 -- 172.21.15.22:0/344177288 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d663d9bb0 con 0x556d66518be0 2023-09-20T21:54:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.184+0000 7f37a4ff9700 1 -- 172.21.15.22:0/344177288 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f37b409e750 con 0x556d66518be0 2023-09-20T21:54:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.184+0000 7f37d740fb80 10 client.25936 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:54:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.184+0000 7f37d740fb80 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:54:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.184+0000 7f37d740fb80 20 client.25936 populate_metadata read hostname 'smithi022' 2023-09-20T21:54:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.184+0000 7f37d740fb80 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x556d66592ba0 0x556d66594fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:54:46.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.184+0000 7f37d740fb80 4 client.25936 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:54:46.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.184+0000 7f37d740fb80 1 -- 172.21.15.22:0/344177288 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556d66461b80 con 0x556d66592ba0 2023-09-20T21:54:46.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.185+0000 7f37c179b700 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x556d66592ba0 0x556d66594fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:54:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.186+0000 7f37a4ff9700 10 client.25936 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:54:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.186+0000 7f37c179b700 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x556d66592ba0 0x556d66594fc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d10a7168c77b73 server_cookie=9b8d2f0235e04953 in_seq=0 out_seq=0 2023-09-20T21:54:46.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.187+0000 7f37a4ff9700 1 -- 172.21.15.22:0/344177288 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556d66461b80 con 0x556d66592ba0 2023-09-20T21:54:46.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.187+0000 7f37a4ff9700 10 client.25936 handle_command_reply: tid=0 2023-09-20T21:54:46.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.187+0000 7f37d740fb80 1 client.25936 shutdown 2023-09-20T21:54:46.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.187+0000 7f37d740fb80 1 -- 172.21.15.22:0/344177288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 msgr2=0x556d66527b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:46.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37d740fb80 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66527b40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f37b40099e0 tx=0x7f37b40a1d00 comp rx=0 tx=0).stop 2023-09-20T21:54:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37d740fb80 1 -- 172.21.15.22:0/344177288 shutdown_connections 2023-09-20T21:54:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37d740fb80 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x556d66592ba0 0x556d66594fc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37d740fb80 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d66518be0 0x556d66527b40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37d740fb80 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d665176f0 0x556d665245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37d740fb80 1 --2- 172.21.15.22:0/344177288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556d66518050 0x556d66524b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37d740fb80 1 -- 172.21.15.22:0/344177288 >> 172.21.15.22:0/344177288 conn(0x556d66444000 msgr2=0x556d6651fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37a4ff9700 0 client.25936 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:54:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37d740fb80 1 -- 172.21.15.22:0/344177288 shutdown_connections 2023-09-20T21:54:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37d740fb80 1 -- 172.21.15.22:0/344177288 wait complete. 2023-09-20T21:54:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.188+0000 7f37d740fb80 20 client.25936 trim_cache size 0 max 16384 2023-09-20T21:54:46.201 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:54:46.201 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:54:46.201 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:54:46.202 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:54:46.202 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:54:46.202 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:54:46.202 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:54:46.203 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:54:46.203 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:54:46.203 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:54:46.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.191+0000 7f37c39ff700 1 -- 172.21.15.22:0/3554755410 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f37a80a1480 msgr2=0x7f37a80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:46.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.191+0000 7f37c39ff700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f37a80a1480 0x7f37a80a3940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x556d66513010 tx=0x7f37ac005fd0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.191+0000 7f37c39ff700 1 -- 172.21.15.22:0/3554755410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37bc10b0e0 msgr2=0x7f37bc10b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:54:46.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.191+0000 7f37c39ff700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37bc10b0e0 0x7f37bc10b560 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f37b400b780 tx=0x7f37b400ba90 comp rx=0 tx=0).stop 2023-09-20T21:54:46.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.192+0000 7f37c39ff700 1 -- 172.21.15.22:0/3554755410 shutdown_connections 2023-09-20T21:54:46.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.192+0000 7f37c39ff700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37bc10b0e0 0x7f37bc10b560 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.192+0000 7f37c39ff700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f37a80a1480 0x7f37a80a3940 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.192+0000 7f37c39ff700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37bc100040 0x7f37bc105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.192+0000 7f37c39ff700 1 --2- 172.21.15.22:0/3554755410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37bc100a00 0x7f37bc106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:54:46.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.192+0000 7f37c39ff700 1 -- 172.21.15.22:0/3554755410 >> 172.21.15.22:0/3554755410 conn(0x7f37bc0fb240 msgr2=0x7f37bc0fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:54:46.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.192+0000 7f37c39ff700 1 -- 172.21.15.22:0/3554755410 shutdown_connections 2023-09-20T21:54:46.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:54:46.192+0000 7f37c39ff700 1 -- 172.21.15.22:0/3554755410 wait complete. 2023-09-20T21:54:46.225 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:54:46.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:46 smithi123 ceph-mon[134433]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.3 MiB/s wr, 534 op/s 2023-09-20T21:54:46.865 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:46 smithi089 ceph-mon[134090]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.3 MiB/s wr, 534 op/s 2023-09-20T21:54:46.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:46 smithi022 ceph-mon[97651]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.3 MiB/s wr, 534 op/s 2023-09-20T21:54:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 325.0 s, 596.9 tps, lat 168.177 ms stddev 177.448 2023-09-20T21:54:48.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:48 smithi089 ceph-mon[134090]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 578 op/s 2023-09-20T21:54:48.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:48 smithi022 ceph-mon[97651]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 578 op/s 2023-09-20T21:54:49.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:48 smithi123 ceph-mon[134433]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 578 op/s 2023-09-20T21:54:50.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:50 smithi089 ceph-mon[134090]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.2 MiB/s wr, 549 op/s 2023-09-20T21:54:51.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:50 smithi022 ceph-mon[97651]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.2 MiB/s wr, 549 op/s 2023-09-20T21:54:51.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:50 smithi123 ceph-mon[134433]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.2 MiB/s wr, 549 op/s 2023-09-20T21:54:52.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:52 smithi022 ceph-mon[97651]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 577 op/s 2023-09-20T21:54:53.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:52 smithi123 ceph-mon[134433]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 577 op/s 2023-09-20T21:54:53.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:52 smithi089 ceph-mon[134090]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 577 op/s 2023-09-20T21:54:53.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 330.0 s, 574.9 tps, lat 174.414 ms stddev 203.206 2023-09-20T21:54:54.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:53 smithi022 ceph-mon[97651]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 484 op/s 2023-09-20T21:54:54.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:53 smithi123 ceph-mon[134433]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 484 op/s 2023-09-20T21:54:54.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:53 smithi089 ceph-mon[134090]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 484 op/s 2023-09-20T21:54:56.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:56 smithi123 ceph-mon[134433]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 MiB/s wr, 521 op/s 2023-09-20T21:54:56.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:56 smithi089 ceph-mon[134090]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 MiB/s wr, 521 op/s 2023-09-20T21:54:56.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:56 smithi022 ceph-mon[97651]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.2 MiB/s wr, 521 op/s 2023-09-20T21:54:58.570 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 335.0 s, 621.9 tps, lat 156.958 ms stddev 204.899 2023-09-20T21:54:58.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:54:58 smithi123 ceph-mon[134433]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 596 op/s 2023-09-20T21:54:58.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:54:58 smithi089 ceph-mon[134090]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 596 op/s 2023-09-20T21:54:58.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:54:58 smithi022 ceph-mon[97651]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 596 op/s 2023-09-20T21:55:01.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:00 smithi022 ceph-mon[97651]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.7 MiB/s wr, 589 op/s 2023-09-20T21:55:01.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:00 smithi123 ceph-mon[134433]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.7 MiB/s wr, 589 op/s 2023-09-20T21:55:01.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:00 smithi089 ceph-mon[134090]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.7 MiB/s wr, 589 op/s 2023-09-20T21:55:03.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:02 smithi022 ceph-mon[97651]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 634 op/s 2023-09-20T21:55:03.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:02 smithi123 ceph-mon[134433]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 634 op/s 2023-09-20T21:55:03.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:02 smithi089 ceph-mon[134090]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 634 op/s 2023-09-20T21:55:03.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 340.0 s, 628.6 tps, lat 157.604 ms stddev 182.288 2023-09-20T21:55:04.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:03 smithi022 ceph-mon[97651]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.8 MiB/s wr, 541 op/s 2023-09-20T21:55:04.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:03 smithi123 ceph-mon[134433]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.8 MiB/s wr, 541 op/s 2023-09-20T21:55:04.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:03 smithi089 ceph-mon[134090]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.8 MiB/s wr, 541 op/s 2023-09-20T21:55:06.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:06 smithi123 ceph-mon[134433]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.2 MiB/s wr, 581 op/s 2023-09-20T21:55:06.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:06 smithi089 ceph-mon[134090]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.2 MiB/s wr, 581 op/s 2023-09-20T21:55:06.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:06 smithi022 ceph-mon[97651]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.2 MiB/s wr, 581 op/s 2023-09-20T21:55:08.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 345.0 s, 376.9 tps, lat 240.371 ms stddev 332.676 2023-09-20T21:55:08.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:08 smithi123 ceph-mon[134433]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 640 op/s 2023-09-20T21:55:08.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:08 smithi089 ceph-mon[134090]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 640 op/s 2023-09-20T21:55:08.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:08 smithi022 ceph-mon[97651]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 640 op/s 2023-09-20T21:55:11.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:10 smithi022 ceph-mon[97651]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 975 op/s 2023-09-20T21:55:11.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:10 smithi123 ceph-mon[134433]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 975 op/s 2023-09-20T21:55:11.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:10 smithi089 ceph-mon[134090]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 15 MiB/s wr, 975 op/s 2023-09-20T21:55:13.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:12 smithi022 ceph-mon[97651]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-09-20T21:55:13.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:12 smithi123 ceph-mon[134433]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-09-20T21:55:13.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:12 smithi089 ceph-mon[134090]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 1.45k op/s 2023-09-20T21:55:13.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 350.0 s, 645.4 tps, lat 172.578 ms stddev 222.397 2023-09-20T21:55:14.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:13 smithi022 ceph-mon[97651]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-09-20T21:55:14.342 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:13 smithi123 ceph-mon[134433]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-09-20T21:55:14.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:13 smithi089 ceph-mon[134090]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 MiB/s wr, 1.32k op/s 2023-09-20T21:55:16.226 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:55:16.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:16 smithi022 ceph-mon[97651]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-20T21:55:16.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:16 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:55:16.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.797+0000 7fe882bff700 1 -- 172.21.15.22:0/1389084813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c107de0 msgr2=0x7fe87c10a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:16.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.797+0000 7fe882bff700 1 --2- 172.21.15.22:0/1389084813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c107de0 0x7fe87c10a210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009990 tx=0x7fe86c009ca0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.802+0000 7fe882bff700 1 -- 172.21.15.22:0/1389084813 shutdown_connections 2023-09-20T21:55:16.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.802+0000 7fe882bff700 1 --2- 172.21.15.22:0/1389084813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c10d120 0x7fe87c10d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.802+0000 7fe882bff700 1 --2- 172.21.15.22:0/1389084813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c107de0 0x7fe87c10a210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.802+0000 7fe882bff700 1 --2- 172.21.15.22:0/1389084813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe87c10a750 0x7fe87c10cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.802+0000 7fe882bff700 1 -- 172.21.15.22:0/1389084813 >> 172.21.15.22:0/1389084813 conn(0x7fe87c06db90 msgr2=0x7fe87c06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:16.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.813+0000 7fe882bff700 1 -- 172.21.15.22:0/1389084813 shutdown_connections 2023-09-20T21:55:16.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.817+0000 7fe882bff700 1 -- 172.21.15.22:0/1389084813 wait complete. 2023-09-20T21:55:16.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.818+0000 7fe882bff700 1 Processor -- start 2023-09-20T21:55:16.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.819+0000 7fe882bff700 1 -- start start 2023-09-20T21:55:16.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.820+0000 7fe882bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c107de0 0x7fe87c1143d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.820+0000 7fe882bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe87c10a750 0x7fe87c114910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.820+0000 7fe882bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c10d120 0x7fe87c1b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.820+0000 7fe882bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c1179a0 con 0x7fe87c10a750 2023-09-20T21:55:16.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.820+0000 7fe882bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c117b10 con 0x7fe87c10d120 2023-09-20T21:55:16.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.820+0000 7fe882bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c1b60b0 con 0x7fe87c107de0 2023-09-20T21:55:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.831+0000 7fe88119c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c10d120 0x7fe87c1b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.831+0000 7fe88119c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c10d120 0x7fe87c1b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:39548/0 (socket says 172.21.15.22:39548) 2023-09-20T21:55:16.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.831+0000 7fe88119c700 1 -- 172.21.15.22:0/893947015 learned_addr learned my addr 172.21.15.22:0/893947015 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:55:16.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.835+0000 7fe88099b700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c107de0 0x7fe87c1143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:16.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.836+0000 7fe88099b700 1 -- 172.21.15.22:0/893947015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c10d120 msgr2=0x7fe87c1b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:16.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.836+0000 7fe88099b700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c10d120 0x7fe87c1b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.836+0000 7fe88099b700 1 -- 172.21.15.22:0/893947015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe87c10a750 msgr2=0x7fe87c114910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:16.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.836+0000 7fe88099b700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe87c10a750 0x7fe87c114910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.836+0000 7fe88099b700 1 -- 172.21.15.22:0/893947015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c0095f0 con 0x7fe87c107de0 2023-09-20T21:55:16.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.838+0000 7fe88099b700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c107de0 0x7fe87c1143d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0098e0 tx=0x7fe86c00f7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:16.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.840+0000 7fe879ffb700 1 -- 172.21.15.22:0/893947015 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c01d070 con 0x7fe87c107de0 2023-09-20T21:55:16.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.841+0000 7fe879ffb700 1 -- 172.21.15.22:0/893947015 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe86c003e20 con 0x7fe87c107de0 2023-09-20T21:55:16.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.841+0000 7fe879ffb700 1 -- 172.21.15.22:0/893947015 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c022460 con 0x7fe87c107de0 2023-09-20T21:55:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.841+0000 7fe882bff700 1 -- 172.21.15.22:0/893947015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87c1b62b0 con 0x7fe87c107de0 2023-09-20T21:55:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.841+0000 7fe882bff700 1 -- 172.21.15.22:0/893947015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c1b6700 con 0x7fe87c107de0 2023-09-20T21:55:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.848+0000 7fe896431b80 1 Processor -- start 2023-09-20T21:55:16.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.849+0000 7fe896431b80 1 -- start start 2023-09-20T21:55:16.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.849+0000 7fe896431b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda80f6f0 0x55ceda80fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.849+0000 7fe896431b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 0x55ceda8105e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.849+0000 7fe896431b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810be0 0x55ceda811060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.849+0000 7fe896431b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda783ca0 con 0x55ceda80f6f0 2023-09-20T21:55:16.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.849+0000 7fe896431b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda767c70 con 0x55ceda810be0 2023-09-20T21:55:16.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.849+0000 7fe896431b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda765590 con 0x55ceda810050 2023-09-20T21:55:16.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.857+0000 7fe87bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810be0 0x55ceda811060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:16.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.858+0000 7fe87bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810be0 0x55ceda811060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:39552/0 (socket says 172.21.15.22:39552) 2023-09-20T21:55:16.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.858+0000 7fe87bfff700 1 -- 172.21.15.22:0/1801742733 learned_addr learned my addr 172.21.15.22:0/1801742733 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:55:16.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.859+0000 7fe87bfff700 1 -- 172.21.15.22:0/1801742733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 msgr2=0x55ceda8105e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:55:16.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.859+0000 7fe87bfff700 1 --2- 172.21.15.22:0/1801742733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 0x55ceda8105e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.859+0000 7fe87bfff700 1 -- 172.21.15.22:0/1801742733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda80f6f0 msgr2=0x55ceda80fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:16.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.859+0000 7fe87bfff700 1 --2- 172.21.15.22:0/1801742733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda80f6f0 0x55ceda80fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.859+0000 7fe87bfff700 1 -- 172.21.15.22:0/1801742733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceda58bde0 con 0x55ceda810be0 2023-09-20T21:55:16.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.865+0000 7fe87bfff700 1 --2- 172.21.15.22:0/1801742733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810be0 0x55ceda811060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098a0 tx=0x7fe870009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb405e6190e9f9f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.865+0000 7fe86affd700 1 -- 172.21.15.22:0/1801742733 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000b870 con 0x55ceda810be0 2023-09-20T21:55:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.865+0000 7fe86affd700 1 -- 172.21.15.22:0/1801742733 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe87000be90 con 0x55ceda810be0 2023-09-20T21:55:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.865+0000 7fe86affd700 1 -- 172.21.15.22:0/1801742733 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870010380 con 0x55ceda810be0 2023-09-20T21:55:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.866+0000 7fe896431b80 1 -- 172.21.15.22:0/1801742733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810be0 msgr2=0x55ceda811060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.866+0000 7fe896431b80 1 --2- 172.21.15.22:0/1801742733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810be0 0x55ceda811060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098a0 tx=0x7fe870009bb0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.867+0000 7fe896431b80 1 -- 172.21.15.22:0/1801742733 shutdown_connections 2023-09-20T21:55:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.867+0000 7fe896431b80 1 --2- 172.21.15.22:0/1801742733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810be0 0x55ceda811060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.868+0000 7fe896431b80 1 --2- 172.21.15.22:0/1801742733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda80f6f0 0x55ceda80fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.868+0000 7fe896431b80 1 --2- 172.21.15.22:0/1801742733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 0x55ceda8105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.868+0000 7fe896431b80 1 -- 172.21.15.22:0/1801742733 >> 172.21.15.22:0/1801742733 conn(0x55ceda73c000 msgr2=0x55ceda80a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:16.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.870+0000 7fe896431b80 1 -- 172.21.15.22:0/1801742733 shutdown_connections 2023-09-20T21:55:16.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.879+0000 7fe879ffb700 1 -- 172.21.15.22:0/893947015 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe86c00fd00 con 0x7fe87c107de0 2023-09-20T21:55:16.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.879+0000 7fe896431b80 1 -- 172.21.15.22:0/1801742733 wait complete. 2023-09-20T21:55:16.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.880+0000 7fe896431b80 1 -- start start 2023-09-20T21:55:16.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.881+0000 7fe896431b80 1 Processor -- start 2023-09-20T21:55:16.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.883+0000 7fe879ffb700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe8640a1410 0x7fe8640a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.884+0000 7fe879ffb700 1 -- 172.21.15.22:0/893947015 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe86c0cd290 con 0x7fe87c107de0 2023-09-20T21:55:16.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.887+0000 7fe896431b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 0x55ceda819200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.887+0000 7fe896431b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810050 0x55ceda819740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.887+0000 7fe896431b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda810be0 0x55ceda820780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.887+0000 7fe896431b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda760e80 con 0x55ceda810be0 2023-09-20T21:55:16.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.887+0000 7fe896431b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda6fd740 con 0x55ceda810050 2023-09-20T21:55:16.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.887+0000 7fe896431b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda6fd3e0 con 0x55ceda80f6f0 2023-09-20T21:55:16.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.887+0000 7fe896431b80 10 client.0 _ll_register_callbacks cb 0x55ceda647a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:55:16.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.888+0000 7fe88119c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda810be0 0x55ceda820780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:16.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.888+0000 7fe88119c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda810be0 0x55ceda820780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49928/0 (socket says 172.21.15.22:49928) 2023-09-20T21:55:16.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.888+0000 7fe88119c700 1 -- 172.21.15.22:0/3139379581 learned_addr learned my addr 172.21.15.22:0/3139379581 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:55:16.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.888+0000 7fe88099b700 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 0x55ceda819200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:16.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.891+0000 7fe88099b700 1 -- 172.21.15.22:0/3139379581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810050 msgr2=0x55ceda819740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:55:16.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.891+0000 7fe88099b700 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810050 0x55ceda819740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.891+0000 7fe88099b700 1 -- 172.21.15.22:0/3139379581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda810be0 msgr2=0x55ceda820780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:16.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.891+0000 7fe88099b700 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda810be0 0x55ceda820780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.891+0000 7fe88099b700 1 -- 172.21.15.22:0/3139379581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8700094d0 con 0x55ceda80f6f0 2023-09-20T21:55:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.891+0000 7fe88119c700 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda810be0 0x55ceda820780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:55:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.891+0000 7fe88099b700 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 0x55ceda819200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00c010 tx=0x7fe86c09a910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=108961301471fa7d server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:16.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.892+0000 7fe88119c700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe8640a1410 0x7fe8640a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:16.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.894+0000 7fe88119c700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe8640a1410 0x7fe8640a38d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe874007fd0 tx=0x7fe874007990 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:16.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.896+0000 7fe85ffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:55:16.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.896+0000 7fe85ffff700 1 -- 172.21.15.22:0/3139379581 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c09aad0 con 0x55ceda80f6f0 2023-09-20T21:55:16.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.896+0000 7fe85ffff700 1 -- 172.21.15.22:0/3139379581 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe86c09a3d0 con 0x55ceda80f6f0 2023-09-20T21:55:16.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.896+0000 7fe896431b80 1 -- 172.21.15.22:0/3139379581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ceda6d1bb0 con 0x55ceda80f6f0 2023-09-20T21:55:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.900+0000 7fe85ffff700 1 -- 172.21.15.22:0/3139379581 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe86c09e200 con 0x55ceda80f6f0 2023-09-20T21:55:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.903+0000 7fe896431b80 10 client.25945 fetch_fsmap learned FSMap version 32 2023-09-20T21:55:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.903+0000 7fe896431b80 1 -- 172.21.15.22:0/3139379581 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ceda574c60 con 0x55ceda80f6f0 2023-09-20T21:55:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.924+0000 7fe85ffff700 1 -- 172.21.15.22:0/3139379581 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ceda6d1bb0 con 0x55ceda80f6f0 2023-09-20T21:55:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.924+0000 7fe85ffff700 1 -- 172.21.15.22:0/3139379581 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe86c0a1f20 con 0x55ceda80f6f0 2023-09-20T21:55:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.931+0000 7fe896431b80 10 client.25945 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:55:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.931+0000 7fe896431b80 10 client.25945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:55:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.931+0000 7fe896431b80 20 client.25945 populate_metadata read hostname 'smithi022' 2023-09-20T21:55:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.931+0000 7fe896431b80 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ceda824ca0 0x55ceda8270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.931+0000 7fe896431b80 4 client.25945 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:55:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.931+0000 7fe896431b80 1 -- 172.21.15.22:0/3139379581 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ceda7de020 con 0x55ceda824ca0 2023-09-20T21:55:16.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.935+0000 7fe87bfff700 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ceda824ca0 0x55ceda8270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:16.937 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:16 smithi089 ceph-mon[134090]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-20T21:55:16.937 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:16 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:55:16.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.953+0000 7fe87bfff700 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ceda824ca0 0x55ceda8270c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=455aeb4792aee6e8 server_cookie=c917fd3b67880476 in_seq=0 out_seq=0 2023-09-20T21:55:16.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.953+0000 7fe85ffff700 10 client.25945 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:55:16.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.962+0000 7fe85ffff700 1 -- 172.21.15.22:0/3139379581 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ceda7de020 con 0x55ceda824ca0 2023-09-20T21:55:16.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.962+0000 7fe85ffff700 10 client.25945 handle_command_reply: tid=0 2023-09-20T21:55:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.965+0000 7fe896431b80 1 client.25945 shutdown 2023-09-20T21:55:16.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.966+0000 7fe896431b80 1 -- 172.21.15.22:0/3139379581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 msgr2=0x55ceda819200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:16.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.966+0000 7fe896431b80 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 0x55ceda819200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00c010 tx=0x7fe86c09a910 comp rx=0 tx=0).stop 2023-09-20T21:55:16.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.980+0000 7fe896431b80 1 -- 172.21.15.22:0/3139379581 shutdown_connections 2023-09-20T21:55:16.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.980+0000 7fe896431b80 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ceda824ca0 0x55ceda8270c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.980+0000 7fe896431b80 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda810be0 0x55ceda820780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.980+0000 7fe896431b80 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 0x55ceda819200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.980+0000 7fe896431b80 1 --2- 172.21.15.22:0/3139379581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810050 0x55ceda819740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.980+0000 7fe896431b80 1 -- 172.21.15.22:0/3139379581 >> 172.21.15.22:0/3139379581 conn(0x55ceda73c000 msgr2=0x55ceda809880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.982+0000 7fe85ffff700 0 client.25945 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:55:16.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.992+0000 7fe896431b80 1 -- 172.21.15.22:0/3139379581 shutdown_connections 2023-09-20T21:55:16.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.992+0000 7fe896431b80 1 -- 172.21.15.22:0/3139379581 wait complete. 2023-09-20T21:55:16.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:16.992+0000 7fe896431b80 20 client.25945 trim_cache size 0 max 16384 2023-09-20T21:55:17.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.027+0000 7fe896431b80 1 Processor -- start 2023-09-20T21:55:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.027+0000 7fe896431b80 1 -- start start 2023-09-20T21:55:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.027+0000 7fe896431b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda73c000 0x55ceda809880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.027+0000 7fe896431b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda80f6f0 0x55ceda820740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:17.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.027+0000 7fe896431b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 0x55ceda8170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:17.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.027+0000 7fe896431b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda5e1920 con 0x55ceda73c000 2023-09-20T21:55:17.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.027+0000 7fe896431b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda810a40 con 0x55ceda80f6f0 2023-09-20T21:55:17.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.027+0000 7fe896431b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda810430 con 0x55ceda810050 2023-09-20T21:55:17.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.028+0000 7fe88119c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 0x55ceda8170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:17.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.028+0000 7fe88119c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 0x55ceda8170c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:38014/0 (socket says 172.21.15.22:38014) 2023-09-20T21:55:17.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.028+0000 7fe88119c700 1 -- 172.21.15.22:0/3183332847 learned_addr learned my addr 172.21.15.22:0/3183332847 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:55:17.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.028+0000 7fe87bfff700 1 --2- 172.21.15.22:0/3183332847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda80f6f0 0x55ceda820740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.028+0000 7fe88119c700 1 -- 172.21.15.22:0/3183332847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda80f6f0 msgr2=0x55ceda820740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.028+0000 7fe88119c700 1 --2- 172.21.15.22:0/3183332847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda80f6f0 0x55ceda820740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.028+0000 7fe88119c700 1 -- 172.21.15.22:0/3183332847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda73c000 msgr2=0x55ceda809880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.028+0000 7fe88119c700 1 --2- 172.21.15.22:0/3183332847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda73c000 0x55ceda809880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.028+0000 7fe88119c700 1 -- 172.21.15.22:0/3183332847 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceda5b2250 con 0x55ceda810050 2023-09-20T21:55:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.029+0000 7fe87bfff700 1 --2- 172.21.15.22:0/3183332847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda80f6f0 0x55ceda820740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:55:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.029+0000 7fe88119c700 1 --2- 172.21.15.22:0/3183332847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 0x55ceda8170c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe874009b00 tx=0x7fe874014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59dbfa298a9aef44 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.029+0000 7fe86b7fe700 1 -- 172.21.15.22:0/3183332847 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874004bb0 con 0x55ceda810050 2023-09-20T21:55:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.029+0000 7fe86b7fe700 1 -- 172.21.15.22:0/3183332847 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe874004d10 con 0x55ceda810050 2023-09-20T21:55:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.030+0000 7fe86b7fe700 1 -- 172.21.15.22:0/3183332847 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8740239d0 con 0x55ceda810050 2023-09-20T21:55:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.030+0000 7fe896431b80 1 -- 172.21.15.22:0/3183332847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 msgr2=0x55ceda8170c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.030+0000 7fe896431b80 1 --2- 172.21.15.22:0/3183332847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 0x55ceda8170c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe874009b00 tx=0x7fe874014000 comp rx=0 tx=0).stop 2023-09-20T21:55:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.031+0000 7fe896431b80 1 -- 172.21.15.22:0/3183332847 shutdown_connections 2023-09-20T21:55:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.031+0000 7fe896431b80 1 --2- 172.21.15.22:0/3183332847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda810050 0x55ceda8170c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.031+0000 7fe896431b80 1 --2- 172.21.15.22:0/3183332847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda73c000 0x55ceda809880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.031+0000 7fe896431b80 1 --2- 172.21.15.22:0/3183332847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda80f6f0 0x55ceda820740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.031+0000 7fe896431b80 1 -- 172.21.15.22:0/3183332847 >> 172.21.15.22:0/3183332847 conn(0x55ceda80a500 msgr2=0x55ceda813080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:17.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.031+0000 7fe896431b80 1 -- 172.21.15.22:0/3183332847 shutdown_connections 2023-09-20T21:55:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.031+0000 7fe896431b80 1 -- 172.21.15.22:0/3183332847 wait complete. 2023-09-20T21:55:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.032+0000 7fe896431b80 1 -- start start 2023-09-20T21:55:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.032+0000 7fe896431b80 1 Processor -- start 2023-09-20T21:55:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.033+0000 7fe896431b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda73c000 0x55ceda81c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.033+0000 7fe896431b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 0x55ceda81c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.033+0000 7fe896431b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810050 0x55ceda81fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.033+0000 7fe896431b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda81ff40 con 0x55ceda73c000 2023-09-20T21:55:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.033+0000 7fe896431b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda8200b0 con 0x55ceda810050 2023-09-20T21:55:17.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.033+0000 7fe896431b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceda820270 con 0x55ceda80f6f0 2023-09-20T21:55:17.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.033+0000 7fe896431b80 10 client.0 _ll_register_callbacks cb 0x55ceda647a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:55:17.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.033+0000 7fe87bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 0x55ceda81c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:17.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.033+0000 7fe87bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 0x55ceda81c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:38020/0 (socket says 172.21.15.22:38020) 2023-09-20T21:55:17.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.034+0000 7fe87bfff700 1 -- 172.21.15.22:0/2624719289 learned_addr learned my addr 172.21.15.22:0/2624719289 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:55:17.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.034+0000 7fe88099b700 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda73c000 0x55ceda81c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:17.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.034+0000 7fe88099b700 1 -- 172.21.15.22:0/2624719289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 msgr2=0x55ceda81c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:17.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.034+0000 7fe88099b700 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 0x55ceda81c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.034+0000 7fe88099b700 1 -- 172.21.15.22:0/2624719289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810050 msgr2=0x55ceda81fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:17.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.034+0000 7fe88099b700 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810050 0x55ceda81fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.034+0000 7fe88099b700 1 -- 172.21.15.22:0/2624719289 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe874012040 con 0x55ceda73c000 2023-09-20T21:55:17.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.035+0000 7fe88099b700 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda73c000 0x55ceda81c4a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe86c098720 tx=0x7fe86c096210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35586f7bacd02e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:17.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.035+0000 7fe85ffff700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:55:17.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.035+0000 7fe85ffff700 1 -- 172.21.15.22:0/2624719289 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c09aad0 con 0x55ceda73c000 2023-09-20T21:55:17.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.035+0000 7fe85ffff700 1 -- 172.21.15.22:0/2624719289 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe86c09a3d0 con 0x55ceda73c000 2023-09-20T21:55:17.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.035+0000 7fe896431b80 1 -- 172.21.15.22:0/2624719289 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe87000be90 con 0x55ceda73c000 2023-09-20T21:55:17.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.036+0000 7fe85ffff700 1 -- 172.21.15.22:0/2624719289 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe86c09e200 con 0x55ceda73c000 2023-09-20T21:55:17.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.036+0000 7fe896431b80 10 client.15804 fetch_fsmap learned FSMap version 32 2023-09-20T21:55:17.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.036+0000 7fe896431b80 1 -- 172.21.15.22:0/2624719289 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ceda58a2b0 con 0x55ceda73c000 2023-09-20T21:55:17.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.036+0000 7fe85ffff700 1 -- 172.21.15.22:0/2624719289 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe86c0a6070 con 0x55ceda73c000 2023-09-20T21:55:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.036+0000 7fe85ffff700 1 -- 172.21.15.22:0/2624719289 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe86c09ad60 con 0x55ceda73c000 2023-09-20T21:55:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.037+0000 7fe896431b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:55:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.037+0000 7fe896431b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:55:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.037+0000 7fe896431b80 20 client.15804 populate_metadata read hostname 'smithi022' 2023-09-20T21:55:17.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.037+0000 7fe896431b80 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ceda810be0 0x55ceda87b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:17.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.037+0000 7fe896431b80 4 client.15804 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:55:17.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.037+0000 7fe896431b80 1 -- 172.21.15.22:0/2624719289 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ceda759e60 con 0x55ceda810be0 2023-09-20T21:55:17.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.037+0000 7fe87bfff700 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ceda810be0 0x55ceda87b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.038+0000 7fe87bfff700 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ceda810be0 0x55ceda87b4e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a61c30dc712f92c server_cookie=648b0f74391c82e8 in_seq=0 out_seq=0 2023-09-20T21:55:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.038+0000 7fe85ffff700 10 client.15804 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:55:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.038+0000 7fe85ffff700 1 -- 172.21.15.22:0/2624719289 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ceda759e60 con 0x55ceda810be0 2023-09-20T21:55:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.038+0000 7fe85ffff700 10 client.15804 handle_command_reply: tid=0 2023-09-20T21:55:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.038+0000 7fe896431b80 1 client.15804 shutdown 2023-09-20T21:55:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.039+0000 7fe896431b80 1 -- 172.21.15.22:0/2624719289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda73c000 msgr2=0x55ceda81c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.039+0000 7fe896431b80 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda73c000 0x55ceda81c4a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe86c098720 tx=0x7fe86c096210 comp rx=0 tx=0).stop 2023-09-20T21:55:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.039+0000 7fe896431b80 1 -- 172.21.15.22:0/2624719289 shutdown_connections 2023-09-20T21:55:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.039+0000 7fe896431b80 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ceda810be0 0x55ceda87b4e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.039+0000 7fe896431b80 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ceda810050 0x55ceda81fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.039+0000 7fe896431b80 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ceda73c000 0x55ceda81c4a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.039+0000 7fe896431b80 1 --2- 172.21.15.22:0/2624719289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ceda80f6f0 0x55ceda81c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.039+0000 7fe896431b80 1 -- 172.21.15.22:0/2624719289 >> 172.21.15.22:0/2624719289 conn(0x55ceda80a500 msgr2=0x55ceda8170c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.039+0000 7fe85ffff700 0 client.15804 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:55:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.039+0000 7fe896431b80 1 -- 172.21.15.22:0/2624719289 shutdown_connections 2023-09-20T21:55:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.040+0000 7fe896431b80 1 -- 172.21.15.22:0/2624719289 wait complete. 2023-09-20T21:55:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.040+0000 7fe896431b80 20 client.15804 trim_cache size 0 max 16384 2023-09-20T21:55:17.049 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:55:17.049 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:55:17.049 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:55:17.049 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:55:17.050 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:55:17.050 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:55:17.050 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:55:17.050 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:55:17.050 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:55:17.051 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:55:17.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 -- 172.21.15.22:0/893947015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe8640a1410 msgr2=0x7fe8640a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:17.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe8640a1410 0x7fe8640a38d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe874007fd0 tx=0x7fe874007990 comp rx=0 tx=0).stop 2023-09-20T21:55:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 -- 172.21.15.22:0/893947015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c107de0 msgr2=0x7fe87c1143d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c107de0 0x7fe87c1143d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0098e0 tx=0x7fe86c00f7d0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 -- 172.21.15.22:0/893947015 shutdown_connections 2023-09-20T21:55:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe8640a1410 0x7fe8640a38d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe87c10d120 0x7fe87c1b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c107de0 0x7fe87c1143d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 --2- 172.21.15.22:0/893947015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe87c10a750 0x7fe87c114910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 -- 172.21.15.22:0/893947015 >> 172.21.15.22:0/893947015 conn(0x7fe87c06db90 msgr2=0x7fe87c10bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 -- 172.21.15.22:0/893947015 shutdown_connections 2023-09-20T21:55:17.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:17.041+0000 7fe882bff700 1 -- 172.21.15.22:0/893947015 wait complete. 2023-09-20T21:55:17.070 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:55:17.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:16 smithi123 ceph-mon[134433]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-09-20T21:55:17.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:16 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:55:17.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:17.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:17.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:17.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:17.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:55:17.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:55:17.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:17 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:55:17.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:17.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:17.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:18.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:18.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:55:18.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:55:18.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:17 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:55:18.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:18.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:18.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:18.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:55:18.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:55:18.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:55:18.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:17 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:55:18.570 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 355.0 s, 562.5 tps, lat 178.494 ms stddev 241.323 2023-09-20T21:55:19.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:18 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi022 to 222.0M 2023-09-20T21:55:19.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:18 smithi022 ceph-mon[97651]: Unable to set osd_memory_target on smithi022 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-20T21:55:19.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:18 smithi022 ceph-mon[97651]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2023-09-20T21:55:19.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:18 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi022 to 222.0M 2023-09-20T21:55:19.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:18 smithi123 ceph-mon[134433]: Unable to set osd_memory_target on smithi022 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-20T21:55:19.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:18 smithi123 ceph-mon[134433]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2023-09-20T21:55:19.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:18 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi022 to 222.0M 2023-09-20T21:55:19.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:18 smithi089 ceph-mon[134090]: Unable to set osd_memory_target on smithi022 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-20T21:55:19.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:18 smithi089 ceph-mon[134090]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 20 MiB/s wr, 1.40k op/s 2023-09-20T21:55:21.010 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:20 smithi022 ceph-mon[97651]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-09-20T21:55:21.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:20 smithi123 ceph-mon[134433]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-09-20T21:55:21.146 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:20 smithi089 ceph-mon[134090]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 20 MiB/s wr, 1.38k op/s 2023-09-20T21:55:22.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:22 smithi022 ceph-mon[97651]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-09-20T21:55:23.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:22 smithi123 ceph-mon[134433]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-09-20T21:55:23.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:22 smithi089 ceph-mon[134090]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2023-09-20T21:55:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 360.0 s, 547.9 tps, lat 182.776 ms stddev 199.357 2023-09-20T21:55:24.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:23 smithi022 ceph-mon[97651]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.4 MiB/s wr, 466 op/s 2023-09-20T21:55:24.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:23 smithi123 ceph-mon[134433]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.4 MiB/s wr, 466 op/s 2023-09-20T21:55:24.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:23 smithi089 ceph-mon[134090]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.4 MiB/s wr, 466 op/s 2023-09-20T21:55:27.009 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:26 smithi022 ceph-mon[97651]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.1 MiB/s wr, 586 op/s 2023-09-20T21:55:27.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:26 smithi123 ceph-mon[134433]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.1 MiB/s wr, 586 op/s 2023-09-20T21:55:27.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:26 smithi089 ceph-mon[134090]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.1 MiB/s wr, 586 op/s 2023-09-20T21:55:28.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 365.0 s, 555.1 tps, lat 182.354 ms stddev 214.262 2023-09-20T21:55:29.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:28 smithi022 ceph-mon[97651]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.3 MiB/s wr, 653 op/s 2023-09-20T21:55:29.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:28 smithi123 ceph-mon[134433]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.3 MiB/s wr, 653 op/s 2023-09-20T21:55:29.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:28 smithi089 ceph-mon[134090]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.3 MiB/s wr, 653 op/s 2023-09-20T21:55:30.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:30 smithi022 ceph-mon[97651]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 635 op/s 2023-09-20T21:55:31.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:30 smithi123 ceph-mon[134433]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 635 op/s 2023-09-20T21:55:31.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:30 smithi089 ceph-mon[134090]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 635 op/s 2023-09-20T21:55:33.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:32 smithi123 ceph-mon[134433]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 MiB/s wr, 667 op/s 2023-09-20T21:55:33.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:32 smithi089 ceph-mon[134090]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 MiB/s wr, 667 op/s 2023-09-20T21:55:33.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:32 smithi022 ceph-mon[97651]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 MiB/s wr, 667 op/s 2023-09-20T21:55:33.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 370.0 s, 580.2 tps, lat 171.627 ms stddev 185.261 2023-09-20T21:55:34.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:34 smithi123 ceph-mon[134433]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 MiB/s wr, 553 op/s 2023-09-20T21:55:34.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:34 smithi089 ceph-mon[134090]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 MiB/s wr, 553 op/s 2023-09-20T21:55:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:34 smithi022 ceph-mon[97651]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 MiB/s wr, 553 op/s 2023-09-20T21:55:36.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:36 smithi022 ceph-mon[97651]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.8 MiB/s wr, 628 op/s 2023-09-20T21:55:37.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:36 smithi123 ceph-mon[134433]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.8 MiB/s wr, 628 op/s 2023-09-20T21:55:37.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:36 smithi089 ceph-mon[134090]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.8 MiB/s wr, 628 op/s 2023-09-20T21:55:38.573 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 375.0 s, 518.3 tps, lat 192.847 ms stddev 250.770 2023-09-20T21:55:38.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:38 smithi022 ceph-mon[97651]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 619 op/s 2023-09-20T21:55:39.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:38 smithi123 ceph-mon[134433]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 619 op/s 2023-09-20T21:55:39.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:38 smithi089 ceph-mon[134090]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 619 op/s 2023-09-20T21:55:40.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:40 smithi022 ceph-mon[97651]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.6 MiB/s wr, 552 op/s 2023-09-20T21:55:41.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:40 smithi123 ceph-mon[134433]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.6 MiB/s wr, 552 op/s 2023-09-20T21:55:41.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:40 smithi089 ceph-mon[134090]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.6 MiB/s wr, 552 op/s 2023-09-20T21:55:42.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:42 smithi022 ceph-mon[97651]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 MiB/s wr, 557 op/s 2023-09-20T21:55:43.101 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:42 smithi123 ceph-mon[134433]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 MiB/s wr, 557 op/s 2023-09-20T21:55:43.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:42 smithi089 ceph-mon[134090]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 MiB/s wr, 557 op/s 2023-09-20T21:55:43.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 380.0 s, 559.9 tps, lat 175.934 ms stddev 197.910 2023-09-20T21:55:44.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:44 smithi123 ceph-mon[134433]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.9 MiB/s wr, 456 op/s 2023-09-20T21:55:44.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:44 smithi089 ceph-mon[134090]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.9 MiB/s wr, 456 op/s 2023-09-20T21:55:44.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:44 smithi022 ceph-mon[97651]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.9 MiB/s wr, 456 op/s 2023-09-20T21:55:47.014 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:46 smithi022 ceph-mon[97651]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.8 MiB/s wr, 538 op/s 2023-09-20T21:55:47.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:55:47.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:46 smithi123 ceph-mon[134433]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.8 MiB/s wr, 538 op/s 2023-09-20T21:55:47.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:46 smithi089 ceph-mon[134090]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.8 MiB/s wr, 538 op/s 2023-09-20T21:55:47.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.921+0000 7f4f103ff700 1 -- 172.21.15.22:0/1441714413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f0810a750 msgr2=0x7f4f0810cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.921+0000 7f4f103ff700 1 --2- 172.21.15.22:0/1441714413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f0810a750 0x7f4f0810cbe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4f040098f0 tx=0x7f4f04009c00 comp rx=0 tx=0).stop 2023-09-20T21:55:47.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.929+0000 7f4f103ff700 1 -- 172.21.15.22:0/1441714413 shutdown_connections 2023-09-20T21:55:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.930+0000 7f4f103ff700 1 --2- 172.21.15.22:0/1441714413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f0810d120 0x7f4f0810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.930+0000 7f4f103ff700 1 --2- 172.21.15.22:0/1441714413 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f08107de0 0x7f4f0810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.930+0000 7f4f103ff700 1 --2- 172.21.15.22:0/1441714413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f0810a750 0x7f4f0810cbe0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.930+0000 7f4f103ff700 1 -- 172.21.15.22:0/1441714413 >> 172.21.15.22:0/1441714413 conn(0x7f4f0806db90 msgr2=0x7f4f0806ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:47.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.930+0000 7f4f103ff700 1 -- 172.21.15.22:0/1441714413 shutdown_connections 2023-09-20T21:55:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.930+0000 7f4f103ff700 1 -- 172.21.15.22:0/1441714413 wait complete. 2023-09-20T21:55:47.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.932+0000 7f4f103ff700 1 Processor -- start 2023-09-20T21:55:47.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.933+0000 7f4f103ff700 1 -- start start 2023-09-20T21:55:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.933+0000 7f4f103ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f08107de0 0x7f4f081a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.933+0000 7f4f103ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f0810a750 0x7f4f081a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.933+0000 7f4f103ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f0810d120 0x7f4f081ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:47.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.933+0000 7f4f103ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081ad450 con 0x7f4f0810d120 2023-09-20T21:55:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.933+0000 7f4f103ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081ad590 con 0x7f4f08107de0 2023-09-20T21:55:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.933+0000 7f4f103ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081ad720 con 0x7f4f0810a750 2023-09-20T21:55:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.934+0000 7f4f0d99a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f0810a750 0x7f4f081a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.934+0000 7f4f0e19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f08107de0 0x7f4f081a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.934+0000 7f4f0d99a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f0810a750 0x7f4f081a5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:49632/0 (socket says 172.21.15.22:49632) 2023-09-20T21:55:47.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.934+0000 7f4f0e19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f08107de0 0x7f4f081a58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:35240/0 (socket says 172.21.15.22:35240) 2023-09-20T21:55:47.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.934+0000 7f4f0d99a700 1 -- 172.21.15.22:0/459579471 learned_addr learned my addr 172.21.15.22:0/459579471 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:55:47.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.934+0000 7f4f0d99a700 1 -- 172.21.15.22:0/459579471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f08107de0 msgr2=0x7f4f081a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:47.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.934+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f08107de0 0x7f4f081a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.934+0000 7f4f0d99a700 1 -- 172.21.15.22:0/459579471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f0810d120 msgr2=0x7f4f081ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:47.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.934+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f0810d120 0x7f4f081ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.934+0000 7f4f0d99a700 1 -- 172.21.15.22:0/459579471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f04009520 con 0x7f4f0810a750 2023-09-20T21:55:47.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.935+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f0810a750 0x7f4f081a5e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400bc60 tx=0x7f4f0400bc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:47.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.935+0000 7f4eff7fe700 1 -- 172.21.15.22:0/459579471 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04005520 con 0x7f4f0810a750 2023-09-20T21:55:47.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.935+0000 7f4eff7fe700 1 -- 172.21.15.22:0/459579471 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f04005680 con 0x7f4f0810a750 2023-09-20T21:55:47.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.935+0000 7f4eff7fe700 1 -- 172.21.15.22:0/459579471 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04017ad0 con 0x7f4f0810a750 2023-09-20T21:55:47.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.935+0000 7f4f103ff700 1 -- 172.21.15.22:0/459579471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f081ad9a0 con 0x7f4f0810a750 2023-09-20T21:55:47.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.936+0000 7f4f103ff700 1 -- 172.21.15.22:0/459579471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f081add70 con 0x7f4f0810a750 2023-09-20T21:55:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.941+0000 7f4eff7fe700 1 -- 172.21.15.22:0/459579471 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f4f04017c30 con 0x7f4f0810a750 2023-09-20T21:55:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.942+0000 7f4f23cd8b80 1 Processor -- start 2023-09-20T21:55:47.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.943+0000 7f4f23cd8b80 1 -- start start 2023-09-20T21:55:47.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.943+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d570ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:47.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.943+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d570b050 0x55a1d570b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:47.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.943+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570bbe0 0x55a1d570c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:47.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.943+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d567eca0 con 0x55a1d570b050 2023-09-20T21:55:47.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.943+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d5662c70 con 0x55a1d570a6f0 2023-09-20T21:55:47.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.943+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d5660590 con 0x55a1d570bbe0 2023-09-20T21:55:47.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.944+0000 7f4f0d99a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570bbe0 0x55a1d570c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:47.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.944+0000 7f4f0d99a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570bbe0 0x55a1d570c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:49634/0 (socket says 172.21.15.22:49634) 2023-09-20T21:55:47.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.944+0000 7f4f0d99a700 1 -- 172.21.15.22:0/389890949 learned_addr learned my addr 172.21.15.22:0/389890949 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:55:47.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.944+0000 7f4f0e99c700 1 --2- 172.21.15.22:0/389890949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d570ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:47.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.944+0000 7f4f0d99a700 1 -- 172.21.15.22:0/389890949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 msgr2=0x55a1d570ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:47.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.944+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/389890949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d570ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.944+0000 7f4f0d99a700 1 -- 172.21.15.22:0/389890949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d570b050 msgr2=0x55a1d570b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:55:47.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.945+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/389890949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d570b050 0x55a1d570b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.945+0000 7f4f0d99a700 1 -- 172.21.15.22:0/389890949 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1d5486de0 con 0x55a1d570bbe0 2023-09-20T21:55:47.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.945+0000 7f4eff7fe700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4ef40a1400 0x7f4ef40a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:47.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.945+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/389890949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570bbe0 0x55a1d570c060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4f040d6340 tx=0x7f4f040d6370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ec3a50c0215f31 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:47.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.945+0000 7f4efcbfe700 1 -- 172.21.15.22:0/389890949 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04097780 con 0x55a1d570bbe0 2023-09-20T21:55:47.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.945+0000 7f4efcbfe700 1 -- 172.21.15.22:0/389890949 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f04097050 con 0x55a1d570bbe0 2023-09-20T21:55:47.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.945+0000 7f4efcbfe700 1 -- 172.21.15.22:0/389890949 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0409b440 con 0x55a1d570bbe0 2023-09-20T21:55:47.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.946+0000 7f4eff7fe700 1 -- 172.21.15.22:0/459579471 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f4f040ce410 con 0x7f4f0810a750 2023-09-20T21:55:47.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.946+0000 7f4f0e99c700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4ef40a1400 0x7f4ef40a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:47.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.947+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/389890949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570bbe0 msgr2=0x55a1d570c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:47.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.947+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/389890949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570bbe0 0x55a1d570c060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4f040d6340 tx=0x7f4f040d6370 comp rx=0 tx=0).stop 2023-09-20T21:55:47.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.948+0000 7f4f0e99c700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4ef40a1400 0x7f4ef40a38c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55a1d57060b0 tx=0x7f4f00007620 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:47.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.949+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/389890949 shutdown_connections 2023-09-20T21:55:47.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.949+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/389890949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570bbe0 0x55a1d570c060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.949+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/389890949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d570ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.949+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/389890949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d570b050 0x55a1d570b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.949+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/389890949 >> 172.21.15.22:0/389890949 conn(0x55a1d5637000 msgr2=0x55a1d57051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.972+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/389890949 shutdown_connections 2023-09-20T21:55:47.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.973+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/389890949 wait complete. 2023-09-20T21:55:47.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.974+0000 7f4f23cd8b80 1 -- start start 2023-09-20T21:55:47.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.974+0000 7f4f23cd8b80 1 Processor -- start 2023-09-20T21:55:47.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.975+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d5713f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:47.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.975+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 0x55a1d5714460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:47.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.975+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d570bbe0 0x55a1d571b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:47.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.975+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d565be80 con 0x55a1d570bbe0 2023-09-20T21:55:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.975+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d55f8740 con 0x55a1d570a6f0 2023-09-20T21:55:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.975+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d55f83e0 con 0x55a1d570b050 2023-09-20T21:55:47.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.975+0000 7f4f23cd8b80 10 client.0 _ll_register_callbacks cb 0x55a1d5542a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:55:47.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.976+0000 7f4f0d99a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 0x55a1d5714460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:47.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.978+0000 7f4f0e99c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d570bbe0 0x55a1d571b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:47.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.978+0000 7f4f0e99c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d570bbe0 0x55a1d571b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58372/0 (socket says 172.21.15.22:58372) 2023-09-20T21:55:47.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.978+0000 7f4f0e99c700 1 -- 172.21.15.22:0/553990061 learned_addr learned my addr 172.21.15.22:0/553990061 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:55:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.979+0000 7f4f0d99a700 1 -- 172.21.15.22:0/553990061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 msgr2=0x55a1d5713f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.979+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d5713f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.979+0000 7f4f0d99a700 1 -- 172.21.15.22:0/553990061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d570bbe0 msgr2=0x55a1d571b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:47.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.979+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d570bbe0 0x55a1d571b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:47.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.979+0000 7f4f0d99a700 1 -- 172.21.15.22:0/553990061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f081add70 con 0x55a1d570b050 2023-09-20T21:55:47.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.984+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 0x55a1d5714460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f040d6340 tx=0x7f4f040d1d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e56739e0229b3a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:47.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.984+0000 7f4ef1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:55:47.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.989+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/553990061 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040d1ea0 con 0x55a1d570b050 2023-09-20T21:55:47.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.989+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/553990061 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f4f0409f270 con 0x55a1d570b050 2023-09-20T21:55:47.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:47.989+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/553990061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1d55ccbb0 con 0x55a1d570b050 2023-09-20T21:55:48.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.000+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/553990061 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f040d18c0 con 0x55a1d570b050 2023-09-20T21:55:48.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.004+0000 7f4f23cd8b80 10 client.25960 fetch_fsmap learned FSMap version 32 2023-09-20T21:55:48.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.004+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/553990061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a1d546fc60 con 0x55a1d570b050 2023-09-20T21:55:48.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.009+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/553990061 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a1d55ccbb0 con 0x55a1d570b050 2023-09-20T21:55:48.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.009+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/553990061 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4f0409a930 con 0x55a1d570b050 2023-09-20T21:55:48.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.013+0000 7f4f23cd8b80 10 client.25960 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:55:48.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.013+0000 7f4f23cd8b80 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:55:48.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.013+0000 7f4f23cd8b80 20 client.25960 populate_metadata read hostname 'smithi022' 2023-09-20T21:55:48.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.013+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a1d5730770 0x55a1d5732b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:48.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.013+0000 7f4f23cd8b80 4 client.25960 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:55:48.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.013+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/553990061 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1d56d9020 con 0x55a1d5730770 2023-09-20T21:55:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.017+0000 7f4f0e19b700 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a1d5730770 0x55a1d5732b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:48.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.039+0000 7f4f0e19b700 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a1d5730770 0x55a1d5732b60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=946e377ece446b82 server_cookie=31f1cba02b1872c7 in_seq=0 out_seq=0 2023-09-20T21:55:48.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.039+0000 7f4ef1ffb700 10 client.25960 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:55:48.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.050+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/553990061 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a1d56d9020 con 0x55a1d5730770 2023-09-20T21:55:48.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.050+0000 7f4ef1ffb700 10 client.25960 handle_command_reply: tid=0 2023-09-20T21:55:48.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.050+0000 7f4f23cd8b80 1 client.25960 shutdown 2023-09-20T21:55:48.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.057+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/553990061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 msgr2=0x55a1d5714460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:48.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.057+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 0x55a1d5714460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f040d6340 tx=0x7f4f040d1d10 comp rx=0 tx=0).stop 2023-09-20T21:55:48.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.058+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/553990061 shutdown_connections 2023-09-20T21:55:48.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.058+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a1d5730770 0x55a1d5732b60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.058+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d570bbe0 0x55a1d571b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.058+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d5713f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.058+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/553990061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 0x55a1d5714460 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.058+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/553990061 >> 172.21.15.22:0/553990061 conn(0x55a1d5637000 msgr2=0x55a1d5704880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:48.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.058+0000 7f4ef1ffb700 0 client.25960 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:55:48.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.061+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/553990061 shutdown_connections 2023-09-20T21:55:48.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.064+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/553990061 wait complete. 2023-09-20T21:55:48.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.067+0000 7f4f23cd8b80 20 client.25960 trim_cache size 0 max 16384 2023-09-20T21:55:48.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.099+0000 7f4f23cd8b80 1 Processor -- start 2023-09-20T21:55:48.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.100+0000 7f4f23cd8b80 1 -- start start 2023-09-20T21:55:48.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.100+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 0x55a1d5712080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:48.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.100+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d571b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:48.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.100+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 0x55a1d5704880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:48.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.100+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d54dc920 con 0x55a1d5637000 2023-09-20T21:55:48.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.100+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d5660590 con 0x55a1d570a6f0 2023-09-20T21:55:48.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.100+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d56d9020 con 0x55a1d570b050 2023-09-20T21:55:48.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.100+0000 7f4f0d99a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d571b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:48.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.100+0000 7f4f0d99a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d571b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:35254/0 (socket says 172.21.15.22:35254) 2023-09-20T21:55:48.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.100+0000 7f4f0d99a700 1 -- 172.21.15.22:0/50552778 learned_addr learned my addr 172.21.15.22:0/50552778 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:55:48.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4f0e19b700 1 --2- 172.21.15.22:0/50552778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 0x55a1d5712080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:48.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4f0e99c700 1 --2- 172.21.15.22:0/50552778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 0x55a1d5704880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:48.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4f0d99a700 1 -- 172.21.15.22:0/50552778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 msgr2=0x55a1d5704880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:48.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/50552778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 0x55a1d5704880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4f0d99a700 1 -- 172.21.15.22:0/50552778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 msgr2=0x55a1d5712080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:48.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/50552778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 0x55a1d5712080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4f0d99a700 1 -- 172.21.15.22:0/50552778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1d54ad250 con 0x55a1d570a6f0 2023-09-20T21:55:48.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4f0e19b700 1 --2- 172.21.15.22:0/50552778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 0x55a1d5712080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:55:48.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/50552778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d571b460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4f04097940 tx=0x7f4f040d6370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfcc506ffe3b624d server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:48.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4efd3ff700 1 -- 172.21.15.22:0/50552778 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040d18c0 con 0x55a1d570a6f0 2023-09-20T21:55:48.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4efd3ff700 1 -- 172.21.15.22:0/50552778 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0409bb40 con 0x55a1d570a6f0 2023-09-20T21:55:48.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.101+0000 7f4efd3ff700 1 -- 172.21.15.22:0/50552778 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0409b210 con 0x55a1d570a6f0 2023-09-20T21:55:48.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.102+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/50552778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 msgr2=0x55a1d571b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:48.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.102+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/50552778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d571b460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4f04097940 tx=0x7f4f040d6370 comp rx=0 tx=0).stop 2023-09-20T21:55:48.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.103+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/50552778 shutdown_connections 2023-09-20T21:55:48.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.103+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/50552778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570b050 0x55a1d5704880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.103+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/50552778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 0x55a1d5712080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.103+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/50552778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570a6f0 0x55a1d571b460 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.103+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/50552778 >> 172.21.15.22:0/50552778 conn(0x55a1d5705500 msgr2=0x55a1d5712bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.103+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/50552778 shutdown_connections 2023-09-20T21:55:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.103+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/50552778 wait complete. 2023-09-20T21:55:48.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.104+0000 7f4f23cd8b80 1 -- start start 2023-09-20T21:55:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.104+0000 7f4f23cd8b80 1 Processor -- start 2023-09-20T21:55:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 0x55a1d571ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570a6f0 0x55a1d571b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f23cd8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570b050 0x55a1d5719380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:48.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d5719a60 con 0x55a1d5637000 2023-09-20T21:55:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d5719bd0 con 0x55a1d570b050 2023-09-20T21:55:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f23cd8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d5719d90 con 0x55a1d570a6f0 2023-09-20T21:55:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f23cd8b80 10 client.0 _ll_register_callbacks cb 0x55a1d5542a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:55:48.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f0d99a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570a6f0 0x55a1d571b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:48.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f0d99a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570a6f0 0x55a1d571b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:49660/0 (socket says 172.21.15.22:49660) 2023-09-20T21:55:48.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f0d99a700 1 -- 172.21.15.22:0/1186464891 learned_addr learned my addr 172.21.15.22:0/1186464891 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:55:48.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.105+0000 7f4f0e99c700 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570b050 0x55a1d5719380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:48.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4f0e19b700 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 0x55a1d571ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:48.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4f0d99a700 1 -- 172.21.15.22:0/1186464891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570b050 msgr2=0x55a1d5719380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:48.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570b050 0x55a1d5719380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4f0d99a700 1 -- 172.21.15.22:0/1186464891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 msgr2=0x55a1d571ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:55:48.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 0x55a1d571ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4f0d99a700 1 -- 172.21.15.22:0/1186464891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1d546fc60 con 0x55a1d570a6f0 2023-09-20T21:55:48.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4f0e19b700 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 0x55a1d571ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:55:48.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4f0d99a700 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570a6f0 0x55a1d571b2b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f0409ad30 tx=0x7f4f040a3420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b4bb9e784068740 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:55:48.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4ef1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:55:48.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/1186464891 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0409ada0 con 0x55a1d570a6f0 2023-09-20T21:55:48.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.106+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/1186464891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f04097050 con 0x55a1d570a6f0 2023-09-20T21:55:48.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/1186464891 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f4f040d18c0 con 0x55a1d570a6f0 2023-09-20T21:55:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/1186464891 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f0409b210 con 0x55a1d570a6f0 2023-09-20T21:55:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4f23cd8b80 10 client.25966 fetch_fsmap learned FSMap version 32 2023-09-20T21:55:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/1186464891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a1d54852b0 con 0x55a1d570a6f0 2023-09-20T21:55:48.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/1186464891 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f04097050 con 0x55a1d570a6f0 2023-09-20T21:55:48.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/1186464891 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4f040a10b0 con 0x55a1d570a6f0 2023-09-20T21:55:48.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4f23cd8b80 10 client.25966 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:55:48.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4f23cd8b80 10 client.25966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:55:48.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4f23cd8b80 20 client.25966 populate_metadata read hostname 'smithi022' 2023-09-20T21:55:48.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a1d570bbe0 0x55a1d57871c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:55:48.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4f23cd8b80 4 client.25966 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:55:48.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.107+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/1186464891 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a1d5654e60 con 0x55a1d570bbe0 2023-09-20T21:55:48.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.108+0000 7f4f0e19b700 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a1d570bbe0 0x55a1d57871c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:55:48.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.108+0000 7f4f0e19b700 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a1d570bbe0 0x55a1d57871c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d29e340fd67ac02 server_cookie=a099cb4f9f812568 in_seq=0 out_seq=0 2023-09-20T21:55:48.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.108+0000 7f4ef1ffb700 10 client.25966 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:55:48.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.108+0000 7f4ef1ffb700 1 -- 172.21.15.22:0/1186464891 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a1d5654e60 con 0x55a1d570bbe0 2023-09-20T21:55:48.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.108+0000 7f4ef1ffb700 10 client.25966 handle_command_reply: tid=0 2023-09-20T21:55:48.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.108+0000 7f4f23cd8b80 1 client.25966 shutdown 2023-09-20T21:55:48.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/1186464891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570a6f0 msgr2=0x55a1d571b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:48.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570a6f0 0x55a1d571b2b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f0409ad30 tx=0x7f4f040a3420 comp rx=0 tx=0).stop 2023-09-20T21:55:48.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/1186464891 shutdown_connections 2023-09-20T21:55:48.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55a1d570bbe0 0x55a1d57871c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a1d570b050 0x55a1d5719380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1d5637000 0x55a1d571ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 1 --2- 172.21.15.22:0/1186464891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a1d570a6f0 0x55a1d571b2b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4ef1ffb700 0 client.25966 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:55:48.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/1186464891 >> 172.21.15.22:0/1186464891 conn(0x55a1d5705500 msgr2=0x55a1d5712080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:48.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/1186464891 shutdown_connections 2023-09-20T21:55:48.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 1 -- 172.21.15.22:0/1186464891 wait complete. 2023-09-20T21:55:48.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.109+0000 7f4f23cd8b80 20 client.25966 trim_cache size 0 max 16384 2023-09-20T21:55:48.125 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:55:48.125 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:55:48.126 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:55:48.126 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:55:48.126 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:55:48.126 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:55:48.126 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:55:48.127 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:55:48.127 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:55:48.127 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:55:48.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.110+0000 7f4f103ff700 1 -- 172.21.15.22:0/459579471 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4ef40a1400 msgr2=0x7f4ef40a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:48.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.110+0000 7f4f103ff700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4ef40a1400 0x7f4ef40a38c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55a1d57060b0 tx=0x7f4f00007620 comp rx=0 tx=0).stop 2023-09-20T21:55:48.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.110+0000 7f4f103ff700 1 -- 172.21.15.22:0/459579471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f0810a750 msgr2=0x7f4f081a5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:55:48.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.110+0000 7f4f103ff700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f0810a750 0x7f4f081a5e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400bc60 tx=0x7f4f0400bc90 comp rx=0 tx=0).stop 2023-09-20T21:55:48.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.110+0000 7f4f103ff700 1 -- 172.21.15.22:0/459579471 shutdown_connections 2023-09-20T21:55:48.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.110+0000 7f4f103ff700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4f0810d120 0x7f4f081ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.110+0000 7f4f103ff700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f4ef40a1400 0x7f4ef40a38c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.110+0000 7f4f103ff700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f08107de0 0x7f4f081a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.110+0000 7f4f103ff700 1 --2- 172.21.15.22:0/459579471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f0810a750 0x7f4f081a5e10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:55:48.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.110+0000 7f4f103ff700 1 -- 172.21.15.22:0/459579471 >> 172.21.15.22:0/459579471 conn(0x7f4f0806db90 msgr2=0x7f4f08108a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:55:48.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.111+0000 7f4f103ff700 1 -- 172.21.15.22:0/459579471 shutdown_connections 2023-09-20T21:55:48.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:55:48.111+0000 7f4f103ff700 1 -- 172.21.15.22:0/459579471 wait complete. 2023-09-20T21:55:48.142 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:55:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 385.0 s, 547.2 tps, lat 185.518 ms stddev 201.054 2023-09-20T21:55:49.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:48 smithi022 ceph-mon[97651]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 583 op/s 2023-09-20T21:55:49.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:48 smithi123 ceph-mon[134433]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 583 op/s 2023-09-20T21:55:49.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:48 smithi089 ceph-mon[134090]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 MiB/s wr, 583 op/s 2023-09-20T21:55:51.023 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:50 smithi022 ceph-mon[97651]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 555 op/s 2023-09-20T21:55:51.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:50 smithi123 ceph-mon[134433]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 555 op/s 2023-09-20T21:55:51.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:50 smithi089 ceph-mon[134090]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 555 op/s 2023-09-20T21:55:53.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:52 smithi123 ceph-mon[134433]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 580 op/s 2023-09-20T21:55:53.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:52 smithi089 ceph-mon[134090]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 580 op/s 2023-09-20T21:55:53.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:52 smithi022 ceph-mon[97651]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 580 op/s 2023-09-20T21:55:53.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 390.0 s, 507.4 tps, lat 170.425 ms stddev 226.882 2023-09-20T21:55:54.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:54 smithi123 ceph-mon[134433]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 541 op/s 2023-09-20T21:55:54.364 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:54 smithi089 ceph-mon[134090]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 541 op/s 2023-09-20T21:55:54.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:54 smithi022 ceph-mon[97651]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 541 op/s 2023-09-20T21:55:56.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:56 smithi022 ceph-mon[97651]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 811 op/s 2023-09-20T21:55:57.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:56 smithi123 ceph-mon[134433]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 811 op/s 2023-09-20T21:55:57.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:56 smithi089 ceph-mon[134090]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 811 op/s 2023-09-20T21:55:58.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 395.0 s, 458.6 tps, lat 240.480 ms stddev 309.531 2023-09-20T21:55:59.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:55:58 smithi022 ceph-mon[97651]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-09-20T21:55:59.100 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:55:58 smithi123 ceph-mon[134433]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-09-20T21:55:59.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:55:58 smithi089 ceph-mon[134090]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-09-20T21:56:01.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:00 smithi123 ceph-mon[134433]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2023-09-20T21:56:01.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:00 smithi089 ceph-mon[134090]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2023-09-20T21:56:01.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:00 smithi022 ceph-mon[97651]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2023-09-20T21:56:02.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:01 smithi123 ceph-mon[134433]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2023-09-20T21:56:02.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:01 smithi089 ceph-mon[134090]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2023-09-20T21:56:02.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:01 smithi022 ceph-mon[97651]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2023-09-20T21:56:03.575 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 400.0 s, 526.3 tps, lat 197.528 ms stddev 217.579 2023-09-20T21:56:04.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:04 smithi123 ceph-mon[134433]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 936 op/s 2023-09-20T21:56:04.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:04 smithi089 ceph-mon[134090]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 936 op/s 2023-09-20T21:56:04.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:04 smithi022 ceph-mon[97651]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 936 op/s 2023-09-20T21:56:06.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:06 smithi022 ceph-mon[97651]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 971 op/s 2023-09-20T21:56:07.102 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:06 smithi123 ceph-mon[134433]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 971 op/s 2023-09-20T21:56:07.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:06 smithi089 ceph-mon[134090]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 971 op/s 2023-09-20T21:56:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 405.0 s, 582.8 tps, lat 172.531 ms stddev 197.598 2023-09-20T21:56:08.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:08 smithi022 ceph-mon[97651]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.6 MiB/s wr, 775 op/s 2023-09-20T21:56:09.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:08 smithi123 ceph-mon[134433]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.6 MiB/s wr, 775 op/s 2023-09-20T21:56:09.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:08 smithi089 ceph-mon[134090]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.6 MiB/s wr, 775 op/s 2023-09-20T21:56:10.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:10 smithi022 ceph-mon[97651]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 509 op/s 2023-09-20T21:56:11.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:10 smithi123 ceph-mon[134433]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 509 op/s 2023-09-20T21:56:11.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:10 smithi089 ceph-mon[134090]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.3 MiB/s wr, 509 op/s 2023-09-20T21:56:13.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:12 smithi123 ceph-mon[134433]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 555 op/s 2023-09-20T21:56:13.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:12 smithi089 ceph-mon[134090]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 555 op/s 2023-09-20T21:56:13.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:12 smithi022 ceph-mon[97651]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 555 op/s 2023-09-20T21:56:13.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 410.0 s, 507.4 tps, lat 195.647 ms stddev 237.838 2023-09-20T21:56:14.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:14 smithi123 ceph-mon[134433]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.1 MiB/s wr, 483 op/s 2023-09-20T21:56:14.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:14 smithi089 ceph-mon[134090]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.1 MiB/s wr, 483 op/s 2023-09-20T21:56:14.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:14 smithi022 ceph-mon[97651]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.1 MiB/s wr, 483 op/s 2023-09-20T21:56:16.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:16 smithi022 ceph-mon[97651]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.2 MiB/s wr, 572 op/s 2023-09-20T21:56:17.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:16 smithi123 ceph-mon[134433]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.2 MiB/s wr, 572 op/s 2023-09-20T21:56:17.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:16 smithi089 ceph-mon[134090]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.2 MiB/s wr, 572 op/s 2023-09-20T21:56:18.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:56:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 415.0 s, 464.0 tps, lat 206.866 ms stddev 233.395 2023-09-20T21:56:19.108 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:18 smithi123 ceph-mon[134433]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 617 op/s 2023-09-20T21:56:19.109 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:18 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:56:19.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:18 smithi089 ceph-mon[134090]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 617 op/s 2023-09-20T21:56:19.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:18 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:56:19.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:18 smithi022 ceph-mon[97651]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 617 op/s 2023-09-20T21:56:19.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:18 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:56:19.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.853+0000 7f05da1ff700 1 -- 172.21.15.22:0/330927954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d4107de0 msgr2=0x7f05d410a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:19.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.854+0000 7f05da1ff700 1 --2- 172.21.15.22:0/330927954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d4107de0 0x7f05d410a210 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f05c401a9a0 tx=0x7f05c401acb0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.854+0000 7f05da1ff700 1 -- 172.21.15.22:0/330927954 shutdown_connections 2023-09-20T21:56:19.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.854+0000 7f05da1ff700 1 --2- 172.21.15.22:0/330927954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d410d120 0x7f05d410d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.854+0000 7f05da1ff700 1 --2- 172.21.15.22:0/330927954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d4107de0 0x7f05d410a210 secure :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f05c401a9a0 tx=0x7f05c401acb0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.854+0000 7f05da1ff700 1 --2- 172.21.15.22:0/330927954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05d410a750 0x7f05d410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.854+0000 7f05da1ff700 1 -- 172.21.15.22:0/330927954 >> 172.21.15.22:0/330927954 conn(0x7f05d406db90 msgr2=0x7f05d406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:19.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.871+0000 7f05da1ff700 1 -- 172.21.15.22:0/330927954 shutdown_connections 2023-09-20T21:56:19.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.881+0000 7f05da1ff700 1 -- 172.21.15.22:0/330927954 wait complete. 2023-09-20T21:56:19.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.883+0000 7f05da1ff700 1 Processor -- start 2023-09-20T21:56:19.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.885+0000 7f05da1ff700 1 -- start start 2023-09-20T21:56:19.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.893+0000 7f05da1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d410a750 0x7f05d419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:19.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.893+0000 7f05da1ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05d410d120 0x7f05d419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:19.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.893+0000 7f05da1ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d41a2a20 0x7f05d41a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:19.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.893+0000 7f05da1ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d41a53e0 con 0x7f05d41a2a20 2023-09-20T21:56:19.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.893+0000 7f05da1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d41a5520 con 0x7f05d410a750 2023-09-20T21:56:19.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.893+0000 7f05da1ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d41a56b0 con 0x7f05d410d120 2023-09-20T21:56:19.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.894+0000 7f05d3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d41a2a20 0x7f05d41a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:19.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.894+0000 7f05d3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d41a2a20 0x7f05d41a4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43560/0 (socket says 172.21.15.22:43560) 2023-09-20T21:56:19.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.894+0000 7f05d3fff700 1 -- 172.21.15.22:0/2086137132 learned_addr learned my addr 172.21.15.22:0/2086137132 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:56:19.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.895+0000 7f05d3fff700 1 -- 172.21.15.22:0/2086137132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05d410d120 msgr2=0x7f05d419d9c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:56:19.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.895+0000 7f05d3fff700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05d410d120 0x7f05d419d9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.895+0000 7f05d3fff700 1 -- 172.21.15.22:0/2086137132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d410a750 msgr2=0x7f05d419d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:56:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.895+0000 7f05d3fff700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d410a750 0x7f05d419d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.895+0000 7f05d3fff700 1 -- 172.21.15.22:0/2086137132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c401a600 con 0x7f05d41a2a20 2023-09-20T21:56:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.895+0000 7f05d3fff700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d41a2a20 0x7f05d41a4e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00b780 tx=0x7f05cc00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:19.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.900+0000 7f05d0ff9700 1 -- 172.21.15.22:0/2086137132 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc00f840 con 0x7f05d41a2a20 2023-09-20T21:56:19.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.900+0000 7f05d0ff9700 1 -- 172.21.15.22:0/2086137132 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05cc00fe60 con 0x7f05d41a2a20 2023-09-20T21:56:19.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.900+0000 7f05d0ff9700 1 -- 172.21.15.22:0/2086137132 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc00d3d0 con 0x7f05d41a2a20 2023-09-20T21:56:19.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.900+0000 7f05da1ff700 1 -- 172.21.15.22:0/2086137132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d41a5990 con 0x7f05d41a2a20 2023-09-20T21:56:19.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.900+0000 7f05da1ff700 1 -- 172.21.15.22:0/2086137132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d41a5ed0 con 0x7f05d41a2a20 2023-09-20T21:56:19.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.907+0000 7f05edafeb80 1 Processor -- start 2023-09-20T21:56:19.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.908+0000 7f05edafeb80 1 -- start start 2023-09-20T21:56:19.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.908+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 0x5561b819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:19.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.908+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 0x5561b819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:19.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.908+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819fbe0 0x5561b81a0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:19.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.908+0000 7f05edafeb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b8112ca0 con 0x5561b819f050 2023-09-20T21:56:19.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.908+0000 7f05edafeb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b80f6c70 con 0x5561b819fbe0 2023-09-20T21:56:19.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.908+0000 7f05edafeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b80f4590 con 0x5561b819e6f0 2023-09-20T21:56:19.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.909+0000 7f05d2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 0x5561b819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:19.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.909+0000 7f05d2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 0x5561b819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:37678/0 (socket says 172.21.15.22:37678) 2023-09-20T21:56:19.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.909+0000 7f05d2ffd700 1 -- 172.21.15.22:0/639042946 learned_addr learned my addr 172.21.15.22:0/639042946 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:56:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.913+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/639042946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819fbe0 0x5561b81a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:19.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.925+0000 7f05d0ff9700 1 -- 172.21.15.22:0/2086137132 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f05cc017690 con 0x7f05d41a2a20 2023-09-20T21:56:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.925+0000 7f05d2ffd700 1 -- 172.21.15.22:0/639042946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819fbe0 msgr2=0x5561b81a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.925+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/639042946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819fbe0 0x5561b81a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.925+0000 7f05d2ffd700 1 -- 172.21.15.22:0/639042946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 msgr2=0x5561b819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:19.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.925+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/639042946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 0x5561b819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.925+0000 7f05d2ffd700 1 -- 172.21.15.22:0/639042946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561b7f1ade0 con 0x5561b819e6f0 2023-09-20T21:56:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.927+0000 7f05d0ff9700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05bc0a1500 0x7f05bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.928+0000 7f05d0ff9700 1 -- 172.21.15.22:0/2086137132 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f05cc0cce80 con 0x7f05d41a2a20 2023-09-20T21:56:19.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.931+0000 7f05d37fe700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05bc0a1500 0x7f05bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.932+0000 7f05d37fe700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05bc0a1500 0x7f05bc0a39c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5561b819a010 tx=0x7f05c401c9f0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:19.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.937+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/639042946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 0x5561b819eb10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009eb0 tx=0x7f05c8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f9103205486f81d server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:19.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.940+0000 7f05b9ffb700 1 -- 172.21.15.22:0/639042946 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8017400 con 0x5561b819e6f0 2023-09-20T21:56:19.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.940+0000 7f05b9ffb700 1 -- 172.21.15.22:0/639042946 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05c8009460 con 0x5561b819e6f0 2023-09-20T21:56:19.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.940+0000 7f05b9ffb700 1 -- 172.21.15.22:0/639042946 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8012740 con 0x5561b819e6f0 2023-09-20T21:56:19.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.941+0000 7f05edafeb80 1 -- 172.21.15.22:0/639042946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 msgr2=0x5561b819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:19.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.941+0000 7f05edafeb80 1 --2- 172.21.15.22:0/639042946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 0x5561b819eb10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009eb0 tx=0x7f05c8009310 comp rx=0 tx=0).stop 2023-09-20T21:56:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.954+0000 7f05edafeb80 1 -- 172.21.15.22:0/639042946 shutdown_connections 2023-09-20T21:56:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.954+0000 7f05edafeb80 1 --2- 172.21.15.22:0/639042946 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819fbe0 0x5561b81a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.954+0000 7f05edafeb80 1 --2- 172.21.15.22:0/639042946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 0x5561b819eb10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.954+0000 7f05edafeb80 1 --2- 172.21.15.22:0/639042946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 0x5561b819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.954+0000 7f05edafeb80 1 -- 172.21.15.22:0/639042946 >> 172.21.15.22:0/639042946 conn(0x5561b80cb000 msgr2=0x5561b81991d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.956+0000 7f05edafeb80 1 -- 172.21.15.22:0/639042946 shutdown_connections 2023-09-20T21:56:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.958+0000 7f05edafeb80 1 -- 172.21.15.22:0/639042946 wait complete. 2023-09-20T21:56:19.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.959+0000 7f05edafeb80 1 -- start start 2023-09-20T21:56:19.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.959+0000 7f05edafeb80 1 Processor -- start 2023-09-20T21:56:19.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.969+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 0x5561b81a8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:19.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.969+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819f050 0x5561b81a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:19.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.969+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819fbe0 0x5561b81af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:19.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.969+0000 7f05edafeb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b80efe80 con 0x5561b819fbe0 2023-09-20T21:56:19.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.969+0000 7f05edafeb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b808c740 con 0x5561b819f050 2023-09-20T21:56:19.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.969+0000 7f05edafeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b808c3e0 con 0x5561b819e6f0 2023-09-20T21:56:19.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.970+0000 7f05edafeb80 10 client.0 _ll_register_callbacks cb 0x5561b7fd6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:56:19.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.971+0000 7f05d2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819f050 0x5561b81a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:19.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.971+0000 7f05d2ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819f050 0x5561b81a87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:56120/0 (socket says 172.21.15.22:56120) 2023-09-20T21:56:19.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.971+0000 7f05d2ffd700 1 -- 172.21.15.22:0/1187000109 learned_addr learned my addr 172.21.15.22:0/1187000109 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:56:19.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.975+0000 7f05d2ffd700 1 -- 172.21.15.22:0/1187000109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 msgr2=0x5561b81a8260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:56:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.975+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 0x5561b81a8260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.975+0000 7f05d2ffd700 1 -- 172.21.15.22:0/1187000109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819fbe0 msgr2=0x5561b81af7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:56:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.975+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819fbe0 0x5561b81af7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.975+0000 7f05d2ffd700 1 -- 172.21.15.22:0/1187000109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c8014290 con 0x5561b819f050 2023-09-20T21:56:19.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.991+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819f050 0x5561b81a87a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f05c8002040 tx=0x7f05c8017ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbbf6051d39f2f30 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:19.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.991+0000 7f05b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:56:19.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.992+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1187000109 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8012480 con 0x5561b819f050 2023-09-20T21:56:19.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.992+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1187000109 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f05c80125e0 con 0x5561b819f050 2023-09-20T21:56:19.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.992+0000 7f05edafeb80 1 -- 172.21.15.22:0/1187000109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561b816d020 con 0x5561b819f050 2023-09-20T21:56:20.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:19.998+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1187000109 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05c80127a0 con 0x5561b819f050 2023-09-20T21:56:20.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.003+0000 7f05edafeb80 10 client.25961 fetch_fsmap learned FSMap version 32 2023-09-20T21:56:20.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.003+0000 7f05edafeb80 1 -- 172.21.15.22:0/1187000109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5561b7f03c60 con 0x5561b819f050 2023-09-20T21:56:20.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.006+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1187000109 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561b816d020 con 0x5561b819f050 2023-09-20T21:56:20.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.006+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1187000109 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f05c8022660 con 0x5561b819f050 2023-09-20T21:56:20.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.006+0000 7f05edafeb80 10 client.25961 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:56:20.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.006+0000 7f05edafeb80 10 client.25961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:56:20.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.006+0000 7f05edafeb80 20 client.25961 populate_metadata read hostname 'smithi022' 2023-09-20T21:56:20.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.006+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5561b81b3e40 0x5561b81b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:20.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.006+0000 7f05edafeb80 4 client.25961 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:56:20.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.006+0000 7f05edafeb80 1 -- 172.21.15.22:0/1187000109 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561b80e8e60 con 0x5561b81b3e40 2023-09-20T21:56:20.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.007+0000 7f05d37fe700 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5561b81b3e40 0x5561b81b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:20.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.013+0000 7f05d37fe700 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5561b81b3e40 0x5561b81b6260 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a60e49814fe356c server_cookie=5903c103a6dc3910 in_seq=0 out_seq=0 2023-09-20T21:56:20.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.013+0000 7f05b2ffd700 10 client.25961 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:56:20.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.024+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1187000109 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5561b80e8e60 con 0x5561b81b3e40 2023-09-20T21:56:20.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.024+0000 7f05b2ffd700 10 client.25961 handle_command_reply: tid=0 2023-09-20T21:56:20.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.025+0000 7f05edafeb80 1 client.25961 shutdown 2023-09-20T21:56:20.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.028+0000 7f05edafeb80 1 -- 172.21.15.22:0/1187000109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819f050 msgr2=0x5561b81a87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:20.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.028+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819f050 0x5561b81a87a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f05c8002040 tx=0x7f05c8017ac0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.032+0000 7f05edafeb80 1 -- 172.21.15.22:0/1187000109 shutdown_connections 2023-09-20T21:56:20.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.032+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5561b81b3e40 0x5561b81b6260 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.033+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819fbe0 0x5561b81af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.033+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819e6f0 0x5561b81a8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.033+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1187000109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819f050 0x5561b81a87a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.033+0000 7f05edafeb80 1 -- 172.21.15.22:0/1187000109 >> 172.21.15.22:0/1187000109 conn(0x5561b80cb000 msgr2=0x5561b8198880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:20.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.033+0000 7f05b2ffd700 0 client.25961 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:56:20.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.033+0000 7f05edafeb80 1 -- 172.21.15.22:0/1187000109 shutdown_connections 2023-09-20T21:56:20.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.033+0000 7f05edafeb80 1 -- 172.21.15.22:0/1187000109 wait complete. 2023-09-20T21:56:20.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.034+0000 7f05edafeb80 20 client.25961 trim_cache size 0 max 16384 2023-09-20T21:56:20.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.071+0000 7f05edafeb80 1 Processor -- start 2023-09-20T21:56:20.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.071+0000 7f05edafeb80 1 -- start start 2023-09-20T21:56:20.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.072+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b80cb000 0x5561b81b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:20.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.072+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819e6f0 0x5561b81af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:20.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.072+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 0x5561b8198880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:20.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.072+0000 7f05edafeb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b7f70920 con 0x5561b819f050 2023-09-20T21:56:20.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.072+0000 7f05edafeb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b819ab90 con 0x5561b819e6f0 2023-09-20T21:56:20.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.072+0000 7f05edafeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b81a7760 con 0x5561b80cb000 2023-09-20T21:56:20.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.073+0000 7f05d3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 0x5561b8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:20.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.073+0000 7f05d3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 0x5561b8198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43584/0 (socket says 172.21.15.22:43584) 2023-09-20T21:56:20.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.073+0000 7f05d3fff700 1 -- 172.21.15.22:0/634095915 learned_addr learned my addr 172.21.15.22:0/634095915 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:56:20.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.073+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/634095915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819e6f0 0x5561b81af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:20.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.073+0000 7f05d3fff700 1 -- 172.21.15.22:0/634095915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b80cb000 msgr2=0x5561b81b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:20.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.073+0000 7f05d3fff700 1 --2- 172.21.15.22:0/634095915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b80cb000 0x5561b81b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.073+0000 7f05d3fff700 1 -- 172.21.15.22:0/634095915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819e6f0 msgr2=0x5561b81af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:20.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.073+0000 7f05d3fff700 1 --2- 172.21.15.22:0/634095915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819e6f0 0x5561b81af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.073+0000 7f05d3fff700 1 -- 172.21.15.22:0/634095915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561b7f41250 con 0x5561b819f050 2023-09-20T21:56:20.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.073+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/634095915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819e6f0 0x5561b81af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:56:20.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.074+0000 7f05d3fff700 1 --2- 172.21.15.22:0/634095915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 0x5561b8198880 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f05cc096320 tx=0x7f05cc099800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3fb6d022faff42 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:20.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.074+0000 7f05ba7fc700 1 -- 172.21.15.22:0/634095915 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc0b2220 con 0x5561b819f050 2023-09-20T21:56:20.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.074+0000 7f05ba7fc700 1 -- 172.21.15.22:0/634095915 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05cc09dc50 con 0x5561b819f050 2023-09-20T21:56:20.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.074+0000 7f05ba7fc700 1 -- 172.21.15.22:0/634095915 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc09a080 con 0x5561b819f050 2023-09-20T21:56:20.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.075+0000 7f05edafeb80 1 -- 172.21.15.22:0/634095915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 msgr2=0x5561b8198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:20.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.075+0000 7f05edafeb80 1 --2- 172.21.15.22:0/634095915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 0x5561b8198880 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f05cc096320 tx=0x7f05cc099800 comp rx=0 tx=0).stop 2023-09-20T21:56:20.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.075+0000 7f05edafeb80 1 -- 172.21.15.22:0/634095915 shutdown_connections 2023-09-20T21:56:20.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.075+0000 7f05edafeb80 1 --2- 172.21.15.22:0/634095915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819f050 0x5561b8198880 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.075+0000 7f05edafeb80 1 --2- 172.21.15.22:0/634095915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b80cb000 0x5561b81b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.075+0000 7f05edafeb80 1 --2- 172.21.15.22:0/634095915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b819e6f0 0x5561b81af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.075+0000 7f05edafeb80 1 -- 172.21.15.22:0/634095915 >> 172.21.15.22:0/634095915 conn(0x5561b8199500 msgr2=0x5561b81a6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:20.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.075+0000 7f05edafeb80 1 -- 172.21.15.22:0/634095915 shutdown_connections 2023-09-20T21:56:20.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.076+0000 7f05edafeb80 1 -- 172.21.15.22:0/634095915 wait complete. 2023-09-20T21:56:20.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.076+0000 7f05edafeb80 1 -- start start 2023-09-20T21:56:20.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.076+0000 7f05edafeb80 1 Processor -- start 2023-09-20T21:56:20.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.077+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b80cb000 0x5561b81af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:20.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.077+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819e6f0 0x5561b81af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:20.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.077+0000 7f05edafeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819f050 0x5561b81ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:20.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.077+0000 7f05edafeb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b81ade30 con 0x5561b819e6f0 2023-09-20T21:56:20.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.077+0000 7f05edafeb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b81adfa0 con 0x5561b80cb000 2023-09-20T21:56:20.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.077+0000 7f05edafeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561b81ae160 con 0x5561b819f050 2023-09-20T21:56:20.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.077+0000 7f05edafeb80 10 client.0 _ll_register_callbacks cb 0x5561b7fd6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:56:20.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.077+0000 7f05d37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b80cb000 0x5561b81af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:20.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.077+0000 7f05d37fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b80cb000 0x5561b81af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:56134/0 (socket says 172.21.15.22:56134) 2023-09-20T21:56:20.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.077+0000 7f05d37fe700 1 -- 172.21.15.22:0/1288653791 learned_addr learned my addr 172.21.15.22:0/1288653791 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:56:20.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.078+0000 7f05d3fff700 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819f050 0x5561b81ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:20.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.078+0000 7f05d37fe700 1 -- 172.21.15.22:0/1288653791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819f050 msgr2=0x5561b81ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:20.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.078+0000 7f05d37fe700 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819f050 0x5561b81ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.078+0000 7f05d37fe700 1 -- 172.21.15.22:0/1288653791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819e6f0 msgr2=0x5561b81af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:20.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.078+0000 7f05d37fe700 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819e6f0 0x5561b81af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.078+0000 7f05d37fe700 1 -- 172.21.15.22:0/1288653791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d41a5ed0 con 0x5561b80cb000 2023-09-20T21:56:20.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.078+0000 7f05d37fe700 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b80cb000 0x5561b81af150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f05c401af50 tx=0x7f05c4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a509f895cf3e414 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:20.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.078+0000 7f05b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:56:20.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.079+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1288653791 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c402d070 con 0x5561b80cb000 2023-09-20T21:56:20.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.079+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1288653791 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f05c403dcb0 con 0x5561b80cb000 2023-09-20T21:56:20.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.079+0000 7f05edafeb80 1 -- 172.21.15.22:0/1288653791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561b8060bb0 con 0x5561b80cb000 2023-09-20T21:56:20.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.081+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1288653791 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05c403de70 con 0x5561b80cb000 2023-09-20T21:56:20.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.081+0000 7f05edafeb80 10 client.25964 fetch_fsmap learned FSMap version 32 2023-09-20T21:56:20.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.081+0000 7f05edafeb80 1 -- 172.21.15.22:0/1288653791 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5561b7f192b0 con 0x5561b80cb000 2023-09-20T21:56:20.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.082+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1288653791 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561b8060bb0 con 0x5561b80cb000 2023-09-20T21:56:20.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.082+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1288653791 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f05c4046070 con 0x5561b80cb000 2023-09-20T21:56:20.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.083+0000 7f05edafeb80 10 client.25964 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:56:20.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.083+0000 7f05edafeb80 10 client.25964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:56:20.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.083+0000 7f05edafeb80 20 client.25964 populate_metadata read hostname 'smithi022' 2023-09-20T21:56:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.083+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5561b819fbe0 0x5561b820b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.083+0000 7f05edafeb80 4 client.25964 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:56:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.083+0000 7f05edafeb80 1 -- 172.21.15.22:0/1288653791 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5561b80e8b80 con 0x5561b819fbe0 2023-09-20T21:56:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.083+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5561b819fbe0 0x5561b820b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.084+0000 7f05b2ffd700 10 client.25964 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:56:20.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.084+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5561b819fbe0 0x5561b820b000 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e40ebb6b76491a6 server_cookie=f86b326529ca52cb in_seq=0 out_seq=0 2023-09-20T21:56:20.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.084+0000 7f05b2ffd700 1 -- 172.21.15.22:0/1288653791 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5561b80e8b80 con 0x5561b819fbe0 2023-09-20T21:56:20.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.084+0000 7f05b2ffd700 10 client.25964 handle_command_reply: tid=0 2023-09-20T21:56:20.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.084+0000 7f05edafeb80 1 client.25964 shutdown 2023-09-20T21:56:20.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.084+0000 7f05edafeb80 1 -- 172.21.15.22:0/1288653791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b80cb000 msgr2=0x5561b81af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:20.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.084+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b80cb000 0x5561b81af150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f05c401af50 tx=0x7f05c4000bc0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.085+0000 7f05edafeb80 1 -- 172.21.15.22:0/1288653791 shutdown_connections 2023-09-20T21:56:20.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.085+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5561b819fbe0 0x5561b820b000 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.085+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561b819f050 0x5561b81ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.085+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561b80cb000 0x5561b81af150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.085+0000 7f05edafeb80 1 --2- 172.21.15.22:0/1288653791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5561b819e6f0 0x5561b81af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.085+0000 7f05edafeb80 1 -- 172.21.15.22:0/1288653791 >> 172.21.15.22:0/1288653791 conn(0x5561b8199500 msgr2=0x5561b81b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:20.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.085+0000 7f05b2ffd700 0 client.25964 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:56:20.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.085+0000 7f05edafeb80 1 -- 172.21.15.22:0/1288653791 shutdown_connections 2023-09-20T21:56:20.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.085+0000 7f05edafeb80 1 -- 172.21.15.22:0/1288653791 wait complete. 2023-09-20T21:56:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.088+0000 7f05edafeb80 20 client.25964 trim_cache size 0 max 16384 2023-09-20T21:56:20.094 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:56:20.094 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:56:20.094 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:56:20.095 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:56:20.095 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:56:20.095 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:56:20.095 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:56:20.095 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:56:20.095 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:56:20.096 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:56:20.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:19 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.095+0000 7f05da1ff700 1 -- 172.21.15.22:0/2086137132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05bc0a1500 msgr2=0x7f05bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:20.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.095+0000 7f05da1ff700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05bc0a1500 0x7f05bc0a39c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5561b819a010 tx=0x7f05c401c9f0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.095+0000 7f05da1ff700 1 -- 172.21.15.22:0/2086137132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d41a2a20 msgr2=0x7f05d41a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:20.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.095+0000 7f05da1ff700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d41a2a20 0x7f05d41a4e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00b780 tx=0x7f05cc00bb40 comp rx=0 tx=0).stop 2023-09-20T21:56:20.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.097+0000 7f05da1ff700 1 -- 172.21.15.22:0/2086137132 shutdown_connections 2023-09-20T21:56:20.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.097+0000 7f05da1ff700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d41a2a20 0x7f05d41a4e10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.097+0000 7f05da1ff700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f05bc0a1500 0x7f05bc0a39c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.097+0000 7f05da1ff700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f05d410a750 0x7f05d419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.097+0000 7f05da1ff700 1 --2- 172.21.15.22:0/2086137132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05d410d120 0x7f05d419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:20.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.097+0000 7f05da1ff700 1 -- 172.21.15.22:0/2086137132 >> 172.21.15.22:0/2086137132 conn(0x7f05d406db90 msgr2=0x7f05d406ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:20.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.098+0000 7f05da1ff700 1 -- 172.21.15.22:0/2086137132 shutdown_connections 2023-09-20T21:56:20.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:20.098+0000 7f05da1ff700 1 -- 172.21.15.22:0/2086137132 wait complete. 2023-09-20T21:56:20.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:19 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.146 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:56:20.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.147 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:20.148 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:19 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:20 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi123 to 196.2M 2023-09-20T21:56:21.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:20 smithi123 ceph-mon[134433]: Unable to set osd_memory_target on smithi123 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-20T21:56:21.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:20 smithi123 ceph-mon[134433]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.9 MiB/s wr, 566 op/s 2023-09-20T21:56:21.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:20 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:20 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi123 to 196.2M 2023-09-20T21:56:21.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:20 smithi089 ceph-mon[134090]: Unable to set osd_memory_target on smithi123 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-20T21:56:21.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:20 smithi089 ceph-mon[134090]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.9 MiB/s wr, 566 op/s 2023-09-20T21:56:21.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:20 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:20 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi123 to 196.2M 2023-09-20T21:56:21.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:20 smithi022 ceph-mon[97651]: Unable to set osd_memory_target on smithi123 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-09-20T21:56:21.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:20 smithi022 ceph-mon[97651]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.9 MiB/s wr, 566 op/s 2023-09-20T21:56:21.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:21.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:20 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-09-20T21:56:22.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:21 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi089 to 196.1M 2023-09-20T21:56:22.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:21 smithi123 ceph-mon[134433]: Unable to set osd_memory_target on smithi089 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-09-20T21:56:22.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:56:22.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:56:22.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:56:22.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:21 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi089 to 196.1M 2023-09-20T21:56:22.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:21 smithi089 ceph-mon[134090]: Unable to set osd_memory_target on smithi089 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-09-20T21:56:22.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:56:22.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:56:22.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:56:22.255 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:21 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi089 to 196.1M 2023-09-20T21:56:22.255 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:21 smithi022 ceph-mon[97651]: Unable to set osd_memory_target on smithi089 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-09-20T21:56:22.255 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:56:22.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:56:22.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:56:23.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:22 smithi022 ceph-mon[97651]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.1 MiB/s wr, 650 op/s 2023-09-20T21:56:23.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:22 smithi123 ceph-mon[134433]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.1 MiB/s wr, 650 op/s 2023-09-20T21:56:23.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:22 smithi089 ceph-mon[134090]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.1 MiB/s wr, 650 op/s 2023-09-20T21:56:23.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 420.0 s, 557.3 tps, lat 188.273 ms stddev 238.913 2023-09-20T21:56:24.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:24 smithi123 ceph-mon[134433]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.6 MiB/s wr, 540 op/s 2023-09-20T21:56:24.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:24 smithi089 ceph-mon[134090]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.6 MiB/s wr, 540 op/s 2023-09-20T21:56:24.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:24 smithi022 ceph-mon[97651]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.6 MiB/s wr, 540 op/s 2023-09-20T21:56:26.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:26 smithi022 ceph-mon[97651]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.0 MiB/s wr, 584 op/s 2023-09-20T21:56:27.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:26 smithi123 ceph-mon[134433]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.0 MiB/s wr, 584 op/s 2023-09-20T21:56:27.118 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:26 smithi089 ceph-mon[134090]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.0 MiB/s wr, 584 op/s 2023-09-20T21:56:28.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 425.0 s, 567.7 tps, lat 171.544 ms stddev 176.864 2023-09-20T21:56:29.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:28 smithi123 ceph-mon[134433]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 583 op/s 2023-09-20T21:56:29.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:28 smithi089 ceph-mon[134090]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 583 op/s 2023-09-20T21:56:29.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:28 smithi022 ceph-mon[97651]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 583 op/s 2023-09-20T21:56:31.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:30 smithi089 ceph-mon[134090]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.3 MiB/s wr, 496 op/s 2023-09-20T21:56:31.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:30 smithi022 ceph-mon[97651]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.3 MiB/s wr, 496 op/s 2023-09-20T21:56:31.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:30 smithi123 ceph-mon[134433]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.3 MiB/s wr, 496 op/s 2023-09-20T21:56:33.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:32 smithi022 ceph-mon[97651]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.6 MiB/s wr, 591 op/s 2023-09-20T21:56:33.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:32 smithi123 ceph-mon[134433]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.6 MiB/s wr, 591 op/s 2023-09-20T21:56:33.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:32 smithi089 ceph-mon[134090]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.6 MiB/s wr, 591 op/s 2023-09-20T21:56:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 430.0 s, 544.8 tps, lat 186.281 ms stddev 196.615 2023-09-20T21:56:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:34 smithi022 ceph-mon[97651]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 475 op/s 2023-09-20T21:56:34.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:34 smithi123 ceph-mon[134433]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 475 op/s 2023-09-20T21:56:34.619 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:34 smithi089 ceph-mon[134090]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 475 op/s 2023-09-20T21:56:37.003 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:36 smithi022 ceph-mon[97651]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.8 MiB/s wr, 531 op/s 2023-09-20T21:56:37.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:36 smithi123 ceph-mon[134433]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.8 MiB/s wr, 531 op/s 2023-09-20T21:56:37.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:36 smithi089 ceph-mon[134090]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.8 MiB/s wr, 531 op/s 2023-09-20T21:56:38.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 435.0 s, 763.8 tps, lat 130.495 ms stddev 157.828 2023-09-20T21:56:39.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:38 smithi123 ceph-mon[134433]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 577 op/s 2023-09-20T21:56:39.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:38 smithi089 ceph-mon[134090]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 577 op/s 2023-09-20T21:56:39.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:38 smithi022 ceph-mon[97651]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.4 MiB/s wr, 577 op/s 2023-09-20T21:56:41.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:40 smithi123 ceph-mon[134433]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 494 op/s 2023-09-20T21:56:41.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:40 smithi089 ceph-mon[134090]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 494 op/s 2023-09-20T21:56:41.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:40 smithi022 ceph-mon[97651]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 494 op/s 2023-09-20T21:56:43.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:42 smithi022 ceph-mon[97651]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.1 MiB/s wr, 615 op/s 2023-09-20T21:56:43.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:42 smithi123 ceph-mon[134433]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.1 MiB/s wr, 615 op/s 2023-09-20T21:56:43.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:42 smithi089 ceph-mon[134090]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.1 MiB/s wr, 615 op/s 2023-09-20T21:56:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 440.0 s, 475.4 tps, lat 210.482 ms stddev 259.871 2023-09-20T21:56:44.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:44 smithi022 ceph-mon[97651]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 497 op/s 2023-09-20T21:56:44.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:44 smithi123 ceph-mon[134433]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 497 op/s 2023-09-20T21:56:44.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:44 smithi089 ceph-mon[134090]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 497 op/s 2023-09-20T21:56:47.010 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:46 smithi022 ceph-mon[97651]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 563 op/s 2023-09-20T21:56:47.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:46 smithi123 ceph-mon[134433]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 563 op/s 2023-09-20T21:56:47.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:46 smithi089 ceph-mon[134090]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 563 op/s 2023-09-20T21:56:48.573 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 445.0 s, 474.1 tps, lat 208.712 ms stddev 227.358 2023-09-20T21:56:49.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:48 smithi123 ceph-mon[134433]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 617 op/s 2023-09-20T21:56:49.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:48 smithi089 ceph-mon[134090]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 617 op/s 2023-09-20T21:56:49.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:48 smithi022 ceph-mon[97651]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 617 op/s 2023-09-20T21:56:50.148 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:56:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.641+0000 7f44b91ff700 1 -- 172.21.15.22:0/4035111709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44b4108740 msgr2=0x7f44b4108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:50.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.641+0000 7f44b91ff700 1 --2- 172.21.15.22:0/4035111709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44b4108740 0x7f44b4108bc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44ac014a30 tx=0x7f44ac014d40 comp rx=0 tx=0).stop 2023-09-20T21:56:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.642+0000 7f44b91ff700 1 -- 172.21.15.22:0/4035111709 shutdown_connections 2023-09-20T21:56:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.642+0000 7f44b91ff700 1 --2- 172.21.15.22:0/4035111709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b4109100 0x7f44b41119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.642+0000 7f44b91ff700 1 --2- 172.21.15.22:0/4035111709 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44b4107de0 0x7f44b4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.642+0000 7f44b91ff700 1 --2- 172.21.15.22:0/4035111709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44b4108740 0x7f44b4108bc0 secure :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f44ac014a30 tx=0x7f44ac014d40 comp rx=0 tx=0).stop 2023-09-20T21:56:50.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.642+0000 7f44b91ff700 1 -- 172.21.15.22:0/4035111709 >> 172.21.15.22:0/4035111709 conn(0x7f44b406db90 msgr2=0x7f44b406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:50.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.657+0000 7f44b91ff700 1 -- 172.21.15.22:0/4035111709 shutdown_connections 2023-09-20T21:56:50.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.657+0000 7f44b91ff700 1 -- 172.21.15.22:0/4035111709 wait complete. 2023-09-20T21:56:50.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.659+0000 7f44b91ff700 1 Processor -- start 2023-09-20T21:56:50.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.660+0000 7f44b91ff700 1 -- start start 2023-09-20T21:56:50.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.660+0000 7f44b91ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44b4107de0 0x7f44b41bf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.660+0000 7f44b91ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44b4109100 0x7f44b41bf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.660+0000 7f44b91ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41c4970 0x7f44b41c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.660+0000 7f44b91ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b41c7330 con 0x7f44b4107de0 2023-09-20T21:56:50.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.660+0000 7f44b91ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b41c7470 con 0x7f44b4109100 2023-09-20T21:56:50.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.660+0000 7f44b91ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b41c7600 con 0x7f44b41c4970 2023-09-20T21:56:50.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.665+0000 7f44b359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41c4970 0x7f44b41c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.665+0000 7f44b359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41c4970 0x7f44b41c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:38742/0 (socket says 172.21.15.22:38742) 2023-09-20T21:56:50.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.665+0000 7f44b359e700 1 -- 172.21.15.22:0/706846573 learned_addr learned my addr 172.21.15.22:0/706846573 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:56:50.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.666+0000 7f44b2d9d700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44b4107de0 0x7f44b41bf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.666+0000 7f44b359e700 1 -- 172.21.15.22:0/706846573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44b4109100 msgr2=0x7f44b41bf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:50.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.666+0000 7f44b359e700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44b4109100 0x7f44b41bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.666+0000 7f44b359e700 1 -- 172.21.15.22:0/706846573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44b4107de0 msgr2=0x7f44b41bf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:50.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.666+0000 7f44b359e700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44b4107de0 0x7f44b41bf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.666+0000 7f44b359e700 1 -- 172.21.15.22:0/706846573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac014660 con 0x7f44b41c4970 2023-09-20T21:56:50.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.667+0000 7f44b359e700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41c4970 0x7f44b41c6d60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f44a800b550 tx=0x7f44a800b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:50.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.667+0000 7f449bfff700 1 -- 172.21.15.22:0/706846573 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8011830 con 0x7f44b41c4970 2023-09-20T21:56:50.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.668+0000 7f449bfff700 1 -- 172.21.15.22:0/706846573 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a8011e50 con 0x7f44b41c4970 2023-09-20T21:56:50.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.668+0000 7f449bfff700 1 -- 172.21.15.22:0/706846573 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a800f2e0 con 0x7f44b41c4970 2023-09-20T21:56:50.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.668+0000 7f44b91ff700 1 -- 172.21.15.22:0/706846573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b41c78e0 con 0x7f44b41c4970 2023-09-20T21:56:50.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.668+0000 7f44b91ff700 1 -- 172.21.15.22:0/706846573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b41c7da0 con 0x7f44b41c4970 2023-09-20T21:56:50.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.676+0000 7f449bfff700 1 -- 172.21.15.22:0/706846573 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f44a800f440 con 0x7f44b41c4970 2023-09-20T21:56:50.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.679+0000 7f449bfff700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f449c0a14b0 0x7f449c0a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.679+0000 7f449bfff700 1 -- 172.21.15.22:0/706846573 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f44a80cd940 con 0x7f44b41c4970 2023-09-20T21:56:50.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.682+0000 7f44b2d9d700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f449c0a14b0 0x7f449c0a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.717+0000 7f44b2d9d700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f449c0a14b0 0x7f449c0a3970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f44b4072f10 tx=0x7f44a400b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:50.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.719+0000 7f44ccac8b80 1 Processor -- start 2023-09-20T21:56:50.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.719+0000 7f44ccac8b80 1 -- start start 2023-09-20T21:56:50.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.719+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 0x560401aedb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.719+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 0x560401aee5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.719+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401aef060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.719+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560401a61ca0 con 0x560401aed6f0 2023-09-20T21:56:50.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.719+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560401a45c70 con 0x560401aee050 2023-09-20T21:56:50.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.719+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560401a43590 con 0x560401aeebe0 2023-09-20T21:56:50.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.720+0000 7f44b2d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 0x560401aedb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.720+0000 7f44b359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401aef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.720+0000 7f44b359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401aef060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:38758/0 (socket says 172.21.15.22:38758) 2023-09-20T21:56:50.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.720+0000 7f44b2d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 0x560401aedb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34066/0 (socket says 172.21.15.22:34066) 2023-09-20T21:56:50.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.720+0000 7f44b359e700 1 -- 172.21.15.22:0/3713201923 learned_addr learned my addr 172.21.15.22:0/3713201923 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:56:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.721+0000 7f44b359e700 1 -- 172.21.15.22:0/3713201923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 msgr2=0x560401aee5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:56:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.721+0000 7f44b259c700 1 --2- 172.21.15.22:0/3713201923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 0x560401aee5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.721+0000 7f44b359e700 1 --2- 172.21.15.22:0/3713201923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 0x560401aee5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.721+0000 7f44b359e700 1 -- 172.21.15.22:0/3713201923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 msgr2=0x560401aedb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.721+0000 7f44b359e700 1 --2- 172.21.15.22:0/3713201923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 0x560401aedb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.721+0000 7f44b359e700 1 -- 172.21.15.22:0/3713201923 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560401869de0 con 0x560401aeebe0 2023-09-20T21:56:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.721+0000 7f44b259c700 1 --2- 172.21.15.22:0/3713201923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 0x560401aee5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:56:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.721+0000 7f44b2d9d700 1 --2- 172.21.15.22:0/3713201923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 0x560401aedb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:56:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.722+0000 7f44b359e700 1 --2- 172.21.15.22:0/3713201923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401aef060 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f44a800b520 tx=0x7f44a809a240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5fb4629ef8d321f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.722+0000 7f44997fa700 1 -- 172.21.15.22:0/3713201923 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a809a8e0 con 0x560401aeebe0 2023-09-20T21:56:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.722+0000 7f44997fa700 1 -- 172.21.15.22:0/3713201923 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a809e710 con 0x560401aeebe0 2023-09-20T21:56:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.722+0000 7f44997fa700 1 -- 172.21.15.22:0/3713201923 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8099dd0 con 0x560401aeebe0 2023-09-20T21:56:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.723+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3713201923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 msgr2=0x560401aef060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.723+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/3713201923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401aef060 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f44a800b520 tx=0x7f44a809a240 comp rx=0 tx=0).stop 2023-09-20T21:56:50.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.724+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3713201923 shutdown_connections 2023-09-20T21:56:50.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.724+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/3713201923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401aef060 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.724+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/3713201923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 0x560401aedb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.724+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/3713201923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 0x560401aee5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.724+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3713201923 >> 172.21.15.22:0/3713201923 conn(0x560401a1a000 msgr2=0x560401ae81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.730+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3713201923 shutdown_connections 2023-09-20T21:56:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.730+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3713201923 wait complete. 2023-09-20T21:56:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.731+0000 7f44ccac8b80 1 -- start start 2023-09-20T21:56:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.731+0000 7f44ccac8b80 1 Processor -- start 2023-09-20T21:56:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.732+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 0x560401af7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.732+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aee050 0x560401af7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.732+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aeebe0 0x560401afe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.732+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560401a3ee80 con 0x560401aed6f0 2023-09-20T21:56:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.732+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604019db740 con 0x560401aeebe0 2023-09-20T21:56:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.732+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604019db3e0 con 0x560401aee050 2023-09-20T21:56:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.732+0000 7f44ccac8b80 10 client.0 _ll_register_callbacks cb 0x560401925a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:56:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.732+0000 7f44b259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aee050 0x560401af7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.732+0000 7f44b259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aee050 0x560401af7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:38768/0 (socket says 172.21.15.22:38768) 2023-09-20T21:56:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.732+0000 7f44b259c700 1 -- 172.21.15.22:0/245660759 learned_addr learned my addr 172.21.15.22:0/245660759 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:56:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.733+0000 7f44b2d9d700 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 0x560401af7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.734+0000 7f44b359e700 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aeebe0 0x560401afe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.734+0000 7f44b359e700 1 -- 172.21.15.22:0/245660759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aee050 msgr2=0x560401af7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:50.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.734+0000 7f44b359e700 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aee050 0x560401af7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.734+0000 7f44b359e700 1 -- 172.21.15.22:0/245660759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 msgr2=0x560401af7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:50.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.734+0000 7f44b359e700 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 0x560401af7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.734+0000 7f44b359e700 1 -- 172.21.15.22:0/245660759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b41c7da0 con 0x560401aeebe0 2023-09-20T21:56:50.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.735+0000 7f44b359e700 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aeebe0 0x560401afe7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f44a809a870 tx=0x7f44a809aec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e07479367bd98c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:50.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.735+0000 7f4496ff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:56:50.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.735+0000 7f4496ff5700 1 -- 172.21.15.22:0/245660759 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8099dd0 con 0x560401aeebe0 2023-09-20T21:56:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.736+0000 7f4496ff5700 1 -- 172.21.15.22:0/245660759 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f44a80b8030 con 0x560401aeebe0 2023-09-20T21:56:50.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.736+0000 7f44ccac8b80 1 -- 172.21.15.22:0/245660759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604019afbb0 con 0x560401aeebe0 2023-09-20T21:56:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.738+0000 7f4496ff5700 1 -- 172.21.15.22:0/245660759 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44a8099f30 con 0x560401aeebe0 2023-09-20T21:56:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.738+0000 7f44ccac8b80 10 client.25970 fetch_fsmap learned FSMap version 32 2023-09-20T21:56:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.738+0000 7f44ccac8b80 1 -- 172.21.15.22:0/245660759 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560401852c60 con 0x560401aeebe0 2023-09-20T21:56:50.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.739+0000 7f4496ff5700 1 -- 172.21.15.22:0/245660759 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604019afbb0 con 0x560401aeebe0 2023-09-20T21:56:50.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.740+0000 7f4496ff5700 1 -- 172.21.15.22:0/245660759 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f44a80b8b30 con 0x560401aeebe0 2023-09-20T21:56:50.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.740+0000 7f44ccac8b80 10 client.25970 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:56:50.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.740+0000 7f44ccac8b80 10 client.25970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:56:50.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.740+0000 7f44ccac8b80 20 client.25970 populate_metadata read hostname 'smithi022' 2023-09-20T21:56:50.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.740+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560401b02c80 0x560401b050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.740+0000 7f44ccac8b80 4 client.25970 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:56:50.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.740+0000 7f44ccac8b80 1 -- 172.21.15.22:0/245660759 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560401abc020 con 0x560401b02c80 2023-09-20T21:56:50.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.740+0000 7f44b2d9d700 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560401b02c80 0x560401b050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.742+0000 7f4496ff5700 10 client.25970 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:56:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.742+0000 7f44b2d9d700 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560401b02c80 0x560401b050a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c0831076b76b9ba server_cookie=beef7f363f567a4c in_seq=0 out_seq=0 2023-09-20T21:56:50.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f4496ff5700 1 -- 172.21.15.22:0/245660759 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560401abc020 con 0x560401b02c80 2023-09-20T21:56:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f4496ff5700 10 client.25970 handle_command_reply: tid=0 2023-09-20T21:56:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f44ccac8b80 1 client.25970 shutdown 2023-09-20T21:56:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f44ccac8b80 1 -- 172.21.15.22:0/245660759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aeebe0 msgr2=0x560401afe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aeebe0 0x560401afe7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f44a809a870 tx=0x7f44a809aec0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f44ccac8b80 1 -- 172.21.15.22:0/245660759 shutdown_connections 2023-09-20T21:56:50.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560401b02c80 0x560401b050a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aeebe0 0x560401afe7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aed6f0 0x560401af7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/245660759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aee050 0x560401af7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.743+0000 7f44ccac8b80 1 -- 172.21.15.22:0/245660759 >> 172.21.15.22:0/245660759 conn(0x560401a1a000 msgr2=0x560401ae7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.744+0000 7f4496ff5700 0 client.25970 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:56:50.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.744+0000 7f44ccac8b80 1 -- 172.21.15.22:0/245660759 shutdown_connections 2023-09-20T21:56:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.744+0000 7f44ccac8b80 1 -- 172.21.15.22:0/245660759 wait complete. 2023-09-20T21:56:50.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.744+0000 7f44ccac8b80 20 client.25970 trim_cache size 0 max 16384 2023-09-20T21:56:50.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.785+0000 7f44ccac8b80 1 Processor -- start 2023-09-20T21:56:50.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.785+0000 7f44ccac8b80 1 -- start start 2023-09-20T21:56:50.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.785+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aed6f0 0x560401aeefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.785+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aee050 0x560401afe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.786+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401af50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.786+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604018bf920 con 0x560401aee050 2023-09-20T21:56:50.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.786+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560401ae2c20 con 0x560401aed6f0 2023-09-20T21:56:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.786+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560401af6700 con 0x560401aeebe0 2023-09-20T21:56:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.786+0000 7f44b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aee050 0x560401afe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.786+0000 7f44b259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aee050 0x560401afe6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34082/0 (socket says 172.21.15.22:34082) 2023-09-20T21:56:50.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.786+0000 7f44b259c700 1 -- 172.21.15.22:0/2575404320 learned_addr learned my addr 172.21.15.22:0/2575404320 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:56:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.786+0000 7f44b359e700 1 --2- 172.21.15.22:0/2575404320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401af50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.787+0000 7f44b259c700 1 -- 172.21.15.22:0/2575404320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 msgr2=0x560401af50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.787+0000 7f44b259c700 1 --2- 172.21.15.22:0/2575404320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401af50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.787+0000 7f44b259c700 1 -- 172.21.15.22:0/2575404320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aed6f0 msgr2=0x560401aeefc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:56:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.787+0000 7f44b259c700 1 --2- 172.21.15.22:0/2575404320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aed6f0 0x560401aeefc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.787+0000 7f44b259c700 1 -- 172.21.15.22:0/2575404320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560401890250 con 0x560401aee050 2023-09-20T21:56:50.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.787+0000 7f44b259c700 1 --2- 172.21.15.22:0/2575404320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aee050 0x560401afe6c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f44ac011010 tx=0x7f44ac005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a97e8972c146f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:50.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.788+0000 7f4499ffb700 1 -- 172.21.15.22:0/2575404320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac013af0 con 0x560401aee050 2023-09-20T21:56:50.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.788+0000 7f4499ffb700 1 -- 172.21.15.22:0/2575404320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44ac013cf0 con 0x560401aee050 2023-09-20T21:56:50.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.788+0000 7f4499ffb700 1 -- 172.21.15.22:0/2575404320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00f3b0 con 0x560401aee050 2023-09-20T21:56:50.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.789+0000 7f44ccac8b80 1 -- 172.21.15.22:0/2575404320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aee050 msgr2=0x560401afe6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:50.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.789+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/2575404320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aee050 0x560401afe6c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f44ac011010 tx=0x7f44ac005040 comp rx=0 tx=0).stop 2023-09-20T21:56:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.789+0000 7f44ccac8b80 1 -- 172.21.15.22:0/2575404320 shutdown_connections 2023-09-20T21:56:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.789+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/2575404320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401af50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.789+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/2575404320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aed6f0 0x560401aeefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.789+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/2575404320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401aee050 0x560401afe6c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.789+0000 7f44ccac8b80 1 -- 172.21.15.22:0/2575404320 >> 172.21.15.22:0/2575404320 conn(0x560401a1a000 msgr2=0x560401ae7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:50.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.791+0000 7f44ccac8b80 1 -- 172.21.15.22:0/2575404320 shutdown_connections 2023-09-20T21:56:50.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.791+0000 7f44ccac8b80 1 -- 172.21.15.22:0/2575404320 wait complete. 2023-09-20T21:56:50.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.791+0000 7f44ccac8b80 1 -- start start 2023-09-20T21:56:50.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.792+0000 7f44ccac8b80 1 Processor -- start 2023-09-20T21:56:50.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.792+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 0x560401afd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.792+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401afd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.792+0000 7f44ccac8b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401afdaa0 0x560401b56a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.792+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560401afeb40 con 0x560401afdaa0 2023-09-20T21:56:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.792+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560401b56f70 con 0x560401aee050 2023-09-20T21:56:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.792+0000 7f44ccac8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560401b570b0 con 0x560401aeebe0 2023-09-20T21:56:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.792+0000 7f44ccac8b80 10 client.0 _ll_register_callbacks cb 0x560401925a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:56:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.793+0000 7f44b2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 0x560401afd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.793+0000 7f44b259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401afd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.793+0000 7f44b259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401afd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:38792/0 (socket says 172.21.15.22:38792) 2023-09-20T21:56:50.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.793+0000 7f44b259c700 1 -- 172.21.15.22:0/3617268516 learned_addr learned my addr 172.21.15.22:0/3617268516 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:56:50.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.793+0000 7f44b359e700 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401afdaa0 0x560401b56a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.793+0000 7f44b259c700 1 -- 172.21.15.22:0/3617268516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 msgr2=0x560401afd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:50.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.793+0000 7f44b259c700 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 0x560401afd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.793+0000 7f44b259c700 1 -- 172.21.15.22:0/3617268516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401afdaa0 msgr2=0x560401b56a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:56:50.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.793+0000 7f44b259c700 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401afdaa0 0x560401b56a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.793+0000 7f44b259c700 1 -- 172.21.15.22:0/3617268516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b4109bd0 con 0x560401aeebe0 2023-09-20T21:56:50.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.794+0000 7f44b259c700 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401afd560 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f44ac014950 tx=0x7f44ac014300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=133147a8c3ea8731 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:56:50.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.794+0000 7f4496ff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:56:50.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.794+0000 7f4496ff5700 1 -- 172.21.15.22:0/3617268516 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac013af0 con 0x560401aeebe0 2023-09-20T21:56:50.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.794+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3617268516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44a809e710 con 0x560401aeebe0 2023-09-20T21:56:50.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.794+0000 7f4496ff5700 1 -- 172.21.15.22:0/3617268516 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f44ac01c7e0 con 0x560401aeebe0 2023-09-20T21:56:50.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.794+0000 7f4496ff5700 1 -- 172.21.15.22:0/3617268516 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44ac00f3b0 con 0x560401aeebe0 2023-09-20T21:56:50.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.794+0000 7f44ccac8b80 10 client.25987 fetch_fsmap learned FSMap version 32 2023-09-20T21:56:50.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.794+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3617268516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5604018682b0 con 0x560401aeebe0 2023-09-20T21:56:50.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.795+0000 7f4496ff5700 1 -- 172.21.15.22:0/3617268516 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a809e710 con 0x560401aeebe0 2023-09-20T21:56:50.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.795+0000 7f4496ff5700 1 -- 172.21.15.22:0/3617268516 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f44ac005490 con 0x560401aeebe0 2023-09-20T21:56:50.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.795+0000 7f44ccac8b80 10 client.25987 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:56:50.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.795+0000 7f44ccac8b80 10 client.25987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:56:50.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.795+0000 7f44ccac8b80 20 client.25987 populate_metadata read hostname 'smithi022' 2023-09-20T21:56:50.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.795+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560401b58810 0x560401b5ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:56:50.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.795+0000 7f44ccac8b80 4 client.25987 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:56:50.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.795+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3617268516 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560401a37e60 con 0x560401b58810 2023-09-20T21:56:50.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.796+0000 7f44b2d9d700 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560401b58810 0x560401b5ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:56:50.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.797+0000 7f44b2d9d700 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560401b58810 0x560401b5ac30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=150a3d36d2492e22 server_cookie=b595423b5e0c5f26 in_seq=0 out_seq=0 2023-09-20T21:56:50.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.797+0000 7f4496ff5700 10 client.25987 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:56:50.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.798+0000 7f4496ff5700 1 -- 172.21.15.22:0/3617268516 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560401a37e60 con 0x560401b58810 2023-09-20T21:56:50.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.798+0000 7f4496ff5700 10 client.25987 handle_command_reply: tid=0 2023-09-20T21:56:50.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.801+0000 7f44ccac8b80 1 client.25987 shutdown 2023-09-20T21:56:50.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.802+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3617268516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 msgr2=0x560401afd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:50.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.802+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401afd560 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f44ac014950 tx=0x7f44ac014300 comp rx=0 tx=0).stop 2023-09-20T21:56:50.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.802+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3617268516 shutdown_connections 2023-09-20T21:56:50.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.802+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560401b58810 0x560401b5ac30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.802+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560401afdaa0 0x560401b56a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.802+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560401aee050 0x560401afd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.802+0000 7f44ccac8b80 1 --2- 172.21.15.22:0/3617268516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560401aeebe0 0x560401afd560 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.802+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3617268516 >> 172.21.15.22:0/3617268516 conn(0x560401a1a000 msgr2=0x560401af50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:50.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.802+0000 7f4496ff5700 0 client.25987 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:56:50.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.803+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3617268516 shutdown_connections 2023-09-20T21:56:50.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.803+0000 7f44ccac8b80 1 -- 172.21.15.22:0/3617268516 wait complete. 2023-09-20T21:56:50.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.803+0000 7f44ccac8b80 20 client.25987 trim_cache size 0 max 16384 2023-09-20T21:56:50.811 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:56:50.811 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:56:50.812 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:56:50.812 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:56:50.812 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:56:50.813 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:56:50.813 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:56:50.813 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:56:50.814 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:56:50.814 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:56:50.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.805+0000 7f44b91ff700 1 -- 172.21.15.22:0/706846573 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f449c0a14b0 msgr2=0x7f449c0a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:50.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.805+0000 7f44b91ff700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f449c0a14b0 0x7f449c0a3970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f44b4072f10 tx=0x7f44a400b410 comp rx=0 tx=0).stop 2023-09-20T21:56:50.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.805+0000 7f44b91ff700 1 -- 172.21.15.22:0/706846573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41c4970 msgr2=0x7f44b41c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:56:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.805+0000 7f44b91ff700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41c4970 0x7f44b41c6d60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f44a800b550 tx=0x7f44a800b860 comp rx=0 tx=0).stop 2023-09-20T21:56:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.806+0000 7f44b91ff700 1 -- 172.21.15.22:0/706846573 shutdown_connections 2023-09-20T21:56:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.806+0000 7f44b91ff700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44b41c4970 0x7f44b41c6d60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.806+0000 7f44b91ff700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44b4107de0 0x7f44b41bf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.806+0000 7f44b91ff700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f449c0a14b0 0x7f449c0a3970 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.806+0000 7f44b91ff700 1 --2- 172.21.15.22:0/706846573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f44b4109100 0x7f44b41bf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:56:50.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.806+0000 7f44b91ff700 1 -- 172.21.15.22:0/706846573 >> 172.21.15.22:0/706846573 conn(0x7f44b406db90 msgr2=0x7f44b410d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:56:50.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.806+0000 7f44b91ff700 1 -- 172.21.15.22:0/706846573 shutdown_connections 2023-09-20T21:56:50.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:56:50.806+0000 7f44b91ff700 1 -- 172.21.15.22:0/706846573 wait complete. 2023-09-20T21:56:50.847 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:56:51.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:50 smithi123 ceph-mon[134433]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 533 op/s 2023-09-20T21:56:51.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:50 smithi089 ceph-mon[134090]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 533 op/s 2023-09-20T21:56:51.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:50 smithi022 ceph-mon[97651]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 533 op/s 2023-09-20T21:56:53.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:52 smithi123 ceph-mon[134433]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 723 op/s 2023-09-20T21:56:53.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:52 smithi089 ceph-mon[134090]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 723 op/s 2023-09-20T21:56:53.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:52 smithi022 ceph-mon[97651]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 723 op/s 2023-09-20T21:56:53.570 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 450.0 s, 529.3 tps, lat 188.968 ms stddev 227.302 2023-09-20T21:56:54.347 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:54 smithi123 ceph-mon[134433]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 MiB/s wr, 611 op/s 2023-09-20T21:56:54.366 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:54 smithi089 ceph-mon[134090]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 MiB/s wr, 611 op/s 2023-09-20T21:56:54.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:54 smithi022 ceph-mon[97651]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 MiB/s wr, 611 op/s 2023-09-20T21:56:57.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:56 smithi022 ceph-mon[97651]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 664 op/s 2023-09-20T21:56:57.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:56 smithi123 ceph-mon[134433]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 664 op/s 2023-09-20T21:56:57.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:56 smithi089 ceph-mon[134090]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 664 op/s 2023-09-20T21:56:58.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 455.0 s, 612.3 tps, lat 163.019 ms stddev 190.614 2023-09-20T21:56:58.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:56:58 smithi022 ceph-mon[97651]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 675 op/s 2023-09-20T21:56:59.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:56:58 smithi123 ceph-mon[134433]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 675 op/s 2023-09-20T21:56:59.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:56:58 smithi089 ceph-mon[134090]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 675 op/s 2023-09-20T21:57:00.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:00 smithi022 ceph-mon[97651]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 577 op/s 2023-09-20T21:57:01.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:00 smithi123 ceph-mon[134433]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 577 op/s 2023-09-20T21:57:01.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:00 smithi089 ceph-mon[134090]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 577 op/s 2023-09-20T21:57:03.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:02 smithi123 ceph-mon[134433]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.9 MiB/s wr, 691 op/s 2023-09-20T21:57:03.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:02 smithi089 ceph-mon[134090]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.9 MiB/s wr, 691 op/s 2023-09-20T21:57:03.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:02 smithi022 ceph-mon[97651]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.9 MiB/s wr, 691 op/s 2023-09-20T21:57:03.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 460.0 s, 582.1 tps, lat 174.917 ms stddev 203.078 2023-09-20T21:57:04.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:04 smithi022 ceph-mon[97651]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.3 MiB/s wr, 504 op/s 2023-09-20T21:57:04.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:04 smithi123 ceph-mon[134433]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.3 MiB/s wr, 504 op/s 2023-09-20T21:57:04.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:04 smithi089 ceph-mon[134090]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.3 MiB/s wr, 504 op/s 2023-09-20T21:57:06.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:06 smithi022 ceph-mon[97651]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 MiB/s wr, 524 op/s 2023-09-20T21:57:07.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:06 smithi123 ceph-mon[134433]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 MiB/s wr, 524 op/s 2023-09-20T21:57:07.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:06 smithi089 ceph-mon[134090]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 MiB/s wr, 524 op/s 2023-09-20T21:57:08.576 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 465.0 s, 528.6 tps, lat 187.124 ms stddev 209.617 2023-09-20T21:57:09.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:08 smithi123 ceph-mon[134433]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 567 op/s 2023-09-20T21:57:09.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:08 smithi089 ceph-mon[134090]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 567 op/s 2023-09-20T21:57:09.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:08 smithi022 ceph-mon[97651]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 567 op/s 2023-09-20T21:57:11.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:10 smithi022 ceph-mon[97651]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.3 MiB/s wr, 526 op/s 2023-09-20T21:57:11.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:10 smithi123 ceph-mon[134433]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.3 MiB/s wr, 526 op/s 2023-09-20T21:57:11.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:10 smithi089 ceph-mon[134090]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.3 MiB/s wr, 526 op/s 2023-09-20T21:57:13.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:12 smithi022 ceph-mon[97651]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 627 op/s 2023-09-20T21:57:13.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:12 smithi123 ceph-mon[134433]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 627 op/s 2023-09-20T21:57:13.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:12 smithi089 ceph-mon[134090]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 627 op/s 2023-09-20T21:57:13.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 470.0 s, 591.9 tps, lat 168.494 ms stddev 192.503 2023-09-20T21:57:14.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:14 smithi022 ceph-mon[97651]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.7 MiB/s wr, 517 op/s 2023-09-20T21:57:14.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:14 smithi123 ceph-mon[134433]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.7 MiB/s wr, 517 op/s 2023-09-20T21:57:14.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:14 smithi089 ceph-mon[134090]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.7 MiB/s wr, 517 op/s 2023-09-20T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:16 smithi123 ceph-mon[134433]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.6 MiB/s wr, 573 op/s 2023-09-20T21:57:17.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:16 smithi089 ceph-mon[134090]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.6 MiB/s wr, 573 op/s 2023-09-20T21:57:17.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:16 smithi022 ceph-mon[97651]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.6 MiB/s wr, 573 op/s 2023-09-20T21:57:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 475.0 s, 506.4 tps, lat 200.417 ms stddev 218.798 2023-09-20T21:57:19.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:18 smithi089 ceph-mon[134090]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 596 op/s 2023-09-20T21:57:19.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:18 smithi022 ceph-mon[97651]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 596 op/s 2023-09-20T21:57:19.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:18 smithi123 ceph-mon[134433]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 596 op/s 2023-09-20T21:57:20.848 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:57:21.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:20 smithi022 ceph-mon[97651]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 515 op/s 2023-09-20T21:57:21.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:20 smithi123 ceph-mon[134433]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 515 op/s 2023-09-20T21:57:21.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:20 smithi089 ceph-mon[134090]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 515 op/s 2023-09-20T21:57:21.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.490+0000 7f8415dff700 1 -- 172.21.15.22:0/3532761389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f841006ca10 msgr2=0x7f841006ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:21.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.490+0000 7f8415dff700 1 --2- 172.21.15.22:0/3532761389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f841006ca10 0x7f841006ce30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f84080098f0 tx=0x7f8408009c00 comp rx=0 tx=0).stop 2023-09-20T21:57:21.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.492+0000 7f8415dff700 1 -- 172.21.15.22:0/3532761389 shutdown_connections 2023-09-20T21:57:21.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.492+0000 7f8415dff700 1 --2- 172.21.15.22:0/3532761389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841006e610 0x7f841006ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.492+0000 7f8415dff700 1 --2- 172.21.15.22:0/3532761389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f841006ca10 0x7f841006ce30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.492+0000 7f8415dff700 1 --2- 172.21.15.22:0/3532761389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f841006dc50 0x7f841006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.493+0000 7f8415dff700 1 -- 172.21.15.22:0/3532761389 >> 172.21.15.22:0/3532761389 conn(0x7f841006b490 msgr2=0x7f841006b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:21.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.504+0000 7f8415dff700 1 -- 172.21.15.22:0/3532761389 shutdown_connections 2023-09-20T21:57:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.507+0000 7f8415dff700 1 -- 172.21.15.22:0/3532761389 wait complete. 2023-09-20T21:57:21.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.508+0000 7f8415dff700 1 Processor -- start 2023-09-20T21:57:21.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.511+0000 7f8415dff700 1 -- start start 2023-09-20T21:57:21.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.511+0000 7f8415dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841006ca10 0x7f841019d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.511+0000 7f8415dff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f841006dc50 0x7f841019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.511+0000 7f8415dff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f841006e610 0x7f84101a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.511+0000 7f8415dff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84101a4d80 con 0x7f841006e610 2023-09-20T21:57:21.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.511+0000 7f8415dff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84101a4ec0 con 0x7f841006ca10 2023-09-20T21:57:21.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.511+0000 7f8415dff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84101a5050 con 0x7f841006dc50 2023-09-20T21:57:21.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.512+0000 7f840f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841006ca10 0x7f841019d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.512+0000 7f840f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841006ca10 0x7f841019d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:35184/0 (socket says 172.21.15.22:35184) 2023-09-20T21:57:21.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.512+0000 7f840f7fe700 1 -- 172.21.15.22:0/3234656373 learned_addr learned my addr 172.21.15.22:0/3234656373 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:57:21.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.513+0000 7f840f7fe700 1 -- 172.21.15.22:0/3234656373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f841006dc50 msgr2=0x7f841019d740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:57:21.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.513+0000 7f840f7fe700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f841006dc50 0x7f841019d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.513+0000 7f840f7fe700 1 -- 172.21.15.22:0/3234656373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f841006e610 msgr2=0x7f84101a47b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:57:21.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.513+0000 7f840f7fe700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f841006e610 0x7f84101a47b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.513+0000 7f840f7fe700 1 -- 172.21.15.22:0/3234656373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8408009520 con 0x7f841006ca10 2023-09-20T21:57:21.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.513+0000 7f840f7fe700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841006ca10 0x7f841019d200 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8408009230 tx=0x7f840800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:21.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.513+0000 7f840cff9700 1 -- 172.21.15.22:0/3234656373 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840801c070 con 0x7f841006ca10 2023-09-20T21:57:21.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.514+0000 7f840cff9700 1 -- 172.21.15.22:0/3234656373 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84080054f0 con 0x7f841006ca10 2023-09-20T21:57:21.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.514+0000 7f840cff9700 1 -- 172.21.15.22:0/3234656373 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408017400 con 0x7f841006ca10 2023-09-20T21:57:21.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.514+0000 7f8415dff700 1 -- 172.21.15.22:0/3234656373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84101a52d0 con 0x7f841006ca10 2023-09-20T21:57:21.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.514+0000 7f8415dff700 1 -- 172.21.15.22:0/3234656373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84101a5720 con 0x7f841006ca10 2023-09-20T21:57:21.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.521+0000 7f8429741b80 1 Processor -- start 2023-09-20T21:57:21.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.522+0000 7f840cff9700 1 -- 172.21.15.22:0/3234656373 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f8408017560 con 0x7f841006ca10 2023-09-20T21:57:21.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.523+0000 7f8429741b80 1 -- start start 2023-09-20T21:57:21.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.524+0000 7f8429741b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296834b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.524+0000 7f8429741b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835050 0x5592968355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.524+0000 7f8429741b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835be0 0x559296836060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.524+0000 7f8429741b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592967a8ca0 con 0x559296835050 2023-09-20T21:57:21.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.524+0000 7f8429741b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929678cc70 con 0x559296835be0 2023-09-20T21:57:21.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.524+0000 7f8429741b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929678a590 con 0x5592968346f0 2023-09-20T21:57:21.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.524+0000 7f840ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296834b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.524+0000 7f840ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296834b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:54534/0 (socket says 172.21.15.22:54534) 2023-09-20T21:57:21.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.524+0000 7f840ffff700 1 -- 172.21.15.22:0/35504397 learned_addr learned my addr 172.21.15.22:0/35504397 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:57:21.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.525+0000 7f840effd700 1 --2- 172.21.15.22:0/35504397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835be0 0x559296836060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.525+0000 7f840ffff700 1 -- 172.21.15.22:0/35504397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835be0 msgr2=0x559296836060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:21.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.525+0000 7f840ffff700 1 --2- 172.21.15.22:0/35504397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835be0 0x559296836060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.525+0000 7f840ffff700 1 -- 172.21.15.22:0/35504397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835050 msgr2=0x5592968355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:21.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.525+0000 7f840ffff700 1 --2- 172.21.15.22:0/35504397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835050 0x5592968355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.525+0000 7f840ffff700 1 -- 172.21.15.22:0/35504397 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592965b0de0 con 0x5592968346f0 2023-09-20T21:57:21.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.526+0000 7f840cff9700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f83f80a1500 0x7f83f80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.526+0000 7f840cff9700 1 -- 172.21.15.22:0/3234656373 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f84080ce8c0 con 0x7f841006ca10 2023-09-20T21:57:21.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.526+0000 7f840ffff700 1 --2- 172.21.15.22:0/35504397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296834b10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f840401cae0 tx=0x7f840401cdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4cc781f6496a4d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:21.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.527+0000 7f83f5ffb700 1 -- 172.21.15.22:0/35504397 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840401e9f0 con 0x5592968346f0 2023-09-20T21:57:21.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.527+0000 7f83f5ffb700 1 -- 172.21.15.22:0/35504397 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8404004830 con 0x5592968346f0 2023-09-20T21:57:21.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.527+0000 7f83f5ffb700 1 -- 172.21.15.22:0/35504397 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84040052f0 con 0x5592968346f0 2023-09-20T21:57:21.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.528+0000 7f840f7fe700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f83f80a1500 0x7f83f80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.528+0000 7f8429741b80 1 -- 172.21.15.22:0/35504397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 msgr2=0x559296834b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:21.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.528+0000 7f8429741b80 1 --2- 172.21.15.22:0/35504397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296834b10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f840401cae0 tx=0x7f840401cdf0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.528+0000 7f8429741b80 1 -- 172.21.15.22:0/35504397 shutdown_connections 2023-09-20T21:57:21.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.529+0000 7f8429741b80 1 --2- 172.21.15.22:0/35504397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835be0 0x559296836060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.529+0000 7f8429741b80 1 --2- 172.21.15.22:0/35504397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296834b10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.529+0000 7f8429741b80 1 --2- 172.21.15.22:0/35504397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835050 0x5592968355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.529+0000 7f8429741b80 1 -- 172.21.15.22:0/35504397 >> 172.21.15.22:0/35504397 conn(0x559296761000 msgr2=0x55929682f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:21.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.529+0000 7f840f7fe700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f83f80a1500 0x7f83f80a39c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x559296830150 tx=0x7f8408009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:21.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.529+0000 7f8429741b80 1 -- 172.21.15.22:0/35504397 shutdown_connections 2023-09-20T21:57:21.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.530+0000 7f8429741b80 1 -- 172.21.15.22:0/35504397 wait complete. 2023-09-20T21:57:21.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.530+0000 7f8429741b80 1 -- start start 2023-09-20T21:57:21.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.530+0000 7f8429741b80 1 Processor -- start 2023-09-20T21:57:21.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.531+0000 7f8429741b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x55929683e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.531+0000 7f8429741b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x55929683e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.531+0000 7f8429741b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835be0 0x5592968457e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.531+0000 7f8429741b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559296785e80 con 0x559296835be0 2023-09-20T21:57:21.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.531+0000 7f8429741b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x559296722740 con 0x559296835050 2023-09-20T21:57:21.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.531+0000 7f8429741b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592967223e0 con 0x5592968346f0 2023-09-20T21:57:21.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.532+0000 7f8429741b80 10 client.0 _ll_register_callbacks cb 0x55929666ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:57:21.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.532+0000 7f840ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x55929683e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.532+0000 7f840ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x55929683e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:54548/0 (socket says 172.21.15.22:54548) 2023-09-20T21:57:21.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.532+0000 7f840ffff700 1 -- 172.21.15.22:0/1671756220 learned_addr learned my addr 172.21.15.22:0/1671756220 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:57:21.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.533+0000 7f840effd700 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x55929683e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.533+0000 7f840ffff700 1 -- 172.21.15.22:0/1671756220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 msgr2=0x55929683e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:21.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.533+0000 7f840ffff700 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x55929683e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.533+0000 7f840ffff700 1 -- 172.21.15.22:0/1671756220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835be0 msgr2=0x5592968457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:21.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.533+0000 7f840ffff700 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835be0 0x5592968457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.533+0000 7f840ffff700 1 -- 172.21.15.22:0/1671756220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840401c710 con 0x5592968346f0 2023-09-20T21:57:21.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.533+0000 7f840effd700 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x55929683e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:57:21.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.533+0000 7f840ffff700 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x55929683e260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f840401e5d0 tx=0x7f84040053d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7085352db0b96350 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:21.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.534+0000 7f83eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:57:21.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.534+0000 7f83eeffd700 1 -- 172.21.15.22:0/1671756220 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8404031070 con 0x5592968346f0 2023-09-20T21:57:21.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.534+0000 7f83eeffd700 1 -- 172.21.15.22:0/1671756220 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f840401ec00 con 0x5592968346f0 2023-09-20T21:57:21.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.534+0000 7f8429741b80 1 -- 172.21.15.22:0/1671756220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592966f6bb0 con 0x5592968346f0 2023-09-20T21:57:21.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.534+0000 7f83eeffd700 1 -- 172.21.15.22:0/1671756220 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8404004d30 con 0x5592968346f0 2023-09-20T21:57:21.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.534+0000 7f8429741b80 10 client.25993 fetch_fsmap learned FSMap version 32 2023-09-20T21:57:21.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.535+0000 7f8429741b80 1 -- 172.21.15.22:0/1671756220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559296599c60 con 0x5592968346f0 2023-09-20T21:57:21.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.535+0000 7f83eeffd700 1 -- 172.21.15.22:0/1671756220 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592966f6bb0 con 0x5592968346f0 2023-09-20T21:57:21.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.535+0000 7f83eeffd700 1 -- 172.21.15.22:0/1671756220 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f840402a460 con 0x5592968346f0 2023-09-20T21:57:21.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.535+0000 7f8429741b80 10 client.25993 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:57:21.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.535+0000 7f8429741b80 10 client.25993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:57:21.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.536+0000 7f8429741b80 20 client.25993 populate_metadata read hostname 'smithi022' 2023-09-20T21:57:21.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.536+0000 7f8429741b80 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x559296849fe0 0x55929684c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.536+0000 7f8429741b80 4 client.25993 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:57:21.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.536+0000 7f8429741b80 1 -- 172.21.15.22:0/1671756220 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559296803020 con 0x559296849fe0 2023-09-20T21:57:21.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.536+0000 7f840effd700 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x559296849fe0 0x55929684c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.538+0000 7f840effd700 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x559296849fe0 0x55929684c400 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b05b26e61f43b292 server_cookie=550c5b65e858fb52 in_seq=0 out_seq=0 2023-09-20T21:57:21.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.538+0000 7f83eeffd700 10 client.25993 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:57:21.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.541+0000 7f83eeffd700 1 -- 172.21.15.22:0/1671756220 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559296803020 con 0x559296849fe0 2023-09-20T21:57:21.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.541+0000 7f83eeffd700 10 client.25993 handle_command_reply: tid=0 2023-09-20T21:57:21.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.541+0000 7f8429741b80 1 client.25993 shutdown 2023-09-20T21:57:21.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.545+0000 7f8429741b80 1 -- 172.21.15.22:0/1671756220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 msgr2=0x55929683e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:21.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.545+0000 7f8429741b80 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x55929683e260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f840401e5d0 tx=0x7f84040053d0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.548+0000 7f8429741b80 1 -- 172.21.15.22:0/1671756220 shutdown_connections 2023-09-20T21:57:21.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.548+0000 7f8429741b80 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x559296849fe0 0x55929684c400 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.548+0000 7f8429741b80 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835be0 0x5592968457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.548+0000 7f8429741b80 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x55929683e260 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.548+0000 7f8429741b80 1 --2- 172.21.15.22:0/1671756220 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x55929683e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.548+0000 7f8429741b80 1 -- 172.21.15.22:0/1671756220 >> 172.21.15.22:0/1671756220 conn(0x559296761000 msgr2=0x55929682e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:21.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.548+0000 7f83eeffd700 0 client.25993 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:57:21.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.549+0000 7f8429741b80 1 -- 172.21.15.22:0/1671756220 shutdown_connections 2023-09-20T21:57:21.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.549+0000 7f8429741b80 1 -- 172.21.15.22:0/1671756220 wait complete. 2023-09-20T21:57:21.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.550+0000 7f8429741b80 20 client.25993 trim_cache size 0 max 16384 2023-09-20T21:57:21.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.583+0000 7f8429741b80 1 Processor -- start 2023-09-20T21:57:21.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.583+0000 7f8429741b80 1 -- start start 2023-09-20T21:57:21.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f8429741b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592968346f0 0x559296835fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f8429741b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x5592968457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f8429741b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559296835be0 0x559296838bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f8429741b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559296606920 con 0x5592968346f0 2023-09-20T21:57:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f8429741b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592966f6bb0 con 0x559296835050 2023-09-20T21:57:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f8429741b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592968305c0 con 0x559296835be0 2023-09-20T21:57:21.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f840effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x5592968457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f840effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x5592968457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:35206/0 (socket says 172.21.15.22:35206) 2023-09-20T21:57:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f840effd700 1 -- 172.21.15.22:0/3626235809 learned_addr learned my addr 172.21.15.22:0/3626235809 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:57:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f840effd700 1 -- 172.21.15.22:0/3626235809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559296835be0 msgr2=0x559296838bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T21:57:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.584+0000 7f840effd700 1 --2- 172.21.15.22:0/3626235809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559296835be0 0x559296838bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.585+0000 7f840effd700 1 -- 172.21.15.22:0/3626235809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592968346f0 msgr2=0x559296835fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.585+0000 7f840effd700 1 --2- 172.21.15.22:0/3626235809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592968346f0 0x559296835fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.585+0000 7f840effd700 1 -- 172.21.15.22:0/3626235809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592965d7250 con 0x559296835050 2023-09-20T21:57:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.585+0000 7f840effd700 1 --2- 172.21.15.22:0/3626235809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x5592968457a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8400000f80 tx=0x7f8400018d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce730d018ceceb30 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.585+0000 7f83f67fc700 1 -- 172.21.15.22:0/3626235809 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400018330 con 0x559296835050 2023-09-20T21:57:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.585+0000 7f83f67fc700 1 -- 172.21.15.22:0/3626235809 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f840001c070 con 0x559296835050 2023-09-20T21:57:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.585+0000 7f83f67fc700 1 -- 172.21.15.22:0/3626235809 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400021980 con 0x559296835050 2023-09-20T21:57:21.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.586+0000 7f8429741b80 1 -- 172.21.15.22:0/3626235809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 msgr2=0x5592968457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:21.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.587+0000 7f8429741b80 1 --2- 172.21.15.22:0/3626235809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x5592968457a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8400000f80 tx=0x7f8400018d80 comp rx=0 tx=0).stop 2023-09-20T21:57:21.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.587+0000 7f8429741b80 1 -- 172.21.15.22:0/3626235809 shutdown_connections 2023-09-20T21:57:21.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.587+0000 7f8429741b80 1 --2- 172.21.15.22:0/3626235809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559296835be0 0x559296838bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.587+0000 7f8429741b80 1 --2- 172.21.15.22:0/3626235809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5592968346f0 0x559296835fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.587+0000 7f8429741b80 1 --2- 172.21.15.22:0/3626235809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x5592968457a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.587+0000 7f8429741b80 1 -- 172.21.15.22:0/3626235809 >> 172.21.15.22:0/3626235809 conn(0x559296761000 msgr2=0x55929683c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:21.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.587+0000 7f8429741b80 1 -- 172.21.15.22:0/3626235809 shutdown_connections 2023-09-20T21:57:21.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.587+0000 7f8429741b80 1 -- 172.21.15.22:0/3626235809 wait complete. 2023-09-20T21:57:21.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.588+0000 7f8429741b80 1 -- start start 2023-09-20T21:57:21.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.588+0000 7f8429741b80 1 Processor -- start 2023-09-20T21:57:21.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.589+0000 7f8429741b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296843e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.589+0000 7f8429741b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x559296844380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.589+0000 7f8429741b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835be0 0x5592968448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.589+0000 7f8429741b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x559296845ac0 con 0x559296835be0 2023-09-20T21:57:21.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.589+0000 7f8429741b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929689de00 con 0x559296835050 2023-09-20T21:57:21.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.589+0000 7f8429741b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929689df40 con 0x5592968346f0 2023-09-20T21:57:21.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.589+0000 7f8429741b80 10 client.0 _ll_register_callbacks cb 0x55929666ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:57:21.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.589+0000 7f840ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296843e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.590+0000 7f840ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296843e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:54556/0 (socket says 172.21.15.22:54556) 2023-09-20T21:57:21.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.590+0000 7f840ffff700 1 -- 172.21.15.22:0/1031934910 learned_addr learned my addr 172.21.15.22:0/1031934910 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:57:21.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.590+0000 7f840ffff700 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835be0 0x5592968448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.590+0000 7f840ffff700 1 -- 172.21.15.22:0/1031934910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 msgr2=0x559296844380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.590+0000 7f840ffff700 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x559296844380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.590+0000 7f840ffff700 1 -- 172.21.15.22:0/1031934910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835be0 msgr2=0x5592968448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.590+0000 7f840ffff700 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835be0 0x5592968448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.590+0000 7f840ffff700 1 -- 172.21.15.22:0/1031934910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592965af2b0 con 0x5592968346f0 2023-09-20T21:57:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.590+0000 7f840ffff700 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296843e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f84040052f0 tx=0x7f840401cdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f1f97b249c2e4bc server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.591+0000 7f83eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:57:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.591+0000 7f83eeffd700 1 -- 172.21.15.22:0/1031934910 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8404031040 con 0x5592968346f0 2023-09-20T21:57:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.591+0000 7f83eeffd700 1 -- 172.21.15.22:0/1031934910 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f8404004e90 con 0x5592968346f0 2023-09-20T21:57:21.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.591+0000 7f8429741b80 1 -- 172.21.15.22:0/1031934910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8404004830 con 0x5592968346f0 2023-09-20T21:57:21.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.591+0000 7f83eeffd700 1 -- 172.21.15.22:0/1031934910 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8404005050 con 0x5592968346f0 2023-09-20T21:57:21.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.591+0000 7f8429741b80 10 client.25996 fetch_fsmap learned FSMap version 32 2023-09-20T21:57:21.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.591+0000 7f8429741b80 1 -- 172.21.15.22:0/1031934910 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55929667da80 con 0x5592968346f0 2023-09-20T21:57:21.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.592+0000 7f83eeffd700 1 -- 172.21.15.22:0/1031934910 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8404004830 con 0x5592968346f0 2023-09-20T21:57:21.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.592+0000 7f83eeffd700 1 -- 172.21.15.22:0/1031934910 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f84040409d0 con 0x5592968346f0 2023-09-20T21:57:21.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.592+0000 7f8429741b80 10 client.25996 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:57:21.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.592+0000 7f8429741b80 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:57:21.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.592+0000 7f8429741b80 20 client.25996 populate_metadata read hostname 'smithi022' 2023-09-20T21:57:21.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.592+0000 7f8429741b80 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55929689f580 0x5592968a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:21.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.592+0000 7f8429741b80 4 client.25996 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.592+0000 7f8429741b80 1 -- 172.21.15.22:0/1031934910 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55929677ee60 con 0x55929689f580 2023-09-20T21:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.592+0000 7f840effd700 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55929689f580 0x5592968a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.593+0000 7f840effd700 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55929689f580 0x5592968a19a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ea718a81e0645f server_cookie=7c7eaa364db8df33 in_seq=0 out_seq=0 2023-09-20T21:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.594+0000 7f83eeffd700 10 client.25996 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:57:21.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.594+0000 7f83eeffd700 1 -- 172.21.15.22:0/1031934910 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55929677ee60 con 0x55929689f580 2023-09-20T21:57:21.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.594+0000 7f83eeffd700 10 client.25996 handle_command_reply: tid=0 2023-09-20T21:57:21.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.594+0000 7f8429741b80 1 client.25996 shutdown 2023-09-20T21:57:21.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.594+0000 7f8429741b80 1 -- 172.21.15.22:0/1031934910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 msgr2=0x559296843e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:21.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.594+0000 7f8429741b80 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296843e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f84040052f0 tx=0x7f840401cdf0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.595+0000 7f8429741b80 1 -- 172.21.15.22:0/1031934910 shutdown_connections 2023-09-20T21:57:21.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.595+0000 7f8429741b80 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55929689f580 0x5592968a19a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.595+0000 7f8429741b80 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559296835be0 0x5592968448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.595+0000 7f8429741b80 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592968346f0 0x559296843e40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.595+0000 7f8429741b80 1 --2- 172.21.15.22:0/1031934910 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559296835050 0x559296844380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.595+0000 7f8429741b80 1 -- 172.21.15.22:0/1031934910 >> 172.21.15.22:0/1031934910 conn(0x559296761000 msgr2=0x559296838bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:21.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.595+0000 7f83eeffd700 0 client.25996 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:57:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.608+0000 7f8429741b80 1 -- 172.21.15.22:0/1031934910 shutdown_connections 2023-09-20T21:57:21.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.608+0000 7f8429741b80 1 -- 172.21.15.22:0/1031934910 wait complete. 2023-09-20T21:57:21.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.617+0000 7f8429741b80 20 client.25996 trim_cache size 0 max 16384 2023-09-20T21:57:21.627 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:57:21.628 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:57:21.628 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:57:21.628 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:57:21.628 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:57:21.629 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:57:21.629 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:57:21.629 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:57:21.629 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:57:21.629 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:57:21.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.634+0000 7f8415dff700 1 -- 172.21.15.22:0/3234656373 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f83f80a1500 msgr2=0x7f83f80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:21.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.634+0000 7f8415dff700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f83f80a1500 0x7f83f80a39c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x559296830150 tx=0x7f8408009260 comp rx=0 tx=0).stop 2023-09-20T21:57:21.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.634+0000 7f8415dff700 1 -- 172.21.15.22:0/3234656373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841006ca10 msgr2=0x7f841019d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:21.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.634+0000 7f8415dff700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841006ca10 0x7f841019d200 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8408009230 tx=0x7f840800b8d0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.644+0000 7f8415dff700 1 -- 172.21.15.22:0/3234656373 shutdown_connections 2023-09-20T21:57:21.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.644+0000 7f8415dff700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f841006e610 0x7f84101a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.644+0000 7f8415dff700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f83f80a1500 0x7f83f80a39c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.644+0000 7f8415dff700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f841006ca10 0x7f841019d200 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.644+0000 7f8415dff700 1 --2- 172.21.15.22:0/3234656373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f841006dc50 0x7f841019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:21.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.644+0000 7f8415dff700 1 -- 172.21.15.22:0/3234656373 >> 172.21.15.22:0/3234656373 conn(0x7f841006b490 msgr2=0x7f8410109fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:21.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.658+0000 7f8415dff700 1 -- 172.21.15.22:0/3234656373 shutdown_connections 2023-09-20T21:57:21.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:21.666+0000 7f8415dff700 1 -- 172.21.15.22:0/3234656373 wait complete. 2023-09-20T21:57:21.702 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:57:22.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:21 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:21 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:57:22.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:21 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:57:23.254 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:22 smithi022 ceph-mon[97651]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 7.7 MiB/s wr, 559 op/s 2023-09-20T21:57:23.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:22 smithi123 ceph-mon[134433]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 7.7 MiB/s wr, 559 op/s 2023-09-20T21:57:23.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:22 smithi089 ceph-mon[134090]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 7.7 MiB/s wr, 559 op/s 2023-09-20T21:57:23.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 480.0 s, 527.0 tps, lat 185.105 ms stddev 220.665 2023-09-20T21:57:24.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:57:24.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:57:24.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:57:24.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:24 smithi022 ceph-mon[97651]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 7.0 MiB/s wr, 509 op/s 2023-09-20T21:57:24.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:57:24.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:57:24.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:57:24.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:24 smithi123 ceph-mon[134433]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 7.0 MiB/s wr, 509 op/s 2023-09-20T21:57:24.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:57:24.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:57:24.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:57:24.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:24 smithi089 ceph-mon[134090]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 7.0 MiB/s wr, 509 op/s 2023-09-20T21:57:27.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:26 smithi123 ceph-mon[134433]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 7.1 MiB/s wr, 550 op/s 2023-09-20T21:57:27.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:26 smithi089 ceph-mon[134090]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 7.1 MiB/s wr, 550 op/s 2023-09-20T21:57:27.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:26 smithi022 ceph-mon[97651]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 7.1 MiB/s wr, 550 op/s 2023-09-20T21:57:28.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 485.0 s, 566.6 tps, lat 180.150 ms stddev 226.689 2023-09-20T21:57:29.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:28 smithi089 ceph-mon[134090]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 7.0 MiB/s wr, 561 op/s 2023-09-20T21:57:29.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:28 smithi022 ceph-mon[97651]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 7.0 MiB/s wr, 561 op/s 2023-09-20T21:57:29.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:28 smithi123 ceph-mon[134433]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 7.0 MiB/s wr, 561 op/s 2023-09-20T21:57:31.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:30 smithi022 ceph-mon[97651]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 6.3 MiB/s wr, 491 op/s 2023-09-20T21:57:31.516 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:30 smithi089 ceph-mon[134090]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 6.3 MiB/s wr, 491 op/s 2023-09-20T21:57:31.516 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:30 smithi123 ceph-mon[134433]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 6.3 MiB/s wr, 491 op/s 2023-09-20T21:57:33.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:32 smithi022 ceph-mon[97651]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 8.3 MiB/s wr, 568 op/s 2023-09-20T21:57:33.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:32 smithi123 ceph-mon[134433]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 8.3 MiB/s wr, 568 op/s 2023-09-20T21:57:33.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:32 smithi089 ceph-mon[134090]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 8.3 MiB/s wr, 568 op/s 2023-09-20T21:57:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 490.0 s, 479.5 tps, lat 202.291 ms stddev 240.246 2023-09-20T21:57:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:34 smithi022 ceph-mon[97651]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 554 op/s 2023-09-20T21:57:34.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:34 smithi123 ceph-mon[134433]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 554 op/s 2023-09-20T21:57:34.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:34 smithi089 ceph-mon[134090]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.9 MiB/s wr, 554 op/s 2023-09-20T21:57:37.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:36 smithi123 ceph-mon[134433]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 545 op/s 2023-09-20T21:57:37.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:36 smithi089 ceph-mon[134090]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 545 op/s 2023-09-20T21:57:37.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:36 smithi022 ceph-mon[97651]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 545 op/s 2023-09-20T21:57:38.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 495.0 s, 538.1 tps, lat 188.538 ms stddev 233.700 2023-09-20T21:57:39.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:38 smithi123 ceph-mon[134433]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 567 op/s 2023-09-20T21:57:39.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:38 smithi089 ceph-mon[134090]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 567 op/s 2023-09-20T21:57:39.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:38 smithi022 ceph-mon[97651]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 567 op/s 2023-09-20T21:57:41.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:40 smithi123 ceph-mon[134433]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 MiB/s wr, 521 op/s 2023-09-20T21:57:41.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:40 smithi089 ceph-mon[134090]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 MiB/s wr, 521 op/s 2023-09-20T21:57:41.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:40 smithi022 ceph-mon[97651]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 MiB/s wr, 521 op/s 2023-09-20T21:57:43.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:42 smithi123 ceph-mon[134433]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 581 op/s 2023-09-20T21:57:43.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:42 smithi089 ceph-mon[134090]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 581 op/s 2023-09-20T21:57:43.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:42 smithi022 ceph-mon[97651]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 581 op/s 2023-09-20T21:57:43.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 500.0 s, 485.5 tps, lat 206.346 ms stddev 239.806 2023-09-20T21:57:44.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:44 smithi022 ceph-mon[97651]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.6 MiB/s wr, 559 op/s 2023-09-20T21:57:44.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:44 smithi123 ceph-mon[134433]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.6 MiB/s wr, 559 op/s 2023-09-20T21:57:44.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:44 smithi089 ceph-mon[134090]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.6 MiB/s wr, 559 op/s 2023-09-20T21:57:47.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:46 smithi123 ceph-mon[134433]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 530 op/s 2023-09-20T21:57:47.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:46 smithi089 ceph-mon[134090]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 530 op/s 2023-09-20T21:57:47.254 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:46 smithi022 ceph-mon[97651]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 530 op/s 2023-09-20T21:57:48.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 505.0 s, 519.5 tps, lat 193.882 ms stddev 222.287 2023-09-20T21:57:49.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:48 smithi123 ceph-mon[134433]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.1 MiB/s wr, 555 op/s 2023-09-20T21:57:49.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:48 smithi089 ceph-mon[134090]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.1 MiB/s wr, 555 op/s 2023-09-20T21:57:49.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:48 smithi022 ceph-mon[97651]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.1 MiB/s wr, 555 op/s 2023-09-20T21:57:51.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:50 smithi123 ceph-mon[134433]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 499 op/s 2023-09-20T21:57:51.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:50 smithi089 ceph-mon[134090]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 499 op/s 2023-09-20T21:57:51.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:50 smithi022 ceph-mon[97651]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 499 op/s 2023-09-20T21:57:51.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:57:52.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.236+0000 7fb13bdff700 1 -- 172.21.15.22:0/1050647752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1340ff640 msgr2=0x7fb134101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:52.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.236+0000 7fb13bdff700 1 --2- 172.21.15.22:0/1050647752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1340ff640 0x7fb134101a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb130009990 tx=0x7fb130009ca0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.236+0000 7fb13bdff700 1 -- 172.21.15.22:0/1050647752 shutdown_connections 2023-09-20T21:57:52.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.236+0000 7fb13bdff700 1 --2- 172.21.15.22:0/1050647752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb134104980 0x7fb134104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.236+0000 7fb13bdff700 1 --2- 172.21.15.22:0/1050647752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb1340ff640 0x7fb134101a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.236+0000 7fb13bdff700 1 --2- 172.21.15.22:0/1050647752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb134101fb0 0x7fb134104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.236+0000 7fb13bdff700 1 -- 172.21.15.22:0/1050647752 >> 172.21.15.22:0/1050647752 conn(0x7fb1340fb240 msgr2=0x7fb1340fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.237+0000 7fb13bdff700 1 -- 172.21.15.22:0/1050647752 shutdown_connections 2023-09-20T21:57:52.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.237+0000 7fb13bdff700 1 -- 172.21.15.22:0/1050647752 wait complete. 2023-09-20T21:57:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.238+0000 7fb13bdff700 1 Processor -- start 2023-09-20T21:57:52.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.238+0000 7fb13bdff700 1 -- start start 2023-09-20T21:57:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.239+0000 7fb13bdff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1340ff640 0x7fb13419d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.239+0000 7fb13bdff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb134101fb0 0x7fb13419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.239+0000 7fb13bdff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb134104980 0x7fb1341a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.239+0000 7fb13bdff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1341a4be0 con 0x7fb1340ff640 2023-09-20T21:57:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.239+0000 7fb13bdff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1341a4d20 con 0x7fb134101fb0 2023-09-20T21:57:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.239+0000 7fb13bdff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1341a4eb0 con 0x7fb134104980 2023-09-20T21:57:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.240+0000 7fb13a39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb134104980 0x7fb1341a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.240+0000 7fb13a39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb134104980 0x7fb1341a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:39082/0 (socket says 172.21.15.22:39082) 2023-09-20T21:57:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.240+0000 7fb13a39c700 1 -- 172.21.15.22:0/1781053219 learned_addr learned my addr 172.21.15.22:0/1781053219 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:57:52.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.240+0000 7fb13939a700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb134101fb0 0x7fb13419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.241+0000 7fb13a39c700 1 -- 172.21.15.22:0/1781053219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb134101fb0 msgr2=0x7fb13419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.241+0000 7fb13a39c700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb134101fb0 0x7fb13419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.241+0000 7fb13a39c700 1 -- 172.21.15.22:0/1781053219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1340ff640 msgr2=0x7fb13419d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.241+0000 7fb13a39c700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1340ff640 0x7fb13419d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.241+0000 7fb13a39c700 1 -- 172.21.15.22:0/1781053219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1300095f0 con 0x7fb134104980 2023-09-20T21:57:52.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.242+0000 7fb13a39c700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb134104980 0x7fb1341a4610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00ef30 tx=0x7fb12c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:52.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.243+0000 7fb12affd700 1 -- 172.21.15.22:0/1781053219 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c018070 con 0x7fb134104980 2023-09-20T21:57:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.243+0000 7fb12affd700 1 -- 172.21.15.22:0/1781053219 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb12c00f040 con 0x7fb134104980 2023-09-20T21:57:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.243+0000 7fb12affd700 1 -- 172.21.15.22:0/1781053219 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c007880 con 0x7fb134104980 2023-09-20T21:57:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.243+0000 7fb13bdff700 1 -- 172.21.15.22:0/1781053219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1341a5130 con 0x7fb134104980 2023-09-20T21:57:52.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.244+0000 7fb13bdff700 1 -- 172.21.15.22:0/1781053219 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1341a55f0 con 0x7fb134104980 2023-09-20T21:57:52.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.250+0000 7fb12affd700 1 -- 172.21.15.22:0/1781053219 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fb12c00c9f0 con 0x7fb134104980 2023-09-20T21:57:52.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.251+0000 7fb14f7f0b80 1 Processor -- start 2023-09-20T21:57:52.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.252+0000 7fb14f7f0b80 1 -- start start 2023-09-20T21:57:52.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.252+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb185fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.252+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860050 0x563bb18605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.252+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860be0 0x563bb1861060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.252+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb17d3ca0 con 0x563bb185f6f0 2023-09-20T21:57:52.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.252+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb17b7c70 con 0x563bb1860be0 2023-09-20T21:57:52.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.252+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb17b5590 con 0x563bb1860050 2023-09-20T21:57:52.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.253+0000 7fb139b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860050 0x563bb18605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.253+0000 7fb139b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860050 0x563bb18605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:39092/0 (socket says 172.21.15.22:39092) 2023-09-20T21:57:52.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.253+0000 7fb139b9b700 1 -- 172.21.15.22:0/1291947454 learned_addr learned my addr 172.21.15.22:0/1291947454 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:57:52.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.253+0000 7fb13939a700 1 --2- 172.21.15.22:0/1291947454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860be0 0x563bb1861060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.253+0000 7fb13939a700 1 --2- 172.21.15.22:0/1291947454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb185fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.253+0000 7fb13939a700 1 -- 172.21.15.22:0/1291947454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860050 msgr2=0x563bb18605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:52.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.253+0000 7fb13939a700 1 --2- 172.21.15.22:0/1291947454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860050 0x563bb18605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.253+0000 7fb13939a700 1 -- 172.21.15.22:0/1291947454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 msgr2=0x563bb185fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:52.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.253+0000 7fb13939a700 1 --2- 172.21.15.22:0/1291947454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb185fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.253+0000 7fb13939a700 1 -- 172.21.15.22:0/1291947454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb15dbde0 con 0x563bb1860be0 2023-09-20T21:57:52.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.254+0000 7fb13939a700 1 --2- 172.21.15.22:0/1291947454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860be0 0x563bb1861060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb1240115d0 tx=0x7fb12400a740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99b0f8edfc8c27fd server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:52.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.254+0000 7fb11ffff700 1 -- 172.21.15.22:0/1291947454 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124004220 con 0x563bb1860be0 2023-09-20T21:57:52.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.254+0000 7fb11ffff700 1 -- 172.21.15.22:0/1291947454 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb124004cf0 con 0x563bb1860be0 2023-09-20T21:57:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.254+0000 7fb11ffff700 1 -- 172.21.15.22:0/1291947454 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124005550 con 0x563bb1860be0 2023-09-20T21:57:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.255+0000 7fb12affd700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1200a1500 0x7fb1200a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.255+0000 7fb12affd700 1 -- 172.21.15.22:0/1781053219 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fb12c0cc610 con 0x7fb134104980 2023-09-20T21:57:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.255+0000 7fb139b9b700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1200a1500 0x7fb1200a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.256+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1291947454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860be0 msgr2=0x563bb1861060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.256+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/1291947454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860be0 0x563bb1861060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb1240115d0 tx=0x7fb12400a740 comp rx=0 tx=0).stop 2023-09-20T21:57:52.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.256+0000 7fb139b9b700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1200a1500 0x7fb1200a39c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x563bb185b150 tx=0x7fb13000b280 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.257+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1291947454 shutdown_connections 2023-09-20T21:57:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.257+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/1291947454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860be0 0x563bb1861060 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.257+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/1291947454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb185fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.257+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/1291947454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860050 0x563bb18605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.257+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1291947454 >> 172.21.15.22:0/1291947454 conn(0x563bb178c000 msgr2=0x563bb185a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.257+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1291947454 shutdown_connections 2023-09-20T21:57:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.257+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1291947454 wait complete. 2023-09-20T21:57:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.258+0000 7fb14f7f0b80 1 -- start start 2023-09-20T21:57:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.259+0000 7fb14f7f0b80 1 Processor -- start 2023-09-20T21:57:52.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb1869290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb18697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb1870810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb17b0e80 con 0x563bb185f6f0 2023-09-20T21:57:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb174d740 con 0x563bb1860050 2023-09-20T21:57:52.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb174d3e0 con 0x563bb1860be0 2023-09-20T21:57:52.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb14f7f0b80 10 client.0 _ll_register_callbacks cb 0x563bb1697a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:57:52.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb13a39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb1870810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb13a39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb1870810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:39104/0 (socket says 172.21.15.22:39104) 2023-09-20T21:57:52.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb13a39c700 1 -- 172.21.15.22:0/1342653356 learned_addr learned my addr 172.21.15.22:0/1342653356 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:57:52.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.260+0000 7fb13939a700 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb18697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb139b9b700 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb1869290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb13a39c700 1 -- 172.21.15.22:0/1342653356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 msgr2=0x563bb18697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb13a39c700 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb18697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb13a39c700 1 -- 172.21.15.22:0/1342653356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 msgr2=0x563bb1869290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb13a39c700 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb1869290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb13a39c700 1 -- 172.21.15.22:0/1342653356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12400ae50 con 0x563bb1860be0 2023-09-20T21:57:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb13a39c700 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb1870810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb12c095ab0 tx=0x7fb12c00cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15b457d90f15c7a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:52.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb11d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:57:52.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb11d7fa700 1 -- 172.21.15.22:0/1342653356 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0b0070 con 0x563bb1860be0 2023-09-20T21:57:52.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb11d7fa700 1 -- 172.21.15.22:0/1342653356 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fb12c099cb0 con 0x563bb1860be0 2023-09-20T21:57:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.261+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1342653356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bb1721bb0 con 0x563bb1860be0 2023-09-20T21:57:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.262+0000 7fb11d7fa700 1 -- 172.21.15.22:0/1342653356 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb12c0995b0 con 0x563bb1860be0 2023-09-20T21:57:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.262+0000 7fb14f7f0b80 10 client.26008 fetch_fsmap learned FSMap version 32 2023-09-20T21:57:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.262+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1342653356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563bb15c4c60 con 0x563bb1860be0 2023-09-20T21:57:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.262+0000 7fb11d7fa700 1 -- 172.21.15.22:0/1342653356 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563bb1721bb0 con 0x563bb1860be0 2023-09-20T21:57:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.263+0000 7fb11d7fa700 1 -- 172.21.15.22:0/1342653356 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb12c098aa0 con 0x563bb1860be0 2023-09-20T21:57:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.263+0000 7fb14f7f0b80 10 client.26008 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:57:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.263+0000 7fb14f7f0b80 10 client.26008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:57:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.263+0000 7fb14f7f0b80 20 client.26008 populate_metadata read hostname 'smithi022' 2023-09-20T21:57:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.263+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x563bb1874e50 0x563bb1877290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.263+0000 7fb14f7f0b80 4 client.26008 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:57:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.263+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1342653356 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bb182e020 con 0x563bb1874e50 2023-09-20T21:57:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.263+0000 7fb139b9b700 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x563bb1874e50 0x563bb1877290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.266+0000 7fb11d7fa700 10 client.26008 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:57:52.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.266+0000 7fb139b9b700 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x563bb1874e50 0x563bb1877290 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c29fa685f47a350 server_cookie=4e8d3b2e7a1d3e1d in_seq=0 out_seq=0 2023-09-20T21:57:52.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb11d7fa700 1 -- 172.21.15.22:0/1342653356 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563bb182e020 con 0x563bb1874e50 2023-09-20T21:57:52.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb11d7fa700 10 client.26008 handle_command_reply: tid=0 2023-09-20T21:57:52.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb14f7f0b80 1 client.26008 shutdown 2023-09-20T21:57:52.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1342653356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 msgr2=0x563bb1870810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:52.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb1870810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb12c095ab0 tx=0x7fb12c00cf80 comp rx=0 tx=0).stop 2023-09-20T21:57:52.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1342653356 shutdown_connections 2023-09-20T21:57:52.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x563bb1874e50 0x563bb1877290 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb1870810 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb1869290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/1342653356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb18697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1342653356 >> 172.21.15.22:0/1342653356 conn(0x563bb178c000 msgr2=0x563bb1859880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:52.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.267+0000 7fb11d7fa700 0 client.26008 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:57:52.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.268+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1342653356 shutdown_connections 2023-09-20T21:57:52.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.268+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/1342653356 wait complete. 2023-09-20T21:57:52.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.268+0000 7fb14f7f0b80 20 client.26008 trim_cache size 0 max 16384 2023-09-20T21:57:52.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb14f7f0b80 1 Processor -- start 2023-09-20T21:57:52.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb14f7f0b80 1 -- start start 2023-09-20T21:57:52.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb1876e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb1863be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb18707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb1631920 con 0x563bb185f6f0 2023-09-20T21:57:52.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb1872500 con 0x563bb1860050 2023-09-20T21:57:52.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb1868790 con 0x563bb1860be0 2023-09-20T21:57:52.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb139b9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb1876e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb13a39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb18707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.299+0000 7fb13939a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb1863be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.300+0000 7fb13a39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb18707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:39120/0 (socket says 172.21.15.22:39120) 2023-09-20T21:57:52.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.300+0000 7fb139b9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb1876e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57130/0 (socket says 172.21.15.22:57130) 2023-09-20T21:57:52.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.300+0000 7fb13a39c700 1 -- 172.21.15.22:0/4082403634 learned_addr learned my addr 172.21.15.22:0/4082403634 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:57:52.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.300+0000 7fb13a39c700 1 -- 172.21.15.22:0/4082403634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 msgr2=0x563bb1863be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:52.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.300+0000 7fb13a39c700 1 --2- 172.21.15.22:0/4082403634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb1863be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.300+0000 7fb13a39c700 1 -- 172.21.15.22:0/4082403634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 msgr2=0x563bb1876e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:52.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.300+0000 7fb13a39c700 1 --2- 172.21.15.22:0/4082403634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb1876e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.300+0000 7fb13a39c700 1 -- 172.21.15.22:0/4082403634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb1602250 con 0x563bb1860be0 2023-09-20T21:57:52.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.301+0000 7fb13a39c700 1 --2- 172.21.15.22:0/4082403634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb18707d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00c7a0 tx=0x7fb12c004dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=658d75adbe1d7d10 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.301+0000 7fb128bff700 1 -- 172.21.15.22:0/4082403634 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0b0070 con 0x563bb1860be0 2023-09-20T21:57:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.301+0000 7fb128bff700 1 -- 172.21.15.22:0/4082403634 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb12c099cb0 con 0x563bb1860be0 2023-09-20T21:57:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.301+0000 7fb128bff700 1 -- 172.21.15.22:0/4082403634 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0b1e10 con 0x563bb1860be0 2023-09-20T21:57:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.302+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/4082403634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 msgr2=0x563bb18707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:52.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.302+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/4082403634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb18707d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00c7a0 tx=0x7fb12c004dc0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.302+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/4082403634 shutdown_connections 2023-09-20T21:57:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.302+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/4082403634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb1860be0 0x563bb18707d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.302+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/4082403634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb185f6f0 0x563bb1876e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.302+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/4082403634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb1863be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.302+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/4082403634 >> 172.21.15.22:0/4082403634 conn(0x563bb178c000 msgr2=0x563bb18670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.306+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/4082403634 shutdown_connections 2023-09-20T21:57:52.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.306+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/4082403634 wait complete. 2023-09-20T21:57:52.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.307+0000 7fb14f7f0b80 1 -- start start 2023-09-20T21:57:52.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.307+0000 7fb14f7f0b80 1 Processor -- start 2023-09-20T21:57:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.308+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb185f6f0 0x563bb186c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.308+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb186ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.308+0000 7fb14f7f0b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb1860be0 0x563bb186fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.308+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb1870210 con 0x563bb1860be0 2023-09-20T21:57:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.308+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb1870380 con 0x563bb1860050 2023-09-20T21:57:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.308+0000 7fb14f7f0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb1870540 con 0x563bb185f6f0 2023-09-20T21:57:52.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.309+0000 7fb14f7f0b80 10 client.0 _ll_register_callbacks cb 0x563bb1697a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:57:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.309+0000 7fb139b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb185f6f0 0x563bb186c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.309+0000 7fb139b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb185f6f0 0x563bb186c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:39132/0 (socket says 172.21.15.22:39132) 2023-09-20T21:57:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.309+0000 7fb139b9b700 1 -- 172.21.15.22:0/2429836176 learned_addr learned my addr 172.21.15.22:0/2429836176 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:57:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.309+0000 7fb13939a700 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb186ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.310+0000 7fb139b9b700 1 -- 172.21.15.22:0/2429836176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 msgr2=0x563bb186ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.310+0000 7fb139b9b700 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb186ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.310+0000 7fb139b9b700 1 -- 172.21.15.22:0/2429836176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb1860be0 msgr2=0x563bb186fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:57:52.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.310+0000 7fb139b9b700 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb1860be0 0x563bb186fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.310+0000 7fb139b9b700 1 -- 172.21.15.22:0/2429836176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bb15c4c60 con 0x563bb185f6f0 2023-09-20T21:57:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.310+0000 7fb139b9b700 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb185f6f0 0x563bb186c770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb13000b680 tx=0x7fb130009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89c845c94fcf681 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:57:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.310+0000 7fb11d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:57:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.311+0000 7fb11d7fa700 1 -- 172.21.15.22:0/2429836176 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130030910 con 0x563bb185f6f0 2023-09-20T21:57:52.314 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.311+0000 7fb11d7fa700 1 -- 172.21.15.22:0/2429836176 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fb130030a70 con 0x563bb185f6f0 2023-09-20T21:57:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.311+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/2429836176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb124004cf0 con 0x563bb185f6f0 2023-09-20T21:57:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.312+0000 7fb11d7fa700 1 -- 172.21.15.22:0/2429836176 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb130030c30 con 0x563bb185f6f0 2023-09-20T21:57:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.312+0000 7fb14f7f0b80 10 client.26014 fetch_fsmap learned FSMap version 32 2023-09-20T21:57:52.315 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.312+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/2429836176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563bb15da2b0 con 0x563bb185f6f0 2023-09-20T21:57:52.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.312+0000 7fb11d7fa700 1 -- 172.21.15.22:0/2429836176 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb124004cf0 con 0x563bb185f6f0 2023-09-20T21:57:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.313+0000 7fb11d7fa700 1 -- 172.21.15.22:0/2429836176 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb13000f460 con 0x563bb185f6f0 2023-09-20T21:57:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.313+0000 7fb14f7f0b80 10 client.26014 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:57:52.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.313+0000 7fb14f7f0b80 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:57:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.313+0000 7fb14f7f0b80 20 client.26014 populate_metadata read hostname 'smithi022' 2023-09-20T21:57:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.313+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x563bb18c9f30 0x563bb18cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:57:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.313+0000 7fb14f7f0b80 4 client.26014 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:57:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.313+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/2429836176 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563bb17a9e60 con 0x563bb18c9f30 2023-09-20T21:57:52.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.314+0000 7fb13939a700 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x563bb18c9f30 0x563bb18cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:57:52.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.314+0000 7fb13939a700 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x563bb18c9f30 0x563bb18cc350 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda8e3a8df54ad4c server_cookie=aca35fda814942ef in_seq=0 out_seq=0 2023-09-20T21:57:52.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.314+0000 7fb11d7fa700 10 client.26014 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:57:52.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb11d7fa700 1 -- 172.21.15.22:0/2429836176 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563bb17a9e60 con 0x563bb18c9f30 2023-09-20T21:57:52.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb11d7fa700 10 client.26014 handle_command_reply: tid=0 2023-09-20T21:57:52.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb14f7f0b80 1 client.26014 shutdown 2023-09-20T21:57:52.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/2429836176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb185f6f0 msgr2=0x563bb186c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:52.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb185f6f0 0x563bb186c770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb13000b680 tx=0x7fb130009310 comp rx=0 tx=0).stop 2023-09-20T21:57:52.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/2429836176 shutdown_connections 2023-09-20T21:57:52.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x563bb18c9f30 0x563bb18cc350 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563bb1860be0 0x563bb186fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bb185f6f0 0x563bb186c770 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb14f7f0b80 1 --2- 172.21.15.22:0/2429836176 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb1860050 0x563bb186ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/2429836176 >> 172.21.15.22:0/2429836176 conn(0x563bb178c000 msgr2=0x563bb1876e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:52.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.316+0000 7fb11d7fa700 0 client.26014 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:57:52.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.317+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/2429836176 shutdown_connections 2023-09-20T21:57:52.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.317+0000 7fb14f7f0b80 1 -- 172.21.15.22:0/2429836176 wait complete. 2023-09-20T21:57:52.322 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.317+0000 7fb14f7f0b80 20 client.26014 trim_cache size 0 max 16384 2023-09-20T21:57:52.322 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:57:52.322 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:57:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:57:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:57:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:57:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:57:52.323 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:57:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:57:52.324 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:57:52.324 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:57:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 -- 172.21.15.22:0/1781053219 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1200a1500 msgr2=0x7fb1200a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1200a1500 0x7fb1200a39c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x563bb185b150 tx=0x7fb13000b280 comp rx=0 tx=0).stop 2023-09-20T21:57:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 -- 172.21.15.22:0/1781053219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb134104980 msgr2=0x7fb1341a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:57:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb134104980 0x7fb1341a4610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00ef30 tx=0x7fb12c00c3a0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 -- 172.21.15.22:0/1781053219 shutdown_connections 2023-09-20T21:57:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb134104980 0x7fb1341a4610 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1340ff640 0x7fb13419d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb1200a1500 0x7fb1200a39c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 --2- 172.21.15.22:0/1781053219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb134101fb0 0x7fb13419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:57:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 -- 172.21.15.22:0/1781053219 >> 172.21.15.22:0/1781053219 conn(0x7fb1340fb240 msgr2=0x7fb134102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:57:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 -- 172.21.15.22:0/1781053219 shutdown_connections 2023-09-20T21:57:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:57:52.318+0000 7fb13bdff700 1 -- 172.21.15.22:0/1781053219 wait complete. 2023-09-20T21:57:52.362 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:57:53.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:52 smithi022 ceph-mon[97651]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.9 MiB/s wr, 539 op/s 2023-09-20T21:57:53.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:52 smithi123 ceph-mon[134433]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.9 MiB/s wr, 539 op/s 2023-09-20T21:57:53.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:52 smithi089 ceph-mon[134090]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.9 MiB/s wr, 539 op/s 2023-09-20T21:57:53.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 510.0 s, 519.0 tps, lat 190.927 ms stddev 232.595 2023-09-20T21:57:54.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:54 smithi022 ceph-mon[97651]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.7 MiB/s wr, 529 op/s 2023-09-20T21:57:54.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:54 smithi123 ceph-mon[134433]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.7 MiB/s wr, 529 op/s 2023-09-20T21:57:54.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:54 smithi089 ceph-mon[134090]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.7 MiB/s wr, 529 op/s 2023-09-20T21:57:57.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:56 smithi123 ceph-mon[134433]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 468 op/s 2023-09-20T21:57:57.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:56 smithi089 ceph-mon[134090]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 468 op/s 2023-09-20T21:57:57.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:56 smithi022 ceph-mon[97651]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 468 op/s 2023-09-20T21:57:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 515.0 s, 526.1 tps, lat 189.195 ms stddev 227.625 2023-09-20T21:57:59.105 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:57:58 smithi123 ceph-mon[134433]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 536 op/s 2023-09-20T21:57:59.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:57:58 smithi089 ceph-mon[134090]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 536 op/s 2023-09-20T21:57:59.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:57:58 smithi022 ceph-mon[97651]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 536 op/s 2023-09-20T21:58:01.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:00 smithi123 ceph-mon[134433]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 474 op/s 2023-09-20T21:58:01.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:00 smithi089 ceph-mon[134090]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 474 op/s 2023-09-20T21:58:01.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:00 smithi022 ceph-mon[97651]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 474 op/s 2023-09-20T21:58:03.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:02 smithi089 ceph-mon[134090]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 516 op/s 2023-09-20T21:58:03.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:02 smithi022 ceph-mon[97651]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 516 op/s 2023-09-20T21:58:03.347 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:02 smithi123 ceph-mon[134433]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 516 op/s 2023-09-20T21:58:03.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 520.0 s, 505.8 tps, lat 202.249 ms stddev 295.159 2023-09-20T21:58:04.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:04 smithi022 ceph-mon[97651]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 520 op/s 2023-09-20T21:58:04.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:04 smithi123 ceph-mon[134433]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 520 op/s 2023-09-20T21:58:04.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:04 smithi089 ceph-mon[134090]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 520 op/s 2023-09-20T21:58:07.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:06 smithi123 ceph-mon[134433]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 443 op/s 2023-09-20T21:58:07.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:06 smithi089 ceph-mon[134090]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 443 op/s 2023-09-20T21:58:07.264 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:06 smithi022 ceph-mon[97651]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 443 op/s 2023-09-20T21:58:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 525.0 s, 552.3 tps, lat 178.043 ms stddev 180.287 2023-09-20T21:58:09.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:08 smithi123 ceph-mon[134433]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 571 op/s 2023-09-20T21:58:09.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:08 smithi089 ceph-mon[134090]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 571 op/s 2023-09-20T21:58:09.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:08 smithi022 ceph-mon[97651]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.1 MiB/s wr, 571 op/s 2023-09-20T21:58:11.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:10 smithi123 ceph-mon[134433]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2023-09-20T21:58:11.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:10 smithi089 ceph-mon[134090]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2023-09-20T21:58:11.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:10 smithi022 ceph-mon[97651]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2023-09-20T21:58:13.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:12 smithi123 ceph-mon[134433]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 527 op/s 2023-09-20T21:58:13.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:12 smithi089 ceph-mon[134090]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 527 op/s 2023-09-20T21:58:13.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:12 smithi022 ceph-mon[97651]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 527 op/s 2023-09-20T21:58:13.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 530.0 s, 505.7 tps, lat 191.714 ms stddev 258.211 2023-09-20T21:58:14.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:14 smithi123 ceph-mon[134433]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2023-09-20T21:58:14.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:14 smithi089 ceph-mon[134090]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2023-09-20T21:58:14.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:14 smithi022 ceph-mon[97651]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2023-09-20T21:58:17.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:16 smithi089 ceph-mon[134090]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 479 op/s 2023-09-20T21:58:17.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:16 smithi022 ceph-mon[97651]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 479 op/s 2023-09-20T21:58:17.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:16 smithi123 ceph-mon[134433]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 479 op/s 2023-09-20T21:58:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 535.0 s, 561.7 tps, lat 182.151 ms stddev 216.004 2023-09-20T21:58:19.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:18 smithi022 ceph-mon[97651]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 6.4 MiB/s wr, 565 op/s 2023-09-20T21:58:19.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:18 smithi123 ceph-mon[134433]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 6.4 MiB/s wr, 565 op/s 2023-09-20T21:58:19.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:18 smithi089 ceph-mon[134090]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 6.4 MiB/s wr, 565 op/s 2023-09-20T21:58:21.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:20 smithi022 ceph-mon[97651]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 5.6 MiB/s wr, 502 op/s 2023-09-20T21:58:21.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:20 smithi123 ceph-mon[134433]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 5.6 MiB/s wr, 502 op/s 2023-09-20T21:58:21.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:20 smithi089 ceph-mon[134090]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 5.6 MiB/s wr, 502 op/s 2023-09-20T21:58:22.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:58:22.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.783+0000 7f0da93ff700 1 -- 172.21.15.22:0/2425120733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da410d120 msgr2=0x7f0da410d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:22.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.783+0000 7f0da93ff700 1 --2- 172.21.15.22:0/2425120733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da410d120 0x7f0da410d5a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c01c0d0 tx=0x7f0d9c01c3e0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.784+0000 7f0da93ff700 1 -- 172.21.15.22:0/2425120733 shutdown_connections 2023-09-20T21:58:22.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.784+0000 7f0da93ff700 1 --2- 172.21.15.22:0/2425120733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da410d120 0x7f0da410d5a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.784+0000 7f0da93ff700 1 --2- 172.21.15.22:0/2425120733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da4107de0 0x7f0da410a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.784+0000 7f0da93ff700 1 --2- 172.21.15.22:0/2425120733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da410a750 0x7f0da410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.784+0000 7f0da93ff700 1 -- 172.21.15.22:0/2425120733 >> 172.21.15.22:0/2425120733 conn(0x7f0da406db90 msgr2=0x7f0da406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:22.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.784+0000 7f0da93ff700 1 -- 172.21.15.22:0/2425120733 shutdown_connections 2023-09-20T21:58:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.785+0000 7f0da93ff700 1 -- 172.21.15.22:0/2425120733 wait complete. 2023-09-20T21:58:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.786+0000 7f0da93ff700 1 Processor -- start 2023-09-20T21:58:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.786+0000 7f0da93ff700 1 -- start start 2023-09-20T21:58:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.787+0000 7f0da93ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da4107de0 0x7f0da419d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.787+0000 7f0da93ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da410a750 0x7f0da419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.787+0000 7f0da93ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da410d120 0x7f0da41a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.787+0000 7f0da93ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da41a4cd0 con 0x7f0da410a750 2023-09-20T21:58:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.787+0000 7f0da93ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da41a4e10 con 0x7f0da4107de0 2023-09-20T21:58:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.787+0000 7f0da93ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da41a4fa0 con 0x7f0da410d120 2023-09-20T21:58:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.788+0000 7f0da37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da410d120 0x7f0da41a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.788+0000 7f0da37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da410d120 0x7f0da41a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:34486/0 (socket says 172.21.15.22:34486) 2023-09-20T21:58:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.788+0000 7f0da37fe700 1 -- 172.21.15.22:0/2666843215 learned_addr learned my addr 172.21.15.22:0/2666843215 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:58:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.788+0000 7f0da2ffd700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da4107de0 0x7f0da419d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.788+0000 7f0da37fe700 1 -- 172.21.15.22:0/2666843215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da4107de0 msgr2=0x7f0da419d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.788+0000 7f0da37fe700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da4107de0 0x7f0da419d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.788+0000 7f0da37fe700 1 -- 172.21.15.22:0/2666843215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da410a750 msgr2=0x7f0da419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.788+0000 7f0da37fe700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da410a750 0x7f0da419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.788+0000 7f0da37fe700 1 -- 172.21.15.22:0/2666843215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c01ad10 con 0x7f0da410d120 2023-09-20T21:58:22.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.791+0000 7f0da37fe700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da410d120 0x7f0da41a4700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c018f20 tx=0x7f0d9c003ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:22.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.792+0000 7f0d8bfff700 1 -- 172.21.15.22:0/2666843215 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c01f030 con 0x7f0da410d120 2023-09-20T21:58:22.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.792+0000 7f0d8bfff700 1 -- 172.21.15.22:0/2666843215 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d9c005980 con 0x7f0da410d120 2023-09-20T21:58:22.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.792+0000 7f0d8bfff700 1 -- 172.21.15.22:0/2666843215 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c02be90 con 0x7f0da410d120 2023-09-20T21:58:22.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.792+0000 7f0da93ff700 1 -- 172.21.15.22:0/2666843215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da41a5220 con 0x7f0da410d120 2023-09-20T21:58:22.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.792+0000 7f0da93ff700 1 -- 172.21.15.22:0/2666843215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da41a5650 con 0x7f0da410d120 2023-09-20T21:58:22.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.800+0000 7f0d8bfff700 1 -- 172.21.15.22:0/2666843215 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f0d9c034410 con 0x7f0da410d120 2023-09-20T21:58:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.802+0000 7f0dbcd93b80 1 Processor -- start 2023-09-20T21:58:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.803+0000 7f0d8bfff700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0d8c0a1480 0x7f0d8c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.803+0000 7f0d8bfff700 1 -- 172.21.15.22:0/2666843215 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f0d9c020020 con 0x7f0da410d120 2023-09-20T21:58:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.804+0000 7f0dbcd93b80 1 -- start start 2023-09-20T21:58:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.805+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbb6f0 0x55ef48bbbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.805+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48bbc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.805+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bbcbe0 0x55ef48bbd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.805+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48b2fca0 con 0x55ef48bbc050 2023-09-20T21:58:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.805+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48b13c70 con 0x55ef48bbcbe0 2023-09-20T21:58:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.805+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48b11590 con 0x55ef48bbb6f0 2023-09-20T21:58:22.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.819+0000 7f0da2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbb6f0 0x55ef48bbbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.819+0000 7f0da2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbb6f0 0x55ef48bbbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:34496/0 (socket says 172.21.15.22:34496) 2023-09-20T21:58:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.819+0000 7f0da2ffd700 1 -- 172.21.15.22:0/2122103405 learned_addr learned my addr 172.21.15.22:0/2122103405 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:58:22.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.820+0000 7f0da27fc700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0d8c0a1480 0x7f0d8c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.820+0000 7f0da27fc700 1 --2- 172.21.15.22:0/2122103405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48bbc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.820+0000 7f0da37fe700 1 --2- 172.21.15.22:0/2122103405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bbcbe0 0x55ef48bbd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.827+0000 7f0da2ffd700 1 -- 172.21.15.22:0/2122103405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bbcbe0 msgr2=0x55ef48bbd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:22.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.827+0000 7f0da2ffd700 1 --2- 172.21.15.22:0/2122103405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bbcbe0 0x55ef48bbd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.828+0000 7f0da2ffd700 1 -- 172.21.15.22:0/2122103405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 msgr2=0x55ef48bbc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:22.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.828+0000 7f0da2ffd700 1 --2- 172.21.15.22:0/2122103405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48bbc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.828+0000 7f0da2ffd700 1 -- 172.21.15.22:0/2122103405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef48937de0 con 0x55ef48bbb6f0 2023-09-20T21:58:22.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.828+0000 7f0da2ffd700 1 --2- 172.21.15.22:0/2122103405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbb6f0 0x55ef48bbbb10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0d98007810 tx=0x7f0d98007b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f2d2b87c0559f5b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:22.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.829+0000 7f0d897fa700 1 -- 172.21.15.22:0/2122103405 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d980125f0 con 0x55ef48bbb6f0 2023-09-20T21:58:22.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.832+0000 7f0d897fa700 1 -- 172.21.15.22:0/2122103405 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d98012c10 con 0x55ef48bbb6f0 2023-09-20T21:58:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.832+0000 7f0d897fa700 1 -- 172.21.15.22:0/2122103405 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98013e00 con 0x55ef48bbb6f0 2023-09-20T21:58:22.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.838+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/2122103405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbb6f0 msgr2=0x55ef48bbbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:22.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.838+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/2122103405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbb6f0 0x55ef48bbbb10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0d98007810 tx=0x7f0d98007b20 comp rx=0 tx=0).stop 2023-09-20T21:58:22.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.841+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/2122103405 shutdown_connections 2023-09-20T21:58:22.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.841+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/2122103405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bbcbe0 0x55ef48bbd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.841+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/2122103405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbb6f0 0x55ef48bbbb10 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0d98007810 tx=0x7f0d98007b20 comp rx=0 tx=0).stop 2023-09-20T21:58:22.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.841+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/2122103405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48bbc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.841+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/2122103405 >> 172.21.15.22:0/2122103405 conn(0x55ef48ae8000 msgr2=0x55ef48bb61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:22.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.843+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/2122103405 shutdown_connections 2023-09-20T21:58:22.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.845+0000 7f0da27fc700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0d8c0a1480 0x7f0d8c0a3940 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55ef48bb7150 tx=0x7f0d9400b250 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:22.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.846+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/2122103405 wait complete. 2023-09-20T21:58:22.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.847+0000 7f0dbcd93b80 1 -- start start 2023-09-20T21:58:22.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.847+0000 7f0dbcd93b80 1 Processor -- start 2023-09-20T21:58:22.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.851+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bbc050 0x55ef48be7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.851+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbcbe0 0x55ef48be79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.851+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bec9e0 0x55ef48beedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.852+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48b0ce80 con 0x55ef48bbcbe0 2023-09-20T21:58:22.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.852+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48aa9740 con 0x55ef48bbc050 2023-09-20T21:58:22.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.852+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48aa93e0 con 0x55ef48bec9e0 2023-09-20T21:58:22.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.852+0000 7f0dbcd93b80 10 client.0 _ll_register_callbacks cb 0x55ef489f3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:58:22.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.855+0000 7f0da27fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbcbe0 0x55ef48be79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.855+0000 7f0da27fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbcbe0 0x55ef48be79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50052/0 (socket says 172.21.15.22:50052) 2023-09-20T21:58:22.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.855+0000 7f0da27fc700 1 -- 172.21.15.22:0/3243020004 learned_addr learned my addr 172.21.15.22:0/3243020004 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:58:22.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.855+0000 7f0da37fe700 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bec9e0 0x55ef48beedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.856+0000 7f0da37fe700 1 -- 172.21.15.22:0/3243020004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bbc050 msgr2=0x55ef48be7470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:58:22.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.856+0000 7f0da37fe700 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bbc050 0x55ef48be7470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.856+0000 7f0da37fe700 1 -- 172.21.15.22:0/3243020004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbcbe0 msgr2=0x55ef48be79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:22.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.856+0000 7f0da37fe700 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbcbe0 0x55ef48be79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.856+0000 7f0da37fe700 1 -- 172.21.15.22:0/3243020004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9800e040 con 0x55ef48bec9e0 2023-09-20T21:58:22.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.856+0000 7f0da37fe700 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bec9e0 0x55ef48beedd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0dcc80 tx=0x7f0d9c0abfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4874da74be496f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:22.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.856+0000 7f0d86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:58:22.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.857+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3243020004 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0b03d0 con 0x55ef48bec9e0 2023-09-20T21:58:22.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.857+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3243020004 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f0d9c0b4580 con 0x55ef48bec9e0 2023-09-20T21:58:22.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.857+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3243020004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef48a7dbb0 con 0x55ef48bec9e0 2023-09-20T21:58:22.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.858+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3243020004 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d9c0df240 con 0x55ef48bec9e0 2023-09-20T21:58:22.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.858+0000 7f0dbcd93b80 10 client.26023 fetch_fsmap learned FSMap version 32 2023-09-20T21:58:22.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.858+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3243020004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ef48920c60 con 0x55ef48bec9e0 2023-09-20T21:58:22.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.859+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3243020004 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef48a7dbb0 con 0x55ef48bec9e0 2023-09-20T21:58:22.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.859+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3243020004 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0d9c0b40f0 con 0x55ef48bec9e0 2023-09-20T21:58:22.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.860+0000 7f0dbcd93b80 10 client.26023 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:58:22.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.860+0000 7f0dbcd93b80 10 client.26023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:58:22.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.860+0000 7f0dbcd93b80 20 client.26023 populate_metadata read hostname 'smithi022' 2023-09-20T21:58:22.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.860+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ef48bf32a0 0x55ef48bf56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.860+0000 7f0dbcd93b80 4 client.26023 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:58:22.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.860+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3243020004 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef48b8a020 con 0x55ef48bf32a0 2023-09-20T21:58:22.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.864+0000 7f0da2ffd700 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ef48bf32a0 0x55ef48bf56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.865+0000 7f0da2ffd700 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ef48bf32a0 0x55ef48bf56c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c2afa1a827ba6a server_cookie=d565842f21ffadf7 in_seq=0 out_seq=0 2023-09-20T21:58:22.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.865+0000 7f0d86ff5700 10 client.26023 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:58:22.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.867+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3243020004 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ef48b8a020 con 0x55ef48bf32a0 2023-09-20T21:58:22.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.867+0000 7f0d86ff5700 10 client.26023 handle_command_reply: tid=0 2023-09-20T21:58:22.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.867+0000 7f0dbcd93b80 1 client.26023 shutdown 2023-09-20T21:58:22.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.868+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3243020004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bec9e0 msgr2=0x55ef48beedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:22.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.868+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bec9e0 0x55ef48beedd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0dcc80 tx=0x7f0d9c0abfb0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.868+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3243020004 shutdown_connections 2023-09-20T21:58:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.868+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ef48bf32a0 0x55ef48bf56c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.868+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bec9e0 0x55ef48beedd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.869+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bbc050 0x55ef48be7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.869+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3243020004 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbcbe0 0x55ef48be79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.869+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3243020004 >> 172.21.15.22:0/3243020004 conn(0x55ef48ae8000 msgr2=0x55ef48bb6110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:22.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.869+0000 7f0d86ff5700 0 client.26023 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:58:22.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.869+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3243020004 shutdown_connections 2023-09-20T21:58:22.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.869+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3243020004 wait complete. 2023-09-20T21:58:22.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.870+0000 7f0dbcd93b80 20 client.26023 trim_cache size 0 max 16384 2023-09-20T21:58:22.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.904+0000 7f0dbcd93b80 1 Processor -- start 2023-09-20T21:58:22.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.904+0000 7f0dbcd93b80 1 -- start start 2023-09-20T21:58:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.905+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48be6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.905+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 0x55ef48bb4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.905+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bec9e0 0x55ef48bb22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.905+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef4898d920 con 0x55ef48bbc050 2023-09-20T21:58:22.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.905+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48a7dbb0 con 0x55ef48bec9e0 2023-09-20T21:58:22.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.905+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48bb75b0 con 0x55ef48bbcbe0 2023-09-20T21:58:22.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.906+0000 7f0da27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 0x55ef48bb4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.906+0000 7f0da27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 0x55ef48bb4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:34512/0 (socket says 172.21.15.22:34512) 2023-09-20T21:58:22.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.906+0000 7f0da27fc700 1 -- 172.21.15.22:0/3593337386 learned_addr learned my addr 172.21.15.22:0/3593337386 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:58:22.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.908+0000 7f0da27fc700 1 -- 172.21.15.22:0/3593337386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bec9e0 msgr2=0x55ef48bb22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:22.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.908+0000 7f0da27fc700 1 --2- 172.21.15.22:0/3593337386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bec9e0 0x55ef48bb22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.908+0000 7f0da27fc700 1 -- 172.21.15.22:0/3593337386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 msgr2=0x55ef48be6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:22.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.908+0000 7f0da27fc700 1 --2- 172.21.15.22:0/3593337386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48be6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.908+0000 7f0da27fc700 1 -- 172.21.15.22:0/3593337386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef4895e250 con 0x55ef48bbcbe0 2023-09-20T21:58:22.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.912+0000 7f0da27fc700 1 --2- 172.21.15.22:0/3593337386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 0x55ef48bb4de0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d940090b0 tx=0x7f0d940069f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c8e4c3252209f83 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:22.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.920+0000 7f0d89ffb700 1 -- 172.21.15.22:0/3593337386 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9400e300 con 0x55ef48bbcbe0 2023-09-20T21:58:22.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.920+0000 7f0d89ffb700 1 -- 172.21.15.22:0/3593337386 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d94006750 con 0x55ef48bbcbe0 2023-09-20T21:58:22.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.920+0000 7f0d89ffb700 1 -- 172.21.15.22:0/3593337386 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9400d420 con 0x55ef48bbcbe0 2023-09-20T21:58:22.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.926+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3593337386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 msgr2=0x55ef48bb4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:22.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.926+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3593337386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 0x55ef48bb4de0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d940090b0 tx=0x7f0d940069f0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.937+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3593337386 shutdown_connections 2023-09-20T21:58:22.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.937+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3593337386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bec9e0 0x55ef48bb22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.937+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3593337386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48be6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.937+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3593337386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 0x55ef48bb4de0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.937+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3593337386 >> 172.21.15.22:0/3593337386 conn(0x55ef48ae8000 msgr2=0x55ef48bfa040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:22.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.948+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3593337386 shutdown_connections 2023-09-20T21:58:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.948+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3593337386 wait complete. 2023-09-20T21:58:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.948+0000 7f0dbcd93b80 1 -- start start 2023-09-20T21:58:22.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.949+0000 7f0dbcd93b80 1 Processor -- start 2023-09-20T21:58:22.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.949+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48c417f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.949+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 0x55ef48c43d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.949+0000 7f0dbcd93b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bec9e0 0x55ef48c46290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.949+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48c03460 con 0x55ef48bbc050 2023-09-20T21:58:22.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.949+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48c467d0 con 0x55ef48bec9e0 2023-09-20T21:58:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.949+0000 7f0dbcd93b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef48c46960 con 0x55ef48bbcbe0 2023-09-20T21:58:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.949+0000 7f0dbcd93b80 10 client.0 _ll_register_callbacks cb 0x55ef489f3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:58:22.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.958+0000 7f0da2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48c417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.958+0000 7f0da2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48c417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50076/0 (socket says 172.21.15.22:50076) 2023-09-20T21:58:22.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.958+0000 7f0da2ffd700 1 -- 172.21.15.22:0/3502081638 learned_addr learned my addr 172.21.15.22:0/3502081638 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:58:22.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.958+0000 7f0da37fe700 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bec9e0 0x55ef48c46290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.958+0000 7f0da27fc700 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 0x55ef48c43d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.959+0000 7f0da37fe700 1 -- 172.21.15.22:0/3502081638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 msgr2=0x55ef48c43d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:22.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.959+0000 7f0da37fe700 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 0x55ef48c43d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.959+0000 7f0da37fe700 1 -- 172.21.15.22:0/3502081638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 msgr2=0x55ef48c417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:22.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.959+0000 7f0da37fe700 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48c417f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.959+0000 7f0da37fe700 1 -- 172.21.15.22:0/3502081638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d94016040 con 0x55ef48bec9e0 2023-09-20T21:58:22.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.959+0000 7f0da37fe700 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bec9e0 0x55ef48c46290 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0ac2a0 tx=0x7f0d9c0abd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65658a8a45405a24 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:22.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.959+0000 7f0d86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:58:22.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.960+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3502081638 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0b4580 con 0x55ef48bec9e0 2023-09-20T21:58:22.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.960+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3502081638 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f0d9c0b03d0 con 0x55ef48bec9e0 2023-09-20T21:58:22.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.960+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3502081638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d98012c10 con 0x55ef48bec9e0 2023-09-20T21:58:22.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.960+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3502081638 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d9c0df240 con 0x55ef48bec9e0 2023-09-20T21:58:22.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.960+0000 7f0dbcd93b80 10 client.26012 fetch_fsmap learned FSMap version 32 2023-09-20T21:58:22.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.960+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3502081638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ef489362b0 con 0x55ef48bec9e0 2023-09-20T21:58:22.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.961+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3502081638 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d9c08d070 con 0x55ef48bec9e0 2023-09-20T21:58:22.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.961+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3502081638 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0d9c0c4b10 con 0x55ef48bec9e0 2023-09-20T21:58:22.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.961+0000 7f0dbcd93b80 10 client.26012 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:58:22.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.961+0000 7f0dbcd93b80 10 client.26012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:58:22.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.961+0000 7f0dbcd93b80 20 client.26012 populate_metadata read hostname 'smithi022' 2023-09-20T21:58:22.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.961+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ef48c47f00 0x55ef48c4a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:22.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.961+0000 7f0dbcd93b80 4 client.26012 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:58:22.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.961+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3502081638 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ef48b05e60 con 0x55ef48c47f00 2023-09-20T21:58:22.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.962+0000 7f0da2ffd700 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ef48c47f00 0x55ef48c4a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:22.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.968+0000 7f0da2ffd700 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ef48c47f00 0x55ef48c4a2f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe3a7e416bb99a3 server_cookie=3a4a5a87e5d28267 in_seq=0 out_seq=0 2023-09-20T21:58:22.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.968+0000 7f0d86ff5700 10 client.26012 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:58:22.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.969+0000 7f0d86ff5700 1 -- 172.21.15.22:0/3502081638 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ef48b05e60 con 0x55ef48c47f00 2023-09-20T21:58:22.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.969+0000 7f0d86ff5700 10 client.26012 handle_command_reply: tid=0 2023-09-20T21:58:22.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.969+0000 7f0dbcd93b80 1 client.26012 shutdown 2023-09-20T21:58:22.996 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:22 smithi022 ceph-mon[97651]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 6.0 MiB/s wr, 554 op/s 2023-09-20T21:58:22.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.989+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3502081638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bec9e0 msgr2=0x55ef48c46290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:22.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.989+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bec9e0 0x55ef48c46290 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0ac2a0 tx=0x7f0d9c0abd90 comp rx=0 tx=0).stop 2023-09-20T21:58:22.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.989+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3502081638 shutdown_connections 2023-09-20T21:58:22.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.989+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55ef48c47f00 0x55ef48c4a2f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.989+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ef48bec9e0 0x55ef48c46290 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.989+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ef48bbc050 0x55ef48c417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.989+0000 7f0dbcd93b80 1 --2- 172.21.15.22:0/3502081638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef48bbcbe0 0x55ef48c43d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:22.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.989+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3502081638 >> 172.21.15.22:0/3502081638 conn(0x55ef48ae8000 msgr2=0x55ef48bb22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:22.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.990+0000 7f0d86ff5700 0 client.26012 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:58:22.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.990+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3502081638 shutdown_connections 2023-09-20T21:58:22.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:22.990+0000 7f0dbcd93b80 1 -- 172.21.15.22:0/3502081638 wait complete. 2023-09-20T21:58:23.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.002+0000 7f0dbcd93b80 20 client.26012 trim_cache size 0 max 16384 2023-09-20T21:58:23.006 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:58:23.006 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:58:23.007 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:58:23.007 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:58:23.008 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:58:23.008 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:58:23.009 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:58:23.009 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:58:23.009 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:58:23.010 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:58:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.014+0000 7f0da93ff700 1 -- 172.21.15.22:0/2666843215 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0d8c0a1480 msgr2=0x7f0d8c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.014+0000 7f0da93ff700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0d8c0a1480 0x7f0d8c0a3940 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55ef48bb7150 tx=0x7f0d9400b250 comp rx=0 tx=0).stop 2023-09-20T21:58:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.014+0000 7f0da93ff700 1 -- 172.21.15.22:0/2666843215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da410d120 msgr2=0x7f0da41a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:23.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.014+0000 7f0da93ff700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da410d120 0x7f0da41a4700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c018f20 tx=0x7f0d9c003ab0 comp rx=0 tx=0).stop 2023-09-20T21:58:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.014+0000 7f0da93ff700 1 -- 172.21.15.22:0/2666843215 shutdown_connections 2023-09-20T21:58:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.014+0000 7f0da93ff700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0da410d120 0x7f0da41a4700 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.014+0000 7f0da93ff700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f0d8c0a1480 0x7f0d8c0a3940 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.014+0000 7f0da93ff700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0da4107de0 0x7f0da419d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.014+0000 7f0da93ff700 1 --2- 172.21.15.22:0/2666843215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0da410a750 0x7f0da419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.014+0000 7f0da93ff700 1 -- 172.21.15.22:0/2666843215 >> 172.21.15.22:0/2666843215 conn(0x7f0da406db90 msgr2=0x7f0da406ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.015+0000 7f0da93ff700 1 -- 172.21.15.22:0/2666843215 shutdown_connections 2023-09-20T21:58:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:23.015+0000 7f0da93ff700 1 -- 172.21.15.22:0/2666843215 wait complete. 2023-09-20T21:58:23.044 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:58:23.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:22 smithi123 ceph-mon[134433]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 6.0 MiB/s wr, 554 op/s 2023-09-20T21:58:23.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:22 smithi089 ceph-mon[134090]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 6.0 MiB/s wr, 554 op/s 2023-09-20T21:58:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 540.0 s, 485.8 tps, lat 207.063 ms stddev 210.017 2023-09-20T21:58:24.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:24 smithi022 ceph-mon[97651]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 5.5 MiB/s wr, 497 op/s 2023-09-20T21:58:24.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:24 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:58:24.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:24 smithi123 ceph-mon[134433]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 5.5 MiB/s wr, 497 op/s 2023-09-20T21:58:24.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:24 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:58:24.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:24 smithi089 ceph-mon[134090]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 5.5 MiB/s wr, 497 op/s 2023-09-20T21:58:24.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:24 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:58:25.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:58:25.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:58:25.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:25 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:58:25.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:25 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:58:25.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:25 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:58:25.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:25 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:58:25.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:25 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:58:25.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:25 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:58:25.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:25 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:58:26.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:26 smithi022 ceph-mon[97651]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 5.1 MiB/s wr, 470 op/s 2023-09-20T21:58:26.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:26 smithi123 ceph-mon[134433]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 5.1 MiB/s wr, 470 op/s 2023-09-20T21:58:26.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:26 smithi089 ceph-mon[134090]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 5.1 MiB/s wr, 470 op/s 2023-09-20T21:58:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 545.0 s, 507.0 tps, lat 199.124 ms stddev 228.391 2023-09-20T21:58:29.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:28 smithi022 ceph-mon[97651]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 5.5 MiB/s wr, 529 op/s 2023-09-20T21:58:29.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:28 smithi123 ceph-mon[134433]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 5.5 MiB/s wr, 529 op/s 2023-09-20T21:58:29.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:28 smithi089 ceph-mon[134090]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 5.5 MiB/s wr, 529 op/s 2023-09-20T21:58:31.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:30 smithi022 ceph-mon[97651]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 532 op/s 2023-09-20T21:58:31.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:30 smithi123 ceph-mon[134433]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 532 op/s 2023-09-20T21:58:31.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:30 smithi089 ceph-mon[134090]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 532 op/s 2023-09-20T21:58:33.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:32 smithi022 ceph-mon[97651]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2023-09-20T21:58:33.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:32 smithi123 ceph-mon[134433]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2023-09-20T21:58:33.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:32 smithi089 ceph-mon[134090]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2023-09-20T21:58:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 550.0 s, 621.0 tps, lat 161.296 ms stddev 164.839 2023-09-20T21:58:34.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:34 smithi022 ceph-mon[97651]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.6 MiB/s wr, 472 op/s 2023-09-20T21:58:34.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:34 smithi123 ceph-mon[134433]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.6 MiB/s wr, 472 op/s 2023-09-20T21:58:34.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:34 smithi089 ceph-mon[134090]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.6 MiB/s wr, 472 op/s 2023-09-20T21:58:37.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:36 smithi022 ceph-mon[97651]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 508 op/s 2023-09-20T21:58:37.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:36 smithi123 ceph-mon[134433]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 508 op/s 2023-09-20T21:58:37.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:36 smithi089 ceph-mon[134090]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 508 op/s 2023-09-20T21:58:38.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 555.0 s, 483.6 tps, lat 204.208 ms stddev 225.192 2023-09-20T21:58:39.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:38 smithi022 ceph-mon[97651]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 538 op/s 2023-09-20T21:58:39.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:38 smithi123 ceph-mon[134433]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 538 op/s 2023-09-20T21:58:39.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:38 smithi089 ceph-mon[134090]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 538 op/s 2023-09-20T21:58:41.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:40 smithi022 ceph-mon[97651]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 534 op/s 2023-09-20T21:58:41.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:40 smithi123 ceph-mon[134433]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 534 op/s 2023-09-20T21:58:41.366 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:40 smithi089 ceph-mon[134090]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 534 op/s 2023-09-20T21:58:43.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:43 smithi123 ceph-mon[134433]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 525 op/s 2023-09-20T21:58:43.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:43 smithi089 ceph-mon[134090]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 525 op/s 2023-09-20T21:58:43.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:43 smithi022 ceph-mon[97651]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 525 op/s 2023-09-20T21:58:43.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 560.0 s, 504.8 tps, lat 198.978 ms stddev 209.326 2023-09-20T21:58:44.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:44 smithi123 ceph-mon[134433]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.8 MiB/s wr, 464 op/s 2023-09-20T21:58:44.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:44 smithi089 ceph-mon[134090]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.8 MiB/s wr, 464 op/s 2023-09-20T21:58:44.762 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:44 smithi022 ceph-mon[97651]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.8 MiB/s wr, 464 op/s 2023-09-20T21:58:47.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:46 smithi022 ceph-mon[97651]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.7 MiB/s wr, 485 op/s 2023-09-20T21:58:47.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:46 smithi123 ceph-mon[134433]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.7 MiB/s wr, 485 op/s 2023-09-20T21:58:47.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:46 smithi089 ceph-mon[134090]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.7 MiB/s wr, 485 op/s 2023-09-20T21:58:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 565.0 s, 457.9 tps, lat 215.261 ms stddev 259.429 2023-09-20T21:58:49.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:48 smithi022 ceph-mon[97651]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 519 op/s 2023-09-20T21:58:49.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:48 smithi123 ceph-mon[134433]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 519 op/s 2023-09-20T21:58:49.369 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:48 smithi089 ceph-mon[134090]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 519 op/s 2023-09-20T21:58:51.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:50 smithi022 ceph-mon[97651]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.5 MiB/s wr, 513 op/s 2023-09-20T21:58:51.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:50 smithi123 ceph-mon[134433]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.5 MiB/s wr, 513 op/s 2023-09-20T21:58:51.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:50 smithi089 ceph-mon[134090]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.5 MiB/s wr, 513 op/s 2023-09-20T21:58:53.046 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:58:53.270 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:53 smithi022 ceph-mon[97651]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2023-09-20T21:58:53.347 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:53 smithi123 ceph-mon[134433]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2023-09-20T21:58:53.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:53 smithi089 ceph-mon[134090]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2023-09-20T21:58:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.527+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/2509620882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb81018d0 msgr2=0x7f7cb80690d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.527+0000 7f7cbf3ff700 1 --2- 172.21.15.22:0/2509620882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb81018d0 0x7f7cb80690d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009990 tx=0x7f7cb0009ca0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.527+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/2509620882 shutdown_connections 2023-09-20T21:58:53.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.527+0000 7f7cbf3ff700 1 --2- 172.21.15.22:0/2509620882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb81018d0 0x7f7cb80690d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.527+0000 7f7cbf3ff700 1 --2- 172.21.15.22:0/2509620882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cb80ffd40 0x7f7cb8100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.527+0000 7f7cbf3ff700 1 --2- 172.21.15.22:0/2509620882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cb8100f30 0x7f7cb8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.527+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/2509620882 >> 172.21.15.22:0/2509620882 conn(0x7f7cb80fb2b0 msgr2=0x7f7cb80fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.527+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/2509620882 shutdown_connections 2023-09-20T21:58:53.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.527+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/2509620882 wait complete. 2023-09-20T21:58:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.529+0000 7f7cbf3ff700 1 Processor -- start 2023-09-20T21:58:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.529+0000 7f7cbf3ff700 1 -- start start 2023-09-20T21:58:53.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.529+0000 7f7cbf3ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cb80ffd40 0x7f7cb8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.529+0000 7f7cbf3ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb8100f30 0x7f7cb8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.529+0000 7f7cbf3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cb81018d0 0x7f7cb81a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.529+0000 7f7cbf3ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81a0860 con 0x7f7cb80ffd40 2023-09-20T21:58:53.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.530+0000 7f7cbf3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb806a830 con 0x7f7cb81018d0 2023-09-20T21:58:53.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.530+0000 7f7cbf3ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb806a9f0 con 0x7f7cb8100f30 2023-09-20T21:58:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.530+0000 7f7cbc99a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb8100f30 0x7f7cb8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.530+0000 7f7cbd99c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cb81018d0 0x7f7cb81a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.530+0000 7f7cbc99a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb8100f30 0x7f7cb8199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:33636/0 (socket says 172.21.15.22:33636) 2023-09-20T21:58:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.530+0000 7f7cbc99a700 1 -- 172.21.15.22:0/125473386 learned_addr learned my addr 172.21.15.22:0/125473386 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:58:53.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.531+0000 7f7cbd99c700 1 -- 172.21.15.22:0/125473386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb8100f30 msgr2=0x7f7cb8199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:53.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.531+0000 7f7cbd99c700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb8100f30 0x7f7cb8199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.531+0000 7f7cbd99c700 1 -- 172.21.15.22:0/125473386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cb80ffd40 msgr2=0x7f7cb8198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:53.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.531+0000 7f7cbd99c700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cb80ffd40 0x7f7cb8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.531+0000 7f7cbd99c700 1 -- 172.21.15.22:0/125473386 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00095f0 con 0x7f7cb81018d0 2023-09-20T21:58:53.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.531+0000 7f7cbd99c700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cb81018d0 0x7f7cb81a0290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000b2e0 tx=0x7f7cb0005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:53.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.531+0000 7f7cae7fc700 1 -- 172.21.15.22:0/125473386 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb001d070 con 0x7f7cb81018d0 2023-09-20T21:58:53.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.531+0000 7f7cae7fc700 1 -- 172.21.15.22:0/125473386 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cb000fbc0 con 0x7f7cb81018d0 2023-09-20T21:58:53.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.531+0000 7f7cae7fc700 1 -- 172.21.15.22:0/125473386 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0021ea0 con 0x7f7cb81018d0 2023-09-20T21:58:53.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.532+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/125473386 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb806ac70 con 0x7f7cb81018d0 2023-09-20T21:58:53.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.532+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/125473386 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb806b150 con 0x7f7cb81018d0 2023-09-20T21:58:53.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.537+0000 7f7cd2d5ab80 1 Processor -- start 2023-09-20T21:58:53.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.539+0000 7f7cd2d5ab80 1 -- start start 2023-09-20T21:58:53.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.539+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f2ffd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.539+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffd860 0x5616f2ffde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.539+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f2ffe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.539+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f2f68ca0 con 0x5616f2ffcf00 2023-09-20T21:58:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.539+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f2f4cc70 con 0x5616f2ffe400 2023-09-20T21:58:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.539+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f2f4a590 con 0x5616f2ffd860 2023-09-20T21:58:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.540+0000 7f7cae7fc700 1 -- 172.21.15.22:0/125473386 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f7cb0021410 con 0x7f7cb81018d0 2023-09-20T21:58:53.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.543+0000 7f7cbc99a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f2ffe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.543+0000 7f7cbc99a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f2ffd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.543+0000 7f7cbc99a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f2ffd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44708/0 (socket says 172.21.15.22:44708) 2023-09-20T21:58:53.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.543+0000 7f7cbc99a700 1 -- 172.21.15.22:0/2414157266 learned_addr learned my addr 172.21.15.22:0/2414157266 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:58:53.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.543+0000 7f7cae7fc700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7ca40a14d0 0x7f7ca40a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.543+0000 7f7cae7fc700 1 -- 172.21.15.22:0/125473386 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f7cb00cdff0 con 0x7f7cb81018d0 2023-09-20T21:58:53.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.555+0000 7f7cbd99c700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7ca40a14d0 0x7f7ca40a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.557+0000 7f7cbd19b700 1 --2- 172.21.15.22:0/2414157266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffd860 0x5616f2ffde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.559+0000 7f7cbd99c700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7ca40a14d0 0x7f7ca40a3990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0006010 tx=0x7f7cb00a1040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:53.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.564+0000 7f7cbc99a700 1 -- 172.21.15.22:0/2414157266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffd860 msgr2=0x5616f2ffde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:53.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.564+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/2414157266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffd860 0x5616f2ffde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.564+0000 7f7cbc99a700 1 -- 172.21.15.22:0/2414157266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 msgr2=0x5616f2ffe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:53.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.564+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/2414157266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f2ffe880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.564+0000 7f7cbc99a700 1 -- 172.21.15.22:0/2414157266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616f2d70de0 con 0x5616f2ffcf00 2023-09-20T21:58:53.573 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 570.0 s, 568.5 tps, lat 180.260 ms stddev 194.904 2023-09-20T21:58:53.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.577+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/2414157266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f2ffd320 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8015480 tx=0x7f7ca8009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc6ec951293b185 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:53.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.582+0000 7f7ca37fe700 1 -- 172.21.15.22:0/2414157266 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8017670 con 0x5616f2ffcf00 2023-09-20T21:58:53.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.582+0000 7f7ca37fe700 1 -- 172.21.15.22:0/2414157266 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca8015e70 con 0x5616f2ffcf00 2023-09-20T21:58:53.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.582+0000 7f7ca37fe700 1 -- 172.21.15.22:0/2414157266 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8016990 con 0x5616f2ffcf00 2023-09-20T21:58:53.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.590+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/2414157266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 msgr2=0x5616f2ffd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:53.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.590+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/2414157266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f2ffd320 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8015480 tx=0x7f7ca8009210 comp rx=0 tx=0).stop 2023-09-20T21:58:53.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.591+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/2414157266 shutdown_connections 2023-09-20T21:58:53.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.591+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/2414157266 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f2ffe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.591+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/2414157266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f2ffd320 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.591+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/2414157266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffd860 0x5616f2ffde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.591+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/2414157266 >> 172.21.15.22:0/2414157266 conn(0x5616f2f21000 msgr2=0x5616f2fef1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:53.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.597+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/2414157266 shutdown_connections 2023-09-20T21:58:53.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.602+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/2414157266 wait complete. 2023-09-20T21:58:53.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.602+0000 7f7cd2d5ab80 1 -- start start 2023-09-20T21:58:53.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.603+0000 7f7cd2d5ab80 1 Processor -- start 2023-09-20T21:58:53.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.604+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffcf00 0x5616f2ff5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.604+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffd860 0x5616f2ff6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.604+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f2ffb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.604+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f2f45e80 con 0x5616f2ffd860 2023-09-20T21:58:53.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.604+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f2ee2740 con 0x5616f2ffe400 2023-09-20T21:58:53.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.604+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f2ee23e0 con 0x5616f2ffcf00 2023-09-20T21:58:53.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.604+0000 7f7cd2d5ab80 10 client.0 _ll_register_callbacks cb 0x5616f2e2ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:58:53.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.606+0000 7f7cbd19b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffd860 0x5616f2ff6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.606+0000 7f7cbd19b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffd860 0x5616f2ff6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44724/0 (socket says 172.21.15.22:44724) 2023-09-20T21:58:53.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.606+0000 7f7cbd19b700 1 -- 172.21.15.22:0/3207807983 learned_addr learned my addr 172.21.15.22:0/3207807983 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:58:53.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.607+0000 7f7cbd19b700 1 -- 172.21.15.22:0/3207807983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffcf00 msgr2=0x5616f2ff5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:53.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.607+0000 7f7cbd19b700 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffcf00 0x5616f2ff5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.607+0000 7f7cbd19b700 1 -- 172.21.15.22:0/3207807983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 msgr2=0x5616f2ffb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:53.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.607+0000 7f7cbd19b700 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f2ffb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.607+0000 7f7cbd19b700 1 -- 172.21.15.22:0/3207807983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca8009d40 con 0x5616f2ffd860 2023-09-20T21:58:53.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.608+0000 7f7cbd19b700 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffd860 0x5616f2ff6200 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011d10 tx=0x7f7cb400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aeaf282a203908b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:53.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.608+0000 7f7ca0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T21:58:53.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.610+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/3207807983 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400e040 con 0x5616f2ffd860 2023-09-20T21:58:53.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.610+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/3207807983 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f7cb400f9c0 con 0x5616f2ffd860 2023-09-20T21:58:53.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.610+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/3207807983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616f2eb6bb0 con 0x5616f2ffd860 2023-09-20T21:58:53.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.613+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/3207807983 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cb400fb80 con 0x5616f2ffd860 2023-09-20T21:58:53.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.620+0000 7f7cd2d5ab80 10 client.15876 fetch_fsmap learned FSMap version 32 2023-09-20T21:58:53.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.620+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/3207807983 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5616f2d59c60 con 0x5616f2ffd860 2023-09-20T21:58:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.633+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/3207807983 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5616f2eb6bb0 con 0x5616f2ffd860 2023-09-20T21:58:53.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.633+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/3207807983 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7cb401e440 con 0x5616f2ffd860 2023-09-20T21:58:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.644+0000 7f7cd2d5ab80 10 client.15876 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:58:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.644+0000 7f7cd2d5ab80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:58:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.644+0000 7f7cd2d5ab80 20 client.15876 populate_metadata read hostname 'smithi022' 2023-09-20T21:58:53.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.644+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5616f301b020 0x5616f301d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.644+0000 7f7cd2d5ab80 4 client.15876 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:58:53.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.644+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/3207807983 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616f2fc3020 con 0x5616f301b020 2023-09-20T21:58:53.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.653+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5616f301b020 0x5616f301d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.667+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5616f301b020 0x5616f301d440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5232d586c5f728f0 server_cookie=2a5f65c990aa3782 in_seq=0 out_seq=0 2023-09-20T21:58:53.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7ca0ff9700 10 client.15876 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:58:53.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/3207807983 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5616f2fc3020 con 0x5616f301b020 2023-09-20T21:58:53.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7ca0ff9700 10 client.15876 handle_command_reply: tid=0 2023-09-20T21:58:53.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7cd2d5ab80 1 client.15876 shutdown 2023-09-20T21:58:53.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/3207807983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffd860 msgr2=0x5616f2ff6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:53.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffd860 0x5616f2ff6200 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011d10 tx=0x7f7cb400f490 comp rx=0 tx=0).stop 2023-09-20T21:58:53.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/3207807983 shutdown_connections 2023-09-20T21:58:53.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5616f301b020 0x5616f301d440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f2ffb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffcf00 0x5616f2ff5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/3207807983 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffd860 0x5616f2ff6200 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011d10 tx=0x7f7cb400f490 comp rx=0 tx=0).stop 2023-09-20T21:58:53.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.668+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/3207807983 >> 172.21.15.22:0/3207807983 conn(0x5616f2f21000 msgr2=0x5616f2fee880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:53.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.669+0000 7f7ca0ff9700 0 client.15876 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:58:53.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.675+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/3207807983 shutdown_connections 2023-09-20T21:58:53.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.675+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/3207807983 wait complete. 2023-09-20T21:58:53.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.681+0000 7f7cd2d5ab80 20 client.15876 trim_cache size 0 max 16384 2023-09-20T21:58:53.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.727+0000 7f7cd2d5ab80 1 Processor -- start 2023-09-20T21:58:53.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.727+0000 7f7cd2d5ab80 1 -- start start 2023-09-20T21:58:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.727+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f302ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.727+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffe400 0x5616f2ff48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.727+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f3021da0 0x5616f301d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.727+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f2dc6920 con 0x5616f2ffcf00 2023-09-20T21:58:53.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.727+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f2ff5170 con 0x5616f3021da0 2023-09-20T21:58:53.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.727+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f301adc0 con 0x5616f2ffe400 2023-09-20T21:58:53.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.730+0000 7f7cbc99a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f3021da0 0x5616f301d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.730+0000 7f7cbc99a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f3021da0 0x5616f301d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:39226/0 (socket says 172.21.15.22:39226) 2023-09-20T21:58:53.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.730+0000 7f7cbc99a700 1 -- 172.21.15.22:0/2626063534 learned_addr learned my addr 172.21.15.22:0/2626063534 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:58:53.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.731+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/2626063534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f302ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.731+0000 7f7cbd19b700 1 --2- 172.21.15.22:0/2626063534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffe400 0x5616f2ff48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.731+0000 7f7cbc99a700 1 -- 172.21.15.22:0/2626063534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffe400 msgr2=0x5616f2ff48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:53.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.731+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/2626063534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffe400 0x5616f2ff48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.731+0000 7f7cbc99a700 1 -- 172.21.15.22:0/2626063534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 msgr2=0x5616f302ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:53.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.731+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/2626063534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f302ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.731+0000 7f7cbc99a700 1 -- 172.21.15.22:0/2626063534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616f2d97250 con 0x5616f3021da0 2023-09-20T21:58:53.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.731+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/2626063534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f3021da0 0x5616f301d030 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8018fe0 tx=0x7f7ca8009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c3029ea2b63029 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:53.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.732+0000 7f7ca3fff700 1 -- 172.21.15.22:0/2626063534 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8025070 con 0x5616f3021da0 2023-09-20T21:58:53.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.732+0000 7f7ca3fff700 1 -- 172.21.15.22:0/2626063534 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca8028810 con 0x5616f3021da0 2023-09-20T21:58:53.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.732+0000 7f7ca3fff700 1 -- 172.21.15.22:0/2626063534 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8016870 con 0x5616f3021da0 2023-09-20T21:58:53.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.733+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/2626063534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f3021da0 msgr2=0x5616f301d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:53.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.733+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/2626063534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f3021da0 0x5616f301d030 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8018fe0 tx=0x7f7ca8009280 comp rx=0 tx=0).stop 2023-09-20T21:58:53.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.733+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/2626063534 shutdown_connections 2023-09-20T21:58:53.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.733+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/2626063534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f3021da0 0x5616f301d030 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.733+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/2626063534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f302ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.733+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/2626063534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f2ffe400 0x5616f2ff48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.733+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/2626063534 >> 172.21.15.22:0/2626063534 conn(0x5616f2f21000 msgr2=0x5616f30048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:53.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.733+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/2626063534 shutdown_connections 2023-09-20T21:58:53.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.733+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/2626063534 wait complete. 2023-09-20T21:58:53.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.734+0000 7f7cd2d5ab80 1 -- start start 2023-09-20T21:58:53.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.734+0000 7f7cd2d5ab80 1 Processor -- start 2023-09-20T21:58:53.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.735+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f3069550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.735+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f306baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.735+0000 7f7cd2d5ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f3021da0 0x5616f306dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.735+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f306e700 con 0x5616f2ffcf00 2023-09-20T21:58:53.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.735+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f306e870 con 0x5616f2ffe400 2023-09-20T21:58:53.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.735+0000 7f7cd2d5ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f306ea30 con 0x5616f3021da0 2023-09-20T21:58:53.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.735+0000 7f7cd2d5ab80 10 client.0 _ll_register_callbacks cb 0x5616f2e2ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:58:53.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbd19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f306baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbd19b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f306baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:39242/0 (socket says 172.21.15.22:39242) 2023-09-20T21:58:53.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbd19b700 1 -- 172.21.15.22:0/229571977 learned_addr learned my addr 172.21.15.22:0/229571977 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:58:53.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f3069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f3021da0 0x5616f306dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbd19b700 1 -- 172.21.15.22:0/229571977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f3021da0 msgr2=0x5616f306dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:53.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbd19b700 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f3021da0 0x5616f306dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbd19b700 1 -- 172.21.15.22:0/229571977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 msgr2=0x5616f3069550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:58:53.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbd19b700 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f3069550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbd19b700 1 -- 172.21.15.22:0/229571977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616f2d70de0 con 0x5616f2ffe400 2023-09-20T21:58:53.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.736+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f3069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:58:53.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.737+0000 7f7cbd19b700 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f306baa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4002230 tx=0x7f7cb40095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48c231fe67f543b server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:58:53.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.744+0000 7f7ca0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:58:53.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.754+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/229571977 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400e040 con 0x5616f2ffe400 2023-09-20T21:58:53.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.754+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/229571977 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f7cb4004bb0 con 0x5616f2ffe400 2023-09-20T21:58:53.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.759+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/229571977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ca8015e70 con 0x5616f2ffe400 2023-09-20T21:58:53.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.780+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/229571977 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cb401ebc0 con 0x5616f2ffe400 2023-09-20T21:58:53.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.780+0000 7f7cd2d5ab80 10 client.26024 fetch_fsmap learned FSMap version 32 2023-09-20T21:58:53.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.780+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/229571977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5616f2d6f2b0 con 0x5616f2ffe400 2023-09-20T21:58:53.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.782+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/229571977 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ca8015e70 con 0x5616f2ffe400 2023-09-20T21:58:53.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.782+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/229571977 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7cb4030030 con 0x5616f2ffe400 2023-09-20T21:58:53.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.782+0000 7f7cd2d5ab80 10 client.26024 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:58:53.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.782+0000 7f7cd2d5ab80 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:58:53.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.782+0000 7f7cd2d5ab80 20 client.26024 populate_metadata read hostname 'smithi022' 2023-09-20T21:58:53.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.782+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5616f3070420 0x5616f3072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:58:53.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.782+0000 7f7cd2d5ab80 4 client.26024 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:58:53.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.782+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/229571977 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5616f2f3ee60 con 0x5616f3070420 2023-09-20T21:58:53.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.787+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5616f3070420 0x5616f3072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:58:53.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.792+0000 7f7cbc99a700 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5616f3070420 0x5616f3072840 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47635c2bfae4788b server_cookie=1707309955f2f225 in_seq=0 out_seq=0 2023-09-20T21:58:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.792+0000 7f7ca0ff9700 10 client.26024 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:58:53.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.807+0000 7f7ca0ff9700 1 -- 172.21.15.22:0/229571977 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5616f2f3ee60 con 0x5616f3070420 2023-09-20T21:58:53.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.807+0000 7f7ca0ff9700 10 client.26024 handle_command_reply: tid=0 2023-09-20T21:58:53.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.811+0000 7f7cd2d5ab80 1 client.26024 shutdown 2023-09-20T21:58:53.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.820+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/229571977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 msgr2=0x5616f306baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:53.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.820+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f306baa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4002230 tx=0x7f7cb40095e0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.825+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/229571977 shutdown_connections 2023-09-20T21:58:53.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.825+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5616f3070420 0x5616f3072840 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.825+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5616f3021da0 0x5616f306dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.825+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5616f2ffcf00 0x5616f3069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.825+0000 7f7cd2d5ab80 1 --2- 172.21.15.22:0/229571977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5616f2ffe400 0x5616f306baa0 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4002230 tx=0x7f7cb40095e0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.825+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/229571977 >> 172.21.15.22:0/229571977 conn(0x5616f2f21000 msgr2=0x5616f2ff48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.825+0000 7f7ca0ff9700 0 client.26024 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:58:53.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.826+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/229571977 shutdown_connections 2023-09-20T21:58:53.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.826+0000 7f7cd2d5ab80 1 -- 172.21.15.22:0/229571977 wait complete. 2023-09-20T21:58:53.831 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:58:53.832 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:58:53.832 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:58:53.832 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:58:53.832 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:58:53.832 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:58:53.833 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:58:53.833 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:58:53.833 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:58:53.833 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:58:53.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.831+0000 7f7cd2d5ab80 20 client.26024 trim_cache size 0 max 16384 2023-09-20T21:58:53.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.841+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/125473386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7ca40a14d0 msgr2=0x7f7ca40a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:53.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.841+0000 7f7cbf3ff700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7ca40a14d0 0x7f7ca40a3990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0006010 tx=0x7f7cb00a1040 comp rx=0 tx=0).stop 2023-09-20T21:58:53.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.841+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/125473386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cb81018d0 msgr2=0x7f7cb81a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:58:53.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.841+0000 7f7cbf3ff700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cb81018d0 0x7f7cb81a0290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000b2e0 tx=0x7f7cb0005f00 comp rx=0 tx=0).stop 2023-09-20T21:58:53.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.842+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/125473386 shutdown_connections 2023-09-20T21:58:53.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.842+0000 7f7cbf3ff700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f7ca40a14d0 0x7f7ca40a3990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.842+0000 7f7cbf3ff700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7cb81018d0 0x7f7cb81a0290 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.842+0000 7f7cbf3ff700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cb80ffd40 0x7f7cb8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.842+0000 7f7cbf3ff700 1 --2- 172.21.15.22:0/125473386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cb8100f30 0x7f7cb8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:58:53.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.842+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/125473386 >> 172.21.15.22:0/125473386 conn(0x7f7cb80fb2b0 msgr2=0x7f7cb80fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:58:53.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.849+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/125473386 shutdown_connections 2023-09-20T21:58:53.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:58:53.853+0000 7f7cbf3ff700 1 -- 172.21.15.22:0/125473386 wait complete. 2023-09-20T21:58:53.888 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:58:54.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:54 smithi123 ceph-mon[134433]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.6 MiB/s wr, 457 op/s 2023-09-20T21:58:54.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:54 smithi089 ceph-mon[134090]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.6 MiB/s wr, 457 op/s 2023-09-20T21:58:54.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:54 smithi022 ceph-mon[97651]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.6 MiB/s wr, 457 op/s 2023-09-20T21:58:57.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:56 smithi022 ceph-mon[97651]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 493 op/s 2023-09-20T21:58:57.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:56 smithi123 ceph-mon[134433]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 493 op/s 2023-09-20T21:58:57.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:56 smithi089 ceph-mon[134090]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.0 MiB/s wr, 493 op/s 2023-09-20T21:58:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 575.0 s, 517.2 tps, lat 195.658 ms stddev 203.717 2023-09-20T21:58:59.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:58:59 smithi123 ceph-mon[134433]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.6 MiB/s wr, 535 op/s 2023-09-20T21:58:59.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:58:59 smithi089 ceph-mon[134090]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.6 MiB/s wr, 535 op/s 2023-09-20T21:58:59.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:58:59 smithi022 ceph-mon[97651]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.6 MiB/s wr, 535 op/s 2023-09-20T21:59:00.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:00 smithi123 ceph-mon[134433]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2023-09-20T21:59:00.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:00 smithi089 ceph-mon[134090]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2023-09-20T21:59:00.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:00 smithi022 ceph-mon[97651]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.7 MiB/s wr, 536 op/s 2023-09-20T21:59:03.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:02 smithi022 ceph-mon[97651]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.3 MiB/s wr, 511 op/s 2023-09-20T21:59:03.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:02 smithi123 ceph-mon[134433]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.3 MiB/s wr, 511 op/s 2023-09-20T21:59:03.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:02 smithi089 ceph-mon[134090]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.3 MiB/s wr, 511 op/s 2023-09-20T21:59:03.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 580.0 s, 520.2 tps, lat 194.597 ms stddev 212.648 2023-09-20T21:59:04.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:04 smithi123 ceph-mon[134433]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.2 MiB/s wr, 483 op/s 2023-09-20T21:59:04.619 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:04 smithi089 ceph-mon[134090]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.2 MiB/s wr, 483 op/s 2023-09-20T21:59:04.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:04 smithi022 ceph-mon[97651]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.2 MiB/s wr, 483 op/s 2023-09-20T21:59:07.256 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:06 smithi022 ceph-mon[97651]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.0 MiB/s wr, 499 op/s 2023-09-20T21:59:07.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:06 smithi123 ceph-mon[134433]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.0 MiB/s wr, 499 op/s 2023-09-20T21:59:07.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:06 smithi089 ceph-mon[134090]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.0 MiB/s wr, 499 op/s 2023-09-20T21:59:08.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 585.0 s, 540.3 tps, lat 180.791 ms stddev 177.784 2023-09-20T21:59:09.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:09 smithi022 ceph-mon[97651]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 MiB/s wr, 546 op/s 2023-09-20T21:59:09.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:09 smithi123 ceph-mon[134433]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 MiB/s wr, 546 op/s 2023-09-20T21:59:09.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:09 smithi089 ceph-mon[134090]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 MiB/s wr, 546 op/s 2023-09-20T21:59:11.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:11 smithi022 ceph-mon[97651]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.4 MiB/s wr, 534 op/s 2023-09-20T21:59:11.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:11 smithi123 ceph-mon[134433]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.4 MiB/s wr, 534 op/s 2023-09-20T21:59:11.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:11 smithi089 ceph-mon[134090]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.4 MiB/s wr, 534 op/s 2023-09-20T21:59:12.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:12 smithi123 ceph-mon[134433]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.0 MiB/s wr, 503 op/s 2023-09-20T21:59:12.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:12 smithi089 ceph-mon[134090]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.0 MiB/s wr, 503 op/s 2023-09-20T21:59:12.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:12 smithi022 ceph-mon[97651]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.0 MiB/s wr, 503 op/s 2023-09-20T21:59:13.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 590.0 s, 472.1 tps, lat 211.898 ms stddev 243.520 2023-09-20T21:59:14.602 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:14 smithi123 ceph-mon[134433]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 MiB/s wr, 497 op/s 2023-09-20T21:59:14.616 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:14 smithi089 ceph-mon[134090]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 MiB/s wr, 497 op/s 2023-09-20T21:59:14.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:14 smithi022 ceph-mon[97651]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 MiB/s wr, 497 op/s 2023-09-20T21:59:17.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:16 smithi022 ceph-mon[97651]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 494 op/s 2023-09-20T21:59:17.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:16 smithi123 ceph-mon[134433]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 494 op/s 2023-09-20T21:59:17.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:16 smithi089 ceph-mon[134090]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.0 MiB/s wr, 494 op/s 2023-09-20T21:59:18.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 595.0 s, 467.9 tps, lat 210.353 ms stddev 265.762 2023-09-20T21:59:19.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:19 smithi123 ceph-mon[134433]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 530 op/s 2023-09-20T21:59:19.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:19 smithi089 ceph-mon[134090]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 530 op/s 2023-09-20T21:59:19.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:19 smithi022 ceph-mon[97651]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 530 op/s 2023-09-20T21:59:21.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:21 smithi123 ceph-mon[134433]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 506 op/s 2023-09-20T21:59:21.618 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:21 smithi089 ceph-mon[134090]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 506 op/s 2023-09-20T21:59:21.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:21 smithi022 ceph-mon[97651]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.0 MiB/s wr, 506 op/s 2023-09-20T21:59:22.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:22 smithi123 ceph-mon[134433]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.7 MiB/s wr, 464 op/s 2023-09-20T21:59:22.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:22 smithi089 ceph-mon[134090]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.7 MiB/s wr, 464 op/s 2023-09-20T21:59:22.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:22 smithi022 ceph-mon[97651]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.7 MiB/s wr, 464 op/s 2023-09-20T21:59:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 600.0 s, 521.4 tps, lat 192.509 ms stddev 242.412 2023-09-20T21:59:23.889 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:59:24.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.358+0000 7f8b431ff700 1 -- 172.21.15.22:0/1047268871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c104180 msgr2=0x7f8b3c106570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:24.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.358+0000 7f8b431ff700 1 --2- 172.21.15.22:0/1047268871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c104180 0x7f8b3c106570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009990 tx=0x7f8b28009ca0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.359+0000 7f8b431ff700 1 -- 172.21.15.22:0/1047268871 shutdown_connections 2023-09-20T21:59:24.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.359+0000 7f8b431ff700 1 --2- 172.21.15.22:0/1047268871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b3c106ab0 0x7f8b3c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.359+0000 7f8b431ff700 1 --2- 172.21.15.22:0/1047268871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b3c101850 0x7f8b3c103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.359+0000 7f8b431ff700 1 --2- 172.21.15.22:0/1047268871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c104180 0x7f8b3c106570 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.359+0000 7f8b431ff700 1 -- 172.21.15.22:0/1047268871 >> 172.21.15.22:0/1047268871 conn(0x7f8b3c0fb240 msgr2=0x7f8b3c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:24.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.359+0000 7f8b431ff700 1 -- 172.21.15.22:0/1047268871 shutdown_connections 2023-09-20T21:59:24.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.361+0000 7f8b431ff700 1 -- 172.21.15.22:0/1047268871 wait complete. 2023-09-20T21:59:24.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.362+0000 7f8b431ff700 1 Processor -- start 2023-09-20T21:59:24.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.363+0000 7f8b431ff700 1 -- start start 2023-09-20T21:59:24.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.363+0000 7f8b431ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b3c101850 0x7f8b3c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.363+0000 7f8b431ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b3c104180 0x7f8b3c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.363+0000 7f8b431ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c106ab0 0x7f8b3c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.363+0000 7f8b431ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c1a0830 con 0x7f8b3c101850 2023-09-20T21:59:24.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.363+0000 7f8b431ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c1a09a0 con 0x7f8b3c104180 2023-09-20T21:59:24.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.363+0000 7f8b431ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c1a0b60 con 0x7f8b3c106ab0 2023-09-20T21:59:24.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.363+0000 7f8b4179c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c106ab0 0x7f8b3c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.363+0000 7f8b4179c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c106ab0 0x7f8b3c1a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:38334/0 (socket says 172.21.15.22:38334) 2023-09-20T21:59:24.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.363+0000 7f8b4179c700 1 -- 172.21.15.22:0/2281762716 learned_addr learned my addr 172.21.15.22:0/2281762716 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:59:24.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.364+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b3c101850 0x7f8b3c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.364+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b3c104180 0x7f8b3c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.364+0000 7f8b4179c700 1 -- 172.21.15.22:0/2281762716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b3c104180 msgr2=0x7f8b3c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:24.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.364+0000 7f8b4179c700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b3c104180 0x7f8b3c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.364+0000 7f8b4179c700 1 -- 172.21.15.22:0/2281762716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b3c101850 msgr2=0x7f8b3c198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:24.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.364+0000 7f8b4179c700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b3c101850 0x7f8b3c198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.364+0000 7f8b4179c700 1 -- 172.21.15.22:0/2281762716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280095f0 con 0x7f8b3c106ab0 2023-09-20T21:59:24.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.364+0000 7f8b4179c700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c106ab0 0x7f8b3c1a0260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400ef30 tx=0x7f8b3400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:24.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.365+0000 7f8b39ffb700 1 -- 172.21.15.22:0/2281762716 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34018070 con 0x7f8b3c106ab0 2023-09-20T21:59:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.365+0000 7f8b39ffb700 1 -- 172.21.15.22:0/2281762716 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3400f040 con 0x7f8b3c106ab0 2023-09-20T21:59:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.365+0000 7f8b431ff700 1 -- 172.21.15.22:0/2281762716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3c1a0de0 con 0x7f8b3c106ab0 2023-09-20T21:59:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.365+0000 7f8b39ffb700 1 -- 172.21.15.22:0/2281762716 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34003dc0 con 0x7f8b3c106ab0 2023-09-20T21:59:24.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.365+0000 7f8b431ff700 1 -- 172.21.15.22:0/2281762716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c1a1200 con 0x7f8b3c106ab0 2023-09-20T21:59:24.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.370+0000 7f8b39ffb700 1 -- 172.21.15.22:0/2281762716 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f8b3400caa0 con 0x7f8b3c106ab0 2023-09-20T21:59:24.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.372+0000 7f8b56b1ab80 1 Processor -- start 2023-09-20T21:59:24.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.372+0000 7f8b56b1ab80 1 -- start start 2023-09-20T21:59:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.372+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.372+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd37b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.372+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd37c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.372+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd2eeca0 con 0x5608fd37b050 2023-09-20T21:59:24.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.372+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd2d2c70 con 0x5608fd37bbe0 2023-09-20T21:59:24.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.372+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd2d0590 con 0x5608fd37a6f0 2023-09-20T21:59:24.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.373+0000 7f8b3bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.373+0000 7f8b3bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:38338/0 (socket says 172.21.15.22:38338) 2023-09-20T21:59:24.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.373+0000 7f8b3bfff700 1 -- 172.21.15.22:0/1014610755 learned_addr learned my addr 172.21.15.22:0/1014610755 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:59:24.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.373+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/1014610755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd37c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.373+0000 7f8b3bfff700 1 -- 172.21.15.22:0/1014610755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 msgr2=0x5608fd37c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.373+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/1014610755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd37c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.373+0000 7f8b3bfff700 1 -- 172.21.15.22:0/1014610755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 msgr2=0x5608fd37b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.373+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/1014610755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd37b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.373+0000 7f8b3bfff700 1 -- 172.21.15.22:0/1014610755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608fd0f6de0 con 0x5608fd37a6f0 2023-09-20T21:59:24.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.373+0000 7f8b39ffb700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8b2c0a9b90 0x7f8b2c0ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.374+0000 7f8b39ffb700 1 -- 172.21.15.22:0/2281762716 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f8b340ccba0 con 0x7f8b3c106ab0 2023-09-20T21:59:24.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.374+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/1014610755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37ab10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800b2e0 tx=0x7f8b2801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bce7d4ed3d15caa server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:24.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.374+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8b2c0a9b90 0x7f8b2c0ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.374+0000 7f8b26ffd700 1 -- 172.21.15.22:0/1014610755 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800bea0 con 0x5608fd37a6f0 2023-09-20T21:59:24.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.374+0000 7f8b26ffd700 1 -- 172.21.15.22:0/1014610755 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b2801e070 con 0x5608fd37a6f0 2023-09-20T21:59:24.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.374+0000 7f8b26ffd700 1 -- 172.21.15.22:0/1014610755 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28004e00 con 0x5608fd37a6f0 2023-09-20T21:59:24.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.375+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/1014610755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 msgr2=0x5608fd37ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:24.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.375+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/1014610755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37ab10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800b2e0 tx=0x7f8b2801a040 comp rx=0 tx=0).stop 2023-09-20T21:59:24.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.375+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/1014610755 shutdown_connections 2023-09-20T21:59:24.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.375+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/1014610755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd37c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.375+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/1014610755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37ab10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.375+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/1014610755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd37b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.375+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/1014610755 >> 172.21.15.22:0/1014610755 conn(0x5608fd2a7000 msgr2=0x5608fd3751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:24.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.386+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8b2c0a9b90 0x7f8b2c0ac050 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8b30005d90 tx=0x7f8b30005ce0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:24.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.387+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/1014610755 shutdown_connections 2023-09-20T21:59:24.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.397+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/1014610755 wait complete. 2023-09-20T21:59:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.398+0000 7f8b56b1ab80 1 -- start start 2023-09-20T21:59:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.398+0000 7f8b56b1ab80 1 Processor -- start 2023-09-20T21:59:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.400+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd3841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.400+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37b050 0x5608fd384730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.400+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37bbe0 0x5608fd38b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.400+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd2cbe80 con 0x5608fd37bbe0 2023-09-20T21:59:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.400+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd268740 con 0x5608fd37b050 2023-09-20T21:59:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.400+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd2683e0 con 0x5608fd37a6f0 2023-09-20T21:59:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.400+0000 7f8b56b1ab80 10 client.0 _ll_register_callbacks cb 0x5608fd1b2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:59:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.402+0000 7f8b3bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37b050 0x5608fd384730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.402+0000 7f8b4179c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37bbe0 0x5608fd38b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.402+0000 7f8b4179c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37bbe0 0x5608fd38b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50234/0 (socket says 172.21.15.22:50234) 2023-09-20T21:59:24.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.402+0000 7f8b4179c700 1 -- 172.21.15.22:0/4033549181 learned_addr learned my addr 172.21.15.22:0/4033549181 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:59:24.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.410+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd3841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.419+0000 7f8b40f9b700 1 -- 172.21.15.22:0/4033549181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37b050 msgr2=0x5608fd384730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:24.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.419+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37b050 0x5608fd384730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.419+0000 7f8b40f9b700 1 -- 172.21.15.22:0/4033549181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37bbe0 msgr2=0x5608fd38b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:24.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.419+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37bbe0 0x5608fd38b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.419+0000 7f8b40f9b700 1 -- 172.21.15.22:0/4033549181 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c109b10 con 0x5608fd37a6f0 2023-09-20T21:59:24.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.429+0000 7f8b4179c700 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37bbe0 0x5608fd38b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T21:59:24.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.429+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37b050 0x5608fd384730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:59:24.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.443+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd3841f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x5608fd376150 tx=0x7f8b30005f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db480e6b42540f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:24.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.443+0000 7f8b1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:59:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.454+0000 7f8b1bfff700 1 -- 172.21.15.22:0/4033549181 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30011db0 con 0x5608fd37a6f0 2023-09-20T21:59:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.454+0000 7f8b1bfff700 1 -- 172.21.15.22:0/4033549181 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f8b3001e450 con 0x5608fd37a6f0 2023-09-20T21:59:24.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.454+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/4033549181 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608fd23cbb0 con 0x5608fd37a6f0 2023-09-20T21:59:24.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.460+0000 7f8b1bfff700 1 -- 172.21.15.22:0/4033549181 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b3001e610 con 0x5608fd37a6f0 2023-09-20T21:59:24.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.461+0000 7f8b56b1ab80 10 client.26056 fetch_fsmap learned FSMap version 32 2023-09-20T21:59:24.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.461+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/4033549181 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5608fd0dfc60 con 0x5608fd37a6f0 2023-09-20T21:59:24.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.476+0000 7f8b1bfff700 1 -- 172.21.15.22:0/4033549181 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5608fd23cbb0 con 0x5608fd37a6f0 2023-09-20T21:59:24.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.476+0000 7f8b1bfff700 1 -- 172.21.15.22:0/4033549181 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8b30027900 con 0x5608fd37a6f0 2023-09-20T21:59:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.479+0000 7f8b56b1ab80 10 client.26056 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:59:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.479+0000 7f8b56b1ab80 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:59:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.479+0000 7f8b56b1ab80 20 client.26056 populate_metadata read hostname 'smithi022' 2023-09-20T21:59:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.479+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5608fd390010 0x5608fd392400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.479+0000 7f8b56b1ab80 4 client.26056 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:59:24.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.479+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/4033549181 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608fd349020 con 0x5608fd390010 2023-09-20T21:59:24.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.486+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5608fd390010 0x5608fd392400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.517+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5608fd390010 0x5608fd392400 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f30cd7edd7a95c server_cookie=693c1a5aacdda1d9 in_seq=0 out_seq=0 2023-09-20T21:59:24.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.517+0000 7f8b1bfff700 10 client.26056 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:59:24.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.519+0000 7f8b1bfff700 1 -- 172.21.15.22:0/4033549181 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5608fd349020 con 0x5608fd390010 2023-09-20T21:59:24.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.519+0000 7f8b1bfff700 10 client.26056 handle_command_reply: tid=0 2023-09-20T21:59:24.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.520+0000 7f8b56b1ab80 1 client.26056 shutdown 2023-09-20T21:59:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.530+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/4033549181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 msgr2=0x5608fd3841f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:24.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.530+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd3841f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x5608fd376150 tx=0x7f8b30005f60 comp rx=0 tx=0).stop 2023-09-20T21:59:24.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.536+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/4033549181 shutdown_connections 2023-09-20T21:59:24.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.536+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5608fd390010 0x5608fd392400 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.536+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37bbe0 0x5608fd38b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.536+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd3841f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.536+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/4033549181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37b050 0x5608fd384730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.536+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/4033549181 >> 172.21.15.22:0/4033549181 conn(0x5608fd2a7000 msgr2=0x5608fd374880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:24.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.536+0000 7f8b1bfff700 0 client.26056 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:59:24.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.536+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/4033549181 shutdown_connections 2023-09-20T21:59:24.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.536+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/4033549181 wait complete. 2023-09-20T21:59:24.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.538+0000 7f8b56b1ab80 20 client.26056 trim_cache size 0 max 16384 2023-09-20T21:59:24.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.570+0000 7f8b56b1ab80 1 Processor -- start 2023-09-20T21:59:24.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.570+0000 7f8b56b1ab80 1 -- start start 2023-09-20T21:59:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.570+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.570+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd38b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd38bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd14c920 con 0x5608fd37b050 2023-09-20T21:59:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd383510 con 0x5608fd37bbe0 2023-09-20T21:59:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd349020 con 0x5608fd37a6f0 2023-09-20T21:59:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b40f9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b40f9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:38354/0 (socket says 172.21.15.22:38354) 2023-09-20T21:59:24.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b40f9b700 1 -- 172.21.15.22:0/2501684369 learned_addr learned my addr 172.21.15.22:0/2501684369 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:59:24.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b4179c700 1 --2- 172.21.15.22:0/2501684369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd38bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/2501684369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd38b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b40f9b700 1 -- 172.21.15.22:0/2501684369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 msgr2=0x5608fd38bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/2501684369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd38bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b40f9b700 1 -- 172.21.15.22:0/2501684369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 msgr2=0x5608fd38b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/2501684369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd38b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b40f9b700 1 -- 172.21.15.22:0/2501684369 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608fd11d250 con 0x5608fd37a6f0 2023-09-20T21:59:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.571+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/2501684369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd38b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T21:59:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.572+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/2501684369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37bfc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b30006010 tx=0x7f8b3000b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55cac4ee8f64570a server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.572+0000 7f8b277fe700 1 -- 172.21.15.22:0/2501684369 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30011d30 con 0x5608fd37a6f0 2023-09-20T21:59:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.572+0000 7f8b277fe700 1 -- 172.21.15.22:0/2501684369 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b30028e90 con 0x5608fd37a6f0 2023-09-20T21:59:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.572+0000 7f8b277fe700 1 -- 172.21.15.22:0/2501684369 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30026b60 con 0x5608fd37a6f0 2023-09-20T21:59:24.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.573+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/2501684369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 msgr2=0x5608fd37bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.573+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/2501684369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37bfc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b30006010 tx=0x7f8b3000b9f0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.575+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/2501684369 shutdown_connections 2023-09-20T21:59:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.575+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/2501684369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd38bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.575+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/2501684369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd37bfc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.575+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/2501684369 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd38b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.575+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/2501684369 >> 172.21.15.22:0/2501684369 conn(0x5608fd2a7000 msgr2=0x5608fd382080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.575+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/2501684369 shutdown_connections 2023-09-20T21:59:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.575+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/2501684369 wait complete. 2023-09-20T21:59:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.575+0000 7f8b56b1ab80 1 -- start start 2023-09-20T21:59:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.576+0000 7f8b56b1ab80 1 Processor -- start 2023-09-20T21:59:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.577+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd389460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.577+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd3899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.577+0000 7f8b56b1ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd38a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.577+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd38aef0 con 0x5608fd37b050 2023-09-20T21:59:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.577+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd38b060 con 0x5608fd37bbe0 2023-09-20T21:59:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.577+0000 7f8b56b1ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fd38b220 con 0x5608fd37a6f0 2023-09-20T21:59:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.577+0000 7f8b56b1ab80 10 client.0 _ll_register_callbacks cb 0x5608fd1b2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:59:24.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b40f9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd389460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b40f9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd389460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:38358/0 (socket says 172.21.15.22:38358) 2023-09-20T21:59:24.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b40f9b700 1 -- 172.21.15.22:0/862660137 learned_addr learned my addr 172.21.15.22:0/862660137 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:59:24.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd3899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b40f9b700 1 -- 172.21.15.22:0/862660137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 msgr2=0x5608fd38a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:24.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd38a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b40f9b700 1 -- 172.21.15.22:0/862660137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 msgr2=0x5608fd3899a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:24.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd3899a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b40f9b700 1 -- 172.21.15.22:0/862660137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608fd0dfc60 con 0x5608fd37a6f0 2023-09-20T21:59:24.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd3899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T21:59:24.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b40f9b700 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd389460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b30010010 tx=0x7f8b300052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=308a9c1bf5f4292f server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:24.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.578+0000 7f8b1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:59:24.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b1bfff700 1 -- 172.21.15.22:0/862660137 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30027850 con 0x5608fd37a6f0 2023-09-20T21:59:24.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b1bfff700 1 -- 172.21.15.22:0/862660137 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f8b30005590 con 0x5608fd37a6f0 2023-09-20T21:59:24.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/862660137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b2801e070 con 0x5608fd37a6f0 2023-09-20T21:59:24.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b1bfff700 1 -- 172.21.15.22:0/862660137 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b300279b0 con 0x5608fd37a6f0 2023-09-20T21:59:24.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b56b1ab80 10 client.26062 fetch_fsmap learned FSMap version 32 2023-09-20T21:59:24.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/862660137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5608fd0f52b0 con 0x5608fd37a6f0 2023-09-20T21:59:24.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b1bfff700 1 -- 172.21.15.22:0/862660137 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b2801e070 con 0x5608fd37a6f0 2023-09-20T21:59:24.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b1bfff700 1 -- 172.21.15.22:0/862660137 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8b30004b10 con 0x5608fd37a6f0 2023-09-20T21:59:24.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b56b1ab80 10 client.26062 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:59:24.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b56b1ab80 10 client.26062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:59:24.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.579+0000 7f8b56b1ab80 20 client.26062 populate_metadata read hostname 'smithi022' 2023-09-20T21:59:24.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.580+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5608fd3e49a0 0x5608fd3e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:24.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.580+0000 7f8b56b1ab80 4 client.26062 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:59:24.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.580+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/862660137 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5608fd2c4e60 con 0x5608fd3e49a0 2023-09-20T21:59:24.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.582+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5608fd3e49a0 0x5608fd3e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:24.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.592+0000 7f8b3bfff700 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5608fd3e49a0 0x5608fd3e6dc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5f586187cc401f server_cookie=5da7eaa17f9bddd9 in_seq=0 out_seq=0 2023-09-20T21:59:24.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.598+0000 7f8b1bfff700 10 client.26062 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:59:24.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.606+0000 7f8b1bfff700 1 -- 172.21.15.22:0/862660137 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5608fd2c4e60 con 0x5608fd3e49a0 2023-09-20T21:59:24.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.606+0000 7f8b1bfff700 10 client.26062 handle_command_reply: tid=0 2023-09-20T21:59:24.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.608+0000 7f8b56b1ab80 1 client.26062 shutdown 2023-09-20T21:59:24.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.615+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/862660137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 msgr2=0x5608fd389460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.615+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd389460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b30010010 tx=0x7f8b300052f0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.621+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/862660137 shutdown_connections 2023-09-20T21:59:24.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.621+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x5608fd3e49a0 0x5608fd3e6dc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.621+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5608fd37bbe0 0x5608fd38a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.621+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608fd37a6f0 0x5608fd389460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.621+0000 7f8b56b1ab80 1 --2- 172.21.15.22:0/862660137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608fd37b050 0x5608fd3899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.621+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/862660137 >> 172.21.15.22:0/862660137 conn(0x5608fd2a7000 msgr2=0x5608fd38bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:24.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.621+0000 7f8b1bfff700 0 client.26062 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:59:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.633+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/862660137 shutdown_connections 2023-09-20T21:59:24.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.640+0000 7f8b56b1ab80 1 -- 172.21.15.22:0/862660137 wait complete. 2023-09-20T21:59:24.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.641+0000 7f8b56b1ab80 20 client.26062 trim_cache size 0 max 16384 2023-09-20T21:59:24.643 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:59:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:59:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:59:24.643 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:59:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:59:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:59:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:59:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:59:24.644 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:59:24.644 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:59:24.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.648+0000 7f8b431ff700 1 -- 172.21.15.22:0/2281762716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8b2c0a9b90 msgr2=0x7f8b2c0ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:24.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.648+0000 7f8b431ff700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8b2c0a9b90 0x7f8b2c0ac050 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8b30005d90 tx=0x7f8b30005ce0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.648+0000 7f8b431ff700 1 -- 172.21.15.22:0/2281762716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c106ab0 msgr2=0x7f8b3c1a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:24.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.648+0000 7f8b431ff700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c106ab0 0x7f8b3c1a0260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400ef30 tx=0x7f8b3400c450 comp rx=0 tx=0).stop 2023-09-20T21:59:24.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.649+0000 7f8b431ff700 1 -- 172.21.15.22:0/2281762716 shutdown_connections 2023-09-20T21:59:24.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.649+0000 7f8b431ff700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b3c106ab0 0x7f8b3c1a0260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.649+0000 7f8b431ff700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b3c101850 0x7f8b3c198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.649+0000 7f8b431ff700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8b2c0a9b90 0x7f8b2c0ac050 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.649+0000 7f8b431ff700 1 --2- 172.21.15.22:0/2281762716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b3c104180 0x7f8b3c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:24.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.649+0000 7f8b431ff700 1 -- 172.21.15.22:0/2281762716 >> 172.21.15.22:0/2281762716 conn(0x7f8b3c0fb240 msgr2=0x7f8b3c106390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:24.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.654+0000 7f8b431ff700 1 -- 172.21.15.22:0/2281762716 shutdown_connections 2023-09-20T21:59:24.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:24.655+0000 7f8b431ff700 1 -- 172.21.15.22:0/2281762716 wait complete. 2023-09-20T21:59:24.696 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:59:25.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:25 smithi123 ceph-mon[134433]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.2 MiB/s wr, 444 op/s 2023-09-20T21:59:25.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:25 smithi089 ceph-mon[134090]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.2 MiB/s wr, 444 op/s 2023-09-20T21:59:25.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:25 smithi022 ceph-mon[97651]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.2 MiB/s wr, 444 op/s 2023-09-20T21:59:26.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:26 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:59:26.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:26 smithi123 ceph-mon[134433]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2023-09-20T21:59:26.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:26 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:59:26.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:26 smithi089 ceph-mon[134090]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2023-09-20T21:59:26.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:26 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T21:59:26.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:26 smithi022 ceph-mon[97651]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2023-09-20T21:59:28.570 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 605.0 s, 438.5 tps, lat 228.170 ms stddev 261.194 2023-09-20T21:59:29.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:29 smithi123 ceph-mon[134433]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 5.0 MiB/s wr, 505 op/s 2023-09-20T21:59:29.595 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:29 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:29.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:29 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:29.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:29 smithi089 ceph-mon[134090]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 5.0 MiB/s wr, 505 op/s 2023-09-20T21:59:29.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:29 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:29.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:29 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:29.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:29 smithi022 ceph-mon[97651]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 5.0 MiB/s wr, 505 op/s 2023-09-20T21:59:29.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:29 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:29.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:29 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:31.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:31 smithi022 ceph-mon[97651]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.0 MiB/s wr, 507 op/s 2023-09-20T21:59:31.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:31.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:31 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:31 smithi123 ceph-mon[134433]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.0 MiB/s wr, 507 op/s 2023-09-20T21:59:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:31.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:31 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:31.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:31 smithi089 ceph-mon[134090]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.0 MiB/s wr, 507 op/s 2023-09-20T21:59:31.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:31.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:31 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:33.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:33 smithi022 ceph-mon[97651]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.6 MiB/s wr, 456 op/s 2023-09-20T21:59:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 610.0 s, 488.5 tps, lat 201.857 ms stddev 225.149 2023-09-20T21:59:33.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:33 smithi123 ceph-mon[134433]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.6 MiB/s wr, 456 op/s 2023-09-20T21:59:33.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:33 smithi089 ceph-mon[134090]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.6 MiB/s wr, 456 op/s 2023-09-20T21:59:35.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:34 smithi022 ceph-mon[97651]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 489 op/s 2023-09-20T21:59:35.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:35.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:34 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:35.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:34 smithi123 ceph-mon[134433]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 489 op/s 2023-09-20T21:59:35.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:35.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:34 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:35.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:34 smithi089 ceph-mon[134090]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 489 op/s 2023-09-20T21:59:35.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:35.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:34 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:36.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:59:36.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:59:36.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:35 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:36.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:59:36.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:59:36.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:35 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:36.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T21:59:36.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T21:59:36.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:35 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T21:59:37.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:37 smithi123 ceph-mon[134433]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.0 MiB/s wr, 497 op/s 2023-09-20T21:59:37.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:37 smithi089 ceph-mon[134090]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.0 MiB/s wr, 497 op/s 2023-09-20T21:59:37.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:37 smithi022 ceph-mon[97651]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.0 MiB/s wr, 497 op/s 2023-09-20T21:59:38.574 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 615.0 s, 499.3 tps, lat 204.852 ms stddev 244.241 2023-09-20T21:59:39.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:39 smithi123 ceph-mon[134433]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.6 MiB/s wr, 531 op/s 2023-09-20T21:59:39.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:39 smithi089 ceph-mon[134090]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.6 MiB/s wr, 531 op/s 2023-09-20T21:59:39.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:39 smithi022 ceph-mon[97651]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.6 MiB/s wr, 531 op/s 2023-09-20T21:59:41.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:41 smithi123 ceph-mon[134433]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.2 MiB/s wr, 526 op/s 2023-09-20T21:59:41.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:41 smithi089 ceph-mon[134090]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.2 MiB/s wr, 526 op/s 2023-09-20T21:59:41.756 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:41 smithi022 ceph-mon[97651]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.2 MiB/s wr, 526 op/s 2023-09-20T21:59:42.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:42 smithi089 ceph-mon[134090]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.4 MiB/s wr, 460 op/s 2023-09-20T21:59:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:42 smithi022 ceph-mon[97651]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.4 MiB/s wr, 460 op/s 2023-09-20T21:59:42.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:42 smithi123 ceph-mon[134433]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.4 MiB/s wr, 460 op/s 2023-09-20T21:59:43.570 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 620.0 s, 466.9 tps, lat 211.248 ms stddev 242.166 2023-09-20T21:59:45.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:45 smithi123 ceph-mon[134433]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 4.5 MiB/s wr, 490 op/s 2023-09-20T21:59:45.612 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:45 smithi089 ceph-mon[134090]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 4.5 MiB/s wr, 490 op/s 2023-09-20T21:59:45.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:45 smithi022 ceph-mon[97651]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 4.5 MiB/s wr, 490 op/s 2023-09-20T21:59:47.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:47 smithi123 ceph-mon[134433]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2023-09-20T21:59:47.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:47 smithi089 ceph-mon[134090]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2023-09-20T21:59:47.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:47 smithi022 ceph-mon[97651]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2023-09-20T21:59:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 625.0 s, 516.2 tps, lat 198.216 ms stddev 224.157 2023-09-20T21:59:49.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:49 smithi089 ceph-mon[134090]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.0 MiB/s wr, 511 op/s 2023-09-20T21:59:49.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:49 smithi022 ceph-mon[97651]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.0 MiB/s wr, 511 op/s 2023-09-20T21:59:49.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:49 smithi123 ceph-mon[134433]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.0 MiB/s wr, 511 op/s 2023-09-20T21:59:50.755 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:50 smithi022 ceph-mon[97651]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.2 MiB/s wr, 457 op/s 2023-09-20T21:59:50.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:50 smithi123 ceph-mon[134433]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.2 MiB/s wr, 457 op/s 2023-09-20T21:59:50.866 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:50 smithi089 ceph-mon[134090]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.2 MiB/s wr, 457 op/s 2023-09-20T21:59:53.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:52 smithi022 ceph-mon[97651]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 452 op/s 2023-09-20T21:59:53.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:52 smithi123 ceph-mon[134433]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 452 op/s 2023-09-20T21:59:53.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:52 smithi089 ceph-mon[134090]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 452 op/s 2023-09-20T21:59:53.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 630.0 s, 530.7 tps, lat 188.317 ms stddev 198.831 2023-09-20T21:59:54.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T21:59:55.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.346+0000 7fea943ff700 1 -- 172.21.15.22:0/2474444782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea8c108740 msgr2=0x7fea8c10f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:55.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.346+0000 7fea943ff700 1 --2- 172.21.15.22:0/2474444782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea8c108740 0x7fea8c10f120 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0098f0 tx=0x7fea7c009c00 comp rx=0 tx=0).stop 2023-09-20T21:59:55.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.346+0000 7fea943ff700 1 -- 172.21.15.22:0/2474444782 shutdown_connections 2023-09-20T21:59:55.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.346+0000 7fea943ff700 1 --2- 172.21.15.22:0/2474444782 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea8c10f660 0x7fea8c111a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.346+0000 7fea943ff700 1 --2- 172.21.15.22:0/2474444782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c107de0 0x7fea8c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.346+0000 7fea943ff700 1 --2- 172.21.15.22:0/2474444782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea8c108740 0x7fea8c10f120 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0098f0 tx=0x7fea7c009c00 comp rx=0 tx=0).stop 2023-09-20T21:59:55.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.346+0000 7fea943ff700 1 -- 172.21.15.22:0/2474444782 >> 172.21.15.22:0/2474444782 conn(0x7fea8c06db90 msgr2=0x7fea8c06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:55.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.353+0000 7fea943ff700 1 -- 172.21.15.22:0/2474444782 shutdown_connections 2023-09-20T21:59:55.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.368+0000 7fea943ff700 1 -- 172.21.15.22:0/2474444782 wait complete. 2023-09-20T21:59:55.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.369+0000 7fea943ff700 1 Processor -- start 2023-09-20T21:59:55.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.371+0000 7fea943ff700 1 -- start start 2023-09-20T21:59:55.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.371+0000 7fea943ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea8c107de0 0x7fea8c1a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.371+0000 7fea943ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea8c10f660 0x7fea8c1a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.371+0000 7fea943ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c1ab1c0 0x7fea8c1ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.371+0000 7fea943ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c1adb80 con 0x7fea8c107de0 2023-09-20T21:59:55.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.371+0000 7fea943ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c1adcf0 con 0x7fea8c1ab1c0 2023-09-20T21:59:55.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.371+0000 7fea943ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c1adeb0 con 0x7fea8c10f660 2023-09-20T21:59:55.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.372+0000 7fea9299c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c1ab1c0 0x7fea8c1ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.373+0000 7fea9219b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea8c107de0 0x7fea8c1a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.373+0000 7fea9219b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea8c107de0 0x7fea8c1a5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60710/0 (socket says 172.21.15.22:60710) 2023-09-20T21:59:55.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.373+0000 7fea9219b700 1 -- 172.21.15.22:0/2627009935 learned_addr learned my addr 172.21.15.22:0/2627009935 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:59:55.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.374+0000 7fea9219b700 1 -- 172.21.15.22:0/2627009935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea8c10f660 msgr2=0x7fea8c1a61a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T21:59:55.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.374+0000 7fea9219b700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea8c10f660 0x7fea8c1a61a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.374+0000 7fea9219b700 1 -- 172.21.15.22:0/2627009935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c1ab1c0 msgr2=0x7fea8c1ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:55.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.374+0000 7fea9219b700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c1ab1c0 0x7fea8c1ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.374+0000 7fea9219b700 1 -- 172.21.15.22:0/2627009935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea84009d00 con 0x7fea8c107de0 2023-09-20T21:59:55.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.375+0000 7fea9219b700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea8c107de0 0x7fea8c1a5c60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fea8800b530 tx=0x7fea8800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:55.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.375+0000 7fea837fe700 1 -- 172.21.15.22:0/2627009935 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea880118f0 con 0x7fea8c107de0 2023-09-20T21:59:55.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.375+0000 7fea837fe700 1 -- 172.21.15.22:0/2627009935 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea88010380 con 0x7fea8c107de0 2023-09-20T21:59:55.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.375+0000 7fea837fe700 1 -- 172.21.15.22:0/2627009935 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8800f540 con 0x7fea8c107de0 2023-09-20T21:59:55.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.376+0000 7fea943ff700 1 -- 172.21.15.22:0/2627009935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7c009520 con 0x7fea8c107de0 2023-09-20T21:59:55.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.379+0000 7fea943ff700 1 -- 172.21.15.22:0/2627009935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8c1ae540 con 0x7fea8c107de0 2023-09-20T21:59:55.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.395+0000 7fea837fe700 1 -- 172.21.15.22:0/2627009935 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fea8800f6a0 con 0x7fea8c107de0 2023-09-20T21:59:55.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.395+0000 7feaa7d48b80 1 Processor -- start 2023-09-20T21:59:55.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.396+0000 7feaa7d48b80 1 -- start start 2023-09-20T21:59:55.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.397+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f467b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.397+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468050 0x55f00f4685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.397+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468be0 0x55f00f469060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.397+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f3dbca0 con 0x55f00f468050 2023-09-20T21:59:55.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.397+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f3bfc70 con 0x55f00f468be0 2023-09-20T21:59:55.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.397+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f3bd590 con 0x55f00f4676f0 2023-09-20T21:59:55.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.397+0000 7fea9219b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468050 0x55f00f4685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.397+0000 7fea9219b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468050 0x55f00f4685e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60722/0 (socket says 172.21.15.22:60722) 2023-09-20T21:59:55.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.397+0000 7fea9219b700 1 -- 172.21.15.22:0/4243001034 learned_addr learned my addr 172.21.15.22:0/4243001034 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:59:55.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.397+0000 7fea9299c700 1 --2- 172.21.15.22:0/4243001034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f467b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.398+0000 7fea9199a700 1 --2- 172.21.15.22:0/4243001034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468be0 0x55f00f469060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.398+0000 7fea9299c700 1 -- 172.21.15.22:0/4243001034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468be0 msgr2=0x55f00f469060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:55.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.398+0000 7fea9299c700 1 --2- 172.21.15.22:0/4243001034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468be0 0x55f00f469060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.398+0000 7fea9299c700 1 -- 172.21.15.22:0/4243001034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468050 msgr2=0x55f00f4685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:55.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.398+0000 7fea9299c700 1 --2- 172.21.15.22:0/4243001034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468050 0x55f00f4685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.398+0000 7fea9299c700 1 -- 172.21.15.22:0/4243001034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f00f1e3de0 con 0x55f00f4676f0 2023-09-20T21:59:55.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.398+0000 7fea9299c700 1 --2- 172.21.15.22:0/4243001034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f467b10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea84009cd0 tx=0x7fea840099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6019b74c703c8ac9 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:55.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.399+0000 7fea80bfe700 1 -- 172.21.15.22:0/4243001034 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400e030 con 0x55f00f4676f0 2023-09-20T21:59:55.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.399+0000 7fea80bfe700 1 -- 172.21.15.22:0/4243001034 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea8400f040 con 0x55f00f4676f0 2023-09-20T21:59:55.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.399+0000 7fea80bfe700 1 -- 172.21.15.22:0/4243001034 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84005280 con 0x55f00f4676f0 2023-09-20T21:59:55.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.399+0000 7fea837fe700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fea780a1460 0x7fea780a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.399+0000 7fea837fe700 1 -- 172.21.15.22:0/2627009935 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fea880cda60 con 0x7fea8c107de0 2023-09-20T21:59:55.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.400+0000 7fea9219b700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fea780a1460 0x7fea780a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.400+0000 7feaa7d48b80 1 -- 172.21.15.22:0/4243001034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 msgr2=0x55f00f467b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:55.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.400+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/4243001034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f467b10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea84009cd0 tx=0x7fea840099e0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.400+0000 7feaa7d48b80 1 -- 172.21.15.22:0/4243001034 shutdown_connections 2023-09-20T21:59:55.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.400+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/4243001034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468be0 0x55f00f469060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.400+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/4243001034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f467b10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.400+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/4243001034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468050 0x55f00f4685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.400+0000 7feaa7d48b80 1 -- 172.21.15.22:0/4243001034 >> 172.21.15.22:0/4243001034 conn(0x55f00f394000 msgr2=0x55f00f4621d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:55.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.400+0000 7fea9219b700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fea780a1460 0x7fea780a3920 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55f00f463150 tx=0x7fea88004860 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:55.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.400+0000 7feaa7d48b80 1 -- 172.21.15.22:0/4243001034 shutdown_connections 2023-09-20T21:59:55.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.401+0000 7feaa7d48b80 1 -- 172.21.15.22:0/4243001034 wait complete. 2023-09-20T21:59:55.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.401+0000 7feaa7d48b80 1 -- start start 2023-09-20T21:59:55.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.402+0000 7feaa7d48b80 1 Processor -- start 2023-09-20T21:59:55.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.402+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f471230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.402+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f471770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.402+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468be0 0x55f00f4787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.402+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f3b8e80 con 0x55f00f468be0 2023-09-20T21:59:55.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.402+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f355740 con 0x55f00f468050 2023-09-20T21:59:55.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.402+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f3553e0 con 0x55f00f4676f0 2023-09-20T21:59:55.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.402+0000 7feaa7d48b80 10 client.0 _ll_register_callbacks cb 0x55f00f29fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:59:55.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9199a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f471770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9199a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f471770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:58382/0 (socket says 172.21.15.22:58382) 2023-09-20T21:59:55.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9299c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f471230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9199a700 1 -- 172.21.15.22:0/220360106 learned_addr learned my addr 172.21.15.22:0/220360106 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:59:55.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9299c700 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468be0 0x55f00f4787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9199a700 1 -- 172.21.15.22:0/220360106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 msgr2=0x55f00f471230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:55.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9199a700 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f471230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9199a700 1 -- 172.21.15.22:0/220360106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468be0 msgr2=0x55f00f4787b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:55.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9199a700 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468be0 0x55f00f4787b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9199a700 1 -- 172.21.15.22:0/220360106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea84009160 con 0x55f00f468050 2023-09-20T21:59:55.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea9199a700 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f471770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea7c000c00 tx=0x7fea7c00f960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee810de3fb6d76b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:55.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.403+0000 7fea75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T21:59:55.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.404+0000 7fea75ffb700 1 -- 172.21.15.22:0/220360106 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c01c070 con 0x55f00f468050 2023-09-20T21:59:55.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.404+0000 7feaa7d48b80 1 -- 172.21.15.22:0/220360106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f00f329bb0 con 0x55f00f468050 2023-09-20T21:59:55.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.404+0000 7fea75ffb700 1 -- 172.21.15.22:0/220360106 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fea7c00ba20 con 0x55f00f468050 2023-09-20T21:59:55.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.404+0000 7fea75ffb700 1 -- 172.21.15.22:0/220360106 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea7c00bbe0 con 0x55f00f468050 2023-09-20T21:59:55.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.404+0000 7feaa7d48b80 10 client.26045 fetch_fsmap learned FSMap version 32 2023-09-20T21:59:55.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.404+0000 7feaa7d48b80 1 -- 172.21.15.22:0/220360106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f00f1ccc60 con 0x55f00f468050 2023-09-20T21:59:55.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.404+0000 7fea75ffb700 1 -- 172.21.15.22:0/220360106 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f00f329bb0 con 0x55f00f468050 2023-09-20T21:59:55.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.405+0000 7fea75ffb700 1 -- 172.21.15.22:0/220360106 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fea7c015020 con 0x55f00f468050 2023-09-20T21:59:55.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.405+0000 7feaa7d48b80 10 client.26045 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:59:55.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.405+0000 7feaa7d48b80 10 client.26045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:59:55.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.405+0000 7feaa7d48b80 20 client.26045 populate_metadata read hostname 'smithi022' 2023-09-20T21:59:55.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.405+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f00f47d0b0 0x55f00f47f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.405+0000 7feaa7d48b80 4 client.26045 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T21:59:55.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.405+0000 7feaa7d48b80 1 -- 172.21.15.22:0/220360106 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f00f436020 con 0x55f00f47d0b0 2023-09-20T21:59:55.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.405+0000 7fea9299c700 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f00f47d0b0 0x55f00f47f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.406+0000 7fea9299c700 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f00f47d0b0 0x55f00f47f4d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403541c2500c9089 server_cookie=86a0931815b22293 in_seq=0 out_seq=0 2023-09-20T21:59:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.406+0000 7fea75ffb700 10 client.26045 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:59:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7fea75ffb700 1 -- 172.21.15.22:0/220360106 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x7fea8402d730 con 0x55f00f47d0b0 2023-09-20T21:59:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7fea75ffb700 10 client.26045 handle_command_reply: tid=0 2023-09-20T21:59:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7feaa7d48b80 1 client.26045 shutdown 2023-09-20T21:59:55.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7feaa7d48b80 1 -- 172.21.15.22:0/220360106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 msgr2=0x55f00f471770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f471770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea7c000c00 tx=0x7fea7c00f960 comp rx=0 tx=0).stop 2023-09-20T21:59:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7feaa7d48b80 1 -- 172.21.15.22:0/220360106 shutdown_connections 2023-09-20T21:59:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f00f47d0b0 0x55f00f47f4d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468be0 0x55f00f4787b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f471230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/220360106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f471770 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7fea75ffb700 0 client.26045 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:59:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.407+0000 7feaa7d48b80 1 -- 172.21.15.22:0/220360106 >> 172.21.15.22:0/220360106 conn(0x55f00f394000 msgr2=0x55f00f461880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.408+0000 7feaa7d48b80 1 -- 172.21.15.22:0/220360106 shutdown_connections 2023-09-20T21:59:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.408+0000 7feaa7d48b80 1 -- 172.21.15.22:0/220360106 wait complete. 2023-09-20T21:59:55.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.408+0000 7feaa7d48b80 20 client.26045 trim_cache size 0 max 16384 2023-09-20T21:59:55.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.444+0000 7feaa7d48b80 1 Processor -- start 2023-09-20T21:59:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.444+0000 7feaa7d48b80 1 -- start start 2023-09-20T21:59:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.445+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f46ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.445+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f478770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.445+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468be0 0x55f00f471230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.445+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f239920 con 0x55f00f468be0 2023-09-20T21:59:55.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.445+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f46edf0 con 0x55f00f468050 2023-09-20T21:59:55.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.445+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f46f750 con 0x55f00f4676f0 2023-09-20T21:59:55.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.445+0000 7fea9299c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f46ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.446+0000 7fea9299c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f46ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:46770/0 (socket says 172.21.15.22:46770) 2023-09-20T21:59:55.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.446+0000 7fea9299c700 1 -- 172.21.15.22:0/2871203794 learned_addr learned my addr 172.21.15.22:0/2871203794 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:59:55.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.446+0000 7fea9299c700 1 --2- 172.21.15.22:0/2871203794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468be0 0x55f00f471230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.446+0000 7fea9299c700 1 -- 172.21.15.22:0/2871203794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 msgr2=0x55f00f478770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T21:59:55.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.446+0000 7fea9299c700 1 --2- 172.21.15.22:0/2871203794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f478770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.446+0000 7fea9299c700 1 -- 172.21.15.22:0/2871203794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468be0 msgr2=0x55f00f471230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:55.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.446+0000 7fea9299c700 1 --2- 172.21.15.22:0/2871203794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468be0 0x55f00f471230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.446+0000 7fea9299c700 1 -- 172.21.15.22:0/2871203794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f00f20a250 con 0x55f00f4676f0 2023-09-20T21:59:55.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.446+0000 7fea9299c700 1 --2- 172.21.15.22:0/2871203794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f46ff20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fea84009450 tx=0x7fea7c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8223253b9a3886bb server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:55.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.448+0000 7fea813ff700 1 -- 172.21.15.22:0/2871203794 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400e040 con 0x55f00f4676f0 2023-09-20T21:59:55.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.448+0000 7fea813ff700 1 -- 172.21.15.22:0/2871203794 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea84029040 con 0x55f00f4676f0 2023-09-20T21:59:55.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.448+0000 7fea813ff700 1 -- 172.21.15.22:0/2871203794 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84005570 con 0x55f00f4676f0 2023-09-20T21:59:55.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.449+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2871203794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 msgr2=0x55f00f46ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:55.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.449+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/2871203794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f46ff20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fea84009450 tx=0x7fea7c00f790 comp rx=0 tx=0).stop 2023-09-20T21:59:55.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.458+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2871203794 shutdown_connections 2023-09-20T21:59:55.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.458+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/2871203794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f468be0 0x55f00f471230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.458+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/2871203794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f4676f0 0x55f00f46ff20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.458+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/2871203794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f478770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.458+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2871203794 >> 172.21.15.22:0/2871203794 conn(0x55f00f394000 msgr2=0x55f00f46f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:55.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.458+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2871203794 shutdown_connections 2023-09-20T21:59:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.458+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2871203794 wait complete. 2023-09-20T21:59:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.459+0000 7feaa7d48b80 1 -- start start 2023-09-20T21:59:55.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.459+0000 7feaa7d48b80 1 Processor -- start 2023-09-20T21:59:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.460+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f4676f0 0x55f00f478510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.460+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f474fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.460+0000 7feaa7d48b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f468be0 0x55f00f4754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.460+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f475a70 con 0x55f00f4676f0 2023-09-20T21:59:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.460+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f475bb0 con 0x55f00f468050 2023-09-20T21:59:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.460+0000 7feaa7d48b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f00f4e1ee0 con 0x55f00f468be0 2023-09-20T21:59:55.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.460+0000 7feaa7d48b80 10 client.0 _ll_register_callbacks cb 0x55f00f29fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T21:59:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.461+0000 7fea9299c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f4676f0 0x55f00f478510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.461+0000 7fea9299c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f468be0 0x55f00f4754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.461+0000 7fea9299c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f468be0 0x55f00f4754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:46778/0 (socket says 172.21.15.22:46778) 2023-09-20T21:59:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.461+0000 7fea9299c700 1 -- 172.21.15.22:0/2833260813 learned_addr learned my addr 172.21.15.22:0/2833260813 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T21:59:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.462+0000 7fea9299c700 1 -- 172.21.15.22:0/2833260813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 msgr2=0x55f00f474fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.462+0000 7fea9299c700 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f474fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.462+0000 7fea9299c700 1 -- 172.21.15.22:0/2833260813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f4676f0 msgr2=0x55f00f478510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T21:59:55.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.462+0000 7fea9299c700 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f4676f0 0x55f00f478510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.462+0000 7fea9299c700 1 -- 172.21.15.22:0/2833260813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f00f1e3de0 con 0x55f00f468be0 2023-09-20T21:59:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.462+0000 7fea9299c700 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f468be0 0x55f00f4754f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fea84003c60 tx=0x7fea84000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e03d2c429f588adf server_cookie=0 in_seq=0 out_seq=0 2023-09-20T21:59:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.463+0000 7fea75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T21:59:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.463+0000 7fea75ffb700 1 -- 172.21.15.22:0/2833260813 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400e040 con 0x55f00f468be0 2023-09-20T21:59:55.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.463+0000 7fea75ffb700 1 -- 172.21.15.22:0/2833260813 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fea840054f0 con 0x55f00f468be0 2023-09-20T21:59:55.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.463+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2833260813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea8400f040 con 0x55f00f468be0 2023-09-20T21:59:55.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.478+0000 7fea75ffb700 1 -- 172.21.15.22:0/2833260813 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea840056b0 con 0x55f00f468be0 2023-09-20T21:59:55.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.479+0000 7feaa7d48b80 10 client.26077 fetch_fsmap learned FSMap version 32 2023-09-20T21:59:55.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.479+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2833260813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f00f1e22b0 con 0x55f00f468be0 2023-09-20T21:59:55.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.485+0000 7fea75ffb700 1 -- 172.21.15.22:0/2833260813 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea84040030 con 0x55f00f468be0 2023-09-20T21:59:55.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.485+0000 7fea75ffb700 1 -- 172.21.15.22:0/2833260813 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fea8403a060 con 0x55f00f468be0 2023-09-20T21:59:55.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.485+0000 7feaa7d48b80 10 client.26077 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T21:59:55.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.485+0000 7feaa7d48b80 10 client.26077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T21:59:55.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.485+0000 7feaa7d48b80 20 client.26077 populate_metadata read hostname 'smithi022' 2023-09-20T21:59:55.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.485+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f00f4e3350 0x55f00f4e5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T21:59:55.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.485+0000 7feaa7d48b80 4 client.26077 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T21:59:55.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.485+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2833260813 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f00f3b1e60 con 0x55f00f4e3350 2023-09-20T21:59:55.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.487+0000 7fea9199a700 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f00f4e3350 0x55f00f4e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T21:59:55.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.491+0000 7fea9199a700 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f00f4e3350 0x55f00f4e5740 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96df35ecd52043cf server_cookie=9f5549fe33870e0c in_seq=0 out_seq=0 2023-09-20T21:59:55.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.497+0000 7fea75ffb700 10 client.26077 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T21:59:55.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.497+0000 7fea75ffb700 1 -- 172.21.15.22:0/2833260813 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f00f3b1e60 con 0x55f00f4e3350 2023-09-20T21:59:55.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.497+0000 7fea75ffb700 10 client.26077 handle_command_reply: tid=0 2023-09-20T21:59:55.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.497+0000 7feaa7d48b80 1 client.26077 shutdown 2023-09-20T21:59:55.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.514+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2833260813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f468be0 msgr2=0x55f00f4754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:55.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.514+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f468be0 0x55f00f4754f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fea84003c60 tx=0x7fea84000f40 comp rx=0 tx=0).stop 2023-09-20T21:59:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.519+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2833260813 shutdown_connections 2023-09-20T21:59:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.519+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f00f4e3350 0x55f00f4e5740 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.519+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f00f468be0 0x55f00f4754f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.519+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f00f4676f0 0x55f00f478510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.519+0000 7feaa7d48b80 1 --2- 172.21.15.22:0/2833260813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f00f468050 0x55f00f474fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.519+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2833260813 >> 172.21.15.22:0/2833260813 conn(0x55f00f394000 msgr2=0x55f00f46ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:55.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.519+0000 7fea75ffb700 0 client.26077 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T21:59:55.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.533+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2833260813 shutdown_connections 2023-09-20T21:59:55.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.535+0000 7feaa7d48b80 1 -- 172.21.15.22:0/2833260813 wait complete. 2023-09-20T21:59:55.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.538+0000 7feaa7d48b80 20 client.26077 trim_cache size 0 max 16384 2023-09-20T21:59:55.542 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T21:59:55.542 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T21:59:55.542 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T21:59:55.543 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T21:59:55.543 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T21:59:55.543 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T21:59:55.543 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T21:59:55.543 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:59:55.544 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T21:59:55.544 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T21:59:55.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.543+0000 7fea943ff700 1 -- 172.21.15.22:0/2627009935 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fea780a1460 msgr2=0x7fea780a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:55.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.543+0000 7fea943ff700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fea780a1460 0x7fea780a3920 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55f00f463150 tx=0x7fea88004860 comp rx=0 tx=0).stop 2023-09-20T21:59:55.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.543+0000 7fea943ff700 1 -- 172.21.15.22:0/2627009935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea8c107de0 msgr2=0x7fea8c1a5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T21:59:55.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.543+0000 7fea943ff700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea8c107de0 0x7fea8c1a5c60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fea8800b530 tx=0x7fea8800b840 comp rx=0 tx=0).stop 2023-09-20T21:59:55.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.545+0000 7fea943ff700 1 -- 172.21.15.22:0/2627009935 shutdown_connections 2023-09-20T21:59:55.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.545+0000 7fea943ff700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fea780a1460 0x7fea780a3920 secure :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55f00f463150 tx=0x7fea88004860 comp rx=0 tx=0).stop 2023-09-20T21:59:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.545+0000 7fea943ff700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8c1ab1c0 0x7fea8c1ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.545+0000 7fea943ff700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea8c107de0 0x7fea8c1a5c60 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fea8800b530 tx=0x7fea8800b840 comp rx=0 tx=0).stop 2023-09-20T21:59:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.545+0000 7fea943ff700 1 --2- 172.21.15.22:0/2627009935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea8c10f660 0x7fea8c1a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T21:59:55.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.545+0000 7fea943ff700 1 -- 172.21.15.22:0/2627009935 >> 172.21.15.22:0/2627009935 conn(0x7fea8c06db90 msgr2=0x7fea8c06e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T21:59:55.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.550+0000 7fea943ff700 1 -- 172.21.15.22:0/2627009935 shutdown_connections 2023-09-20T21:59:55.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T21:59:55.551+0000 7fea943ff700 1 -- 172.21.15.22:0/2627009935 wait complete. 2023-09-20T21:59:55.579 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T21:59:55.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:55 smithi123 ceph-mon[134433]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.1 MiB/s wr, 508 op/s 2023-09-20T21:59:55.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:55 smithi089 ceph-mon[134090]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.1 MiB/s wr, 508 op/s 2023-09-20T21:59:55.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:55 smithi022 ceph-mon[97651]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.1 MiB/s wr, 508 op/s 2023-09-20T21:59:56.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:56 smithi022 ceph-mon[97651]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.3 MiB/s wr, 513 op/s 2023-09-20T21:59:56.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:56 smithi123 ceph-mon[134433]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.3 MiB/s wr, 513 op/s 2023-09-20T21:59:56.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:56 smithi089 ceph-mon[134090]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.3 MiB/s wr, 513 op/s 2023-09-20T21:59:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 635.0 s, 463.7 tps, lat 206.226 ms stddev 208.806 2023-09-20T21:59:59.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 21:59:59 smithi022 ceph-mon[97651]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.1 MiB/s wr, 568 op/s 2023-09-20T21:59:59.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 21:59:59 smithi123 ceph-mon[134433]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.1 MiB/s wr, 568 op/s 2023-09-20T21:59:59.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 21:59:59 smithi089 ceph-mon[134090]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.1 MiB/s wr, 568 op/s 2023-09-20T22:00:00.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:00 smithi022 ceph-mon[97651]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.7 MiB/s wr, 520 op/s 2023-09-20T22:00:00.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:00 smithi022 ceph-mon[97651]: overall HEALTH_OK 2023-09-20T22:00:00.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:00 smithi123 ceph-mon[134433]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.7 MiB/s wr, 520 op/s 2023-09-20T22:00:00.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:00 smithi123 ceph-mon[134433]: overall HEALTH_OK 2023-09-20T22:00:00.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:00 smithi089 ceph-mon[134090]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.7 MiB/s wr, 520 op/s 2023-09-20T22:00:00.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:00 smithi089 ceph-mon[134090]: overall HEALTH_OK 2023-09-20T22:00:03.164 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:02 smithi089 ceph-mon[134090]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.0 MiB/s wr, 593 op/s 2023-09-20T22:00:03.254 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:02 smithi022 ceph-mon[97651]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.0 MiB/s wr, 593 op/s 2023-09-20T22:00:03.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:02 smithi123 ceph-mon[134433]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.0 MiB/s wr, 593 op/s 2023-09-20T22:00:03.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 640.0 s, 498.6 tps, lat 207.520 ms stddev 209.112 2023-09-20T22:00:05.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:05 smithi123 ceph-mon[134433]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.4 MiB/s wr, 581 op/s 2023-09-20T22:00:05.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:05 smithi089 ceph-mon[134090]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.4 MiB/s wr, 581 op/s 2023-09-20T22:00:05.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:05 smithi022 ceph-mon[97651]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.4 MiB/s wr, 581 op/s 2023-09-20T22:00:07.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:07 smithi022 ceph-mon[97651]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.5 MiB/s wr, 553 op/s 2023-09-20T22:00:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:07 smithi123 ceph-mon[134433]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.5 MiB/s wr, 553 op/s 2023-09-20T22:00:07.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:07 smithi089 ceph-mon[134090]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.5 MiB/s wr, 553 op/s 2023-09-20T22:00:08.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 645.0 s, 565.7 tps, lat 177.156 ms stddev 188.384 2023-09-20T22:00:09.605 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:09 smithi123 ceph-mon[134433]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.0 MiB/s wr, 560 op/s 2023-09-20T22:00:09.611 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:09 smithi089 ceph-mon[134090]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.0 MiB/s wr, 560 op/s 2023-09-20T22:00:09.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:09 smithi022 ceph-mon[97651]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.0 MiB/s wr, 560 op/s 2023-09-20T22:00:11.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:11 smithi022 ceph-mon[97651]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.4 MiB/s wr, 511 op/s 2023-09-20T22:00:11.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:11 smithi123 ceph-mon[134433]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.4 MiB/s wr, 511 op/s 2023-09-20T22:00:11.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:11 smithi089 ceph-mon[134090]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.4 MiB/s wr, 511 op/s 2023-09-20T22:00:13.575 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 650.0 s, 519.3 tps, lat 188.693 ms stddev 190.538 2023-09-20T22:00:13.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:13 smithi022 ceph-mon[97651]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 8.4 MiB/s wr, 551 op/s 2023-09-20T22:00:13.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:13 smithi123 ceph-mon[134433]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 8.4 MiB/s wr, 551 op/s 2023-09-20T22:00:13.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:13 smithi089 ceph-mon[134090]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 8.4 MiB/s wr, 551 op/s 2023-09-20T22:00:15.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:15 smithi022 ceph-mon[97651]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.4 MiB/s wr, 485 op/s 2023-09-20T22:00:15.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:15 smithi123 ceph-mon[134433]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.4 MiB/s wr, 485 op/s 2023-09-20T22:00:15.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:15 smithi089 ceph-mon[134090]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.4 MiB/s wr, 485 op/s 2023-09-20T22:00:17.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:17 smithi022 ceph-mon[97651]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.8 MiB/s wr, 494 op/s 2023-09-20T22:00:17.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:17 smithi123 ceph-mon[134433]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.8 MiB/s wr, 494 op/s 2023-09-20T22:00:17.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:17 smithi089 ceph-mon[134090]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.8 MiB/s wr, 494 op/s 2023-09-20T22:00:18.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 655.0 s, 504.3 tps, lat 199.477 ms stddev 219.611 2023-09-20T22:00:19.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:19 smithi022 ceph-mon[97651]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.4 MiB/s wr, 492 op/s 2023-09-20T22:00:19.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:19 smithi123 ceph-mon[134433]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.4 MiB/s wr, 492 op/s 2023-09-20T22:00:19.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:19 smithi089 ceph-mon[134090]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.4 MiB/s wr, 492 op/s 2023-09-20T22:00:21.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:21 smithi022 ceph-mon[97651]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.0 MiB/s wr, 486 op/s 2023-09-20T22:00:21.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:21 smithi123 ceph-mon[134433]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.0 MiB/s wr, 486 op/s 2023-09-20T22:00:21.875 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:21 smithi089 ceph-mon[134090]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 7.0 MiB/s wr, 486 op/s 2023-09-20T22:00:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 660.0 s, 547.2 tps, lat 184.056 ms stddev 207.629 2023-09-20T22:00:23.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:23 smithi022 ceph-mon[97651]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.9 MiB/s wr, 514 op/s 2023-09-20T22:00:23.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:23 smithi123 ceph-mon[134433]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.9 MiB/s wr, 514 op/s 2023-09-20T22:00:23.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:23 smithi089 ceph-mon[134090]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.9 MiB/s wr, 514 op/s 2023-09-20T22:00:24.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:24 smithi123 ceph-mon[134433]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.6 MiB/s wr, 476 op/s 2023-09-20T22:00:24.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:24 smithi089 ceph-mon[134090]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.6 MiB/s wr, 476 op/s 2023-09-20T22:00:24.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:24 smithi022 ceph-mon[97651]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.6 MiB/s wr, 476 op/s 2023-09-20T22:00:25.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T22:00:26.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.027+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/157192733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c94107de0 msgr2=0x7f1c9410a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:26.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.027+0000 7f1c9a3ff700 1 --2- 172.21.15.22:0/157192733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c94107de0 0x7f1c9410a210 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009990 tx=0x7f1c88009ca0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.033+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/157192733 shutdown_connections 2023-09-20T22:00:26.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.033+0000 7f1c9a3ff700 1 --2- 172.21.15.22:0/157192733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9410d120 0x7f1c9410d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.033+0000 7f1c9a3ff700 1 --2- 172.21.15.22:0/157192733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c94107de0 0x7f1c9410a210 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.033+0000 7f1c9a3ff700 1 --2- 172.21.15.22:0/157192733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9410a750 0x7f1c9410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.033+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/157192733 >> 172.21.15.22:0/157192733 conn(0x7f1c9406db90 msgr2=0x7f1c9406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:26.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.037+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/157192733 shutdown_connections 2023-09-20T22:00:26.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.037+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/157192733 wait complete. 2023-09-20T22:00:26.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.039+0000 7f1c9a3ff700 1 Processor -- start 2023-09-20T22:00:26.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.039+0000 7f1c9a3ff700 1 -- start start 2023-09-20T22:00:26.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.040+0000 7f1c9a3ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c94107de0 0x7f1c94114340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.040+0000 7f1c9a3ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9410a750 0x7f1c94114880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.040+0000 7f1c9a3ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9410d120 0x7f1c941b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.040+0000 7f1c9a3ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94117960 con 0x7f1c94107de0 2023-09-20T22:00:26.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.040+0000 7f1c9a3ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94117ad0 con 0x7f1c9410a750 2023-09-20T22:00:26.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.040+0000 7f1c9a3ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c941b6030 con 0x7f1c9410d120 2023-09-20T22:00:26.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.041+0000 7f1c937fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9410a750 0x7f1c94114880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.041+0000 7f1c937fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9410a750 0x7f1c94114880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:57008/0 (socket says 172.21.15.22:57008) 2023-09-20T22:00:26.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.041+0000 7f1c937fe700 1 -- 172.21.15.22:0/1857004136 learned_addr learned my addr 172.21.15.22:0/1857004136 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:00:26.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.041+0000 7f1c9899c700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9410d120 0x7f1c941b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.041+0000 7f1c93fff700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c94107de0 0x7f1c94114340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.042+0000 7f1c937fe700 1 -- 172.21.15.22:0/1857004136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9410d120 msgr2=0x7f1c941b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:26.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.042+0000 7f1c937fe700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9410d120 0x7f1c941b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.042+0000 7f1c937fe700 1 -- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c94107de0 msgr2=0x7f1c94114340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:26.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.042+0000 7f1c937fe700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c94107de0 0x7f1c94114340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.042+0000 7f1c937fe700 1 -- 172.21.15.22:0/1857004136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c880095f0 con 0x7f1c9410a750 2023-09-20T22:00:26.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.043+0000 7f1c937fe700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9410a750 0x7f1c94114880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400e9a0 tx=0x7f1c8400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:26.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.044+0000 7f1c917fa700 1 -- 172.21.15.22:0/1857004136 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8400cb70 con 0x7f1c9410a750 2023-09-20T22:00:26.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.044+0000 7f1c917fa700 1 -- 172.21.15.22:0/1857004136 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c8400ccd0 con 0x7f1c9410a750 2023-09-20T22:00:26.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.044+0000 7f1c917fa700 1 -- 172.21.15.22:0/1857004136 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c84018680 con 0x7f1c9410a750 2023-09-20T22:00:26.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.046+0000 7f1c93fff700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c94107de0 0x7f1c94114340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T22:00:26.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.049+0000 7f1c9899c700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9410d120 0x7f1c941b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:00:26.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.049+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/1857004136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c941b6230 con 0x7f1c9410a750 2023-09-20T22:00:26.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.049+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/1857004136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c941b66c0 con 0x7f1c9410a750 2023-09-20T22:00:26.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.056+0000 7f1cadd82b80 1 Processor -- start 2023-09-20T22:00:26.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.057+0000 7f1cadd82b80 1 -- start start 2023-09-20T22:00:26.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.057+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fedbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.057+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 0x55e77fedc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.057+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedcbe0 0x55e77fedd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.057+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77fe4fca0 con 0x55e77fedc050 2023-09-20T22:00:26.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.057+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77fe33c70 con 0x55e77fedcbe0 2023-09-20T22:00:26.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.057+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77fe31590 con 0x55e77fedb6f0 2023-09-20T22:00:26.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.059+0000 7f1c9899c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fedbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.060+0000 7f1c9899c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fedbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:44910/0 (socket says 172.21.15.22:44910) 2023-09-20T22:00:26.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.060+0000 7f1c9899c700 1 -- 172.21.15.22:0/545038317 learned_addr learned my addr 172.21.15.22:0/545038317 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:00:26.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.060+0000 7f1c93fff700 1 --2- 172.21.15.22:0/545038317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 0x55e77fedc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.062+0000 7f1c93fff700 1 -- 172.21.15.22:0/545038317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 msgr2=0x55e77fedbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:26.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.062+0000 7f1c93fff700 1 --2- 172.21.15.22:0/545038317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fedbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.062+0000 7f1c93fff700 1 -- 172.21.15.22:0/545038317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedcbe0 msgr2=0x55e77fedd060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T22:00:26.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.062+0000 7f1c93fff700 1 --2- 172.21.15.22:0/545038317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedcbe0 0x55e77fedd060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.062+0000 7f1c93fff700 1 -- 172.21.15.22:0/545038317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e77fc57de0 con 0x55e77fedc050 2023-09-20T22:00:26.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.062+0000 7f1c93fff700 1 --2- 172.21.15.22:0/545038317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 0x55e77fedc5e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009990 tx=0x7f1c8800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28329c5d04a5766a server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:26.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.063+0000 7f1c7a7fc700 1 -- 172.21.15.22:0/545038317 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8801e070 con 0x55e77fedc050 2023-09-20T22:00:26.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.063+0000 7f1c7a7fc700 1 -- 172.21.15.22:0/545038317 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c8800bcd0 con 0x55e77fedc050 2023-09-20T22:00:26.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.063+0000 7f1c7a7fc700 1 -- 172.21.15.22:0/545038317 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c880049c0 con 0x55e77fedc050 2023-09-20T22:00:26.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.064+0000 7f1cadd82b80 1 -- 172.21.15.22:0/545038317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 msgr2=0x55e77fedc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:26.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.064+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/545038317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 0x55e77fedc5e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009990 tx=0x7f1c8800b260 comp rx=0 tx=0).stop 2023-09-20T22:00:26.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.064+0000 7f1cadd82b80 1 -- 172.21.15.22:0/545038317 shutdown_connections 2023-09-20T22:00:26.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.064+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/545038317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedcbe0 0x55e77fedd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.064+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/545038317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fedbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.064+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/545038317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 0x55e77fedc5e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.065+0000 7f1cadd82b80 1 -- 172.21.15.22:0/545038317 >> 172.21.15.22:0/545038317 conn(0x55e77fe08000 msgr2=0x55e77fed61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:26.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.066+0000 7f1c917fa700 1 -- 172.21.15.22:0/1857004136 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1c840187e0 con 0x7f1c9410a750 2023-09-20T22:00:26.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.067+0000 7f1cadd82b80 1 -- 172.21.15.22:0/545038317 shutdown_connections 2023-09-20T22:00:26.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.068+0000 7f1c917fa700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1c7c0a1420 0x7f1c7c0a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.069+0000 7f1c917fa700 1 -- 172.21.15.22:0/1857004136 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1c84014070 con 0x7f1c9410a750 2023-09-20T22:00:26.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.069+0000 7f1cadd82b80 1 -- 172.21.15.22:0/545038317 wait complete. 2023-09-20T22:00:26.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.070+0000 7f1cadd82b80 1 -- start start 2023-09-20T22:00:26.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.071+0000 7f1cadd82b80 1 Processor -- start 2023-09-20T22:00:26.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.072+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fee5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.072+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 0x55e77fee5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.072+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77feea880 0x55e77feecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.072+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77fe2ce80 con 0x55e77fedc050 2023-09-20T22:00:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.072+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77fdc9740 con 0x55e77feea880 2023-09-20T22:00:26.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.072+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77fdc93e0 con 0x55e77fedb6f0 2023-09-20T22:00:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.072+0000 7f1cadd82b80 10 client.0 _ll_register_callbacks cb 0x55e77fd13a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:00:26.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.072+0000 7f1c93fff700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1c7c0a1420 0x7f1c7c0a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.072+0000 7f1c93fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fee5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.073+0000 7f1c93fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fee5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:44916/0 (socket says 172.21.15.22:44916) 2023-09-20T22:00:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.073+0000 7f1c93fff700 1 -- 172.21.15.22:0/2652710795 learned_addr learned my addr 172.21.15.22:0/2652710795 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:00:26.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.073+0000 7f1c937fe700 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 0x55e77fee5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.073+0000 7f1c9899c700 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77feea880 0x55e77feecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.073+0000 7f1c93fff700 1 -- 172.21.15.22:0/2652710795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77feea880 msgr2=0x55e77feecc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:26.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.073+0000 7f1c93fff700 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77feea880 0x55e77feecc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.073+0000 7f1c93fff700 1 -- 172.21.15.22:0/2652710795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 msgr2=0x55e77fee5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:26.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.073+0000 7f1c93fff700 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 0x55e77fee5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.073+0000 7f1c93fff700 1 -- 172.21.15.22:0/2652710795 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9410e120 con 0x55e77fedb6f0 2023-09-20T22:00:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.073+0000 7f1c93fff700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1c7c0a1420 0x7f1c7c0a38e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009990 tx=0x7f1c88005040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.074+0000 7f1c937fe700 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 0x55e77fee5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:00:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.074+0000 7f1c93fff700 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fee5310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802f000 tx=0x7f1c8801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb36eb51386a819 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:26.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.074+0000 7f1c737fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T22:00:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.074+0000 7f1c737fe700 1 -- 172.21.15.22:0/2652710795 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8801c030 con 0x55e77fedb6f0 2023-09-20T22:00:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.074+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2652710795 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e77fd9dbb0 con 0x55e77fedb6f0 2023-09-20T22:00:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.074+0000 7f1c737fe700 1 -- 172.21.15.22:0/2652710795 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1c8800f740 con 0x55e77fedb6f0 2023-09-20T22:00:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.075+0000 7f1c737fe700 1 -- 172.21.15.22:0/2652710795 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c8801e070 con 0x55e77fedb6f0 2023-09-20T22:00:26.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.075+0000 7f1cadd82b80 10 client.26089 fetch_fsmap learned FSMap version 32 2023-09-20T22:00:26.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.075+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2652710795 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e77fc40c60 con 0x55e77fedb6f0 2023-09-20T22:00:26.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.075+0000 7f1c737fe700 1 -- 172.21.15.22:0/2652710795 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e77fd9dbb0 con 0x55e77fedb6f0 2023-09-20T22:00:26.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.076+0000 7f1c737fe700 1 -- 172.21.15.22:0/2652710795 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1c8802da60 con 0x55e77fedb6f0 2023-09-20T22:00:26.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.076+0000 7f1cadd82b80 10 client.26089 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:00:26.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.076+0000 7f1cadd82b80 10 client.26089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:00:26.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.076+0000 7f1cadd82b80 20 client.26089 populate_metadata read hostname 'smithi022' 2023-09-20T22:00:26.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.076+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e77fef0fd0 0x55e77fef33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.076+0000 7f1cadd82b80 4 client.26089 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T22:00:26.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.076+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2652710795 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e77feaa020 con 0x55e77fef0fd0 2023-09-20T22:00:26.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.077+0000 7f1c937fe700 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e77fef0fd0 0x55e77fef33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.078+0000 7f1c937fe700 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e77fef0fd0 0x55e77fef33f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a587a3fdc222d6c2 server_cookie=bcd7ae477489c610 in_seq=0 out_seq=0 2023-09-20T22:00:26.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.078+0000 7f1c737fe700 10 client.26089 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:00:26.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.078+0000 7f1c737fe700 1 -- 172.21.15.22:0/2652710795 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e77feaa020 con 0x55e77fef0fd0 2023-09-20T22:00:26.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.078+0000 7f1c737fe700 10 client.26089 handle_command_reply: tid=0 2023-09-20T22:00:26.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1cadd82b80 1 client.26089 shutdown 2023-09-20T22:00:26.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2652710795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 msgr2=0x55e77fee5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:26.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fee5310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802f000 tx=0x7f1c8801e000 comp rx=0 tx=0).stop 2023-09-20T22:00:26.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2652710795 shutdown_connections 2023-09-20T22:00:26.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e77fef0fd0 0x55e77fef33f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77feea880 0x55e77feecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedb6f0 0x55e77fee5310 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/2652710795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77fedc050 0x55e77fee5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2652710795 >> 172.21.15.22:0/2652710795 conn(0x55e77fe08000 msgr2=0x55e77fed5880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:26.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1c737fe700 0 client.26089 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:00:26.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.079+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2652710795 shutdown_connections 2023-09-20T22:00:26.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.080+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2652710795 wait complete. 2023-09-20T22:00:26.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.080+0000 7f1cadd82b80 20 client.26089 trim_cache size 0 max 16384 2023-09-20T22:00:26.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.107+0000 7f1cadd82b80 1 Processor -- start 2023-09-20T22:00:26.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.107+0000 7f1cadd82b80 1 -- start start 2023-09-20T22:00:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.107+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.107+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77feed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.107+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 0x55e77fedfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.107+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77fcad920 con 0x55e77feea880 2023-09-20T22:00:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.107+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77fd9dbb0 con 0x55e77fedb6f0 2023-09-20T22:00:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.107+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77fed7290 con 0x55e77fedc050 2023-09-20T22:00:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.108+0000 7f1c93fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.108+0000 7f1c93fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:57024/0 (socket says 172.21.15.22:57024) 2023-09-20T22:00:26.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.108+0000 7f1c93fff700 1 -- 172.21.15.22:0/2807378414 learned_addr learned my addr 172.21.15.22:0/2807378414 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:00:26.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.108+0000 7f1c937fe700 1 --2- 172.21.15.22:0/2807378414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77feed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.108+0000 7f1c9899c700 1 --2- 172.21.15.22:0/2807378414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 0x55e77fedfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.109+0000 7f1c937fe700 1 -- 172.21.15.22:0/2807378414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 msgr2=0x55e77ff00bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.109+0000 7f1c937fe700 1 --2- 172.21.15.22:0/2807378414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.109+0000 7f1c937fe700 1 -- 172.21.15.22:0/2807378414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 msgr2=0x55e77fedfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:26.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.109+0000 7f1c937fe700 1 --2- 172.21.15.22:0/2807378414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 0x55e77fedfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.109+0000 7f1c937fe700 1 -- 172.21.15.22:0/2807378414 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e77fc7e250 con 0x55e77fedc050 2023-09-20T22:00:26.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.109+0000 7f1c93fff700 1 --2- 172.21.15.22:0/2807378414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:00:26.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.109+0000 7f1c937fe700 1 --2- 172.21.15.22:0/2807378414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77feed590 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55e77fedc570 tx=0x7f1c840ca8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e8f14e70d4beac server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:26.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.109+0000 7f1c7affd700 1 -- 172.21.15.22:0/2807378414 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c840caea0 con 0x55e77fedc050 2023-09-20T22:00:26.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.109+0000 7f1c7affd700 1 -- 172.21.15.22:0/2807378414 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c8409a270 con 0x55e77fedc050 2023-09-20T22:00:26.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.109+0000 7f1c7affd700 1 -- 172.21.15.22:0/2807378414 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8409a4d0 con 0x55e77fedc050 2023-09-20T22:00:26.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.110+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2807378414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 msgr2=0x55e77feed590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:26.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.110+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/2807378414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77feed590 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55e77fedc570 tx=0x7f1c840ca8b0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.110+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2807378414 shutdown_connections 2023-09-20T22:00:26.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.110+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/2807378414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 0x55e77fedfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.110+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/2807378414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.110+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/2807378414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77feed590 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.110+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2807378414 >> 172.21.15.22:0/2807378414 conn(0x55e77fe08000 msgr2=0x55e77fee3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:26.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.111+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2807378414 shutdown_connections 2023-09-20T22:00:26.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.111+0000 7f1cadd82b80 1 -- 172.21.15.22:0/2807378414 wait complete. 2023-09-20T22:00:26.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.111+0000 7f1cadd82b80 1 -- start start 2023-09-20T22:00:26.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.111+0000 7f1cadd82b80 1 Processor -- start 2023-09-20T22:00:26.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77ff41530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1cadd82b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 0x55e77ff43be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77ff44320 con 0x55e77feea880 2023-09-20T22:00:26.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77ff44490 con 0x55e77fedb6f0 2023-09-20T22:00:26.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1cadd82b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e77ff44650 con 0x55e77fedc050 2023-09-20T22:00:26.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1cadd82b80 10 client.0 _ll_register_callbacks cb 0x55e77fd13a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:00:26.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c93fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c937fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77ff41530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c93fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:57038/0 (socket says 172.21.15.22:57038) 2023-09-20T22:00:26.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c93fff700 1 -- 172.21.15.22:0/1599046670 learned_addr learned my addr 172.21.15.22:0/1599046670 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:00:26.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c937fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77ff41530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:44946/0 (socket says 172.21.15.22:44946) 2023-09-20T22:00:26.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c9899c700 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 0x55e77ff43be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c93fff700 1 -- 172.21.15.22:0/1599046670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 msgr2=0x55e77ff41530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:26.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c93fff700 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77ff41530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c93fff700 1 -- 172.21.15.22:0/1599046670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 msgr2=0x55e77ff43be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:26.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c93fff700 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 0x55e77ff43be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c93fff700 1 -- 172.21.15.22:0/1599046670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e77fc40c60 con 0x55e77fedb6f0 2023-09-20T22:00:26.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c937fe700 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77ff41530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:00:26.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.112+0000 7f1c9899c700 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 0x55e77ff43be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:00:26.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.113+0000 7f1c93fff700 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802de40 tx=0x7f1c880224a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e78783eac5b0e0e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:26.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.113+0000 7f1c737fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T22:00:26.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.113+0000 7f1c737fe700 1 -- 172.21.15.22:0/1599046670 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8801c030 con 0x55e77fedb6f0 2023-09-20T22:00:26.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.113+0000 7f1cadd82b80 1 -- 172.21.15.22:0/1599046670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c8800bcd0 con 0x55e77fedb6f0 2023-09-20T22:00:26.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.113+0000 7f1c737fe700 1 -- 172.21.15.22:0/1599046670 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1c880227f0 con 0x55e77fedb6f0 2023-09-20T22:00:26.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.113+0000 7f1c737fe700 1 -- 172.21.15.22:0/1599046670 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c880229b0 con 0x55e77fedb6f0 2023-09-20T22:00:26.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.113+0000 7f1cadd82b80 10 client.26060 fetch_fsmap learned FSMap version 32 2023-09-20T22:00:26.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.113+0000 7f1cadd82b80 1 -- 172.21.15.22:0/1599046670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f1c941b66c0 con 0x55e77fedb6f0 2023-09-20T22:00:26.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.114+0000 7f1c737fe700 1 -- 172.21.15.22:0/1599046670 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c8800bcd0 con 0x55e77fedb6f0 2023-09-20T22:00:26.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.114+0000 7f1c737fe700 1 -- 172.21.15.22:0/1599046670 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1c8800f740 con 0x55e77fedb6f0 2023-09-20T22:00:26.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.114+0000 7f1cadd82b80 10 client.26060 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:00:26.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.114+0000 7f1cadd82b80 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:00:26.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.114+0000 7f1cadd82b80 20 client.26060 populate_metadata read hostname 'smithi022' 2023-09-20T22:00:26.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.114+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e77ff45c50 0x55e77ff48070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:26.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.114+0000 7f1cadd82b80 4 client.26060 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T22:00:26.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.114+0000 7f1cadd82b80 1 -- 172.21.15.22:0/1599046670 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e77fe25e60 con 0x55e77ff45c50 2023-09-20T22:00:26.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.114+0000 7f1c937fe700 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e77ff45c50 0x55e77ff48070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:26.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.115+0000 7f1c937fe700 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e77ff45c50 0x55e77ff48070 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea2e6875f0e25675 server_cookie=f296b60ebebe184f in_seq=0 out_seq=0 2023-09-20T22:00:26.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.115+0000 7f1c737fe700 10 client.26060 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:00:26.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.115+0000 7f1c737fe700 1 -- 172.21.15.22:0/1599046670 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e77fe25e60 con 0x55e77ff45c50 2023-09-20T22:00:26.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.115+0000 7f1c737fe700 10 client.26060 handle_command_reply: tid=0 2023-09-20T22:00:26.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.115+0000 7f1cadd82b80 1 client.26060 shutdown 2023-09-20T22:00:26.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.116+0000 7f1cadd82b80 1 -- 172.21.15.22:0/1599046670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 msgr2=0x55e77ff00bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:26.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.116+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802de40 tx=0x7f1c880224a0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.116+0000 7f1cadd82b80 1 -- 172.21.15.22:0/1599046670 shutdown_connections 2023-09-20T22:00:26.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.116+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55e77ff45c50 0x55e77ff48070 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.116+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e77feea880 0x55e77ff43be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.116+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e77fedb6f0 0x55e77ff00bd0 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1c8802de40 tx=0x7f1c880224a0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.116+0000 7f1cadd82b80 1 --2- 172.21.15.22:0/1599046670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e77fedc050 0x55e77ff41530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.116+0000 7f1cadd82b80 1 -- 172.21.15.22:0/1599046670 >> 172.21.15.22:0/1599046670 conn(0x55e77fe08000 msgr2=0x55e77feed590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:26.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.116+0000 7f1c737fe700 0 client.26060 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:00:26.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.117+0000 7f1cadd82b80 1 -- 172.21.15.22:0/1599046670 shutdown_connections 2023-09-20T22:00:26.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.117+0000 7f1cadd82b80 1 -- 172.21.15.22:0/1599046670 wait complete. 2023-09-20T22:00:26.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.118+0000 7f1cadd82b80 20 client.26060 trim_cache size 0 max 16384 2023-09-20T22:00:26.132 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:00:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T22:00:26.132 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T22:00:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T22:00:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T22:00:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T22:00:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T22:00:26.133 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:00:26.134 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:00:26.134 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:00:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.118+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1c7c0a1420 msgr2=0x7f1c7c0a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:26.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.118+0000 7f1c9a3ff700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1c7c0a1420 0x7f1c7c0a38e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009990 tx=0x7f1c88005040 comp rx=0 tx=0).stop 2023-09-20T22:00:26.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.118+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/1857004136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9410a750 msgr2=0x7f1c94114880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:26.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.118+0000 7f1c9a3ff700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9410a750 0x7f1c94114880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400e9a0 tx=0x7f1c8400ed60 comp rx=0 tx=0).stop 2023-09-20T22:00:26.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.119+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/1857004136 shutdown_connections 2023-09-20T22:00:26.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.119+0000 7f1c9a3ff700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9410d120 0x7f1c941b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.119+0000 7f1c9a3ff700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c94107de0 0x7f1c94114340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.119+0000 7f1c9a3ff700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1c7c0a1420 0x7f1c7c0a38e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.119+0000 7f1c9a3ff700 1 --2- 172.21.15.22:0/1857004136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c9410a750 0x7f1c94114880 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.119+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/1857004136 >> 172.21.15.22:0/1857004136 conn(0x7f1c9406db90 msgr2=0x7f1c9410b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.119+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/1857004136 shutdown_connections 2023-09-20T22:00:26.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:26.119+0000 7f1c9a3ff700 1 -- 172.21.15.22:0/1857004136 wait complete. 2023-09-20T22:00:26.144 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T22:00:27.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:26 smithi022 ceph-mon[97651]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.6 MiB/s wr, 516 op/s 2023-09-20T22:00:27.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:26 smithi123 ceph-mon[134433]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.6 MiB/s wr, 516 op/s 2023-09-20T22:00:27.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:26 smithi089 ceph-mon[134090]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 7.6 MiB/s wr, 516 op/s 2023-09-20T22:00:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 665.0 s, 483.1 tps, lat 203.197 ms stddev 235.358 2023-09-20T22:00:29.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:29 smithi089 ceph-mon[134090]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.4 MiB/s wr, 461 op/s 2023-09-20T22:00:29.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:29 smithi022 ceph-mon[97651]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.4 MiB/s wr, 461 op/s 2023-09-20T22:00:29.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:29 smithi123 ceph-mon[134433]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.4 MiB/s wr, 461 op/s 2023-09-20T22:00:31.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:31 smithi022 ceph-mon[97651]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.4 MiB/s wr, 509 op/s 2023-09-20T22:00:31.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:31 smithi123 ceph-mon[134433]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.4 MiB/s wr, 509 op/s 2023-09-20T22:00:31.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:31 smithi089 ceph-mon[134090]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.4 MiB/s wr, 509 op/s 2023-09-20T22:00:33.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 670.0 s, 580.1 tps, lat 176.726 ms stddev 183.759 2023-09-20T22:00:33.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:33 smithi022 ceph-mon[97651]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.1 MiB/s wr, 520 op/s 2023-09-20T22:00:33.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:33 smithi123 ceph-mon[134433]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.1 MiB/s wr, 520 op/s 2023-09-20T22:00:33.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:33 smithi089 ceph-mon[134090]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.1 MiB/s wr, 520 op/s 2023-09-20T22:00:34.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:34 smithi022 ceph-mon[97651]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.1 MiB/s wr, 460 op/s 2023-09-20T22:00:34.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:34 smithi123 ceph-mon[134433]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.1 MiB/s wr, 460 op/s 2023-09-20T22:00:34.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:34 smithi089 ceph-mon[134090]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.1 MiB/s wr, 460 op/s 2023-09-20T22:00:37.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:37 smithi123 ceph-mon[134433]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.0 MiB/s wr, 517 op/s 2023-09-20T22:00:37.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:37 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:00:37.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:37 smithi089 ceph-mon[134090]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.0 MiB/s wr, 517 op/s 2023-09-20T22:00:37.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:37 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:00:37.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:37 smithi022 ceph-mon[97651]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.0 MiB/s wr, 517 op/s 2023-09-20T22:00:37.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:37 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:00:38.570 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 675.0 s, 459.9 tps, lat 214.095 ms stddev 235.738 2023-09-20T22:00:38.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:38 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:00:38.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:38 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:00:38.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:38 smithi022 ceph-mon[97651]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.0 MiB/s wr, 465 op/s 2023-09-20T22:00:38.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:38 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:00:38.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:38 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:00:38.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:38 smithi089 ceph-mon[134090]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.0 MiB/s wr, 465 op/s 2023-09-20T22:00:38.951 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:38 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:00:38.951 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:38 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:00:38.951 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:38 smithi123 ceph-mon[134433]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.0 MiB/s wr, 465 op/s 2023-09-20T22:00:39.751 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:00:39.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:00:39.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:39 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:00:39.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:00:39.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:00:39.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:39 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:00:39.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:00:39.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:00:39.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:39 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:00:40.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:40 smithi022 ceph-mon[97651]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.0 MiB/s wr, 529 op/s 2023-09-20T22:00:40.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:40 smithi123 ceph-mon[134433]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.0 MiB/s wr, 529 op/s 2023-09-20T22:00:40.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:40 smithi089 ceph-mon[134090]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.0 MiB/s wr, 529 op/s 2023-09-20T22:00:43.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:42 smithi022 ceph-mon[97651]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.4 MiB/s wr, 512 op/s 2023-09-20T22:00:43.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:42 smithi089 ceph-mon[134090]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.4 MiB/s wr, 512 op/s 2023-09-20T22:00:43.363 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:42 smithi123 ceph-mon[134433]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.4 MiB/s wr, 512 op/s 2023-09-20T22:00:43.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 680.0 s, 447.6 tps, lat 223.283 ms stddev 267.975 2023-09-20T22:00:45.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:45 smithi022 ceph-mon[97651]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.4 MiB/s wr, 463 op/s 2023-09-20T22:00:45.857 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:45 smithi123 ceph-mon[134433]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.4 MiB/s wr, 463 op/s 2023-09-20T22:00:45.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:45 smithi089 ceph-mon[134090]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.4 MiB/s wr, 463 op/s 2023-09-20T22:00:47.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:47 smithi022 ceph-mon[97651]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.3 MiB/s wr, 532 op/s 2023-09-20T22:00:47.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:47 smithi123 ceph-mon[134433]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.3 MiB/s wr, 532 op/s 2023-09-20T22:00:47.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:47 smithi089 ceph-mon[134090]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.3 MiB/s wr, 532 op/s 2023-09-20T22:00:48.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 685.0 s, 496.0 tps, lat 202.851 ms stddev 216.640 2023-09-20T22:00:49.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:49 smithi022 ceph-mon[97651]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.4 MiB/s wr, 465 op/s 2023-09-20T22:00:49.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:49 smithi123 ceph-mon[134433]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.4 MiB/s wr, 465 op/s 2023-09-20T22:00:49.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:49 smithi089 ceph-mon[134090]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.4 MiB/s wr, 465 op/s 2023-09-20T22:00:50.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:50 smithi123 ceph-mon[134433]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.3 MiB/s wr, 496 op/s 2023-09-20T22:00:50.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:50 smithi089 ceph-mon[134090]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.3 MiB/s wr, 496 op/s 2023-09-20T22:00:51.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:50 smithi022 ceph-mon[97651]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.3 MiB/s wr, 496 op/s 2023-09-20T22:00:53.255 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:52 smithi022 ceph-mon[97651]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.3 MiB/s wr, 517 op/s 2023-09-20T22:00:53.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:52 smithi123 ceph-mon[134433]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.3 MiB/s wr, 517 op/s 2023-09-20T22:00:53.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:52 smithi089 ceph-mon[134090]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.3 MiB/s wr, 517 op/s 2023-09-20T22:00:53.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 690.0 s, 456.4 tps, lat 212.358 ms stddev 229.739 2023-09-20T22:00:55.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:55 smithi022 ceph-mon[97651]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2023-09-20T22:00:55.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:55 smithi123 ceph-mon[134433]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2023-09-20T22:00:55.865 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:55 smithi089 ceph-mon[134090]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.2 MiB/s wr, 486 op/s 2023-09-20T22:00:56.145 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T22:00:56.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.611+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3529212080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b0810a750 msgr2=0x7f1b0810cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:56.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.611+0000 7f1b0fbff700 1 --2- 172.21.15.22:0/3529212080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b0810a750 0x7f1b0810cbe0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009f90 tx=0x7f1af800b330 comp rx=0 tx=0).stop 2023-09-20T22:00:56.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.628+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3529212080 shutdown_connections 2023-09-20T22:00:56.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.628+0000 7f1b0fbff700 1 --2- 172.21.15.22:0/3529212080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0810d120 0x7f1b0810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.628+0000 7f1b0fbff700 1 --2- 172.21.15.22:0/3529212080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b08107de0 0x7f1b0810a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.628+0000 7f1b0fbff700 1 --2- 172.21.15.22:0/3529212080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b0810a750 0x7f1b0810cbe0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.628+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3529212080 >> 172.21.15.22:0/3529212080 conn(0x7f1b0806db90 msgr2=0x7f1b0806ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:56.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.642+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3529212080 shutdown_connections 2023-09-20T22:00:56.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.643+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3529212080 wait complete. 2023-09-20T22:00:56.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.644+0000 7f1b0fbff700 1 Processor -- start 2023-09-20T22:00:56.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.644+0000 7f1b0fbff700 1 -- start start 2023-09-20T22:00:56.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.645+0000 7f1b0fbff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b08107de0 0x7f1b081142b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.645+0000 7f1b0fbff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b0810a750 0x7f1b081147f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.645+0000 7f1b0fbff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0810d120 0x7f1b081b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.645+0000 7f1b0fbff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b081178d0 con 0x7f1b08107de0 2023-09-20T22:00:56.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.645+0000 7f1b0fbff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b08117a40 con 0x7f1b0810a750 2023-09-20T22:00:56.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.645+0000 7f1b0fbff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b081b60b0 con 0x7f1b0810d120 2023-09-20T22:00:56.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.651+0000 7f1b0e19c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0810d120 0x7f1b081b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.651+0000 7f1b0e19c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0810d120 0x7f1b081b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:35086/0 (socket says 172.21.15.22:35086) 2023-09-20T22:00:56.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.651+0000 7f1b0e19c700 1 -- 172.21.15.22:0/3823770937 learned_addr learned my addr 172.21.15.22:0/3823770937 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:00:56.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.653+0000 7f1b0d99b700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b08107de0 0x7f1b081142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.658+0000 7f1b0e19c700 1 -- 172.21.15.22:0/3823770937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b0810a750 msgr2=0x7f1b081147f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-20T22:00:56.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.658+0000 7f1b0e19c700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b0810a750 0x7f1b081147f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.658+0000 7f1b0e19c700 1 -- 172.21.15.22:0/3823770937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b08107de0 msgr2=0x7f1b081142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:56.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.658+0000 7f1b0e19c700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b08107de0 0x7f1b081142b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.658+0000 7f1b0e19c700 1 -- 172.21.15.22:0/3823770937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af8009c40 con 0x7f1b0810d120 2023-09-20T22:00:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.659+0000 7f1b0e19c700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0810d120 0x7f1b081b5b70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00d980 tx=0x7f1afc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.661+0000 7f1b06ffd700 1 -- 172.21.15.22:0/3823770937 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc004a30 con 0x7f1b0810d120 2023-09-20T22:00:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.661+0000 7f1b06ffd700 1 -- 172.21.15.22:0/3823770937 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1afc0054e0 con 0x7f1b0810d120 2023-09-20T22:00:56.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.661+0000 7f1b06ffd700 1 -- 172.21.15.22:0/3823770937 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc009bd0 con 0x7f1b0810d120 2023-09-20T22:00:56.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.661+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3823770937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b081b61f0 con 0x7f1b0810d120 2023-09-20T22:00:56.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.661+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3823770937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b081b66a0 con 0x7f1b0810d120 2023-09-20T22:00:56.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.668+0000 7f1b23536b80 1 Processor -- start 2023-09-20T22:00:56.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.675+0000 7f1b06ffd700 1 -- 172.21.15.22:0/3823770937 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1afc009d30 con 0x7f1b0810d120 2023-09-20T22:00:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.675+0000 7f1b23536b80 1 -- start start 2023-09-20T22:00:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.676+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.676+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccd050 0x55d8e0ccd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.676+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.676+0000 7f1b23536b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0c40ca0 con 0x55d8e0ccc6f0 2023-09-20T22:00:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.676+0000 7f1b23536b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0c24c70 con 0x55d8e0ccd050 2023-09-20T22:00:56.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.676+0000 7f1b23536b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0c22590 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.676+0000 7f1b0d19a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.676+0000 7f1b0d19a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.676+0000 7f1b0d19a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36592/0 (socket says 172.21.15.22:36592) 2023-09-20T22:00:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.676+0000 7f1b0d19a700 1 -- 172.21.15.22:0/4217123566 learned_addr learned my addr 172.21.15.22:0/4217123566 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:00:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.677+0000 7f1b0d19a700 1 -- 172.21.15.22:0/4217123566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccd050 msgr2=0x55d8e0ccd5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T22:00:56.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.677+0000 7f1b0d19a700 1 --2- 172.21.15.22:0/4217123566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccd050 0x55d8e0ccd5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.677+0000 7f1b0d19a700 1 -- 172.21.15.22:0/4217123566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccc6f0 msgr2=0x55d8e0cccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.677+0000 7f1b0d19a700 1 --2- 172.21.15.22:0/4217123566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.677+0000 7f1b0d19a700 1 -- 172.21.15.22:0/4217123566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8e0a48de0 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.678+0000 7f1b0d19a700 1 --2- 172.21.15.22:0/4217123566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cce060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1af8005bd0 tx=0x7f1af8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b52b2484722b22d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.678+0000 7f1b06ffd700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1af00a1500 0x7f1af00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.678+0000 7f1b06ffd700 1 -- 172.21.15.22:0/3823770937 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1afc0cca70 con 0x7f1b0810d120 2023-09-20T22:00:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.678+0000 7f1aeffff700 1 -- 172.21.15.22:0/4217123566 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af800e070 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.678+0000 7f1aeffff700 1 -- 172.21.15.22:0/4217123566 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1af800be70 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.678+0000 7f1aeffff700 1 -- 172.21.15.22:0/4217123566 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8005560 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.680+0000 7f1b23536b80 1 -- 172.21.15.22:0/4217123566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 msgr2=0x55d8e0cce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.680+0000 7f1b23536b80 1 --2- 172.21.15.22:0/4217123566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cce060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1af8005bd0 tx=0x7f1af8000f40 comp rx=0 tx=0).stop 2023-09-20T22:00:56.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.691+0000 7f1b23536b80 1 -- 172.21.15.22:0/4217123566 shutdown_connections 2023-09-20T22:00:56.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.691+0000 7f1b23536b80 1 --2- 172.21.15.22:0/4217123566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cce060 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.691+0000 7f1b23536b80 1 --2- 172.21.15.22:0/4217123566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.691+0000 7f1b23536b80 1 --2- 172.21.15.22:0/4217123566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccd050 0x55d8e0ccd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.691+0000 7f1b23536b80 1 -- 172.21.15.22:0/4217123566 >> 172.21.15.22:0/4217123566 conn(0x55d8e0bf9000 msgr2=0x55d8e0cc71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:56.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.695+0000 7f1b0d99b700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1af00a1500 0x7f1af00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.709+0000 7f1b23536b80 1 -- 172.21.15.22:0/4217123566 shutdown_connections 2023-09-20T22:00:56.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.719+0000 7f1b23536b80 1 -- 172.21.15.22:0/4217123566 wait complete. 2023-09-20T22:00:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.720+0000 7f1b23536b80 1 -- start start 2023-09-20T22:00:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.720+0000 7f1b23536b80 1 Processor -- start 2023-09-20T22:00:56.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.725+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cd6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.725+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cd6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.725+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cdd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.725+0000 7f1b23536b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0c1de80 con 0x55d8e0ccd050 2023-09-20T22:00:56.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.726+0000 7f1b23536b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0bba740 con 0x55d8e0ccc6f0 2023-09-20T22:00:56.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.726+0000 7f1b23536b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0bba3e0 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.726+0000 7f1b23536b80 10 client.0 _ll_register_callbacks cb 0x55d8e0b04a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:00:56.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.726+0000 7f1b0d19a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cd6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.726+0000 7f1b0d19a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cd6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36608/0 (socket says 172.21.15.22:36608) 2023-09-20T22:00:56.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.726+0000 7f1b0d19a700 1 -- 172.21.15.22:0/676840080 learned_addr learned my addr 172.21.15.22:0/676840080 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:00:56.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.733+0000 7f1b0d99b700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1af00a1500 0x7f1af00a39c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55d8e0cc8150 tx=0x7f1b00009190 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:56.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.748+0000 7f1b0e19c700 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cdd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.749+0000 7f1b0d99b700 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cd6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.751+0000 7f1b0e19c700 1 -- 172.21.15.22:0/676840080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 msgr2=0x55d8e0cd6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.751+0000 7f1b0e19c700 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cd6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.751+0000 7f1b0e19c700 1 -- 172.21.15.22:0/676840080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 msgr2=0x55d8e0cd6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.751+0000 7f1b0e19c700 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cd6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.751+0000 7f1b0e19c700 1 -- 172.21.15.22:0/676840080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0810e120 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.751+0000 7f1b0e19c700 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cdd6c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0ca180 tx=0x7f1afc0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba4628dbbf332fdb server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.753+0000 7f1aed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T22:00:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.753+0000 7f1aed7fa700 1 -- 172.21.15.22:0/676840080 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc09d840 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.753+0000 7f1aed7fa700 1 -- 172.21.15.22:0/676840080 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1afc0a1a80 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.753+0000 7f1b23536b80 1 -- 172.21.15.22:0/676840080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8e0b8ebb0 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.764+0000 7f1aed7fa700 1 -- 172.21.15.22:0/676840080 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1afc004b90 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.775+0000 7f1b23536b80 10 client.26107 fetch_fsmap learned FSMap version 32 2023-09-20T22:00:56.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.775+0000 7f1b23536b80 1 -- 172.21.15.22:0/676840080 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d8e0a31c60 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.790+0000 7f1aed7fa700 1 -- 172.21.15.22:0/676840080 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8e0b8ebb0 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.790+0000 7f1aed7fa700 1 -- 172.21.15.22:0/676840080 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1afc0a15f0 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.790+0000 7f1b23536b80 10 client.26107 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:00:56.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.790+0000 7f1b23536b80 10 client.26107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:00:56.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.790+0000 7f1b23536b80 20 client.26107 populate_metadata read hostname 'smithi022' 2023-09-20T22:00:56.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.790+0000 7f1b23536b80 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d8e0ce1eb0 0x55d8e0ce42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.790+0000 7f1b23536b80 4 client.26107 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T22:00:56.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.790+0000 7f1b23536b80 1 -- 172.21.15.22:0/676840080 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8e0c9b020 con 0x55d8e0ce1eb0 2023-09-20T22:00:56.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.799+0000 7f1b0d99b700 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d8e0ce1eb0 0x55d8e0ce42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.826+0000 7f1b0d99b700 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d8e0ce1eb0 0x55d8e0ce42a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1455704599589e4 server_cookie=b40ac0fa0bde7c22 in_seq=0 out_seq=0 2023-09-20T22:00:56.828 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.828+0000 7f1aed7fa700 10 client.26107 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:00:56.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.830+0000 7f1aed7fa700 1 -- 172.21.15.22:0/676840080 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d8e0c9b020 con 0x55d8e0ce1eb0 2023-09-20T22:00:56.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.830+0000 7f1aed7fa700 10 client.26107 handle_command_reply: tid=0 2023-09-20T22:00:56.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.831+0000 7f1b23536b80 1 client.26107 shutdown 2023-09-20T22:00:56.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.833+0000 7f1b23536b80 1 -- 172.21.15.22:0/676840080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 msgr2=0x55d8e0cdd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:56.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.833+0000 7f1b23536b80 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cdd6c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0ca180 tx=0x7f1afc0056d0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.842+0000 7f1b23536b80 1 -- 172.21.15.22:0/676840080 shutdown_connections 2023-09-20T22:00:56.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.842+0000 7f1b23536b80 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d8e0ce1eb0 0x55d8e0ce42a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.842+0000 7f1b23536b80 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cdd6c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.842+0000 7f1b23536b80 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cd6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.842+0000 7f1b23536b80 1 --2- 172.21.15.22:0/676840080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cd6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.842+0000 7f1b23536b80 1 -- 172.21.15.22:0/676840080 >> 172.21.15.22:0/676840080 conn(0x55d8e0bf9000 msgr2=0x55d8e0cc6880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.847+0000 7f1aed7fa700 0 client.26107 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:00:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.849+0000 7f1b23536b80 1 -- 172.21.15.22:0/676840080 shutdown_connections 2023-09-20T22:00:56.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.853+0000 7f1b23536b80 1 -- 172.21.15.22:0/676840080 wait complete. 2023-09-20T22:00:56.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.861+0000 7f1b23536b80 20 client.26107 trim_cache size 0 max 16384 2023-09-20T22:00:56.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.907+0000 7f1b23536b80 1 Processor -- start 2023-09-20T22:00:56.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.917+0000 7f1b23536b80 1 -- start start 2023-09-20T22:00:56.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.917+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0ce3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.917+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cdd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.917+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cd0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.917+0000 7f1b23536b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0a9e920 con 0x55d8e0ccd050 2023-09-20T22:00:56.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.917+0000 7f1b23536b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0cc85f0 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.917+0000 7f1b23536b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0cdf3b0 con 0x55d8e0ccc6f0 2023-09-20T22:00:56.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.926+0000 7f1b0d19a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cdd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.926+0000 7f1b0d19a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cdd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36618/0 (socket says 172.21.15.22:36618) 2023-09-20T22:00:56.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.926+0000 7f1b0d19a700 1 -- 172.21.15.22:0/2273464434 learned_addr learned my addr 172.21.15.22:0/2273464434 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:00:56.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.932+0000 7f1b0e19c700 1 --2- 172.21.15.22:0/2273464434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cd0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.938+0000 7f1b0d19a700 1 -- 172.21.15.22:0/2273464434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccc6f0 msgr2=0x55d8e0ce3ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T22:00:56.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.938+0000 7f1b0d19a700 1 --2- 172.21.15.22:0/2273464434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0ce3ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.938+0000 7f1b0d19a700 1 -- 172.21.15.22:0/2273464434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccdbe0 msgr2=0x55d8e0cd0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:56.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.938+0000 7f1b0d19a700 1 --2- 172.21.15.22:0/2273464434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cd0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.938+0000 7f1b0d19a700 1 -- 172.21.15.22:0/2273464434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8e0a6f250 con 0x55d8e0ccd050 2023-09-20T22:00:56.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.939+0000 7f1b0d19a700 1 --2- 172.21.15.22:0/2273464434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cdd680 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1af800bd50 tx=0x7f1af800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4b897f5f32bf87 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:56.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.940+0000 7f1b04bff700 1 -- 172.21.15.22:0/2273464434 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af800e040 con 0x55d8e0ccd050 2023-09-20T22:00:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.940+0000 7f1b04bff700 1 -- 172.21.15.22:0/2273464434 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1af80073e0 con 0x55d8e0ccd050 2023-09-20T22:00:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.940+0000 7f1b04bff700 1 -- 172.21.15.22:0/2273464434 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af80055d0 con 0x55d8e0ccd050 2023-09-20T22:00:56.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.941+0000 7f1b23536b80 1 -- 172.21.15.22:0/2273464434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 msgr2=0x55d8e0cdd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:56.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.941+0000 7f1b23536b80 1 --2- 172.21.15.22:0/2273464434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cdd680 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1af800bd50 tx=0x7f1af800bb70 comp rx=0 tx=0).stop 2023-09-20T22:00:56.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.957+0000 7f1b23536b80 1 -- 172.21.15.22:0/2273464434 shutdown_connections 2023-09-20T22:00:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.958+0000 7f1b23536b80 1 --2- 172.21.15.22:0/2273464434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cd0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.958+0000 7f1b23536b80 1 --2- 172.21.15.22:0/2273464434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0ce3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.958+0000 7f1b23536b80 1 --2- 172.21.15.22:0/2273464434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cdd680 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.958+0000 7f1b23536b80 1 -- 172.21.15.22:0/2273464434 >> 172.21.15.22:0/2273464434 conn(0x55d8e0bf9000 msgr2=0x55d8e0cd4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:56.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.967+0000 7f1b23536b80 1 -- 172.21.15.22:0/2273464434 shutdown_connections 2023-09-20T22:00:56.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.970+0000 7f1b23536b80 1 -- 172.21.15.22:0/2273464434 wait complete. 2023-09-20T22:00:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.971+0000 7f1b23536b80 1 -- start start 2023-09-20T22:00:56.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.971+0000 7f1b23536b80 1 Processor -- start 2023-09-20T22:00:56.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.977+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cdb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.977+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cdbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.977+0000 7f1b23536b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cdcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.977+0000 7f1b23536b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0cdd000 con 0x55d8e0ccd050 2023-09-20T22:00:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.977+0000 7f1b23536b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0cdd170 con 0x55d8e0ccc6f0 2023-09-20T22:00:56.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.977+0000 7f1b23536b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e0cdd330 con 0x55d8e0ccdbe0 2023-09-20T22:00:56.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.977+0000 7f1b23536b80 10 client.0 _ll_register_callbacks cb 0x55d8e0b04a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:00:56.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.977+0000 7f1b0d99b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cdb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.977+0000 7f1b0d99b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cdb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:59404/0 (socket says 172.21.15.22:59404) 2023-09-20T22:00:56.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.978+0000 7f1b0d99b700 1 -- 172.21.15.22:0/3284208059 learned_addr learned my addr 172.21.15.22:0/3284208059 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:00:56.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.978+0000 7f1b0e19c700 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cdcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:56.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.983+0000 7f1b0d99b700 1 -- 172.21.15.22:0/3284208059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 msgr2=0x55d8e0cdcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:56.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.983+0000 7f1b0d99b700 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cdcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.983+0000 7f1b0d99b700 1 -- 172.21.15.22:0/3284208059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 msgr2=0x55d8e0cdbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:00:56.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.984+0000 7f1b0d99b700 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cdbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:56.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.984+0000 7f1b0d99b700 1 -- 172.21.15.22:0/3284208059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8e0a48de0 con 0x55d8e0ccc6f0 2023-09-20T22:00:56.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.986+0000 7f1b0d99b700 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cdb570 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55d8e0cc8010 tx=0x7f1b00004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad12da30ad56d87 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:00:56.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.987+0000 7f1aed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T22:00:56.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.989+0000 7f1aed7fa700 1 -- 172.21.15.22:0/3284208059 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00025440 con 0x55d8e0ccc6f0 2023-09-20T22:00:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.989+0000 7f1aed7fa700 1 -- 172.21.15.22:0/3284208059 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1b0000e810 con 0x55d8e0ccc6f0 2023-09-20T22:00:56.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.990+0000 7f1b23536b80 1 -- 172.21.15.22:0/3284208059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1af800be70 con 0x55d8e0ccc6f0 2023-09-20T22:00:57.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.995+0000 7f1aed7fa700 1 -- 172.21.15.22:0/3284208059 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b000255a0 con 0x55d8e0ccc6f0 2023-09-20T22:00:57.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.995+0000 7f1b23536b80 10 client.26069 fetch_fsmap learned FSMap version 32 2023-09-20T22:00:57.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:56.995+0000 7f1b23536b80 1 -- 172.21.15.22:0/3284208059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d8e0a472b0 con 0x55d8e0ccc6f0 2023-09-20T22:00:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.008+0000 7f1aed7fa700 1 -- 172.21.15.22:0/3284208059 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1af800be70 con 0x55d8e0ccc6f0 2023-09-20T22:00:57.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.008+0000 7f1aed7fa700 1 -- 172.21.15.22:0/3284208059 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1b00015ad0 con 0x55d8e0ccc6f0 2023-09-20T22:00:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.008+0000 7f1b23536b80 10 client.26069 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:00:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.008+0000 7f1b23536b80 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:00:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.008+0000 7f1b23536b80 20 client.26069 populate_metadata read hostname 'smithi022' 2023-09-20T22:00:57.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.008+0000 7f1b23536b80 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d8e0d36df0 0x55d8e0d39210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:00:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.008+0000 7f1b23536b80 4 client.26069 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T22:00:57.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.008+0000 7f1b23536b80 1 -- 172.21.15.22:0/3284208059 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d8e0c16e60 con 0x55d8e0d36df0 2023-09-20T22:00:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.009+0000 7f1b0d19a700 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d8e0d36df0 0x55d8e0d39210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:00:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.010+0000 7f1b0d19a700 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d8e0d36df0 0x55d8e0d39210 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e95ffce194404d0 server_cookie=a5f537ba746942d5 in_seq=0 out_seq=0 2023-09-20T22:00:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.010+0000 7f1aed7fa700 10 client.26069 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:00:57.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.010+0000 7f1aed7fa700 1 -- 172.21.15.22:0/3284208059 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d8e0c16e60 con 0x55d8e0d36df0 2023-09-20T22:00:57.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.010+0000 7f1aed7fa700 10 client.26069 handle_command_reply: tid=0 2023-09-20T22:00:57.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.010+0000 7f1b23536b80 1 client.26069 shutdown 2023-09-20T22:00:57.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.011+0000 7f1b23536b80 1 -- 172.21.15.22:0/3284208059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 msgr2=0x55d8e0cdb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:57.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.011+0000 7f1b23536b80 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cdb570 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55d8e0cc8010 tx=0x7f1b00004860 comp rx=0 tx=0).stop 2023-09-20T22:00:57.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.011+0000 7f1b23536b80 1 -- 172.21.15.22:0/3284208059 shutdown_connections 2023-09-20T22:00:57.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.011+0000 7f1b23536b80 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55d8e0d36df0 0x55d8e0d39210 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:57.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.011+0000 7f1b23536b80 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8e0ccdbe0 0x55d8e0cdcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:57.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.011+0000 7f1b23536b80 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8e0ccc6f0 0x55d8e0cdb570 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:57.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.011+0000 7f1b23536b80 1 --2- 172.21.15.22:0/3284208059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d8e0ccd050 0x55d8e0cdbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:57.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.011+0000 7f1aed7fa700 0 client.26069 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:00:57.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.011+0000 7f1b23536b80 1 -- 172.21.15.22:0/3284208059 >> 172.21.15.22:0/3284208059 conn(0x55d8e0bf9000 msgr2=0x55d8e0ce3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:57.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.011+0000 7f1b23536b80 1 -- 172.21.15.22:0/3284208059 shutdown_connections 2023-09-20T22:00:57.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.012+0000 7f1b23536b80 1 -- 172.21.15.22:0/3284208059 wait complete. 2023-09-20T22:00:57.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.013+0000 7f1b23536b80 20 client.26069 trim_cache size 0 max 16384 2023-09-20T22:00:57.019 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:00:57.020 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T22:00:57.020 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T22:00:57.020 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T22:00:57.020 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T22:00:57.020 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T22:00:57.021 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T22:00:57.021 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:00:57.021 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:00:57.021 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:00:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.020+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3823770937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1af00a1500 msgr2=0x7f1af00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.020+0000 7f1b0fbff700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1af00a1500 0x7f1af00a39c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55d8e0cc8150 tx=0x7f1b00009190 comp rx=0 tx=0).stop 2023-09-20T22:00:57.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.020+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3823770937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0810d120 msgr2=0x7f1b081b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:00:57.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.020+0000 7f1b0fbff700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0810d120 0x7f1b081b5b70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00d980 tx=0x7f1afc00dc90 comp rx=0 tx=0).stop 2023-09-20T22:00:57.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.021+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3823770937 shutdown_connections 2023-09-20T22:00:57.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.021+0000 7f1b0fbff700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b0810d120 0x7f1b081b5b70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:57.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.021+0000 7f1b0fbff700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1b08107de0 0x7f1b081142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:57.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.021+0000 7f1b0fbff700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1af00a1500 0x7f1af00a39c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:57.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.021+0000 7f1b0fbff700 1 --2- 172.21.15.22:0/3823770937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b0810a750 0x7f1b081147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:00:57.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.021+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3823770937 >> 172.21.15.22:0/3823770937 conn(0x7f1b0806db90 msgr2=0x7f1b08108a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:00:57.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.021+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3823770937 shutdown_connections 2023-09-20T22:00:57.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:00:57.021+0000 7f1b0fbff700 1 -- 172.21.15.22:0/3823770937 wait complete. 2023-09-20T22:00:57.047 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T22:00:57.750 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:57 smithi022 ceph-mon[97651]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.3 MiB/s wr, 534 op/s 2023-09-20T22:00:57.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:57 smithi123 ceph-mon[134433]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.3 MiB/s wr, 534 op/s 2023-09-20T22:00:57.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:57 smithi089 ceph-mon[134090]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.3 MiB/s wr, 534 op/s 2023-09-20T22:00:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 695.0 s, 474.8 tps, lat 212.482 ms stddev 252.804 2023-09-20T22:00:59.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:00:59 smithi022 ceph-mon[97651]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.1 MiB/s wr, 474 op/s 2023-09-20T22:00:59.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:00:59 smithi123 ceph-mon[134433]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.1 MiB/s wr, 474 op/s 2023-09-20T22:00:59.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:00:59 smithi089 ceph-mon[134090]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.1 MiB/s wr, 474 op/s 2023-09-20T22:01:00.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:00 smithi123 ceph-mon[134433]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.2 MiB/s wr, 511 op/s 2023-09-20T22:01:00.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:00 smithi089 ceph-mon[134090]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.2 MiB/s wr, 511 op/s 2023-09-20T22:01:00.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:00 smithi022 ceph-mon[97651]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.2 MiB/s wr, 511 op/s 2023-09-20T22:01:03.346 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:03 smithi123 ceph-mon[134433]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 545 op/s 2023-09-20T22:01:03.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:03 smithi089 ceph-mon[134090]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 545 op/s 2023-09-20T22:01:03.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:03 smithi022 ceph-mon[97651]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 545 op/s 2023-09-20T22:01:03.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 700.0 s, 574.4 tps, lat 178.084 ms stddev 206.926 2023-09-20T22:01:05.753 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:05 smithi022 ceph-mon[97651]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.0 MiB/s wr, 473 op/s 2023-09-20T22:01:05.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:05 smithi123 ceph-mon[134433]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.0 MiB/s wr, 473 op/s 2023-09-20T22:01:05.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:05 smithi089 ceph-mon[134090]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.0 MiB/s wr, 473 op/s 2023-09-20T22:01:07.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:07 smithi022 ceph-mon[97651]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.8 MiB/s wr, 507 op/s 2023-09-20T22:01:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:07 smithi123 ceph-mon[134433]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.8 MiB/s wr, 507 op/s 2023-09-20T22:01:07.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:07 smithi089 ceph-mon[134090]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.8 MiB/s wr, 507 op/s 2023-09-20T22:01:08.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 705.0 s, 488.9 tps, lat 196.781 ms stddev 228.988 2023-09-20T22:01:09.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:09 smithi022 ceph-mon[97651]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.1 MiB/s wr, 470 op/s 2023-09-20T22:01:09.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:09 smithi123 ceph-mon[134433]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.1 MiB/s wr, 470 op/s 2023-09-20T22:01:09.867 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:09 smithi089 ceph-mon[134090]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.1 MiB/s wr, 470 op/s 2023-09-20T22:01:10.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:10 smithi123 ceph-mon[134433]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.2 MiB/s wr, 494 op/s 2023-09-20T22:01:10.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:10 smithi089 ceph-mon[134090]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.2 MiB/s wr, 494 op/s 2023-09-20T22:01:10.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:10 smithi022 ceph-mon[97651]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.2 MiB/s wr, 494 op/s 2023-09-20T22:01:13.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:13 smithi123 ceph-mon[134433]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.6 MiB/s wr, 536 op/s 2023-09-20T22:01:13.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:13 smithi089 ceph-mon[134090]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.6 MiB/s wr, 536 op/s 2023-09-20T22:01:13.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:13 smithi022 ceph-mon[97651]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.6 MiB/s wr, 536 op/s 2023-09-20T22:01:13.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 710.0 s, 484.5 tps, lat 209.033 ms stddev 248.383 2023-09-20T22:01:15.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:15 smithi022 ceph-mon[97651]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.1 MiB/s wr, 485 op/s 2023-09-20T22:01:15.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:15 smithi123 ceph-mon[134433]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.1 MiB/s wr, 485 op/s 2023-09-20T22:01:15.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:15 smithi089 ceph-mon[134090]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.1 MiB/s wr, 485 op/s 2023-09-20T22:01:17.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:17 smithi022 ceph-mon[97651]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.9 MiB/s wr, 522 op/s 2023-09-20T22:01:17.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:17 smithi123 ceph-mon[134433]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.9 MiB/s wr, 522 op/s 2023-09-20T22:01:17.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:17 smithi089 ceph-mon[134090]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.9 MiB/s wr, 522 op/s 2023-09-20T22:01:18.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 715.0 s, 532.8 tps, lat 194.544 ms stddev 233.799 2023-09-20T22:01:19.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:19 smithi123 ceph-mon[134433]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.7 MiB/s wr, 474 op/s 2023-09-20T22:01:19.865 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:19 smithi089 ceph-mon[134090]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.7 MiB/s wr, 474 op/s 2023-09-20T22:01:19.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:19 smithi022 ceph-mon[97651]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 5.7 MiB/s wr, 474 op/s 2023-09-20T22:01:20.858 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:20 smithi123 ceph-mon[134433]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.8 MiB/s wr, 504 op/s 2023-09-20T22:01:20.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:20 smithi089 ceph-mon[134090]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.8 MiB/s wr, 504 op/s 2023-09-20T22:01:20.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:20 smithi022 ceph-mon[97651]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.8 MiB/s wr, 504 op/s 2023-09-20T22:01:23.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:23 smithi123 ceph-mon[134433]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.5 MiB/s wr, 551 op/s 2023-09-20T22:01:23.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:23 smithi089 ceph-mon[134090]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.5 MiB/s wr, 551 op/s 2023-09-20T22:01:23.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:23 smithi022 ceph-mon[97651]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.5 MiB/s wr, 551 op/s 2023-09-20T22:01:23.576 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 720.0 s, 508.9 tps, lat 195.074 ms stddev 224.671 2023-09-20T22:01:25.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:25 smithi022 ceph-mon[97651]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 5.3 MiB/s wr, 478 op/s 2023-09-20T22:01:25.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:25 smithi123 ceph-mon[134433]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 5.3 MiB/s wr, 478 op/s 2023-09-20T22:01:25.865 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:25 smithi089 ceph-mon[134090]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 5.3 MiB/s wr, 478 op/s 2023-09-20T22:01:27.048 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T22:01:27.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.518+0000 7f22e89ff700 1 -- 172.21.15.22:0/1112343925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22e40fec70 msgr2=0x7f22e40ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:27.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.518+0000 7f22e89ff700 1 --2- 172.21.15.22:0/1112343925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22e40fec70 0x7f22e40ff090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009990 tx=0x7f22dc009ca0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.525+0000 7f22e89ff700 1 -- 172.21.15.22:0/1112343925 shutdown_connections 2023-09-20T22:01:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.525+0000 7f22e89ff700 1 --2- 172.21.15.22:0/1112343925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e4100760 0x7f22e4105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.525+0000 7f22e89ff700 1 --2- 172.21.15.22:0/1112343925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22e40fec70 0x7f22e40ff090 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.525+0000 7f22e89ff700 1 --2- 172.21.15.22:0/1112343925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40ffda0 0x7f22e4100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.525+0000 7f22e89ff700 1 -- 172.21.15.22:0/1112343925 >> 172.21.15.22:0/1112343925 conn(0x7f22e40fa570 msgr2=0x7f22e40fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:27.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.538+0000 7f22e89ff700 1 -- 172.21.15.22:0/1112343925 shutdown_connections 2023-09-20T22:01:27.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.541+0000 7f22e89ff700 1 -- 172.21.15.22:0/1112343925 wait complete. 2023-09-20T22:01:27.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.542+0000 7f22e89ff700 1 Processor -- start 2023-09-20T22:01:27.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.543+0000 7f22e89ff700 1 -- start start 2023-09-20T22:01:27.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.543+0000 7f22e89ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40fec70 0x7f22e4071480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.543+0000 7f22e89ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22e40ffda0 0x7f22e40719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.543+0000 7f22e89ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e4100760 0x7f22e4071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.543+0000 7f22e89ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e410db00 con 0x7f22e4100760 2023-09-20T22:01:27.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.543+0000 7f22e89ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e410dc70 con 0x7f22e40fec70 2023-09-20T22:01:27.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.543+0000 7f22e89ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e410de30 con 0x7f22e40ffda0 2023-09-20T22:01:27.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.545+0000 7f22e1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22e40ffda0 0x7f22e40719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.545+0000 7f22e1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22e40ffda0 0x7f22e40719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:55688/0 (socket says 172.21.15.22:55688) 2023-09-20T22:01:27.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.545+0000 7f22e1d9b700 1 -- 172.21.15.22:0/87756872 learned_addr learned my addr 172.21.15.22:0/87756872 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:01:27.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.550+0000 7f22e259c700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40fec70 0x7f22e4071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.550+0000 7f22e1d9b700 1 -- 172.21.15.22:0/87756872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40fec70 msgr2=0x7f22e4071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:27.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.550+0000 7f22e1d9b700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40fec70 0x7f22e4071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.550+0000 7f22e1d9b700 1 -- 172.21.15.22:0/87756872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e4100760 msgr2=0x7f22e4071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:27.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.550+0000 7f22e1d9b700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e4100760 0x7f22e4071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.550+0000 7f22e1d9b700 1 -- 172.21.15.22:0/87756872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc0095f0 con 0x7f22e40ffda0 2023-09-20T22:01:27.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.551+0000 7f22e1d9b700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22e40ffda0 0x7f22e40719c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b530 tx=0x7f22cc00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:27.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.555+0000 7f22db7fe700 1 -- 172.21.15.22:0/87756872 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc011830 con 0x7f22e40ffda0 2023-09-20T22:01:27.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.556+0000 7f22db7fe700 1 -- 172.21.15.22:0/87756872 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22cc011e50 con 0x7f22e40ffda0 2023-09-20T22:01:27.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.556+0000 7f22db7fe700 1 -- 172.21.15.22:0/87756872 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00f2e0 con 0x7f22e40ffda0 2023-09-20T22:01:27.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.556+0000 7f22e89ff700 1 -- 172.21.15.22:0/87756872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e410e110 con 0x7f22e40ffda0 2023-09-20T22:01:27.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.556+0000 7f22e89ff700 1 -- 172.21.15.22:0/87756872 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e410e650 con 0x7f22e40ffda0 2023-09-20T22:01:27.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.563+0000 7f22fc3a4b80 1 Processor -- start 2023-09-20T22:01:27.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.566+0000 7f22fc3a4b80 1 -- start start 2023-09-20T22:01:27.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.567+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a46f0 0x55eb271a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.567+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5050 0x55eb271a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.567+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a5be0 0x55eb271a6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.567+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb27118ca0 con 0x55eb271a5be0 2023-09-20T22:01:27.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.567+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb270fcc70 con 0x55eb271a46f0 2023-09-20T22:01:27.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.567+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb270fa590 con 0x55eb271a5050 2023-09-20T22:01:27.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.567+0000 7f22e2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a46f0 0x55eb271a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.567+0000 7f22e2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a46f0 0x55eb271a4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:50522/0 (socket says 172.21.15.22:50522) 2023-09-20T22:01:27.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.567+0000 7f22e2d9d700 1 -- 172.21.15.22:0/1821546409 learned_addr learned my addr 172.21.15.22:0/1821546409 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:01:27.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.567+0000 7f22e259c700 1 --2- 172.21.15.22:0/1821546409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5050 0x55eb271a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.569+0000 7f22e259c700 1 -- 172.21.15.22:0/1821546409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a46f0 msgr2=0x55eb271a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:01:27.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.569+0000 7f22e259c700 1 --2- 172.21.15.22:0/1821546409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a46f0 0x55eb271a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.569+0000 7f22e259c700 1 -- 172.21.15.22:0/1821546409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a5be0 msgr2=0x55eb271a6060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T22:01:27.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.569+0000 7f22e259c700 1 --2- 172.21.15.22:0/1821546409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a5be0 0x55eb271a6060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.569+0000 7f22e259c700 1 -- 172.21.15.22:0/1821546409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb26f20de0 con 0x55eb271a5050 2023-09-20T22:01:27.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.570+0000 7f22e259c700 1 --2- 172.21.15.22:0/1821546409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5050 0x55eb271a55e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0098e0 tx=0x7f22dc009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e08b4b37afc4dd01 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:27.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.580+0000 7f22db7fe700 1 -- 172.21.15.22:0/87756872 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f22cc011990 con 0x7f22e40ffda0 2023-09-20T22:01:27.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.580+0000 7f22d8ff9700 1 -- 172.21.15.22:0/1821546409 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00b9d0 con 0x55eb271a5050 2023-09-20T22:01:27.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.580+0000 7f22d8ff9700 1 -- 172.21.15.22:0/1821546409 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22dc01e040 con 0x55eb271a5050 2023-09-20T22:01:27.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.580+0000 7f22d8ff9700 1 -- 172.21.15.22:0/1821546409 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00f460 con 0x55eb271a5050 2023-09-20T22:01:27.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.581+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1821546409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5050 msgr2=0x55eb271a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:27.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.581+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/1821546409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5050 0x55eb271a55e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0098e0 tx=0x7f22dc009f40 comp rx=0 tx=0).stop 2023-09-20T22:01:27.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.582+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1821546409 shutdown_connections 2023-09-20T22:01:27.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.582+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/1821546409 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a5be0 0x55eb271a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.582+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/1821546409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a46f0 0x55eb271a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.582+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/1821546409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5050 0x55eb271a55e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.582+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1821546409 >> 172.21.15.22:0/1821546409 conn(0x55eb270d1000 msgr2=0x55eb2719f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:27.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.582+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1821546409 shutdown_connections 2023-09-20T22:01:27.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.582+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1821546409 wait complete. 2023-09-20T22:01:27.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.582+0000 7f22db7fe700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22d00a1410 0x7f22d00a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.583+0000 7f22db7fe700 1 -- 172.21.15.22:0/87756872 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f22cc0cc920 con 0x7f22e40ffda0 2023-09-20T22:01:27.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.583+0000 7f22fc3a4b80 1 -- start start 2023-09-20T22:01:27.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.584+0000 7f22fc3a4b80 1 Processor -- start 2023-09-20T22:01:27.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.585+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.585+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 0x55eb271ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.585+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5be0 0x55eb271b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.585+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb270f5e80 con 0x55eb271a46f0 2023-09-20T22:01:27.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.585+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb27092740 con 0x55eb271a5050 2023-09-20T22:01:27.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.585+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb270923e0 con 0x55eb271a5be0 2023-09-20T22:01:27.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.585+0000 7f22fc3a4b80 10 client.0 _ll_register_callbacks cb 0x55eb26fdca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:01:27.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.587+0000 7f22e259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.587+0000 7f22e259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48792/0 (socket says 172.21.15.22:48792) 2023-09-20T22:01:27.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.587+0000 7f22e259c700 1 -- 172.21.15.22:0/3719482149 learned_addr learned my addr 172.21.15.22:0/3719482149 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:01:27.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.588+0000 7f22e259c700 1 -- 172.21.15.22:0/3719482149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5be0 msgr2=0x55eb271b5800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T22:01:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.588+0000 7f22e259c700 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5be0 0x55eb271b5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.588+0000 7f22e259c700 1 -- 172.21.15.22:0/3719482149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 msgr2=0x55eb271ae7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T22:01:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.588+0000 7f22e259c700 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 0x55eb271ae7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.588+0000 7f22e259c700 1 -- 172.21.15.22:0/3719482149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e4105a30 con 0x55eb271a46f0 2023-09-20T22:01:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.589+0000 7f22e259c700 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271ae280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00a010 tx=0x7f22dc0050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7d386a26145054 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:27.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.589+0000 7f22c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T22:01:27.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.590+0000 7f22c9ffb700 1 -- 172.21.15.22:0/3719482149 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc003cb0 con 0x55eb271a46f0 2023-09-20T22:01:27.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.590+0000 7f22c9ffb700 1 -- 172.21.15.22:0/3719482149 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f22dc003e10 con 0x55eb271a46f0 2023-09-20T22:01:27.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.591+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/3719482149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb27066bb0 con 0x55eb271a46f0 2023-09-20T22:01:27.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.591+0000 7f22c9ffb700 1 -- 172.21.15.22:0/3719482149 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22dc00f460 con 0x55eb271a46f0 2023-09-20T22:01:27.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.591+0000 7f22e259c700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22d00a1410 0x7f22d00a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.591+0000 7f22fc3a4b80 10 client.15945 fetch_fsmap learned FSMap version 32 2023-09-20T22:01:27.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.591+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/3719482149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55eb26f09c60 con 0x55eb271a46f0 2023-09-20T22:01:27.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.593+0000 7f22c9ffb700 1 -- 172.21.15.22:0/3719482149 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eb27066bb0 con 0x55eb271a46f0 2023-09-20T22:01:27.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.593+0000 7f22c9ffb700 1 -- 172.21.15.22:0/3719482149 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f22dc04e830 con 0x55eb271a46f0 2023-09-20T22:01:27.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.593+0000 7f22fc3a4b80 10 client.15945 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:01:27.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.593+0000 7f22fc3a4b80 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:01:27.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.593+0000 7f22fc3a4b80 20 client.15945 populate_metadata read hostname 'smithi022' 2023-09-20T22:01:27.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.593+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55eb271b9b60 0x55eb271bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.593+0000 7f22fc3a4b80 4 client.15945 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T22:01:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.593+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/3719482149 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb27173020 con 0x55eb271b9b60 2023-09-20T22:01:27.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.595+0000 7f22e259c700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22d00a1410 0x7f22d00a38d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f22dc005850 tx=0x7f22dc062000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:27.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.600+0000 7f22e1d9b700 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55eb271b9b60 0x55eb271bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.601+0000 7f22e1d9b700 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55eb271b9b60 0x55eb271bbf80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10b85bc7ac41f0f8 server_cookie=b4e3ccf55bbda634 in_seq=0 out_seq=0 2023-09-20T22:01:27.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.601+0000 7f22c9ffb700 10 client.15945 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:01:27.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.603+0000 7f22c9ffb700 1 -- 172.21.15.22:0/3719482149 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eb27173020 con 0x55eb271b9b60 2023-09-20T22:01:27.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.603+0000 7f22c9ffb700 10 client.15945 handle_command_reply: tid=0 2023-09-20T22:01:27.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.604+0000 7f22fc3a4b80 1 client.15945 shutdown 2023-09-20T22:01:27.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.604+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/3719482149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 msgr2=0x55eb271ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:27.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.604+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271ae280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00a010 tx=0x7f22dc0050d0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.606+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/3719482149 shutdown_connections 2023-09-20T22:01:27.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.606+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55eb271b9b60 0x55eb271bbf80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.606+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5be0 0x55eb271b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.606+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271ae280 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.606+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/3719482149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 0x55eb271ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.606+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/3719482149 >> 172.21.15.22:0/3719482149 conn(0x55eb270d1000 msgr2=0x55eb2719e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:27.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.606+0000 7f22c9ffb700 0 client.15945 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:01:27.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.606+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/3719482149 shutdown_connections 2023-09-20T22:01:27.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.606+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/3719482149 wait complete. 2023-09-20T22:01:27.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.613+0000 7f22fc3a4b80 20 client.15945 trim_cache size 0 max 16384 2023-09-20T22:01:27.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.646+0000 7f22fc3a4b80 1 Processor -- start 2023-09-20T22:01:27.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.651+0000 7f22fc3a4b80 1 -- start start 2023-09-20T22:01:27.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.651+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb270d1000 0x55eb271ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.651+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a46f0 0x55eb271b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.651+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 0x55eb271bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.651+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb26f76920 con 0x55eb270d1000 2023-09-20T22:01:27.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.651+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb271ad730 con 0x55eb271a5050 2023-09-20T22:01:27.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.651+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb271af130 con 0x55eb271a46f0 2023-09-20T22:01:27.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.652+0000 7f22e2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 0x55eb271bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.652+0000 7f22e2d9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 0x55eb271bbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:50546/0 (socket says 172.21.15.22:50546) 2023-09-20T22:01:27.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.652+0000 7f22e2d9d700 1 -- 172.21.15.22:0/557786871 learned_addr learned my addr 172.21.15.22:0/557786871 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:01:27.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.656+0000 7f22e259c700 1 --2- 172.21.15.22:0/557786871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb270d1000 0x55eb271ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.656+0000 7f22e1d9b700 1 --2- 172.21.15.22:0/557786871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a46f0 0x55eb271b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.656+0000 7f22e2d9d700 1 -- 172.21.15.22:0/557786871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a46f0 msgr2=0x55eb271b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:01:27.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.656+0000 7f22e2d9d700 1 --2- 172.21.15.22:0/557786871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a46f0 0x55eb271b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.656+0000 7f22e2d9d700 1 -- 172.21.15.22:0/557786871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb270d1000 msgr2=0x55eb271ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:01:27.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.656+0000 7f22e2d9d700 1 --2- 172.21.15.22:0/557786871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb270d1000 0x55eb271ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.656+0000 7f22e2d9d700 1 -- 172.21.15.22:0/557786871 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb26f47250 con 0x55eb271a5050 2023-09-20T22:01:27.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.656+0000 7f22e1d9b700 1 --2- 172.21.15.22:0/557786871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a46f0 0x55eb271b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T22:01:27.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.657+0000 7f22e2d9d700 1 --2- 172.21.15.22:0/557786871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 0x55eb271bbb70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009d90 tx=0x7f22d400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed7b4326b91a80a server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:27.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.657+0000 7f22d97fa700 1 -- 172.21.15.22:0/557786871 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4010040 con 0x55eb271a5050 2023-09-20T22:01:27.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.658+0000 7f22d97fa700 1 -- 172.21.15.22:0/557786871 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d400ec70 con 0x55eb271a5050 2023-09-20T22:01:27.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.658+0000 7f22d97fa700 1 -- 172.21.15.22:0/557786871 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4014ea0 con 0x55eb271a5050 2023-09-20T22:01:27.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.659+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/557786871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 msgr2=0x55eb271bbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:27.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.659+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/557786871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 0x55eb271bbb70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009d90 tx=0x7f22d400b040 comp rx=0 tx=0).stop 2023-09-20T22:01:27.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.671+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/557786871 shutdown_connections 2023-09-20T22:01:27.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.671+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/557786871 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb271a5050 0x55eb271bbb70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.671+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/557786871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb270d1000 0x55eb271ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.671+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/557786871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a46f0 0x55eb271b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.671+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/557786871 >> 172.21.15.22:0/557786871 conn(0x55eb2719f500 msgr2=0x55eb271b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:27.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.679+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/557786871 shutdown_connections 2023-09-20T22:01:27.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.682+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/557786871 wait complete. 2023-09-20T22:01:27.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.682+0000 7f22fc3a4b80 1 -- start start 2023-09-20T22:01:27.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.683+0000 7f22fc3a4b80 1 Processor -- start 2023-09-20T22:01:27.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.683+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb270d1000 0x55eb271b5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.683+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.683+0000 7f22fc3a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5050 0x55eb271b3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.683+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb271b4030 con 0x55eb271a46f0 2023-09-20T22:01:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.683+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb271b41a0 con 0x55eb270d1000 2023-09-20T22:01:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.683+0000 7f22fc3a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb271b4360 con 0x55eb271a5050 2023-09-20T22:01:27.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.683+0000 7f22fc3a4b80 10 client.0 _ll_register_callbacks cb 0x55eb26fdca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:01:27.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.685+0000 7f22e1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.685+0000 7f22e1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48810/0 (socket says 172.21.15.22:48810) 2023-09-20T22:01:27.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.685+0000 7f22e1d9b700 1 -- 172.21.15.22:0/1336828148 learned_addr learned my addr 172.21.15.22:0/1336828148 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:01:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.685+0000 7f22e1d9b700 1 -- 172.21.15.22:0/1336828148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5050 msgr2=0x55eb271b3920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T22:01:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.685+0000 7f22e1d9b700 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5050 0x55eb271b3920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.685+0000 7f22e1d9b700 1 -- 172.21.15.22:0/1336828148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb270d1000 msgr2=0x55eb271b5220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T22:01:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.685+0000 7f22e1d9b700 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb270d1000 0x55eb271b5220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.685+0000 7f22e1d9b700 1 -- 172.21.15.22:0/1336828148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d4009a70 con 0x55eb271a46f0 2023-09-20T22:01:27.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.686+0000 7f22e1d9b700 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271b5760 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00bcf0 tx=0x7f22cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a766e13788ff3972 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:27.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.691+0000 7f22c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-09-20T22:01:27.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.691+0000 7f22c9ffb700 1 -- 172.21.15.22:0/1336828148 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc0a1930 con 0x55eb271a46f0 2023-09-20T22:01:27.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.691+0000 7f22c9ffb700 1 -- 172.21.15.22:0/1336828148 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f22cc097c60 con 0x55eb271a46f0 2023-09-20T22:01:27.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.692+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1336828148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22dc01e040 con 0x55eb271a46f0 2023-09-20T22:01:27.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.693+0000 7f22c9ffb700 1 -- 172.21.15.22:0/1336828148 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22cc0ca140 con 0x55eb271a46f0 2023-09-20T22:01:27.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.697+0000 7f22fc3a4b80 10 client.15951 fetch_fsmap learned FSMap version 32 2023-09-20T22:01:27.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.697+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1336828148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55eb26f1f2b0 con 0x55eb271a46f0 2023-09-20T22:01:27.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.699+0000 7f22c9ffb700 1 -- 172.21.15.22:0/1336828148 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22dc01e040 con 0x55eb271a46f0 2023-09-20T22:01:27.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.699+0000 7f22c9ffb700 1 -- 172.21.15.22:0/1336828148 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f22cc0a14a0 con 0x55eb271a46f0 2023-09-20T22:01:27.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.699+0000 7f22fc3a4b80 10 client.15951 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:01:27.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.699+0000 7f22fc3a4b80 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:01:27.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.699+0000 7f22fc3a4b80 20 client.15951 populate_metadata read hostname 'smithi022' 2023-09-20T22:01:27.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.699+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55eb271a5be0 0x55eb27210e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:27.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.699+0000 7f22fc3a4b80 4 client.15951 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T22:01:27.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.699+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1336828148 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eb270eee60 con 0x55eb271a5be0 2023-09-20T22:01:27.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.705+0000 7f22e259c700 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55eb271a5be0 0x55eb27210e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:27.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.731+0000 7f22e259c700 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55eb271a5be0 0x55eb27210e90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b415dbcb3f472e server_cookie=c1d94d273b4e999a in_seq=0 out_seq=0 2023-09-20T22:01:27.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.731+0000 7f22c9ffb700 10 client.15951 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:01:27.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.745+0000 7f22c9ffb700 1 -- 172.21.15.22:0/1336828148 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55eb270eee60 con 0x55eb271a5be0 2023-09-20T22:01:27.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.745+0000 7f22c9ffb700 10 client.15951 handle_command_reply: tid=0 2023-09-20T22:01:27.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.745+0000 7f22fc3a4b80 1 client.15951 shutdown 2023-09-20T22:01:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.747+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1336828148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 msgr2=0x55eb271b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.747+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271b5760 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00bcf0 tx=0x7f22cc000bc0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.750+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1336828148 shutdown_connections 2023-09-20T22:01:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.750+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55eb271a5be0 0x55eb27210e90 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.750+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eb271a5050 0x55eb271b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.750+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eb270d1000 0x55eb271b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.750+0000 7f22fc3a4b80 1 --2- 172.21.15.22:0/1336828148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55eb271a46f0 0x55eb271b5760 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.750+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1336828148 >> 172.21.15.22:0/1336828148 conn(0x55eb2719f500 msgr2=0x55eb271bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:27.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.751+0000 7f22c9ffb700 0 client.15951 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:01:27.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.751+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1336828148 shutdown_connections 2023-09-20T22:01:27.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.751+0000 7f22fc3a4b80 1 -- 172.21.15.22:0/1336828148 wait complete. 2023-09-20T22:01:27.768 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:01:27.768 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T22:01:27.769 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T22:01:27.769 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T22:01:27.769 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T22:01:27.769 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T22:01:27.769 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T22:01:27.770 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:01:27.770 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:01:27.770 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:01:27.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.766+0000 7f22fc3a4b80 20 client.15951 trim_cache size 0 max 16384 2023-09-20T22:01:27.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.778+0000 7f22e89ff700 1 -- 172.21.15.22:0/87756872 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22d00a1410 msgr2=0x7f22d00a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:27.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.778+0000 7f22e89ff700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22d00a1410 0x7f22d00a38d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f22dc005850 tx=0x7f22dc062000 comp rx=0 tx=0).stop 2023-09-20T22:01:27.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.778+0000 7f22e89ff700 1 -- 172.21.15.22:0/87756872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22e40ffda0 msgr2=0x7f22e40719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:27.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.778+0000 7f22e89ff700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22e40ffda0 0x7f22e40719c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b530 tx=0x7f22cc00b8f0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.791+0000 7f22e89ff700 1 -- 172.21.15.22:0/87756872 shutdown_connections 2023-09-20T22:01:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.791+0000 7f22e89ff700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22e4100760 0x7f22e4071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.791+0000 7f22e89ff700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f22d00a1410 0x7f22d00a38d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.791+0000 7f22e89ff700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22e40fec70 0x7f22e4071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.791+0000 7f22e89ff700 1 --2- 172.21.15.22:0/87756872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22e40ffda0 0x7f22e40719c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:27.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.791+0000 7f22e89ff700 1 -- 172.21.15.22:0/87756872 >> 172.21.15.22:0/87756872 conn(0x7f22e40fa570 msgr2=0x7f22e40fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:27.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.794+0000 7f22e89ff700 1 -- 172.21.15.22:0/87756872 shutdown_connections 2023-09-20T22:01:27.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:27.794+0000 7f22e89ff700 1 -- 172.21.15.22:0/87756872 wait complete. 2023-09-20T22:01:27.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:27 smithi123 ceph-mon[134433]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.5 MiB/s wr, 525 op/s 2023-09-20T22:01:27.849 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T22:01:27.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:27 smithi089 ceph-mon[134090]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.5 MiB/s wr, 525 op/s 2023-09-20T22:01:28.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:27 smithi022 ceph-mon[97651]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.5 MiB/s wr, 525 op/s 2023-09-20T22:01:28.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 725.0 s, 526.0 tps, lat 189.921 ms stddev 234.807 2023-09-20T22:01:29.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:29 smithi022 ceph-mon[97651]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 6.0 MiB/s wr, 488 op/s 2023-09-20T22:01:29.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:29 smithi123 ceph-mon[134433]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 6.0 MiB/s wr, 488 op/s 2023-09-20T22:01:29.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:29 smithi089 ceph-mon[134090]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 6.0 MiB/s wr, 488 op/s 2023-09-20T22:01:31.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:31 smithi022 ceph-mon[97651]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 6.4 MiB/s wr, 519 op/s 2023-09-20T22:01:31.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:31 smithi123 ceph-mon[134433]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 6.4 MiB/s wr, 519 op/s 2023-09-20T22:01:31.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:31 smithi089 ceph-mon[134090]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 6.4 MiB/s wr, 519 op/s 2023-09-20T22:01:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 730.0 s, 556.8 tps, lat 181.416 ms stddev 179.309 2023-09-20T22:01:33.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:33 smithi022 ceph-mon[97651]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 6.2 MiB/s wr, 560 op/s 2023-09-20T22:01:33.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:33 smithi123 ceph-mon[134433]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 6.2 MiB/s wr, 560 op/s 2023-09-20T22:01:33.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:33 smithi089 ceph-mon[134090]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 6.2 MiB/s wr, 560 op/s 2023-09-20T22:01:35.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:35 smithi123 ceph-mon[134433]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.6 MiB/s wr, 486 op/s 2023-09-20T22:01:35.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:35 smithi089 ceph-mon[134090]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.6 MiB/s wr, 486 op/s 2023-09-20T22:01:35.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:35 smithi022 ceph-mon[97651]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.6 MiB/s wr, 486 op/s 2023-09-20T22:01:36.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:36 smithi089 ceph-mon[134090]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 543 op/s 2023-09-20T22:01:36.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:36 smithi022 ceph-mon[97651]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 543 op/s 2023-09-20T22:01:37.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:36 smithi123 ceph-mon[134433]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 543 op/s 2023-09-20T22:01:38.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 735.0 s, 467.6 tps, lat 211.890 ms stddev 261.056 2023-09-20T22:01:39.749 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:39 smithi022 ceph-mon[97651]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.5 MiB/s wr, 475 op/s 2023-09-20T22:01:39.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:39 smithi123 ceph-mon[134433]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.5 MiB/s wr, 475 op/s 2023-09-20T22:01:39.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:39 smithi089 ceph-mon[134090]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.5 MiB/s wr, 475 op/s 2023-09-20T22:01:40.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:40 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:01:40.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:40 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:01:40.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:40 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:01:41.757 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:41 smithi022 ceph-mon[97651]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.5 MiB/s wr, 503 op/s 2023-09-20T22:01:41.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:41 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:41 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:41 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.758 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:41 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:41 smithi123 ceph-mon[134433]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.5 MiB/s wr, 503 op/s 2023-09-20T22:01:41.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:41 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:41 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:41 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.844 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:41 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:41 smithi089 ceph-mon[134090]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.5 MiB/s wr, 503 op/s 2023-09-20T22:01:41.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:41 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:41 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:41 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:41.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:41 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:42.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:01:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:01:42.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:42 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:42.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:01:42.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:01:42.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:42 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:42.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:01:42.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:01:42.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:42 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:01:43.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 740.0 s, 438.8 tps, lat 222.709 ms stddev 241.061 2023-09-20T22:01:43.851 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:43 smithi123 ceph-mon[134433]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 545 op/s 2023-09-20T22:01:43.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:43 smithi089 ceph-mon[134090]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 545 op/s 2023-09-20T22:01:43.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:43 smithi022 ceph-mon[97651]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 545 op/s 2023-09-20T22:01:45.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:45 smithi123 ceph-mon[134433]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 462 op/s 2023-09-20T22:01:45.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:45 smithi089 ceph-mon[134090]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 462 op/s 2023-09-20T22:01:45.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:45 smithi022 ceph-mon[97651]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.4 MiB/s wr, 462 op/s 2023-09-20T22:01:46.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:46 smithi089 ceph-mon[134090]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 526 op/s 2023-09-20T22:01:46.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:46 smithi022 ceph-mon[97651]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 526 op/s 2023-09-20T22:01:47.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:46 smithi123 ceph-mon[134433]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 526 op/s 2023-09-20T22:01:48.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 745.0 s, 514.2 tps, lat 196.053 ms stddev 236.622 2023-09-20T22:01:49.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:49 smithi022 ceph-mon[97651]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 459 op/s 2023-09-20T22:01:49.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:49 smithi123 ceph-mon[134433]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 459 op/s 2023-09-20T22:01:49.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:49 smithi089 ceph-mon[134090]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 459 op/s 2023-09-20T22:01:51.754 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:51 smithi022 ceph-mon[97651]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 488 op/s 2023-09-20T22:01:51.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:51 smithi123 ceph-mon[134433]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 488 op/s 2023-09-20T22:01:51.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:51 smithi089 ceph-mon[134090]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 488 op/s 2023-09-20T22:01:53.573 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 750.0 s, 559.7 tps, lat 177.325 ms stddev 192.319 2023-09-20T22:01:53.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:53 smithi123 ceph-mon[134433]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.8 MiB/s wr, 526 op/s 2023-09-20T22:01:53.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:53 smithi089 ceph-mon[134090]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.8 MiB/s wr, 526 op/s 2023-09-20T22:01:53.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:53 smithi022 ceph-mon[97651]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.8 MiB/s wr, 526 op/s 2023-09-20T22:01:55.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:55 smithi123 ceph-mon[134433]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 454 op/s 2023-09-20T22:01:55.871 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:55 smithi089 ceph-mon[134090]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 454 op/s 2023-09-20T22:01:55.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:55 smithi022 ceph-mon[97651]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.0 MiB/s wr, 454 op/s 2023-09-20T22:01:57.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:57 smithi123 ceph-mon[134433]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 523 op/s 2023-09-20T22:01:57.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T22:01:57.870 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:57 smithi089 ceph-mon[134090]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 523 op/s 2023-09-20T22:01:57.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:57 smithi022 ceph-mon[97651]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 523 op/s 2023-09-20T22:01:58.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.342+0000 7f87e23ff700 1 -- 172.21.15.22:0/1173457240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87dc10d120 msgr2=0x7f87dc10d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:58.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.342+0000 7f87e23ff700 1 --2- 172.21.15.22:0/1173457240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87dc10d120 0x7f87dc10d5a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009990 tx=0x7f87d0009ca0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.353+0000 7f87e23ff700 1 -- 172.21.15.22:0/1173457240 shutdown_connections 2023-09-20T22:01:58.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.353+0000 7f87e23ff700 1 --2- 172.21.15.22:0/1173457240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87dc10d120 0x7f87dc10d5a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.353+0000 7f87e23ff700 1 --2- 172.21.15.22:0/1173457240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87dc107de0 0x7f87dc10a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.353+0000 7f87e23ff700 1 --2- 172.21.15.22:0/1173457240 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87dc10a750 0x7f87dc10cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.353+0000 7f87e23ff700 1 -- 172.21.15.22:0/1173457240 >> 172.21.15.22:0/1173457240 conn(0x7f87dc06db90 msgr2=0x7f87dc06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:58.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.360+0000 7f87e23ff700 1 -- 172.21.15.22:0/1173457240 shutdown_connections 2023-09-20T22:01:58.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.367+0000 7f87e23ff700 1 -- 172.21.15.22:0/1173457240 wait complete. 2023-09-20T22:01:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.368+0000 7f87e23ff700 1 Processor -- start 2023-09-20T22:01:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.369+0000 7f87e23ff700 1 -- start start 2023-09-20T22:01:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.369+0000 7f87e23ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87dc107de0 0x7f87dc114340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.369+0000 7f87e23ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87dc10a750 0x7f87dc114880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.369+0000 7f87e23ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87dc10d120 0x7f87dc1b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.369+0000 7f87e23ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc117960 con 0x7f87dc10a750 2023-09-20T22:01:58.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.369+0000 7f87e23ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc117ad0 con 0x7f87dc10d120 2023-09-20T22:01:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.369+0000 7f87e23ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc1b60f0 con 0x7f87dc107de0 2023-09-20T22:01:58.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.372+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87dc107de0 0x7f87dc114340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.372+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87dc107de0 0x7f87dc114340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:35220/0 (socket says 172.21.15.22:35220) 2023-09-20T22:01:58.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.372+0000 7f87dbfff700 1 -- 172.21.15.22:0/3099552483 learned_addr learned my addr 172.21.15.22:0/3099552483 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:01:58.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.372+0000 7f87db7fe700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87dc10a750 0x7f87dc114880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.376+0000 7f87e099c700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87dc10d120 0x7f87dc1b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.377+0000 7f87dbfff700 1 -- 172.21.15.22:0/3099552483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87dc10d120 msgr2=0x7f87dc1b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:58.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.377+0000 7f87dbfff700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87dc10d120 0x7f87dc1b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.377+0000 7f87dbfff700 1 -- 172.21.15.22:0/3099552483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87dc10a750 msgr2=0x7f87dc114880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:58.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.377+0000 7f87dbfff700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87dc10a750 0x7f87dc114880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.377+0000 7f87dbfff700 1 -- 172.21.15.22:0/3099552483 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00095f0 con 0x7f87dc107de0 2023-09-20T22:01:58.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.377+0000 7f87dbfff700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87dc107de0 0x7f87dc114340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00b530 tx=0x7f87cc00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:58.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.382+0000 7f87e099c700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87dc10d120 0x7f87dc1b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:01:58.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.382+0000 7f87d97fa700 1 -- 172.21.15.22:0/3099552483 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc011830 con 0x7f87dc107de0 2023-09-20T22:01:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.382+0000 7f87d97fa700 1 -- 172.21.15.22:0/3099552483 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87cc011e50 con 0x7f87dc107de0 2023-09-20T22:01:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.382+0000 7f87d97fa700 1 -- 172.21.15.22:0/3099552483 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00f2e0 con 0x7f87dc107de0 2023-09-20T22:01:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.382+0000 7f87e23ff700 1 -- 172.21.15.22:0/3099552483 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc1b6290 con 0x7f87dc107de0 2023-09-20T22:01:58.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.382+0000 7f87e23ff700 1 -- 172.21.15.22:0/3099552483 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc1b67d0 con 0x7f87dc107de0 2023-09-20T22:01:58.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.392+0000 7f87d97fa700 1 -- 172.21.15.22:0/3099552483 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f87cc011990 con 0x7f87dc107de0 2023-09-20T22:01:58.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.395+0000 7f87f5d9cb80 1 Processor -- start 2023-09-20T22:01:58.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.395+0000 7f87f5d9cb80 1 -- start start 2023-09-20T22:01:58.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.396+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b050b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.396+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051050 0x560c1b0515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.396+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051be0 0x560c1b052060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.396+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1afc4ca0 con 0x560c1b051be0 2023-09-20T22:01:58.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.396+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1afa8c70 con 0x560c1b0506f0 2023-09-20T22:01:58.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.396+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1afa6590 con 0x560c1b051050 2023-09-20T22:01:58.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.396+0000 7f87d97fa700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f87c40a1500 0x7f87c40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.396+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b050b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.396+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b050b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:52320/0 (socket says 172.21.15.22:52320) 2023-09-20T22:01:58.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.396+0000 7f87dbfff700 1 -- 172.21.15.22:0/504637768 learned_addr learned my addr 172.21.15.22:0/504637768 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:01:58.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87e099c700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f87c40a1500 0x7f87c40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87dbfff700 1 -- 172.21.15.22:0/504637768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051050 msgr2=0x560c1b0515e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T22:01:58.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87dbfff700 1 --2- 172.21.15.22:0/504637768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051050 0x560c1b0515e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87dbfff700 1 -- 172.21.15.22:0/504637768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051be0 msgr2=0x560c1b052060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:01:58.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87dbfff700 1 --2- 172.21.15.22:0/504637768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051be0 0x560c1b052060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87dbfff700 1 -- 172.21.15.22:0/504637768 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c1adccde0 con 0x560c1b0506f0 2023-09-20T22:01:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87d97fa700 1 -- 172.21.15.22:0/3099552483 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f87cc0cc760 con 0x7f87dc107de0 2023-09-20T22:01:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87dbfff700 1 --2- 172.21.15.22:0/504637768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b050b10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0c9db0 tx=0x7f87cc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3d109d4f2779c35 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87c27fc700 1 -- 172.21.15.22:0/504637768 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc099700 con 0x560c1b0506f0 2023-09-20T22:01:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87c27fc700 1 -- 172.21.15.22:0/504637768 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87cc09d530 con 0x560c1b0506f0 2023-09-20T22:01:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.397+0000 7f87c27fc700 1 -- 172.21.15.22:0/504637768 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc09d0e0 con 0x560c1b0506f0 2023-09-20T22:01:58.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.398+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/504637768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 msgr2=0x560c1b050b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:58.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.398+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/504637768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b050b10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0c9db0 tx=0x7f87cc004750 comp rx=0 tx=0).stop 2023-09-20T22:01:58.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.398+0000 7f87e099c700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f87c40a1500 0x7f87c40a39c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f87d000b5c0 tx=0x7f87d001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:58.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.399+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/504637768 shutdown_connections 2023-09-20T22:01:58.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.399+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/504637768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051be0 0x560c1b052060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.399+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/504637768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b050b10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.399+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/504637768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051050 0x560c1b0515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.399+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/504637768 >> 172.21.15.22:0/504637768 conn(0x560c1af7d000 msgr2=0x560c1b04b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.399+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/504637768 shutdown_connections 2023-09-20T22:01:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.399+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/504637768 wait complete. 2023-09-20T22:01:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.400+0000 7f87f5d9cb80 1 -- start start 2023-09-20T22:01:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.400+0000 7f87f5d9cb80 1 Processor -- start 2023-09-20T22:01:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.401+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b0506f0 0x560c1b05a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.401+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 0x560c1b05a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.401+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 0x560c1b0616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.401+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1afa1e80 con 0x560c1b051050 2023-09-20T22:01:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.401+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1af3e740 con 0x560c1b051be0 2023-09-20T22:01:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.401+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1af3e3e0 con 0x560c1b0506f0 2023-09-20T22:01:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.401+0000 7f87f5d9cb80 10 client.0 _ll_register_callbacks cb 0x560c1ae88a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:01:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.402+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b0506f0 0x560c1b05a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.402+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b0506f0 0x560c1b05a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:35228/0 (socket says 172.21.15.22:35228) 2023-09-20T22:01:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.402+0000 7f87dbfff700 1 -- 172.21.15.22:0/3771621316 learned_addr learned my addr 172.21.15.22:0/3771621316 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:01:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.402+0000 7f87e099c700 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 0x560c1b0616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.402+0000 7f87db7fe700 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 0x560c1b05a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.402+0000 7f87dbfff700 1 -- 172.21.15.22:0/3771621316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 msgr2=0x560c1b0616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:01:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.402+0000 7f87dbfff700 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 0x560c1b0616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.402+0000 7f87dbfff700 1 -- 172.21.15.22:0/3771621316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 msgr2=0x560c1b05a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:01:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.402+0000 7f87dbfff700 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 0x560c1b05a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.402+0000 7f87dbfff700 1 -- 172.21.15.22:0/3771621316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc1b67d0 con 0x560c1b0506f0 2023-09-20T22:01:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.403+0000 7f87e099c700 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 0x560c1b0616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:01:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.403+0000 7f87dbfff700 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b0506f0 0x560c1b05a130 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f87cc006f60 tx=0x7f87cc095ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c27d6d08673357d server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.403+0000 7f87db7fe700 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 0x560c1b05a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T22:01:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.403+0000 7f87bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T22:01:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.403+0000 7f87bb7fe700 1 -- 172.21.15.22:0/3771621316 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc097aa0 con 0x560c1b0506f0 2023-09-20T22:01:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.403+0000 7f87bb7fe700 1 -- 172.21.15.22:0/3771621316 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f87cc0b1040 con 0x560c1b0506f0 2023-09-20T22:01:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.403+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/3771621316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c1af12bb0 con 0x560c1b0506f0 2023-09-20T22:01:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87bb7fe700 1 -- 172.21.15.22:0/3771621316 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87cc0cc430 con 0x560c1b0506f0 2023-09-20T22:01:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87f5d9cb80 10 client.26128 fetch_fsmap learned FSMap version 32 2023-09-20T22:01:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/3771621316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560c1adb5c60 con 0x560c1b0506f0 2023-09-20T22:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87bb7fe700 1 -- 172.21.15.22:0/3771621316 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c1af12bb0 con 0x560c1b0506f0 2023-09-20T22:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87bb7fe700 1 -- 172.21.15.22:0/3771621316 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87cc099960 con 0x560c1b0506f0 2023-09-20T22:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87f5d9cb80 10 client.26128 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87f5d9cb80 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:01:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87f5d9cb80 20 client.26128 populate_metadata read hostname 'smithi022' 2023-09-20T22:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560c1b065bf0 0x560c1b068010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87f5d9cb80 4 client.26128 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T22:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.404+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/3771621316 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c1b01f020 con 0x560c1b065bf0 2023-09-20T22:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.405+0000 7f87db7fe700 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560c1b065bf0 0x560c1b068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.405+0000 7f87bb7fe700 10 client.26128 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.405+0000 7f87db7fe700 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560c1b065bf0 0x560c1b068010 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=516b405c6af14a5a server_cookie=4df91e9c84ae4551 in_seq=0 out_seq=0 2023-09-20T22:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.406+0000 7f87bb7fe700 1 -- 172.21.15.22:0/3771621316 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c1b01f020 con 0x560c1b065bf0 2023-09-20T22:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.406+0000 7f87bb7fe700 10 client.26128 handle_command_reply: tid=0 2023-09-20T22:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.406+0000 7f87f5d9cb80 1 client.26128 shutdown 2023-09-20T22:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/3771621316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b0506f0 msgr2=0x560c1b05a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b0506f0 0x560c1b05a130 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f87cc006f60 tx=0x7f87cc095ac0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/3771621316 shutdown_connections 2023-09-20T22:01:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560c1b065bf0 0x560c1b068010 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 0x560c1b0616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b0506f0 0x560c1b05a130 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/3771621316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 0x560c1b05a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/3771621316 >> 172.21.15.22:0/3771621316 conn(0x560c1af7d000 msgr2=0x560c1b04a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87bb7fe700 0 client.26128 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:01:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/3771621316 shutdown_connections 2023-09-20T22:01:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.407+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/3771621316 wait complete. 2023-09-20T22:01:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.408+0000 7f87f5d9cb80 20 client.26128 trim_cache size 0 max 16384 2023-09-20T22:01:58.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.440+0000 7f87f5d9cb80 1 Processor -- start 2023-09-20T22:01:58.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.440+0000 7f87f5d9cb80 1 -- start start 2023-09-20T22:01:58.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.440+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b0506f0 0x560c1b051fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.440+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051050 0x560c1b061640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.440+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 0x560c1b054b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.440+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1ae22920 con 0x560c1b0506f0 2023-09-20T22:01:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.440+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1b01f020 con 0x560c1b051be0 2023-09-20T22:01:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.440+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1af12bb0 con 0x560c1b051050 2023-09-20T22:01:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.441+0000 7f87e099c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 0x560c1b054b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.441+0000 7f87e099c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 0x560c1b054b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:52348/0 (socket says 172.21.15.22:52348) 2023-09-20T22:01:58.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.441+0000 7f87e099c700 1 -- 172.21.15.22:0/2973401518 learned_addr learned my addr 172.21.15.22:0/2973401518 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:01:58.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.443+0000 7f87db7fe700 1 --2- 172.21.15.22:0/2973401518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051050 0x560c1b061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.443+0000 7f87db7fe700 1 -- 172.21.15.22:0/2973401518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 msgr2=0x560c1b054b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:01:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.443+0000 7f87db7fe700 1 --2- 172.21.15.22:0/2973401518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 0x560c1b054b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.443+0000 7f87db7fe700 1 -- 172.21.15.22:0/2973401518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b0506f0 msgr2=0x560c1b051fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:01:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.443+0000 7f87db7fe700 1 --2- 172.21.15.22:0/2973401518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b0506f0 0x560c1b051fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.443+0000 7f87db7fe700 1 -- 172.21.15.22:0/2973401518 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c1adf3250 con 0x560c1b051050 2023-09-20T22:01:58.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.444+0000 7f87db7fe700 1 --2- 172.21.15.22:0/2973401518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051050 0x560c1b061640 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x560c1b0688c0 tx=0x7f87d4009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6067e06885de02d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:58.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.445+0000 7f87c2ffd700 1 -- 172.21.15.22:0/2973401518 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4033040 con 0x560c1b051050 2023-09-20T22:01:58.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.445+0000 7f87c2ffd700 1 -- 172.21.15.22:0/2973401518 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87d403c070 con 0x560c1b051050 2023-09-20T22:01:58.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.445+0000 7f87c2ffd700 1 -- 172.21.15.22:0/2973401518 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d403f860 con 0x560c1b051050 2023-09-20T22:01:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.446+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2973401518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051050 msgr2=0x560c1b061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.446+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/2973401518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051050 0x560c1b061640 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x560c1b0688c0 tx=0x7f87d4009c90 comp rx=0 tx=0).stop 2023-09-20T22:01:58.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.446+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2973401518 shutdown_connections 2023-09-20T22:01:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.446+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/2973401518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b051be0 0x560c1b054b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.446+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/2973401518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b0506f0 0x560c1b051fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.446+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/2973401518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051050 0x560c1b061640 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.446+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2973401518 >> 172.21.15.22:0/2973401518 conn(0x560c1af7d000 msgr2=0x560c1b057ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:58.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.447+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2973401518 shutdown_connections 2023-09-20T22:01:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.447+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2973401518 wait complete. 2023-09-20T22:01:58.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.448+0000 7f87f5d9cb80 1 -- start start 2023-09-20T22:01:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.448+0000 7f87f5d9cb80 1 Processor -- start 2023-09-20T22:01:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.449+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b061210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.449+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 0x560c1b061750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.449+0000 7f87f5d9cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051be0 0x560c1b05f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.449+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1b060040 con 0x560c1b051050 2023-09-20T22:01:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.449+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1b0601b0 con 0x560c1b0506f0 2023-09-20T22:01:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.449+0000 7f87f5d9cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c1b060370 con 0x560c1b051be0 2023-09-20T22:01:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.449+0000 7f87f5d9cb80 10 client.0 _ll_register_callbacks cb 0x560c1ae88a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:01:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.449+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b061210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.449+0000 7f87dbfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b061210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:52362/0 (socket says 172.21.15.22:52362) 2023-09-20T22:01:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.449+0000 7f87dbfff700 1 -- 172.21.15.22:0/2053696647 learned_addr learned my addr 172.21.15.22:0/2053696647 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:01:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.450+0000 7f87db7fe700 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 0x560c1b061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.450+0000 7f87dbfff700 1 -- 172.21.15.22:0/2053696647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051be0 msgr2=0x560c1b05f960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T22:01:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.450+0000 7f87dbfff700 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051be0 0x560c1b05f960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.450+0000 7f87dbfff700 1 -- 172.21.15.22:0/2053696647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 msgr2=0x560c1b061750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:01:58.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.450+0000 7f87dbfff700 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 0x560c1b061750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.450+0000 7f87dbfff700 1 -- 172.21.15.22:0/2053696647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d4009b00 con 0x560c1b0506f0 2023-09-20T22:01:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.450+0000 7f87dbfff700 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b061210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0aeca0 tx=0x7f87cc0953a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7397163fba13bf8 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:01:58.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.450+0000 7f87bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T22:01:58.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.453+0000 7f87bb7fe700 1 -- 172.21.15.22:0/2053696647 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc098bf0 con 0x560c1b0506f0 2023-09-20T22:01:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.453+0000 7f87bb7fe700 1 -- 172.21.15.22:0/2053696647 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f87cc0cc430 con 0x560c1b0506f0 2023-09-20T22:01:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.453+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2053696647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87cc09d530 con 0x560c1b0506f0 2023-09-20T22:01:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.454+0000 7f87bb7fe700 1 -- 172.21.15.22:0/2053696647 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87cc0aeee0 con 0x560c1b0506f0 2023-09-20T22:01:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.454+0000 7f87f5d9cb80 10 client.26093 fetch_fsmap learned FSMap version 32 2023-09-20T22:01:58.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.454+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2053696647 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560c1adcb2b0 con 0x560c1b0506f0 2023-09-20T22:01:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.454+0000 7f87bb7fe700 1 -- 172.21.15.22:0/2053696647 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87cc09d530 con 0x560c1b0506f0 2023-09-20T22:01:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.454+0000 7f87bb7fe700 1 -- 172.21.15.22:0/2053696647 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f87cc0a12e0 con 0x560c1b0506f0 2023-09-20T22:01:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.455+0000 7f87f5d9cb80 10 client.26093 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:01:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.455+0000 7f87f5d9cb80 10 client.26093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:01:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.455+0000 7f87f5d9cb80 20 client.26093 populate_metadata read hostname 'smithi022' 2023-09-20T22:01:58.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.455+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560c1b0baad0 0x560c1b0bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:01:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.455+0000 7f87f5d9cb80 4 client.26093 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T22:01:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.455+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2053696647 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c1af9ae60 con 0x560c1b0baad0 2023-09-20T22:01:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.455+0000 7f87db7fe700 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560c1b0baad0 0x560c1b0bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:01:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.455+0000 7f87db7fe700 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560c1b0baad0 0x560c1b0bcef0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=982a5db0aa330fcb server_cookie=86f5acc23a6d8c0e in_seq=0 out_seq=0 2023-09-20T22:01:58.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.455+0000 7f87bb7fe700 10 client.26093 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:01:58.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87bb7fe700 1 -- 172.21.15.22:0/2053696647 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560c1af9ae60 con 0x560c1b0baad0 2023-09-20T22:01:58.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87bb7fe700 10 client.26093 handle_command_reply: tid=0 2023-09-20T22:01:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87f5d9cb80 1 client.26093 shutdown 2023-09-20T22:01:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2053696647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 msgr2=0x560c1b061210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b061210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0aeca0 tx=0x7f87cc0953a0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2053696647 shutdown_connections 2023-09-20T22:01:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x560c1b0baad0 0x560c1b0bcef0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c1b051be0 0x560c1b05f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560c1b0506f0 0x560c1b061210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87f5d9cb80 1 --2- 172.21.15.22:0/2053696647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560c1b051050 0x560c1b061750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2053696647 >> 172.21.15.22:0/2053696647 conn(0x560c1af7d000 msgr2=0x560c1b054b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.456+0000 7f87bb7fe700 0 client.26093 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:01:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.457+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2053696647 shutdown_connections 2023-09-20T22:01:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.457+0000 7f87f5d9cb80 1 -- 172.21.15.22:0/2053696647 wait complete. 2023-09-20T22:01:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.457+0000 7f87f5d9cb80 20 client.26093 trim_cache size 0 max 16384 2023-09-20T22:01:58.462 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:01:58.462 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T22:01:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T22:01:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T22:01:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T22:01:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T22:01:58.463 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T22:01:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:01:58.464 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:01:58.464 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:01:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 -- 172.21.15.22:0/3099552483 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f87c40a1500 msgr2=0x7f87c40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f87c40a1500 0x7f87c40a39c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f87d000b5c0 tx=0x7f87d001a040 comp rx=0 tx=0).stop 2023-09-20T22:01:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 -- 172.21.15.22:0/3099552483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87dc107de0 msgr2=0x7f87dc114340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:01:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87dc107de0 0x7f87dc114340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00b530 tx=0x7f87cc00b8f0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 -- 172.21.15.22:0/3099552483 shutdown_connections 2023-09-20T22:01:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f87c40a1500 0x7f87c40a39c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87dc10d120 0x7f87dc1b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87dc107de0 0x7f87dc114340 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 --2- 172.21.15.22:0/3099552483 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87dc10a750 0x7f87dc114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:01:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 -- 172.21.15.22:0/3099552483 >> 172.21.15.22:0/3099552483 conn(0x7f87dc06db90 msgr2=0x7f87dc10b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:01:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 -- 172.21.15.22:0/3099552483 shutdown_connections 2023-09-20T22:01:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:01:58.458+0000 7f87e23ff700 1 -- 172.21.15.22:0/3099552483 wait complete. 2023-09-20T22:01:58.498 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T22:01:58.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 755.0 s, 476.6 tps, lat 210.267 ms stddev 239.692 2023-09-20T22:01:59.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:01:59 smithi123 ceph-mon[134433]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.1 MiB/s wr, 467 op/s 2023-09-20T22:01:59.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:01:59 smithi089 ceph-mon[134090]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.1 MiB/s wr, 467 op/s 2023-09-20T22:02:00.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:01:59 smithi022 ceph-mon[97651]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.1 MiB/s wr, 467 op/s 2023-09-20T22:02:01.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:00 smithi022 ceph-mon[97651]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2023-09-20T22:02:01.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:00 smithi123 ceph-mon[134433]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2023-09-20T22:02:01.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:00 smithi089 ceph-mon[134090]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2023-09-20T22:02:03.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:03 smithi123 ceph-mon[134433]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 553 op/s 2023-09-20T22:02:03.369 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:03 smithi089 ceph-mon[134090]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 553 op/s 2023-09-20T22:02:03.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:03 smithi022 ceph-mon[97651]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 553 op/s 2023-09-20T22:02:03.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 760.0 s, 549.2 tps, lat 187.932 ms stddev 208.411 2023-09-20T22:02:05.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:05 smithi123 ceph-mon[134433]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 475 op/s 2023-09-20T22:02:05.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:05 smithi089 ceph-mon[134090]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 475 op/s 2023-09-20T22:02:05.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:05 smithi022 ceph-mon[97651]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.3 MiB/s wr, 475 op/s 2023-09-20T22:02:07.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:07 smithi089 ceph-mon[134090]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 537 op/s 2023-09-20T22:02:08.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:07 smithi022 ceph-mon[97651]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 537 op/s 2023-09-20T22:02:08.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:07 smithi123 ceph-mon[134433]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 537 op/s 2023-09-20T22:02:08.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 765.0 s, 453.2 tps, lat 213.656 ms stddev 234.347 2023-09-20T22:02:09.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:09 smithi123 ceph-mon[134433]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 464 op/s 2023-09-20T22:02:09.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:09 smithi089 ceph-mon[134090]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 464 op/s 2023-09-20T22:02:09.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:09 smithi022 ceph-mon[97651]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 464 op/s 2023-09-20T22:02:11.845 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:11 smithi123 ceph-mon[134433]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 483 op/s 2023-09-20T22:02:11.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:11 smithi089 ceph-mon[134090]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 483 op/s 2023-09-20T22:02:11.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:11 smithi022 ceph-mon[97651]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 483 op/s 2023-09-20T22:02:13.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 770.0 s, 510.0 tps, lat 198.552 ms stddev 252.781 2023-09-20T22:02:13.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:13 smithi123 ceph-mon[134433]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.2 MiB/s wr, 521 op/s 2023-09-20T22:02:13.865 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:13 smithi089 ceph-mon[134090]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.2 MiB/s wr, 521 op/s 2023-09-20T22:02:13.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:13 smithi022 ceph-mon[97651]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.2 MiB/s wr, 521 op/s 2023-09-20T22:02:15.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:15 smithi123 ceph-mon[134433]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 443 op/s 2023-09-20T22:02:15.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:15 smithi089 ceph-mon[134090]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 443 op/s 2023-09-20T22:02:15.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:15 smithi022 ceph-mon[97651]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 443 op/s 2023-09-20T22:02:17.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:17 smithi022 ceph-mon[97651]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 488 op/s 2023-09-20T22:02:18.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:17 smithi123 ceph-mon[134433]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 488 op/s 2023-09-20T22:02:18.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:17 smithi089 ceph-mon[134090]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 488 op/s 2023-09-20T22:02:18.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 775.0 s, 417.8 tps, lat 231.438 ms stddev 279.560 2023-09-20T22:02:20.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:19 smithi022 ceph-mon[97651]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 443 op/s 2023-09-20T22:02:20.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:19 smithi123 ceph-mon[134433]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 443 op/s 2023-09-20T22:02:20.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:19 smithi089 ceph-mon[134090]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 443 op/s 2023-09-20T22:02:20.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:20 smithi022 ceph-mon[97651]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 472 op/s 2023-09-20T22:02:21.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:20 smithi123 ceph-mon[134433]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 472 op/s 2023-09-20T22:02:21.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:20 smithi089 ceph-mon[134090]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 472 op/s 2023-09-20T22:02:23.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:23 smithi022 ceph-mon[97651]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2023-09-20T22:02:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 780.0 s, 475.3 tps, lat 216.347 ms stddev 246.365 2023-09-20T22:02:23.596 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:23 smithi123 ceph-mon[134433]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2023-09-20T22:02:23.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:23 smithi089 ceph-mon[134090]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.9 MiB/s wr, 507 op/s 2023-09-20T22:02:25.851 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:25 smithi123 ceph-mon[134433]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.5 MiB/s wr, 443 op/s 2023-09-20T22:02:25.864 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:25 smithi089 ceph-mon[134090]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.5 MiB/s wr, 443 op/s 2023-09-20T22:02:25.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:25 smithi022 ceph-mon[97651]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.5 MiB/s wr, 443 op/s 2023-09-20T22:02:27.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:27 smithi123 ceph-mon[134433]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.1 MiB/s wr, 486 op/s 2023-09-20T22:02:27.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:27 smithi089 ceph-mon[134090]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.1 MiB/s wr, 486 op/s 2023-09-20T22:02:28.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:27 smithi022 ceph-mon[97651]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.1 MiB/s wr, 486 op/s 2023-09-20T22:02:28.499 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T22:02:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 785.0 s, 566.1 tps, lat 180.458 ms stddev 211.330 2023-09-20T22:02:29.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:28.998+0000 7fc29adff700 1 -- 172.21.15.22:0/274577575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc294078520 msgr2=0x7fc29407a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:29.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:28.998+0000 7fc29adff700 1 --2- 172.21.15.22:0/274577575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc294078520 0x7fc29407a9b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00d290 tx=0x7fc28c00d5a0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.011+0000 7fc29adff700 1 -- 172.21.15.22:0/274577575 shutdown_connections 2023-09-20T22:02:29.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.011+0000 7fc29adff700 1 --2- 172.21.15.22:0/274577575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc294078520 0x7fc29407a9b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.011+0000 7fc29adff700 1 --2- 172.21.15.22:0/274577575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc294072c70 0x7fc294073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.011+0000 7fc29adff700 1 --2- 172.21.15.22:0/274577575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc294075b50 0x7fc294077fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.011+0000 7fc29adff700 1 -- 172.21.15.22:0/274577575 >> 172.21.15.22:0/274577575 conn(0x7fc29406db90 msgr2=0x7fc29406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:29.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.016+0000 7fc29adff700 1 -- 172.21.15.22:0/274577575 shutdown_connections 2023-09-20T22:02:29.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.021+0000 7fc29adff700 1 -- 172.21.15.22:0/274577575 wait complete. 2023-09-20T22:02:29.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.023+0000 7fc29adff700 1 Processor -- start 2023-09-20T22:02:29.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.023+0000 7fc29adff700 1 -- start start 2023-09-20T22:02:29.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.023+0000 7fc29adff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc294072c70 0x7fc29412be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.023+0000 7fc29adff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc294075b50 0x7fc29412e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.023+0000 7fc29adff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc294078520 0x7fc2941308a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.023+0000 7fc29adff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc294130f00 con 0x7fc294075b50 2023-09-20T22:02:29.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.023+0000 7fc29adff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc294131040 con 0x7fc294078520 2023-09-20T22:02:29.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.023+0000 7fc29adff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc294131200 con 0x7fc294072c70 2023-09-20T22:02:29.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.024+0000 7fc29939c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc294078520 0x7fc2941308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.024+0000 7fc29939c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc294078520 0x7fc2941308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37474/0 (socket says 172.21.15.22:37474) 2023-09-20T22:02:29.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.024+0000 7fc29939c700 1 -- 172.21.15.22:0/2202028166 learned_addr learned my addr 172.21.15.22:0/2202028166 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:02:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.026+0000 7fc29939c700 1 -- 172.21.15.22:0/2202028166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc294072c70 msgr2=0x7fc29412be00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T22:02:29.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.026+0000 7fc29939c700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc294072c70 0x7fc29412be00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.026+0000 7fc29939c700 1 -- 172.21.15.22:0/2202028166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc294075b50 msgr2=0x7fc29412e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.026+0000 7fc29939c700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc294075b50 0x7fc29412e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.026+0000 7fc29939c700 1 -- 172.21.15.22:0/2202028166 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c009c40 con 0x7fc294078520 2023-09-20T22:02:29.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.027+0000 7fc29939c700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc294078520 0x7fc2941308a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc28c008d70 tx=0x7fc28c008e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.028+0000 7fc291ffb700 1 -- 172.21.15.22:0/2202028166 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c003e10 con 0x7fc294078520 2023-09-20T22:02:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.028+0000 7fc291ffb700 1 -- 172.21.15.22:0/2202028166 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc28c011b60 con 0x7fc294078520 2023-09-20T22:02:29.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.028+0000 7fc291ffb700 1 -- 172.21.15.22:0/2202028166 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c005540 con 0x7fc294078520 2023-09-20T22:02:29.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.028+0000 7fc29adff700 1 -- 172.21.15.22:0/2202028166 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc294131480 con 0x7fc294078520 2023-09-20T22:02:29.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.029+0000 7fc29adff700 1 -- 172.21.15.22:0/2202028166 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2941319c0 con 0x7fc294078520 2023-09-20T22:02:29.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.043+0000 7fc2ae7d9b80 1 Processor -- start 2023-09-20T22:02:29.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.044+0000 7fc2ae7d9b80 1 -- start start 2023-09-20T22:02:29.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.045+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a046f0 0x561128a04b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.045+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a05050 0x561128a055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.045+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05be0 0x561128a06060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.045+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561128978ca0 con 0x561128a05be0 2023-09-20T22:02:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.045+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112895cc70 con 0x561128a046f0 2023-09-20T22:02:29.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.045+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56112895a590 con 0x561128a05050 2023-09-20T22:02:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.047+0000 7fc298b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a05050 0x561128a055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.047+0000 7fc291ffb700 1 -- 172.21.15.22:0/2202028166 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fc28c011cd0 con 0x7fc294078520 2023-09-20T22:02:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.047+0000 7fc298b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a05050 0x561128a055e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:59668/0 (socket says 172.21.15.22:59668) 2023-09-20T22:02:29.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.047+0000 7fc298b9b700 1 -- 172.21.15.22:0/2277330995 learned_addr learned my addr 172.21.15.22:0/2277330995 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:02:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.048+0000 7fc293fff700 1 --2- 172.21.15.22:0/2277330995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a046f0 0x561128a04b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.048+0000 7fc298b9b700 1 -- 172.21.15.22:0/2277330995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a046f0 msgr2=0x561128a04b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.048+0000 7fc298b9b700 1 --2- 172.21.15.22:0/2277330995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a046f0 0x561128a04b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.048+0000 7fc298b9b700 1 -- 172.21.15.22:0/2277330995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05be0 msgr2=0x561128a06060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.048+0000 7fc298b9b700 1 --2- 172.21.15.22:0/2277330995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05be0 0x561128a06060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.048+0000 7fc298b9b700 1 -- 172.21.15.22:0/2277330995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561128780de0 con 0x561128a05050 2023-09-20T22:02:29.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.049+0000 7fc298b9b700 1 --2- 172.21.15.22:0/2277330995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a05050 0x561128a055e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc288009830 tx=0x7fc288009b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b35bfc16ea9968d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:29.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.049+0000 7fc27affd700 1 -- 172.21.15.22:0/2277330995 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288011830 con 0x561128a05050 2023-09-20T22:02:29.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.049+0000 7fc27affd700 1 -- 172.21.15.22:0/2277330995 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc288011e50 con 0x561128a05050 2023-09-20T22:02:29.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.049+0000 7fc27affd700 1 -- 172.21.15.22:0/2277330995 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288010380 con 0x561128a05050 2023-09-20T22:02:29.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.050+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/2277330995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a05050 msgr2=0x561128a055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.050+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/2277330995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a05050 0x561128a055e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc288009830 tx=0x7fc288009b40 comp rx=0 tx=0).stop 2023-09-20T22:02:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.050+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/2277330995 shutdown_connections 2023-09-20T22:02:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.050+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/2277330995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05be0 0x561128a06060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.050+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/2277330995 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a046f0 0x561128a04b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.050+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/2277330995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a05050 0x561128a055e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.050+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/2277330995 >> 172.21.15.22:0/2277330995 conn(0x561128931000 msgr2=0x5611289ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.051+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/2277330995 shutdown_connections 2023-09-20T22:02:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.051+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/2277330995 wait complete. 2023-09-20T22:02:29.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.051+0000 7fc291ffb700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc27c0a1410 0x7fc27c0a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.051+0000 7fc2ae7d9b80 1 -- start start 2023-09-20T22:02:29.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.051+0000 7fc291ffb700 1 -- 172.21.15.22:0/2202028166 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc28c0ce9f0 con 0x7fc294078520 2023-09-20T22:02:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.052+0000 7fc293fff700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc27c0a1410 0x7fc27c0a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.052+0000 7fc2ae7d9b80 1 Processor -- start 2023-09-20T22:02:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a046f0 0x561128a0e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a0e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a15800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561128955e80 con 0x561128a05050 2023-09-20T22:02:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611288f2740 con 0x561128a05be0 2023-09-20T22:02:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611288f23e0 con 0x561128a046f0 2023-09-20T22:02:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc2ae7d9b80 10 client.0 _ll_register_callbacks cb 0x56112883ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:02:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc29939c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a15800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc298b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a046f0 0x561128a0e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc29939c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a15800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37490/0 (socket says 172.21.15.22:37490) 2023-09-20T22:02:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc29939c700 1 -- 172.21.15.22:0/1477637694 learned_addr learned my addr 172.21.15.22:0/1477637694 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:02:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.053+0000 7fc298b9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a046f0 0x561128a0e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:59682/0 (socket says 172.21.15.22:59682) 2023-09-20T22:02:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.054+0000 7fc293fff700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc27c0a1410 0x7fc27c0a38d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc284004790 tx=0x7fc284008530 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.054+0000 7fc293fff700 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a0e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.054+0000 7fc298b9b700 1 -- 172.21.15.22:0/1477637694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 msgr2=0x561128a15800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.054+0000 7fc298b9b700 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a15800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.054+0000 7fc298b9b700 1 -- 172.21.15.22:0/1477637694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 msgr2=0x561128a0e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.054+0000 7fc298b9b700 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a0e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.054+0000 7fc298b9b700 1 -- 172.21.15.22:0/1477637694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc288009460 con 0x561128a046f0 2023-09-20T22:02:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.055+0000 7fc298b9b700 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a046f0 0x561128a0e280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x561128a00150 tx=0x7fc28800b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c10ad3b8577bb1b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.055+0000 7fc273fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T22:02:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.056+0000 7fc273fff700 1 -- 172.21.15.22:0/1477637694 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288011830 con 0x561128a046f0 2023-09-20T22:02:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.056+0000 7fc273fff700 1 -- 172.21.15.22:0/1477637694 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc28800be60 con 0x561128a046f0 2023-09-20T22:02:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.056+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/1477637694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611288c6bb0 con 0x561128a046f0 2023-09-20T22:02:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.057+0000 7fc273fff700 1 -- 172.21.15.22:0/1477637694 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc288011990 con 0x561128a046f0 2023-09-20T22:02:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.057+0000 7fc2ae7d9b80 10 client.26140 fetch_fsmap learned FSMap version 32 2023-09-20T22:02:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.057+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/1477637694 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561128769c60 con 0x561128a046f0 2023-09-20T22:02:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.058+0000 7fc273fff700 1 -- 172.21.15.22:0/1477637694 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611288c6bb0 con 0x561128a046f0 2023-09-20T22:02:29.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.058+0000 7fc273fff700 1 -- 172.21.15.22:0/1477637694 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc28802b050 con 0x561128a046f0 2023-09-20T22:02:29.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.058+0000 7fc2ae7d9b80 10 client.26140 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:02:29.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.058+0000 7fc2ae7d9b80 10 client.26140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:02:29.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.058+0000 7fc2ae7d9b80 20 client.26140 populate_metadata read hostname 'smithi022' 2023-09-20T22:02:29.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.058+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561128a19e80 0x561128a1c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.058+0000 7fc2ae7d9b80 4 client.26140 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T22:02:29.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.058+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/1477637694 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611289d3020 con 0x561128a19e80 2023-09-20T22:02:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.058+0000 7fc293fff700 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561128a19e80 0x561128a1c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.059+0000 7fc273fff700 10 client.26140 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:02:29.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.060+0000 7fc293fff700 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561128a19e80 0x561128a1c270 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=777b5da52a968525 server_cookie=6d7c9076e3f05ff3 in_seq=0 out_seq=0 2023-09-20T22:02:29.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.061+0000 7fc273fff700 1 -- 172.21.15.22:0/1477637694 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5611289d3020 con 0x561128a19e80 2023-09-20T22:02:29.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.061+0000 7fc273fff700 10 client.26140 handle_command_reply: tid=0 2023-09-20T22:02:29.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.062+0000 7fc2ae7d9b80 1 client.26140 shutdown 2023-09-20T22:02:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.062+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/1477637694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a046f0 msgr2=0x561128a0e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.062+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a046f0 0x561128a0e280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x561128a00150 tx=0x7fc28800b3e0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.062+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/1477637694 shutdown_connections 2023-09-20T22:02:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.062+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561128a19e80 0x561128a1c270 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.062+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a15800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.062+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561128a046f0 0x561128a0e280 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x561128a00150 tx=0x7fc28800b3e0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.062+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/1477637694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a0e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.062+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/1477637694 >> 172.21.15.22:0/1477637694 conn(0x561128931000 msgr2=0x5611289fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.062+0000 7fc273fff700 0 client.26140 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:02:29.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.063+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/1477637694 shutdown_connections 2023-09-20T22:02:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.063+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/1477637694 wait complete. 2023-09-20T22:02:29.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.063+0000 7fc2ae7d9b80 20 client.26140 trim_cache size 0 max 16384 2023-09-20T22:02:29.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.093+0000 7fc2ae7d9b80 1 Processor -- start 2023-09-20T22:02:29.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.093+0000 7fc2ae7d9b80 1 -- start start 2023-09-20T22:02:29.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a08cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a09200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611289fb2e0 0x561128a157c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611287d6920 con 0x561128a05050 2023-09-20T22:02:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561128a0c810 con 0x561128a05be0 2023-09-20T22:02:29.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611289d3020 con 0x5611289fb2e0 2023-09-20T22:02:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc298b9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a08cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc293fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a09200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc298b9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a08cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:43088/0 (socket says 172.21.15.22:43088) 2023-09-20T22:02:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc298b9b700 1 -- 172.21.15.22:0/135149268 learned_addr learned my addr 172.21.15.22:0/135149268 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:02:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.094+0000 7fc29939c700 1 --2- 172.21.15.22:0/135149268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611289fb2e0 0x561128a157c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.095+0000 7fc298b9b700 1 -- 172.21.15.22:0/135149268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611289fb2e0 msgr2=0x561128a157c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:29.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.095+0000 7fc298b9b700 1 --2- 172.21.15.22:0/135149268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611289fb2e0 0x561128a157c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.095+0000 7fc298b9b700 1 -- 172.21.15.22:0/135149268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 msgr2=0x561128a09200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.095+0000 7fc298b9b700 1 --2- 172.21.15.22:0/135149268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a09200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.095+0000 7fc298b9b700 1 -- 172.21.15.22:0/135149268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611287a7250 con 0x561128a05050 2023-09-20T22:02:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.095+0000 7fc293fff700 1 --2- 172.21.15.22:0/135149268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a09200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:02:29.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.095+0000 7fc298b9b700 1 --2- 172.21.15.22:0/135149268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a08cc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc28802edd0 tx=0x7fc288009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d577126922da78cd server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.095+0000 7fc27b7fe700 1 -- 172.21.15.22:0/135149268 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28800be60 con 0x561128a05050 2023-09-20T22:02:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.096+0000 7fc27b7fe700 1 -- 172.21.15.22:0/135149268 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc28801e070 con 0x561128a05050 2023-09-20T22:02:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.096+0000 7fc27b7fe700 1 -- 172.21.15.22:0/135149268 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28801aab0 con 0x561128a05050 2023-09-20T22:02:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.096+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/135149268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 msgr2=0x561128a08cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:29.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.096+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/135149268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a08cc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc28802edd0 tx=0x7fc288009b40 comp rx=0 tx=0).stop 2023-09-20T22:02:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.096+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/135149268 shutdown_connections 2023-09-20T22:02:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.097+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/135149268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611289fb2e0 0x561128a157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.097+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/135149268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a08cc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.097+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/135149268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a09200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.097+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/135149268 >> 172.21.15.22:0/135149268 conn(0x561128931000 msgr2=0x561128a16030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.097+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/135149268 shutdown_connections 2023-09-20T22:02:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.097+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/135149268 wait complete. 2023-09-20T22:02:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.097+0000 7fc2ae7d9b80 1 -- start start 2023-09-20T22:02:29.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.097+0000 7fc2ae7d9b80 1 Processor -- start 2023-09-20T22:02:29.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611289fb2e0 0x561128a15340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a13950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc2ae7d9b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a13e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x561128a15b60 con 0x561128a05050 2023-09-20T22:02:29.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x561128a143d0 con 0x561128a05be0 2023-09-20T22:02:29.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc2ae7d9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x561128a14560 con 0x5611289fb2e0 2023-09-20T22:02:29.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc2ae7d9b80 10 client.0 _ll_register_callbacks cb 0x56112883ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:02:29.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc29939c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a13e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc29939c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a13e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37508/0 (socket says 172.21.15.22:37508) 2023-09-20T22:02:29.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc29939c700 1 -- 172.21.15.22:0/3192880191 learned_addr learned my addr 172.21.15.22:0/3192880191 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:02:29.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc298b9b700 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611289fb2e0 0x561128a15340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc293fff700 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a13950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc29939c700 1 -- 172.21.15.22:0/3192880191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611289fb2e0 msgr2=0x561128a15340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:29.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc29939c700 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611289fb2e0 0x561128a15340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc29939c700 1 -- 172.21.15.22:0/3192880191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 msgr2=0x561128a13950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:29.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.098+0000 7fc29939c700 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a13950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc29939c700 1 -- 172.21.15.22:0/3192880191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc284008bd0 con 0x561128a05be0 2023-09-20T22:02:29.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc293fff700 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a13950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:02:29.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc29939c700 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a13e90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc28c098490 tx=0x7fc28c00d5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adece5967ff92099 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:29.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc273fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T22:02:29.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc273fff700 1 -- 172.21.15.22:0/3192880191 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c09be80 con 0x561128a05be0 2023-09-20T22:02:29.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc273fff700 1 -- 172.21.15.22:0/3192880191 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fc28c09c120 con 0x561128a05be0 2023-09-20T22:02:29.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/3192880191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc288011e50 con 0x561128a05be0 2023-09-20T22:02:29.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc273fff700 1 -- 172.21.15.22:0/3192880191 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc28c097d60 con 0x561128a05be0 2023-09-20T22:02:29.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc2ae7d9b80 10 client.26111 fetch_fsmap learned FSMap version 32 2023-09-20T22:02:29.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/3192880191 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561128769c60 con 0x561128a05be0 2023-09-20T22:02:29.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.099+0000 7fc273fff700 1 -- 172.21.15.22:0/3192880191 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc288011e50 con 0x561128a05be0 2023-09-20T22:02:29.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.100+0000 7fc273fff700 1 -- 172.21.15.22:0/3192880191 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc28c0b0030 con 0x561128a05be0 2023-09-20T22:02:29.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.100+0000 7fc2ae7d9b80 10 client.26111 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:02:29.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.100+0000 7fc2ae7d9b80 10 client.26111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:02:29.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.100+0000 7fc2ae7d9b80 20 client.26111 populate_metadata read hostname 'smithi022' 2023-09-20T22:02:29.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.100+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561128a6efd0 0x561128a713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:29.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.100+0000 7fc2ae7d9b80 4 client.26111 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T22:02:29.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.100+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/3192880191 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56112894ee60 con 0x561128a6efd0 2023-09-20T22:02:29.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.100+0000 7fc298b9b700 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561128a6efd0 0x561128a713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:29.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc273fff700 10 client.26111 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:02:29.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc298b9b700 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561128a6efd0 0x561128a713f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd1a1f328786ab42 server_cookie=cc1e6b9411db36a5 in_seq=0 out_seq=0 2023-09-20T22:02:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc273fff700 1 -- 172.21.15.22:0/3192880191 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56112894ee60 con 0x561128a6efd0 2023-09-20T22:02:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc273fff700 10 client.26111 handle_command_reply: tid=0 2023-09-20T22:02:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 client.26111 shutdown 2023-09-20T22:02:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/3192880191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 msgr2=0x561128a13e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a13e90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc28c098490 tx=0x7fc28c00d5a0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/3192880191 shutdown_connections 2023-09-20T22:02:29.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x561128a6efd0 0x561128a713f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561128a05be0 0x561128a13e90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611289fb2e0 0x561128a15340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc273fff700 0 client.26111 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:02:29.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 --2- 172.21.15.22:0/3192880191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561128a05050 0x561128a13950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/3192880191 >> 172.21.15.22:0/3192880191 conn(0x561128931000 msgr2=0x561128a08cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:29.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/3192880191 shutdown_connections 2023-09-20T22:02:29.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.101+0000 7fc2ae7d9b80 1 -- 172.21.15.22:0/3192880191 wait complete. 2023-09-20T22:02:29.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc2ae7d9b80 20 client.26111 trim_cache size 0 max 16384 2023-09-20T22:02:29.115 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:02:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T22:02:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T22:02:29.115 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T22:02:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T22:02:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T22:02:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T22:02:29.116 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:02:29.117 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:02:29.117 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:02:29.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 -- 172.21.15.22:0/2202028166 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc27c0a1410 msgr2=0x7fc27c0a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:29.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc27c0a1410 0x7fc27c0a38d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc284004790 tx=0x7fc284008530 comp rx=0 tx=0).stop 2023-09-20T22:02:29.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 -- 172.21.15.22:0/2202028166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc294078520 msgr2=0x7fc2941308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:29.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc294078520 0x7fc2941308a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc28c008d70 tx=0x7fc28c008e50 comp rx=0 tx=0).stop 2023-09-20T22:02:29.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 -- 172.21.15.22:0/2202028166 shutdown_connections 2023-09-20T22:02:29.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fc27c0a1410 0x7fc27c0a38d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc294078520 0x7fc2941308a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc294072c70 0x7fc29412be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 --2- 172.21.15.22:0/2202028166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc294075b50 0x7fc29412e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:29.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 -- 172.21.15.22:0/2202028166 >> 172.21.15.22:0/2202028166 conn(0x7fc29406db90 msgr2=0x7fc294076830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:29.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 -- 172.21.15.22:0/2202028166 shutdown_connections 2023-09-20T22:02:29.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:29.102+0000 7fc29adff700 1 -- 172.21.15.22:0/2202028166 wait complete. 2023-09-20T22:02:29.125 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T22:02:30.006 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:29 smithi022 ceph-mon[97651]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.6 MiB/s wr, 466 op/s 2023-09-20T22:02:30.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:29 smithi123 ceph-mon[134433]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.6 MiB/s wr, 466 op/s 2023-09-20T22:02:30.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:29 smithi089 ceph-mon[134090]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.6 MiB/s wr, 466 op/s 2023-09-20T22:02:30.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:30 smithi022 ceph-mon[97651]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.7 MiB/s wr, 479 op/s 2023-09-20T22:02:31.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:30 smithi123 ceph-mon[134433]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.7 MiB/s wr, 479 op/s 2023-09-20T22:02:31.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:30 smithi089 ceph-mon[134090]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.7 MiB/s wr, 479 op/s 2023-09-20T22:02:33.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:33 smithi022 ceph-mon[97651]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.4 MiB/s wr, 547 op/s 2023-09-20T22:02:33.573 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 790.0 s, 456.6 tps, lat 214.065 ms stddev 239.047 2023-09-20T22:02:33.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:33 smithi123 ceph-mon[134433]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.4 MiB/s wr, 547 op/s 2023-09-20T22:02:33.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:33 smithi089 ceph-mon[134090]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.4 MiB/s wr, 547 op/s 2023-09-20T22:02:35.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:35 smithi123 ceph-mon[134433]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.1 MiB/s wr, 487 op/s 2023-09-20T22:02:35.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:35 smithi089 ceph-mon[134090]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.1 MiB/s wr, 487 op/s 2023-09-20T22:02:35.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:35 smithi022 ceph-mon[97651]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.1 MiB/s wr, 487 op/s 2023-09-20T22:02:37.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:37 smithi022 ceph-mon[97651]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.0 MiB/s wr, 514 op/s 2023-09-20T22:02:38.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:37 smithi123 ceph-mon[134433]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.0 MiB/s wr, 514 op/s 2023-09-20T22:02:38.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:37 smithi089 ceph-mon[134090]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.0 MiB/s wr, 514 op/s 2023-09-20T22:02:38.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 795.0 s, 521.7 tps, lat 196.938 ms stddev 216.934 2023-09-20T22:02:39.846 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:39 smithi123 ceph-mon[134433]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 512 op/s 2023-09-20T22:02:39.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:39 smithi089 ceph-mon[134090]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 512 op/s 2023-09-20T22:02:39.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:39 smithi022 ceph-mon[97651]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 512 op/s 2023-09-20T22:02:41.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:41 smithi123 ceph-mon[134433]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.9 MiB/s wr, 513 op/s 2023-09-20T22:02:41.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:41 smithi089 ceph-mon[134090]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.9 MiB/s wr, 513 op/s 2023-09-20T22:02:41.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:41 smithi022 ceph-mon[97651]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.9 MiB/s wr, 513 op/s 2023-09-20T22:02:43.587 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 800.0 s, 452.4 tps, lat 206.834 ms stddev 206.062 2023-09-20T22:02:43.862 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:43 smithi089 ceph-mon[134090]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.4 MiB/s wr, 557 op/s 2023-09-20T22:02:43.863 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:43 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:02:44.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:43 smithi022 ceph-mon[97651]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.4 MiB/s wr, 557 op/s 2023-09-20T22:02:44.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:43 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:02:44.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:43 smithi123 ceph-mon[134433]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.4 MiB/s wr, 557 op/s 2023-09-20T22:02:44.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:43 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:02:45.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:45 smithi089 ceph-mon[134090]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 MiB/s wr, 459 op/s 2023-09-20T22:02:45.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:45 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:02:45.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:45 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:02:45.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:45 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:02:46.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:45 smithi022 ceph-mon[97651]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 MiB/s wr, 459 op/s 2023-09-20T22:02:46.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:02:46.003 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:02:46.003 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:45 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:02:46.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:45 smithi123 ceph-mon[134433]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.4 MiB/s wr, 459 op/s 2023-09-20T22:02:46.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:45 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:02:46.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:45 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:02:46.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:45 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:02:47.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:47 smithi022 ceph-mon[97651]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.3 MiB/s wr, 444 op/s 2023-09-20T22:02:48.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:47 smithi123 ceph-mon[134433]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.3 MiB/s wr, 444 op/s 2023-09-20T22:02:48.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:47 smithi089 ceph-mon[134090]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.3 MiB/s wr, 444 op/s 2023-09-20T22:02:48.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 805.0 s, 445.6 tps, lat 234.040 ms stddev 314.236 2023-09-20T22:02:50.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:49 smithi022 ceph-mon[97651]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.7 MiB/s wr, 475 op/s 2023-09-20T22:02:50.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:49 smithi123 ceph-mon[134433]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.7 MiB/s wr, 475 op/s 2023-09-20T22:02:50.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:49 smithi089 ceph-mon[134090]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.7 MiB/s wr, 475 op/s 2023-09-20T22:02:51.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:51 smithi022 ceph-mon[97651]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.4 MiB/s wr, 460 op/s 2023-09-20T22:02:52.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:51 smithi123 ceph-mon[134433]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.4 MiB/s wr, 460 op/s 2023-09-20T22:02:52.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:51 smithi089 ceph-mon[134090]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.4 MiB/s wr, 460 op/s 2023-09-20T22:02:52.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:52 smithi022 ceph-mon[97651]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 4.9 MiB/s wr, 508 op/s 2023-09-20T22:02:53.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:52 smithi123 ceph-mon[134433]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 4.9 MiB/s wr, 508 op/s 2023-09-20T22:02:53.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:52 smithi089 ceph-mon[134090]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 4.9 MiB/s wr, 508 op/s 2023-09-20T22:02:53.570 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 810.0 s, 573.6 tps, lat 178.007 ms stddev 231.149 2023-09-20T22:02:55.875 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:55 smithi089 ceph-mon[134090]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 4.0 MiB/s wr, 413 op/s 2023-09-20T22:02:56.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:55 smithi022 ceph-mon[97651]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 4.0 MiB/s wr, 413 op/s 2023-09-20T22:02:56.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:55 smithi123 ceph-mon[134433]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 4.0 MiB/s wr, 413 op/s 2023-09-20T22:02:57.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:57 smithi022 ceph-mon[97651]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 4.4 MiB/s wr, 440 op/s 2023-09-20T22:02:58.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:57 smithi123 ceph-mon[134433]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 4.4 MiB/s wr, 440 op/s 2023-09-20T22:02:58.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:57 smithi089 ceph-mon[134090]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 4.4 MiB/s wr, 440 op/s 2023-09-20T22:02:58.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 815.0 s, 572.7 tps, lat 173.283 ms stddev 188.232 2023-09-20T22:02:59.126 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T22:02:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.549+0000 7fe376bff700 1 -- 172.21.15.22:0/4086843114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe370069b30 msgr2=0x7fe370108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.549+0000 7fe376bff700 1 --2- 172.21.15.22:0/4086843114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe370069b30 0x7fe370108050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe360009f90 tx=0x7fe36000b330 comp rx=0 tx=0).stop 2023-09-20T22:02:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.549+0000 7fe376bff700 1 -- 172.21.15.22:0/4086843114 shutdown_connections 2023-09-20T22:02:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.549+0000 7fe376bff700 1 --2- 172.21.15.22:0/4086843114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3701087b0 0x7fe370108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.549+0000 7fe376bff700 1 --2- 172.21.15.22:0/4086843114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe370069b30 0x7fe370108050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.549+0000 7fe376bff700 1 --2- 172.21.15.22:0/4086843114 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37010ba00 0x7fe37010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.549+0000 7fe376bff700 1 -- 172.21.15.22:0/4086843114 >> 172.21.15.22:0/4086843114 conn(0x7fe37006dfe0 msgr2=0x7fe370070400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:59.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.549+0000 7fe376bff700 1 -- 172.21.15.22:0/4086843114 shutdown_connections 2023-09-20T22:02:59.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.550+0000 7fe376bff700 1 -- 172.21.15.22:0/4086843114 wait complete. 2023-09-20T22:02:59.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.551+0000 7fe376bff700 1 Processor -- start 2023-09-20T22:02:59.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.551+0000 7fe376bff700 1 -- start start 2023-09-20T22:02:59.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe376bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe370069b30 0x7fe3701a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe376bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3701087b0 0x7fe3701a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe376bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37010ba00 0x7fe3701a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe376bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3701a9220 con 0x7fe37010ba00 2023-09-20T22:02:59.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe376bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3701a9360 con 0x7fe3701087b0 2023-09-20T22:02:59.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe376bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3701a94f0 con 0x7fe370069b30 2023-09-20T22:02:59.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe37499b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe370069b30 0x7fe3701a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe36ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3701087b0 0x7fe3701a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe37499b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe370069b30 0x7fe3701a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:46502/0 (socket says 172.21.15.22:46502) 2023-09-20T22:02:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe37499b700 1 -- 172.21.15.22:0/3393975669 learned_addr learned my addr 172.21.15.22:0/3393975669 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:02:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe36ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3701087b0 0x7fe3701a1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37570/0 (socket says 172.21.15.22:37570) 2023-09-20T22:02:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe37499b700 1 -- 172.21.15.22:0/3393975669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3701087b0 msgr2=0x7fe3701a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe37499b700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3701087b0 0x7fe3701a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe37499b700 1 -- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37010ba00 msgr2=0x7fe3701a8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.552+0000 7fe37519c700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37010ba00 0x7fe3701a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.553+0000 7fe37499b700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37010ba00 0x7fe3701a8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.553+0000 7fe37499b700 1 -- 172.21.15.22:0/3393975669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360009c40 con 0x7fe370069b30 2023-09-20T22:02:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.553+0000 7fe36ffff700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3701087b0 0x7fe3701a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:02:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.553+0000 7fe37519c700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37010ba00 0x7fe3701a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T22:02:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.553+0000 7fe37499b700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe370069b30 0x7fe3701a16a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe360000f80 tx=0x7fe360004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:59.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.553+0000 7fe36dffb700 1 -- 172.21.15.22:0/3393975669 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36000e040 con 0x7fe370069b30 2023-09-20T22:02:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.553+0000 7fe36dffb700 1 -- 172.21.15.22:0/3393975669 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36000be00 con 0x7fe370069b30 2023-09-20T22:02:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.553+0000 7fe36dffb700 1 -- 172.21.15.22:0/3393975669 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600053a0 con 0x7fe370069b30 2023-09-20T22:02:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.553+0000 7fe376bff700 1 -- 172.21.15.22:0/3393975669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3701a9770 con 0x7fe370069b30 2023-09-20T22:02:59.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.554+0000 7fe376bff700 1 -- 172.21.15.22:0/3393975669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3701a9ba0 con 0x7fe370069b30 2023-09-20T22:02:59.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.559+0000 7fe36dffb700 1 -- 172.21.15.22:0/3393975669 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe36001d470 con 0x7fe370069b30 2023-09-20T22:02:59.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.561+0000 7fe38a439b80 1 Processor -- start 2023-09-20T22:02:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.561+0000 7fe38a439b80 1 -- start start 2023-09-20T22:02:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.562+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb76f0 0x565144fb7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.562+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fb85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.562+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb8be0 0x565144fb9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.562+0000 7fe38a439b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144f2bca0 con 0x565144fb8be0 2023-09-20T22:02:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.562+0000 7fe38a439b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144f0fc70 con 0x565144fb8050 2023-09-20T22:02:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.562+0000 7fe38a439b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144f0d590 con 0x565144fb76f0 2023-09-20T22:02:59.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.562+0000 7fe36dffb700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe35c0a1380 0x7fe35c0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37519c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb76f0 0x565144fb7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37499b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fb85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37519c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb76f0 0x565144fb7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:46512/0 (socket says 172.21.15.22:46512) 2023-09-20T22:02:59.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37519c700 1 -- 172.21.15.22:0/888218762 learned_addr learned my addr 172.21.15.22:0/888218762 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:02:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe36dffb700 1 -- 172.21.15.22:0/3393975669 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe3600ce2e0 con 0x7fe370069b30 2023-09-20T22:02:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe36ffff700 1 --2- 172.21.15.22:0/888218762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb8be0 0x565144fb9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37519c700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe35c0a1380 0x7fe35c0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37499b700 1 -- 172.21.15.22:0/888218762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb76f0 msgr2=0x565144fb7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37499b700 1 --2- 172.21.15.22:0/888218762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb76f0 0x565144fb7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37499b700 1 -- 172.21.15.22:0/888218762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb8be0 msgr2=0x565144fb9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37499b700 1 --2- 172.21.15.22:0/888218762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb8be0 0x565144fb9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37499b700 1 -- 172.21.15.22:0/888218762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565144d33de0 con 0x565144fb8050 2023-09-20T22:02:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.563+0000 7fe37519c700 1 --2- 172.21.15.22:0/888218762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb76f0 0x565144fb7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:02:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.564+0000 7fe37499b700 1 --2- 172.21.15.22:0/888218762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fb85e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe36009b300 tx=0x7fe36001de00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4959350459ebb0fd server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.564+0000 7fe366ffd700 1 -- 172.21.15.22:0/888218762 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36009aeb0 con 0x565144fb8050 2023-09-20T22:02:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.564+0000 7fe37519c700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe35c0a1380 0x7fe35c0a3840 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe368005950 tx=0x7fe368009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.564+0000 7fe366ffd700 1 -- 172.21.15.22:0/888218762 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36009acd0 con 0x565144fb8050 2023-09-20T22:02:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.564+0000 7fe366ffd700 1 -- 172.21.15.22:0/888218762 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600a3350 con 0x565144fb8050 2023-09-20T22:02:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.565+0000 7fe38a439b80 1 -- 172.21.15.22:0/888218762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 msgr2=0x565144fb85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.565+0000 7fe38a439b80 1 --2- 172.21.15.22:0/888218762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fb85e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe36009b300 tx=0x7fe36001de00 comp rx=0 tx=0).stop 2023-09-20T22:02:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.565+0000 7fe38a439b80 1 -- 172.21.15.22:0/888218762 shutdown_connections 2023-09-20T22:02:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.565+0000 7fe38a439b80 1 --2- 172.21.15.22:0/888218762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb8be0 0x565144fb9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.565+0000 7fe38a439b80 1 --2- 172.21.15.22:0/888218762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb76f0 0x565144fb7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.565+0000 7fe38a439b80 1 --2- 172.21.15.22:0/888218762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fb85e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.565+0000 7fe38a439b80 1 -- 172.21.15.22:0/888218762 >> 172.21.15.22:0/888218762 conn(0x565144ee4000 msgr2=0x565144fb21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.565+0000 7fe38a439b80 1 -- 172.21.15.22:0/888218762 shutdown_connections 2023-09-20T22:02:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.565+0000 7fe38a439b80 1 -- 172.21.15.22:0/888218762 wait complete. 2023-09-20T22:02:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.566+0000 7fe38a439b80 1 -- start start 2023-09-20T22:02:59.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.566+0000 7fe38a439b80 1 Processor -- start 2023-09-20T22:02:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 0x565144fc0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fc14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8be0 0x565144fc8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe38a439b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144f08e80 con 0x565144fb76f0 2023-09-20T22:02:59.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe38a439b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144ea5740 con 0x565144fb8050 2023-09-20T22:02:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe38a439b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144ea53e0 con 0x565144fb8be0 2023-09-20T22:02:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe38a439b80 10 client.0 _ll_register_callbacks cb 0x565144defa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:02:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe36ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fc14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe37519c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8be0 0x565144fc8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe36ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fc14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37598/0 (socket says 172.21.15.22:37598) 2023-09-20T22:02:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.567+0000 7fe36ffff700 1 -- 172.21.15.22:0/38256024 learned_addr learned my addr 172.21.15.22:0/38256024 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:02:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.568+0000 7fe36ffff700 1 -- 172.21.15.22:0/38256024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8be0 msgr2=0x565144fc8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.568+0000 7fe36ffff700 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8be0 0x565144fc8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.568+0000 7fe36ffff700 1 -- 172.21.15.22:0/38256024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 msgr2=0x565144fc0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.568+0000 7fe36ffff700 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 0x565144fc0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.568+0000 7fe36ffff700 1 -- 172.21.15.22:0/38256024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe368013460 con 0x565144fb8050 2023-09-20T22:02:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.569+0000 7fe36ffff700 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fc14e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe35800bed0 tx=0x7fe35800d620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38ae6aa7a2ff8a01 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.569+0000 7fe34ffff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T22:02:59.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.569+0000 7fe34ffff700 1 -- 172.21.15.22:0/38256024 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580203f0 con 0x565144fb8050 2023-09-20T22:02:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.569+0000 7fe34ffff700 1 -- 172.21.15.22:0/38256024 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe358020550 con 0x565144fb8050 2023-09-20T22:02:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.569+0000 7fe38a439b80 1 -- 172.21.15.22:0/38256024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565144e79bb0 con 0x565144fb8050 2023-09-20T22:02:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.570+0000 7fe34ffff700 1 -- 172.21.15.22:0/38256024 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe358020710 con 0x565144fb8050 2023-09-20T22:02:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.570+0000 7fe38a439b80 10 client.26123 fetch_fsmap learned FSMap version 32 2023-09-20T22:02:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.570+0000 7fe38a439b80 1 -- 172.21.15.22:0/38256024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fe3701a9ba0 con 0x565144fb8050 2023-09-20T22:02:59.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.570+0000 7fe34ffff700 1 -- 172.21.15.22:0/38256024 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565144e79bb0 con 0x565144fb8050 2023-09-20T22:02:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.570+0000 7fe34ffff700 1 -- 172.21.15.22:0/38256024 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe35801f070 con 0x565144fb8050 2023-09-20T22:02:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.570+0000 7fe38a439b80 10 client.26123 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:02:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.570+0000 7fe38a439b80 10 client.26123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:02:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.571+0000 7fe38a439b80 20 client.26123 populate_metadata read hostname 'smithi022' 2023-09-20T22:02:59.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.571+0000 7fe38a439b80 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x565144fcced0 0x565144fcf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.571+0000 7fe38a439b80 4 client.26123 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T22:02:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.571+0000 7fe38a439b80 1 -- 172.21.15.22:0/38256024 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565144f86020 con 0x565144fcced0 2023-09-20T22:02:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.571+0000 7fe37499b700 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x565144fcced0 0x565144fcf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.572+0000 7fe37499b700 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x565144fcced0 0x565144fcf2f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e96e187ed05f91a4 server_cookie=daaeb87e9eedc284 in_seq=0 out_seq=0 2023-09-20T22:02:59.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.572+0000 7fe34ffff700 10 client.26123 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:02:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.572+0000 7fe34ffff700 1 -- 172.21.15.22:0/38256024 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565144f86020 con 0x565144fcced0 2023-09-20T22:02:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.572+0000 7fe34ffff700 10 client.26123 handle_command_reply: tid=0 2023-09-20T22:02:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 client.26123 shutdown 2023-09-20T22:02:59.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 -- 172.21.15.22:0/38256024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 msgr2=0x565144fc14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fc14e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe35800bed0 tx=0x7fe35800d620 comp rx=0 tx=0).stop 2023-09-20T22:02:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 -- 172.21.15.22:0/38256024 shutdown_connections 2023-09-20T22:02:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x565144fcced0 0x565144fcf2f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8be0 0x565144fc8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 0x565144fc0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 --2- 172.21.15.22:0/38256024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8050 0x565144fc14e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 -- 172.21.15.22:0/38256024 >> 172.21.15.22:0/38256024 conn(0x565144ee4000 msgr2=0x565144fb1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe34ffff700 0 client.26123 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:02:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 -- 172.21.15.22:0/38256024 shutdown_connections 2023-09-20T22:02:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.573+0000 7fe38a439b80 1 -- 172.21.15.22:0/38256024 wait complete. 2023-09-20T22:02:59.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.574+0000 7fe38a439b80 20 client.26123 trim_cache size 0 max 16384 2023-09-20T22:02:59.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.603+0000 7fe38a439b80 1 Processor -- start 2023-09-20T22:02:59.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.603+0000 7fe38a439b80 1 -- start start 2023-09-20T22:02:59.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 0x565144fbfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8be0 0x565144fbf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe38a439b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144d89920 con 0x565144fb76f0 2023-09-20T22:02:59.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe38a439b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144facc20 con 0x565144fb8be0 2023-09-20T22:02:59.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe38a439b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144fb8a40 con 0x565144fb8050 2023-09-20T22:02:59.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe37499b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 0x565144fbfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe36ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe37499b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 0x565144fbfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36284/0 (socket says 172.21.15.22:36284) 2023-09-20T22:02:59.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe36ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:46528/0 (socket says 172.21.15.22:46528) 2023-09-20T22:02:59.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe37499b700 1 -- 172.21.15.22:0/1207732591 learned_addr learned my addr 172.21.15.22:0/1207732591 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:02:59.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.604+0000 7fe37519c700 1 --2- 172.21.15.22:0/1207732591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8be0 0x565144fbf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.605+0000 7fe36ffff700 1 -- 172.21.15.22:0/1207732591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8be0 msgr2=0x565144fbf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:59.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.605+0000 7fe36ffff700 1 --2- 172.21.15.22:0/1207732591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8be0 0x565144fbf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.605+0000 7fe36ffff700 1 -- 172.21.15.22:0/1207732591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 msgr2=0x565144fbfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:59.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.605+0000 7fe36ffff700 1 --2- 172.21.15.22:0/1207732591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 0x565144fbfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.605+0000 7fe36ffff700 1 -- 172.21.15.22:0/1207732591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565144d1cc60 con 0x565144fb8050 2023-09-20T22:02:59.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.605+0000 7fe37499b700 1 --2- 172.21.15.22:0/1207732591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 0x565144fbfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:02:59.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.605+0000 7fe36ffff700 1 --2- 172.21.15.22:0/1207732591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc84e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe358000c00 tx=0x7fe358005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97766e66e2bd05ae server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:59.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.605+0000 7fe3677fe700 1 -- 172.21.15.22:0/1207732591 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800de60 con 0x565144fb8050 2023-09-20T22:02:59.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.605+0000 7fe3677fe700 1 -- 172.21.15.22:0/1207732591 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe35801c040 con 0x565144fb8050 2023-09-20T22:02:59.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.606+0000 7fe3677fe700 1 -- 172.21.15.22:0/1207732591 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580052d0 con 0x565144fb8050 2023-09-20T22:02:59.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.607+0000 7fe38a439b80 1 -- 172.21.15.22:0/1207732591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 msgr2=0x565144fc84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:59.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.607+0000 7fe38a439b80 1 --2- 172.21.15.22:0/1207732591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc84e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe358000c00 tx=0x7fe358005a20 comp rx=0 tx=0).stop 2023-09-20T22:02:59.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.607+0000 7fe38a439b80 1 -- 172.21.15.22:0/1207732591 shutdown_connections 2023-09-20T22:02:59.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.607+0000 7fe38a439b80 1 --2- 172.21.15.22:0/1207732591 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb8be0 0x565144fbf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.607+0000 7fe38a439b80 1 --2- 172.21.15.22:0/1207732591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb76f0 0x565144fbfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.607+0000 7fe38a439b80 1 --2- 172.21.15.22:0/1207732591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc84e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.607+0000 7fe38a439b80 1 -- 172.21.15.22:0/1207732591 >> 172.21.15.22:0/1207732591 conn(0x565144ee4000 msgr2=0x565144fb1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:59.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.607+0000 7fe38a439b80 1 -- 172.21.15.22:0/1207732591 shutdown_connections 2023-09-20T22:02:59.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.607+0000 7fe38a439b80 1 -- 172.21.15.22:0/1207732591 wait complete. 2023-09-20T22:02:59.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.608+0000 7fe38a439b80 1 -- start start 2023-09-20T22:02:59.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.608+0000 7fe38a439b80 1 Processor -- start 2023-09-20T22:02:59.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.609+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb76f0 0x565144fc7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.609+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.609+0000 7fe38a439b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb8be0 0x565144fc6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.609+0000 7fe38a439b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144fc6ab0 con 0x565144fb8be0 2023-09-20T22:02:59.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.609+0000 7fe38a439b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144fc6c20 con 0x565144fb76f0 2023-09-20T22:02:59.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.609+0000 7fe38a439b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x565144fc6de0 con 0x565144fb8050 2023-09-20T22:02:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.609+0000 7fe38a439b80 10 client.0 _ll_register_callbacks cb 0x565144defa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:02:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.609+0000 7fe37499b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb76f0 0x565144fc7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.609+0000 7fe36ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.610+0000 7fe36ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:46534/0 (socket says 172.21.15.22:46534) 2023-09-20T22:02:59.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.610+0000 7fe37499b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb76f0 0x565144fc7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:37620/0 (socket says 172.21.15.22:37620) 2023-09-20T22:02:59.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.610+0000 7fe36ffff700 1 -- 172.21.15.22:0/3849132436 learned_addr learned my addr 172.21.15.22:0/3849132436 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:02:59.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.610+0000 7fe37519c700 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb8be0 0x565144fc6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.610+0000 7fe36ffff700 1 -- 172.21.15.22:0/3849132436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb76f0 msgr2=0x565144fc7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:59.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.610+0000 7fe36ffff700 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb76f0 0x565144fc7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.610+0000 7fe36ffff700 1 -- 172.21.15.22:0/3849132436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb8be0 msgr2=0x565144fc6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:02:59.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.610+0000 7fe36ffff700 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb8be0 0x565144fc6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.610+0000 7fe36ffff700 1 -- 172.21.15.22:0/3849132436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565144d33de0 con 0x565144fb8050 2023-09-20T22:02:59.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.610+0000 7fe37499b700 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb76f0 0x565144fc7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:02:59.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.611+0000 7fe36ffff700 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc8350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe358018010 tx=0x7fe3580042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e47a6408ac950074 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:02:59.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.611+0000 7fe34ffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T22:02:59.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.611+0000 7fe34ffff700 1 -- 172.21.15.22:0/3849132436 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800de60 con 0x565144fb8050 2023-09-20T22:02:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.611+0000 7fe34ffff700 1 -- 172.21.15.22:0/3849132436 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe358000e40 con 0x565144fb8050 2023-09-20T22:02:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.611+0000 7fe38a439b80 1 -- 172.21.15.22:0/3849132436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe36009acd0 con 0x565144fb8050 2023-09-20T22:02:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.611+0000 7fe34ffff700 1 -- 172.21.15.22:0/3849132436 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3580293b0 con 0x565144fb8050 2023-09-20T22:02:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.611+0000 7fe38a439b80 10 client.26164 fetch_fsmap learned FSMap version 32 2023-09-20T22:02:59.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.611+0000 7fe38a439b80 1 -- 172.21.15.22:0/3849132436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fe3701a9ba0 con 0x565144fb8050 2023-09-20T22:02:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.612+0000 7fe34ffff700 1 -- 172.21.15.22:0/3849132436 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe36009acd0 con 0x565144fb8050 2023-09-20T22:02:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.612+0000 7fe34ffff700 1 -- 172.21.15.22:0/3849132436 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe35803a040 con 0x565144fb8050 2023-09-20T22:02:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.612+0000 7fe38a439b80 10 client.26164 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:02:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.612+0000 7fe38a439b80 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:02:59.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.612+0000 7fe38a439b80 20 client.26164 populate_metadata read hostname 'smithi022' 2023-09-20T22:02:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.612+0000 7fe38a439b80 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x565145021db0 0x5651450241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:02:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.612+0000 7fe38a439b80 4 client.26164 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T22:02:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.612+0000 7fe38a439b80 1 -- 172.21.15.22:0/3849132436 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565144f01e60 con 0x565145021db0 2023-09-20T22:02:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.612+0000 7fe37499b700 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x565145021db0 0x5651450241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:02:59.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe37499b700 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x565145021db0 0x5651450241d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ca48f41dcfae78 server_cookie=4dabef5eef3224bb in_seq=0 out_seq=0 2023-09-20T22:02:59.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe34ffff700 10 client.26164 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:02:59.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe34ffff700 1 -- 172.21.15.22:0/3849132436 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x565144f01e60 con 0x565145021db0 2023-09-20T22:02:59.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe34ffff700 10 client.26164 handle_command_reply: tid=0 2023-09-20T22:02:59.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe38a439b80 1 client.26164 shutdown 2023-09-20T22:02:59.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe38a439b80 1 -- 172.21.15.22:0/3849132436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 msgr2=0x565144fc8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe38a439b80 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc8350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe358018010 tx=0x7fe3580042d0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe38a439b80 1 -- 172.21.15.22:0/3849132436 shutdown_connections 2023-09-20T22:02:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe38a439b80 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x565145021db0 0x5651450241d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe38a439b80 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565144fb8be0 0x565144fc6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe38a439b80 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x565144fb76f0 0x565144fc7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe34ffff700 0 client.26164 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:02:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe38a439b80 1 --2- 172.21.15.22:0/3849132436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565144fb8050 0x565144fc8350 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe38a439b80 1 -- 172.21.15.22:0/3849132436 >> 172.21.15.22:0/3849132436 conn(0x565144ee4000 msgr2=0x565144fbfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.613+0000 7fe38a439b80 1 -- 172.21.15.22:0/3849132436 shutdown_connections 2023-09-20T22:02:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.614+0000 7fe38a439b80 1 -- 172.21.15.22:0/3849132436 wait complete. 2023-09-20T22:02:59.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.614+0000 7fe38a439b80 20 client.26164 trim_cache size 0 max 16384 2023-09-20T22:02:59.626 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:02:59.626 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T22:02:59.626 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T22:02:59.626 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T22:02:59.627 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T22:02:59.627 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T22:02:59.627 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T22:02:59.627 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:02:59.627 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:02:59.627 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:02:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.614+0000 7fe376bff700 1 -- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe35c0a1380 msgr2=0x7fe35c0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.614+0000 7fe376bff700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe35c0a1380 0x7fe35c0a3840 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe368005950 tx=0x7fe368009310 comp rx=0 tx=0).stop 2023-09-20T22:02:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.615+0000 7fe376bff700 1 -- 172.21.15.22:0/3393975669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe370069b30 msgr2=0x7fe3701a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:02:59.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.615+0000 7fe376bff700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe370069b30 0x7fe3701a16a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe360000f80 tx=0x7fe360004ce0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.615+0000 7fe376bff700 1 -- 172.21.15.22:0/3393975669 shutdown_connections 2023-09-20T22:02:59.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.615+0000 7fe376bff700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe37010ba00 0x7fe3701a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.615+0000 7fe376bff700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe370069b30 0x7fe3701a16a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.615+0000 7fe376bff700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe35c0a1380 0x7fe35c0a3840 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.615+0000 7fe376bff700 1 --2- 172.21.15.22:0/3393975669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe3701087b0 0x7fe3701a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:02:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.615+0000 7fe376bff700 1 -- 172.21.15.22:0/3393975669 >> 172.21.15.22:0/3393975669 conn(0x7fe37006dfe0 msgr2=0x7fe370110dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:02:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.615+0000 7fe376bff700 1 -- 172.21.15.22:0/3393975669 shutdown_connections 2023-09-20T22:02:59.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:02:59.616+0000 7fe376bff700 1 -- 172.21.15.22:0/3393975669 wait complete. 2023-09-20T22:02:59.645 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T22:02:59.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:02:59 smithi022 ceph-mon[97651]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.7 MiB/s wr, 527 op/s 2023-09-20T22:03:00.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:02:59 smithi123 ceph-mon[134433]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.7 MiB/s wr, 527 op/s 2023-09-20T22:03:00.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:02:59 smithi089 ceph-mon[134090]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.7 MiB/s wr, 527 op/s 2023-09-20T22:03:02.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:01 smithi123 ceph-mon[134433]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 4.6 MiB/s wr, 507 op/s 2023-09-20T22:03:02.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:01 smithi089 ceph-mon[134090]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 4.6 MiB/s wr, 507 op/s 2023-09-20T22:03:02.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:01 smithi022 ceph-mon[97651]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 4.6 MiB/s wr, 507 op/s 2023-09-20T22:03:03.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:02 smithi123 ceph-mon[134433]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 5.5 MiB/s wr, 586 op/s 2023-09-20T22:03:03.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:02 smithi089 ceph-mon[134090]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 5.5 MiB/s wr, 586 op/s 2023-09-20T22:03:03.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:02 smithi022 ceph-mon[97651]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 5.5 MiB/s wr, 586 op/s 2023-09-20T22:03:03.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 820.0 s, 440.9 tps, lat 223.117 ms stddev 287.768 2023-09-20T22:03:05.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:05 smithi123 ceph-mon[134433]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.6 MiB/s wr, 489 op/s 2023-09-20T22:03:05.861 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:05 smithi089 ceph-mon[134090]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.6 MiB/s wr, 489 op/s 2023-09-20T22:03:05.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:05 smithi022 ceph-mon[97651]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.6 MiB/s wr, 489 op/s 2023-09-20T22:03:07.843 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:07 smithi123 ceph-mon[134433]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.6 MiB/s wr, 515 op/s 2023-09-20T22:03:07.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:07 smithi089 ceph-mon[134090]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.6 MiB/s wr, 515 op/s 2023-09-20T22:03:07.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:07 smithi022 ceph-mon[97651]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.6 MiB/s wr, 515 op/s 2023-09-20T22:03:08.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 825.0 s, 494.1 tps, lat 202.684 ms stddev 219.710 2023-09-20T22:03:09.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:09 smithi089 ceph-mon[134090]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 5.2 MiB/s wr, 584 op/s 2023-09-20T22:03:09.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:09 smithi022 ceph-mon[97651]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 5.2 MiB/s wr, 584 op/s 2023-09-20T22:03:10.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:09 smithi123 ceph-mon[134433]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 5.2 MiB/s wr, 584 op/s 2023-09-20T22:03:11.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:11 smithi022 ceph-mon[97651]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 4.8 MiB/s wr, 529 op/s 2023-09-20T22:03:12.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:11 smithi123 ceph-mon[134433]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 4.8 MiB/s wr, 529 op/s 2023-09-20T22:03:12.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:11 smithi089 ceph-mon[134090]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 4.8 MiB/s wr, 529 op/s 2023-09-20T22:03:13.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 830.0 s, 432.9 tps, lat 229.464 ms stddev 247.364 2023-09-20T22:03:13.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:13 smithi022 ceph-mon[97651]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 5.9 MiB/s wr, 584 op/s 2023-09-20T22:03:14.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:13 smithi123 ceph-mon[134433]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 5.9 MiB/s wr, 584 op/s 2023-09-20T22:03:14.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:13 smithi089 ceph-mon[134090]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 5.9 MiB/s wr, 584 op/s 2023-09-20T22:03:15.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:15 smithi022 ceph-mon[97651]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.6 MiB/s wr, 458 op/s 2023-09-20T22:03:16.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:15 smithi123 ceph-mon[134433]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.6 MiB/s wr, 458 op/s 2023-09-20T22:03:16.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:15 smithi089 ceph-mon[134090]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.6 MiB/s wr, 458 op/s 2023-09-20T22:03:17.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:17 smithi022 ceph-mon[97651]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.9 MiB/s wr, 491 op/s 2023-09-20T22:03:18.102 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:17 smithi123 ceph-mon[134433]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.9 MiB/s wr, 491 op/s 2023-09-20T22:03:18.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:17 smithi089 ceph-mon[134090]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.9 MiB/s wr, 491 op/s 2023-09-20T22:03:18.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 835.0 s, 402.8 tps, lat 249.333 ms stddev 280.803 2023-09-20T22:03:19.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:19 smithi022 ceph-mon[97651]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2023-09-20T22:03:20.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:19 smithi123 ceph-mon[134433]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2023-09-20T22:03:20.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:19 smithi089 ceph-mon[134090]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2023-09-20T22:03:21.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:20 smithi123 ceph-mon[134433]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.4 MiB/s wr, 518 op/s 2023-09-20T22:03:21.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:20 smithi089 ceph-mon[134090]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.4 MiB/s wr, 518 op/s 2023-09-20T22:03:21.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:20 smithi022 ceph-mon[97651]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.4 MiB/s wr, 518 op/s 2023-09-20T22:03:23.498 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:23 smithi022 ceph-mon[97651]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 7.6 MiB/s wr, 580 op/s 2023-09-20T22:03:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 840.0 s, 503.0 tps, lat 203.811 ms stddev 214.641 2023-09-20T22:03:23.592 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:23 smithi123 ceph-mon[134433]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 7.6 MiB/s wr, 580 op/s 2023-09-20T22:03:23.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:23 smithi089 ceph-mon[134090]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 7.6 MiB/s wr, 580 op/s 2023-09-20T22:03:26.007 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:25 smithi022 ceph-mon[97651]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 6.3 MiB/s wr, 467 op/s 2023-09-20T22:03:26.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:25 smithi123 ceph-mon[134433]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 6.3 MiB/s wr, 467 op/s 2023-09-20T22:03:26.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:25 smithi089 ceph-mon[134090]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 6.3 MiB/s wr, 467 op/s 2023-09-20T22:03:28.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:27 smithi022 ceph-mon[97651]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 6.8 MiB/s wr, 491 op/s 2023-09-20T22:03:28.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:27 smithi123 ceph-mon[134433]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 6.8 MiB/s wr, 491 op/s 2023-09-20T22:03:28.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:27 smithi089 ceph-mon[134090]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 6.8 MiB/s wr, 491 op/s 2023-09-20T22:03:28.573 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 845.0 s, 531.2 tps, lat 182.024 ms stddev 195.079 2023-09-20T22:03:29.647 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T22:03:29.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:29 smithi022 ceph-mon[97651]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 8.3 MiB/s wr, 553 op/s 2023-09-20T22:03:30.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:29 smithi123 ceph-mon[134433]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 8.3 MiB/s wr, 553 op/s 2023-09-20T22:03:30.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:29 smithi089 ceph-mon[134090]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 8.3 MiB/s wr, 553 op/s 2023-09-20T22:03:30.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.143+0000 7f69879ff700 1 -- 172.21.15.22:0/1337787204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6980107de0 msgr2=0x7f698010a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:03:30.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.143+0000 7f69879ff700 1 --2- 172.21.15.22:0/1337787204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6980107de0 0x7f698010a210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6970009990 tx=0x7f6970009ca0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.162+0000 7f69879ff700 1 -- 172.21.15.22:0/1337787204 shutdown_connections 2023-09-20T22:03:30.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.162+0000 7f69879ff700 1 --2- 172.21.15.22:0/1337787204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f698010d120 0x7f698010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.162+0000 7f69879ff700 1 --2- 172.21.15.22:0/1337787204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6980107de0 0x7f698010a210 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.162+0000 7f69879ff700 1 --2- 172.21.15.22:0/1337787204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f698010a750 0x7f698010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.162+0000 7f69879ff700 1 -- 172.21.15.22:0/1337787204 >> 172.21.15.22:0/1337787204 conn(0x7f698006db90 msgr2=0x7f698006ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:03:30.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.178+0000 7f69879ff700 1 -- 172.21.15.22:0/1337787204 shutdown_connections 2023-09-20T22:03:30.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.178+0000 7f69879ff700 1 -- 172.21.15.22:0/1337787204 wait complete. 2023-09-20T22:03:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.179+0000 7f69879ff700 1 Processor -- start 2023-09-20T22:03:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.180+0000 7f69879ff700 1 -- start start 2023-09-20T22:03:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.190+0000 7f69879ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6980107de0 0x7f69801142f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.190+0000 7f69879ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f698010a750 0x7f6980114830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.190+0000 7f69879ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f698010d120 0x7f69801a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.190+0000 7f69879ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6980117980 con 0x7f698010a750 2023-09-20T22:03:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.190+0000 7f69879ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6980117af0 con 0x7f6980107de0 2023-09-20T22:03:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.190+0000 7f69879ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69801a5020 con 0x7f698010d120 2023-09-20T22:03:30.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.191+0000 7f6985f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f698010d120 0x7f69801a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.191+0000 7f6985f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f698010d120 0x7f69801a4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:49972/0 (socket says 172.21.15.22:49972) 2023-09-20T22:03:30.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.191+0000 7f6985f9c700 1 -- 172.21.15.22:0/2233375868 learned_addr learned my addr 172.21.15.22:0/2233375868 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:03:30.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.191+0000 7f698579b700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6980107de0 0x7f69801142f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.196+0000 7f698579b700 1 -- 172.21.15.22:0/2233375868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f698010d120 msgr2=0x7f69801a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:03:30.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.196+0000 7f698579b700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f698010d120 0x7f69801a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.196+0000 7f698579b700 1 -- 172.21.15.22:0/2233375868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f698010a750 msgr2=0x7f6980114830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:03:30.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.196+0000 7f698579b700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f698010a750 0x7f6980114830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.196+0000 7f698579b700 1 -- 172.21.15.22:0/2233375868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700095f0 con 0x7f6980107de0 2023-09-20T22:03:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.197+0000 7f698579b700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6980107de0 0x7f69801142f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6970003720 tx=0x7f697000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:03:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.198+0000 7f69767fc700 1 -- 172.21.15.22:0/2233375868 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697001d070 con 0x7f6980107de0 2023-09-20T22:03:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.198+0000 7f69767fc700 1 -- 172.21.15.22:0/2233375868 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6970021410 con 0x7f6980107de0 2023-09-20T22:03:30.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.198+0000 7f69767fc700 1 -- 172.21.15.22:0/2233375868 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970017650 con 0x7f6980107de0 2023-09-20T22:03:30.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.199+0000 7f69879ff700 1 -- 172.21.15.22:0/2233375868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69801a52a0 con 0x7f6980107de0 2023-09-20T22:03:30.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.199+0000 7f69879ff700 1 -- 172.21.15.22:0/2233375868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69801a5700 con 0x7f6980107de0 2023-09-20T22:03:30.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.206+0000 7f699b357b80 1 Processor -- start 2023-09-20T22:03:30.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.207+0000 7f699b357b80 1 -- start start 2023-09-20T22:03:30.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.207+0000 7f699b357b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.207+0000 7f699b357b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0af050 0x55c2ec0af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.207+0000 7f699b357b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.207+0000 7f699b357b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec022ca0 con 0x55c2ec0af050 2023-09-20T22:03:30.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.207+0000 7f699b357b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec006c70 con 0x55c2ec0afbe0 2023-09-20T22:03:30.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.207+0000 7f699b357b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec004590 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.208+0000 7f6985f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.208+0000 7f6985f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:49980/0 (socket says 172.21.15.22:49980) 2023-09-20T22:03:30.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.208+0000 7f6985f9c700 1 -- 172.21.15.22:0/3796245474 learned_addr learned my addr 172.21.15.22:0/3796245474 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:03:30.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.212+0000 7f6985f9c700 1 -- 172.21.15.22:0/3796245474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 msgr2=0x55c2ec0b0060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-20T22:03:30.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.212+0000 7f6985f9c700 1 --2- 172.21.15.22:0/3796245474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0b0060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.212+0000 7f6985f9c700 1 -- 172.21.15.22:0/3796245474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0af050 msgr2=0x55c2ec0af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:03:30.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.212+0000 7f6985f9c700 1 --2- 172.21.15.22:0/3796245474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0af050 0x55c2ec0af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.212+0000 7f6985f9c700 1 -- 172.21.15.22:0/3796245474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2ebe2ade0 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.212+0000 7f69767fc700 1 -- 172.21.15.22:0/2233375868 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f697000fac0 con 0x7f6980107de0 2023-09-20T22:03:30.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.215+0000 7f6985f9c700 1 --2- 172.21.15.22:0/3796245474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0aeb10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6978009fa0 tx=0x7f69780091b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=591134c20cf9e809 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:03:30.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.222+0000 7f696b7fe700 1 -- 172.21.15.22:0/3796245474 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978019070 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.222+0000 7f696b7fe700 1 -- 172.21.15.22:0/3796245474 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697800f040 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.222+0000 7f696b7fe700 1 -- 172.21.15.22:0/3796245474 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69780156e0 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.223+0000 7f699b357b80 1 -- 172.21.15.22:0/3796245474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0ae6f0 msgr2=0x55c2ec0aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:03:30.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.223+0000 7f699b357b80 1 --2- 172.21.15.22:0/3796245474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0aeb10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6978009fa0 tx=0x7f69780091b0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.224+0000 7f69767fc700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f696c0a1500 0x7f696c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.224+0000 7f69767fc700 1 -- 172.21.15.22:0/2233375868 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f69700cfdb0 con 0x7f6980107de0 2023-09-20T22:03:30.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.236+0000 7f6985f9c700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f696c0a1500 0x7f696c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.243+0000 7f699b357b80 1 -- 172.21.15.22:0/3796245474 shutdown_connections 2023-09-20T22:03:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.243+0000 7f699b357b80 1 --2- 172.21.15.22:0/3796245474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.243+0000 7f699b357b80 1 --2- 172.21.15.22:0/3796245474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0aeb10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.243+0000 7f699b357b80 1 --2- 172.21.15.22:0/3796245474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0af050 0x55c2ec0af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.243+0000 7f699b357b80 1 -- 172.21.15.22:0/3796245474 >> 172.21.15.22:0/3796245474 conn(0x55c2ebfdb000 msgr2=0x55c2ec0a91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:03:30.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.260+0000 7f699b357b80 1 -- 172.21.15.22:0/3796245474 shutdown_connections 2023-09-20T22:03:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.261+0000 7f699b357b80 1 -- 172.21.15.22:0/3796245474 wait complete. 2023-09-20T22:03:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.261+0000 7f699b357b80 1 -- start start 2023-09-20T22:03:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.262+0000 7f699b357b80 1 Processor -- start 2023-09-20T22:03:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.271+0000 7f6985f9c700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f696c0a1500 0x7f696c0a39c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6978015040 tx=0x7f6978006040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:03:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.275+0000 7f699b357b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.275+0000 7f699b357b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0af050 0x55c2ec0b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.275+0000 7f699b357b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.275+0000 7f699b357b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ebfffe80 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.275+0000 7f699b357b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ebf9c740 con 0x55c2ec0afbe0 2023-09-20T22:03:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.275+0000 7f699b357b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ebf9c3e0 con 0x55c2ec0af050 2023-09-20T22:03:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.276+0000 7f699b357b80 10 client.0 _ll_register_callbacks cb 0x55c2ebee6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:03:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.285+0000 7f698579b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.285+0000 7f698579b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44460/0 (socket says 172.21.15.22:44460) 2023-09-20T22:03:30.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.285+0000 7f698579b700 1 -- 172.21.15.22:0/1848964676 learned_addr learned my addr 172.21.15.22:0/1848964676 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:03:30.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.286+0000 7f6985f9c700 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.292+0000 7f6985f9c700 1 -- 172.21.15.22:0/1848964676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0af050 msgr2=0x55c2ec0b87a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T22:03:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.292+0000 7f6985f9c700 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0af050 0x55c2ec0b87a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.292+0000 7f6985f9c700 1 -- 172.21.15.22:0/1848964676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 msgr2=0x55c2ec0b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:03:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.292+0000 7f6985f9c700 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.293+0000 7f6985f9c700 1 -- 172.21.15.22:0/1848964676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6978009bd0 con 0x55c2ec0afbe0 2023-09-20T22:03:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.298+0000 7f6985f9c700 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bf7e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6978009f70 tx=0x7f69780091b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc78a9b039b92d09 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:03:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.309+0000 7f6968ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T22:03:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.309+0000 7f6968ff9700 1 -- 172.21.15.22:0/1848964676 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978019040 con 0x55c2ec0afbe0 2023-09-20T22:03:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.309+0000 7f6968ff9700 1 -- 172.21.15.22:0/1848964676 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6978020430 con 0x55c2ec0afbe0 2023-09-20T22:03:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.309+0000 7f699b357b80 1 -- 172.21.15.22:0/1848964676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2ebf70bb0 con 0x55c2ec0afbe0 2023-09-20T22:03:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.318+0000 7f6968ff9700 1 -- 172.21.15.22:0/1848964676 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6978015ea0 con 0x55c2ec0afbe0 2023-09-20T22:03:30.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.318+0000 7f699b357b80 10 client.26135 fetch_fsmap learned FSMap version 32 2023-09-20T22:03:30.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.318+0000 7f699b357b80 1 -- 172.21.15.22:0/1848964676 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c2ebe13c60 con 0x55c2ec0afbe0 2023-09-20T22:03:30.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.324+0000 7f6968ff9700 1 -- 172.21.15.22:0/1848964676 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c2ebf70bb0 con 0x55c2ec0afbe0 2023-09-20T22:03:30.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.324+0000 7f6968ff9700 1 -- 172.21.15.22:0/1848964676 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f697802b580 con 0x55c2ec0afbe0 2023-09-20T22:03:30.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.324+0000 7f699b357b80 10 client.26135 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:03:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.324+0000 7f699b357b80 10 client.26135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:03:30.325 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.324+0000 7f699b357b80 20 client.26135 populate_metadata read hostname 'smithi022' 2023-09-20T22:03:30.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.324+0000 7f699b357b80 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2ec0c3f80 0x55c2ec0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.324+0000 7f699b357b80 4 client.26135 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T22:03:30.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.324+0000 7f699b357b80 1 -- 172.21.15.22:0/1848964676 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2ec07d020 con 0x55c2ec0c3f80 2023-09-20T22:03:30.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.325+0000 7f698579b700 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2ec0c3f80 0x55c2ec0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.325+0000 7f698579b700 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2ec0c3f80 0x55c2ec0c63a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20349a47ba27d380 server_cookie=149a7d7482db37f6 in_seq=0 out_seq=0 2023-09-20T22:03:30.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.325+0000 7f6968ff9700 10 client.26135 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:03:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.326+0000 7f6968ff9700 1 -- 172.21.15.22:0/1848964676 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c2ec07d020 con 0x55c2ec0c3f80 2023-09-20T22:03:30.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.326+0000 7f6968ff9700 10 client.26135 handle_command_reply: tid=0 2023-09-20T22:03:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.326+0000 7f699b357b80 1 client.26135 shutdown 2023-09-20T22:03:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.327+0000 7f699b357b80 1 -- 172.21.15.22:0/1848964676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 msgr2=0x55c2ec0bf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:03:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.327+0000 7f699b357b80 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bf7e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6978009f70 tx=0x7f69780091b0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.328+0000 7f699b357b80 1 -- 172.21.15.22:0/1848964676 shutdown_connections 2023-09-20T22:03:30.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.328+0000 7f699b357b80 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2ec0c3f80 0x55c2ec0c63a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.328+0000 7f699b357b80 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bf7e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.328+0000 7f699b357b80 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.328+0000 7f699b357b80 1 --2- 172.21.15.22:0/1848964676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0af050 0x55c2ec0b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.328+0000 7f699b357b80 1 -- 172.21.15.22:0/1848964676 >> 172.21.15.22:0/1848964676 conn(0x55c2ebfdb000 msgr2=0x55c2ec0a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:03:30.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.328+0000 7f6968ff9700 0 client.26135 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:03:30.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.329+0000 7f699b357b80 1 -- 172.21.15.22:0/1848964676 shutdown_connections 2023-09-20T22:03:30.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.329+0000 7f699b357b80 1 -- 172.21.15.22:0/1848964676 wait complete. 2023-09-20T22:03:30.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.330+0000 7f699b357b80 20 client.26135 trim_cache size 0 max 16384 2023-09-20T22:03:30.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.367+0000 7f699b357b80 1 Processor -- start 2023-09-20T22:03:30.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.367+0000 7f699b357b80 1 -- start start 2023-09-20T22:03:30.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f699b357b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f699b357b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0af050 0x55c2ec0b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f699b357b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f699b357b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ebe80920 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f699b357b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec0b7710 con 0x55c2ec0af050 2023-09-20T22:03:30.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f699b357b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec0b7dd0 con 0x55c2ec0afbe0 2023-09-20T22:03:30.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f6984f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0af050 0x55c2ec0b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f698579b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f698579b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44476/0 (socket says 172.21.15.22:44476) 2023-09-20T22:03:30.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f6984f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0af050 0x55c2ec0b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:51036/0 (socket says 172.21.15.22:51036) 2023-09-20T22:03:30.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.368+0000 7f698579b700 1 -- 172.21.15.22:0/3250641662 learned_addr learned my addr 172.21.15.22:0/3250641662 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:03:30.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.369+0000 7f6985f9c700 1 --2- 172.21.15.22:0/3250641662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.369+0000 7f698579b700 1 -- 172.21.15.22:0/3250641662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0afbe0 msgr2=0x55c2ec0bf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:03:30.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.369+0000 7f698579b700 1 --2- 172.21.15.22:0/3250641662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.369+0000 7f698579b700 1 -- 172.21.15.22:0/3250641662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0af050 msgr2=0x55c2ec0b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:03:30.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.369+0000 7f698579b700 1 --2- 172.21.15.22:0/3250641662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0af050 0x55c2ec0b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.369+0000 7f698579b700 1 -- 172.21.15.22:0/3250641662 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2ebe51250 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.369+0000 7f698579b700 1 --2- 172.21.15.22:0/3250641662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b2bc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f697009cae0 tx=0x7f6970005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473a80e486df7f36 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:03:30.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.370+0000 7f696bfff700 1 -- 172.21.15.22:0/3250641662 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700a0c10 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.370+0000 7f696bfff700 1 -- 172.21.15.22:0/3250641662 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69700a4dc0 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.370+0000 7f696bfff700 1 -- 172.21.15.22:0/3250641662 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697009c530 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.371+0000 7f699b357b80 1 -- 172.21.15.22:0/3250641662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 msgr2=0x55c2ec0b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:03:30.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.371+0000 7f699b357b80 1 --2- 172.21.15.22:0/3250641662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b2bc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f697009cae0 tx=0x7f6970005040 comp rx=0 tx=0).stop 2023-09-20T22:03:30.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.372+0000 7f699b357b80 1 -- 172.21.15.22:0/3250641662 shutdown_connections 2023-09-20T22:03:30.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.372+0000 7f699b357b80 1 --2- 172.21.15.22:0/3250641662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.372+0000 7f699b357b80 1 --2- 172.21.15.22:0/3250641662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0b2bc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.372+0000 7f699b357b80 1 --2- 172.21.15.22:0/3250641662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0af050 0x55c2ec0b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.372+0000 7f699b357b80 1 -- 172.21.15.22:0/3250641662 >> 172.21.15.22:0/3250641662 conn(0x55c2ebfdb000 msgr2=0x55c2ec0c00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:03:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.372+0000 7f699b357b80 1 -- 172.21.15.22:0/3250641662 shutdown_connections 2023-09-20T22:03:30.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.372+0000 7f699b357b80 1 -- 172.21.15.22:0/3250641662 wait complete. 2023-09-20T22:03:30.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.373+0000 7f699b357b80 1 -- start start 2023-09-20T22:03:30.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.373+0000 7f699b357b80 1 Processor -- start 2023-09-20T22:03:30.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.374+0000 7f699b357b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.374+0000 7f699b357b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0af050 0x55c2ec0bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.374+0000 7f699b357b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.374+0000 7f699b357b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec0bde10 con 0x55c2ec0ae6f0 2023-09-20T22:03:30.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.374+0000 7f699b357b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec0bdf80 con 0x55c2ec0afbe0 2023-09-20T22:03:30.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.374+0000 7f699b357b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2ec0be140 con 0x55c2ec0af050 2023-09-20T22:03:30.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.374+0000 7f699b357b80 10 client.0 _ll_register_callbacks cb 0x55c2ebee6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:03:30.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.375+0000 7f6984f9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0af050 0x55c2ec0bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.375+0000 7f6984f9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0af050 0x55c2ec0bf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:49984/0 (socket says 172.21.15.22:49984) 2023-09-20T22:03:30.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.375+0000 7f6984f9a700 1 -- 172.21.15.22:0/2339237482 learned_addr learned my addr 172.21.15.22:0/2339237482 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:03:30.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.375+0000 7f6985f9c700 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.375+0000 7f698579b700 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.376+0000 7f6985f9c700 1 -- 172.21.15.22:0/2339237482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0af050 msgr2=0x55c2ec0bf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:03:30.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.376+0000 7f6985f9c700 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0af050 0x55c2ec0bf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.376+0000 7f6985f9c700 1 -- 172.21.15.22:0/2339237482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 msgr2=0x55c2ec0bf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:03:30.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.376+0000 7f6985f9c700 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0bf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.376+0000 7f6985f9c700 1 -- 172.21.15.22:0/2339237482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69801a5700 con 0x55c2ec0afbe0 2023-09-20T22:03:30.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.376+0000 7f698579b700 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0bf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:03:30.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.376+0000 7f6985f9c700 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bd7f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6978007fd0 tx=0x7f6978034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f419326574164e5c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:03:30.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.376+0000 7f6968ff9700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T22:03:30.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.377+0000 7f6968ff9700 1 -- 172.21.15.22:0/2339237482 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978019040 con 0x55c2ec0afbe0 2023-09-20T22:03:30.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.377+0000 7f699b357b80 1 -- 172.21.15.22:0/2339237482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f697800f040 con 0x55c2ec0afbe0 2023-09-20T22:03:30.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.377+0000 7f6968ff9700 1 -- 172.21.15.22:0/2339237482 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f6978034d20 con 0x55c2ec0afbe0 2023-09-20T22:03:30.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.377+0000 7f6968ff9700 1 -- 172.21.15.22:0/2339237482 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69780156c0 con 0x55c2ec0afbe0 2023-09-20T22:03:30.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.377+0000 7f699b357b80 10 client.26141 fetch_fsmap learned FSMap version 32 2023-09-20T22:03:30.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.377+0000 7f699b357b80 1 -- 172.21.15.22:0/2339237482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c2ebe292b0 con 0x55c2ec0afbe0 2023-09-20T22:03:30.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.377+0000 7f6968ff9700 1 -- 172.21.15.22:0/2339237482 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697800f040 con 0x55c2ec0afbe0 2023-09-20T22:03:30.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.378+0000 7f6968ff9700 1 -- 172.21.15.22:0/2339237482 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f697803e910 con 0x55c2ec0afbe0 2023-09-20T22:03:30.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.378+0000 7f699b357b80 10 client.26141 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:03:30.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.378+0000 7f699b357b80 10 client.26141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:03:30.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.378+0000 7f699b357b80 20 client.26141 populate_metadata read hostname 'smithi022' 2023-09-20T22:03:30.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.378+0000 7f699b357b80 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2ec118d90 0x55c2ec11b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:03:30.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.378+0000 7f699b357b80 4 client.26141 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T22:03:30.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.378+0000 7f699b357b80 1 -- 172.21.15.22:0/2339237482 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c2ebff8e60 con 0x55c2ec118d90 2023-09-20T22:03:30.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.378+0000 7f698579b700 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2ec118d90 0x55c2ec11b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:03:30.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.379+0000 7f698579b700 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2ec118d90 0x55c2ec11b1b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39c9b7e1ed688be7 server_cookie=617362f3b6fee117 in_seq=0 out_seq=0 2023-09-20T22:03:30.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.379+0000 7f6968ff9700 10 client.26141 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:03:30.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.379+0000 7f6968ff9700 1 -- 172.21.15.22:0/2339237482 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c2ebff8e60 con 0x55c2ec118d90 2023-09-20T22:03:30.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.379+0000 7f6968ff9700 10 client.26141 handle_command_reply: tid=0 2023-09-20T22:03:30.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.379+0000 7f699b357b80 1 client.26141 shutdown 2023-09-20T22:03:30.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 1 -- 172.21.15.22:0/2339237482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 msgr2=0x55c2ec0bd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:03:30.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bd7f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6978007fd0 tx=0x7f6978034670 comp rx=0 tx=0).stop 2023-09-20T22:03:30.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 1 -- 172.21.15.22:0/2339237482 shutdown_connections 2023-09-20T22:03:30.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55c2ec118d90 0x55c2ec11b1b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c2ec0afbe0 0x55c2ec0bd7f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c2ec0ae6f0 0x55c2ec0bf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 1 --2- 172.21.15.22:0/2339237482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2ec0af050 0x55c2ec0bf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 1 -- 172.21.15.22:0/2339237482 >> 172.21.15.22:0/2339237482 conn(0x55c2ebfdb000 msgr2=0x55c2ec0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:03:30.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f6968ff9700 0 client.26141 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:03:30.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 1 -- 172.21.15.22:0/2339237482 shutdown_connections 2023-09-20T22:03:30.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 1 -- 172.21.15.22:0/2339237482 wait complete. 2023-09-20T22:03:30.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.380+0000 7f699b357b80 20 client.26141 trim_cache size 0 max 16384 2023-09-20T22:03:30.392 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:03:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T22:03:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T22:03:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T22:03:30.392 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T22:03:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T22:03:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T22:03:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:03:30.393 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:03:30.394 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:03:30.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.381+0000 7f69879ff700 1 -- 172.21.15.22:0/2233375868 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f696c0a1500 msgr2=0x7f696c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:03:30.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.381+0000 7f69879ff700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f696c0a1500 0x7f696c0a39c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6978015040 tx=0x7f6978006040 comp rx=0 tx=0).stop 2023-09-20T22:03:30.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.381+0000 7f69879ff700 1 -- 172.21.15.22:0/2233375868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6980107de0 msgr2=0x7f69801142f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:03:30.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.381+0000 7f69879ff700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6980107de0 0x7f69801142f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6970003720 tx=0x7f697000f690 comp rx=0 tx=0).stop 2023-09-20T22:03:30.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.381+0000 7f69879ff700 1 -- 172.21.15.22:0/2233375868 shutdown_connections 2023-09-20T22:03:30.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.381+0000 7f69879ff700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f698010d120 0x7f69801a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.381+0000 7f69879ff700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f696c0a1500 0x7f696c0a39c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.381+0000 7f69879ff700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6980107de0 0x7f69801142f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.381+0000 7f69879ff700 1 --2- 172.21.15.22:0/2233375868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f698010a750 0x7f6980114830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:03:30.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.381+0000 7f69879ff700 1 -- 172.21.15.22:0/2233375868 >> 172.21.15.22:0/2233375868 conn(0x7f698006db90 msgr2=0x7f698010b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:03:30.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.382+0000 7f69879ff700 1 -- 172.21.15.22:0/2233375868 shutdown_connections 2023-09-20T22:03:30.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:03:30.382+0000 7f69879ff700 1 -- 172.21.15.22:0/2233375868 wait complete. 2023-09-20T22:03:30.413 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T22:03:31.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:31 smithi022 ceph-mon[97651]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.9 MiB/s wr, 486 op/s 2023-09-20T22:03:32.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:31 smithi123 ceph-mon[134433]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.9 MiB/s wr, 486 op/s 2023-09-20T22:03:32.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:31 smithi089 ceph-mon[134090]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.9 MiB/s wr, 486 op/s 2023-09-20T22:03:33.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 850.0 s, 515.2 tps, lat 193.347 ms stddev 232.380 2023-09-20T22:03:34.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:33 smithi123 ceph-mon[134433]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 8.2 MiB/s wr, 529 op/s 2023-09-20T22:03:34.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:33 smithi089 ceph-mon[134090]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 8.2 MiB/s wr, 529 op/s 2023-09-20T22:03:34.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:33 smithi022 ceph-mon[97651]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 8.2 MiB/s wr, 529 op/s 2023-09-20T22:03:36.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:35 smithi123 ceph-mon[134433]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 6.6 MiB/s wr, 419 op/s 2023-09-20T22:03:36.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:35 smithi089 ceph-mon[134090]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 6.6 MiB/s wr, 419 op/s 2023-09-20T22:03:36.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:35 smithi022 ceph-mon[97651]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 6.6 MiB/s wr, 419 op/s 2023-09-20T22:03:37.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:36 smithi022 ceph-mon[97651]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 6.8 MiB/s wr, 437 op/s 2023-09-20T22:03:37.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:36 smithi123 ceph-mon[134433]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 6.8 MiB/s wr, 437 op/s 2023-09-20T22:03:37.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:36 smithi089 ceph-mon[134090]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 6.8 MiB/s wr, 437 op/s 2023-09-20T22:03:38.575 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 855.0 s, 471.0 tps, lat 211.587 ms stddev 248.444 2023-09-20T22:03:39.860 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:39 smithi089 ceph-mon[134090]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 7.6 MiB/s wr, 507 op/s 2023-09-20T22:03:39.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:39 smithi022 ceph-mon[97651]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 7.6 MiB/s wr, 507 op/s 2023-09-20T22:03:40.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:39 smithi123 ceph-mon[134433]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 7.6 MiB/s wr, 507 op/s 2023-09-20T22:03:41.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:41 smithi022 ceph-mon[97651]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 7.1 MiB/s wr, 463 op/s 2023-09-20T22:03:42.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:41 smithi123 ceph-mon[134433]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 7.1 MiB/s wr, 463 op/s 2023-09-20T22:03:42.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:41 smithi089 ceph-mon[134090]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 7.1 MiB/s wr, 463 op/s 2023-09-20T22:03:43.572 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 860.0 s, 514.6 tps, lat 201.474 ms stddev 253.915 2023-09-20T22:03:44.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:43 smithi022 ceph-mon[97651]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 7.7 MiB/s wr, 513 op/s 2023-09-20T22:03:44.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:43 smithi123 ceph-mon[134433]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 7.7 MiB/s wr, 513 op/s 2023-09-20T22:03:44.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:43 smithi089 ceph-mon[134090]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 7.7 MiB/s wr, 513 op/s 2023-09-20T22:03:46.005 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:45 smithi022 ceph-mon[97651]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 6.1 MiB/s wr, 418 op/s 2023-09-20T22:03:46.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:45 smithi123 ceph-mon[134433]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 6.1 MiB/s wr, 418 op/s 2023-09-20T22:03:46.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:45 smithi089 ceph-mon[134090]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 6.1 MiB/s wr, 418 op/s 2023-09-20T22:03:46.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:46 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:03:47.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:46 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:03:47.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:46 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:03:47.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:47 smithi022 ceph-mon[97651]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 6.4 MiB/s wr, 451 op/s 2023-09-20T22:03:47.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:47 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:03:47.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:47 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:03:47.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:47 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:03:48.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:47 smithi123 ceph-mon[134433]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 6.4 MiB/s wr, 451 op/s 2023-09-20T22:03:48.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:47 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:03:48.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:47 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:03:48.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:47 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:03:48.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:47 smithi089 ceph-mon[134090]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 6.4 MiB/s wr, 451 op/s 2023-09-20T22:03:48.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:47 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:03:48.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:47 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:03:48.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:47 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:03:48.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 865.0 s, 454.3 tps, lat 214.523 ms stddev 241.911 2023-09-20T22:03:50.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:49 smithi123 ceph-mon[134433]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 7.7 MiB/s wr, 535 op/s 2023-09-20T22:03:50.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:49 smithi089 ceph-mon[134090]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 7.7 MiB/s wr, 535 op/s 2023-09-20T22:03:50.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:49 smithi022 ceph-mon[97651]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 7.7 MiB/s wr, 535 op/s 2023-09-20T22:03:52.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:51 smithi089 ceph-mon[134090]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 6.9 MiB/s wr, 493 op/s 2023-09-20T22:03:52.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:51 smithi022 ceph-mon[97651]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 6.9 MiB/s wr, 493 op/s 2023-09-20T22:03:52.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:51 smithi123 ceph-mon[134433]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 6.9 MiB/s wr, 493 op/s 2023-09-20T22:03:53.569 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 870.0 s, 486.8 tps, lat 207.310 ms stddev 226.402 2023-09-20T22:03:54.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:53 smithi022 ceph-mon[97651]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 7.4 MiB/s wr, 539 op/s 2023-09-20T22:03:54.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:53 smithi123 ceph-mon[134433]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 7.4 MiB/s wr, 539 op/s 2023-09-20T22:03:54.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:53 smithi089 ceph-mon[134090]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 7.4 MiB/s wr, 539 op/s 2023-09-20T22:03:55.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:54 smithi022 ceph-mon[97651]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 6.1 MiB/s wr, 440 op/s 2023-09-20T22:03:55.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:54 smithi123 ceph-mon[134433]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 6.1 MiB/s wr, 440 op/s 2023-09-20T22:03:55.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:54 smithi089 ceph-mon[134090]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 6.1 MiB/s wr, 440 op/s 2023-09-20T22:03:57.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:57 smithi123 ceph-mon[134433]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 6.5 MiB/s wr, 469 op/s 2023-09-20T22:03:57.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:57 smithi089 ceph-mon[134090]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 6.5 MiB/s wr, 469 op/s 2023-09-20T22:03:57.752 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:57 smithi022 ceph-mon[97651]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 6.5 MiB/s wr, 469 op/s 2023-09-20T22:03:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 875.0 s, 548.5 tps, lat 184.103 ms stddev 204.746 2023-09-20T22:03:59.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:03:59 smithi022 ceph-mon[97651]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 7.0 MiB/s wr, 524 op/s 2023-09-20T22:04:00.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:03:59 smithi123 ceph-mon[134433]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 7.0 MiB/s wr, 524 op/s 2023-09-20T22:04:00.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:03:59 smithi089 ceph-mon[134090]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 7.0 MiB/s wr, 524 op/s 2023-09-20T22:04:00.414 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T22:04:00.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.872+0000 7fef7e5ff700 1 -- 172.21.15.22:0/2310408907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef78107de0 msgr2=0x7fef7810a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.872+0000 7fef7e5ff700 1 --2- 172.21.15.22:0/2310408907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef78107de0 0x7fef7810a210 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009990 tx=0x7fef6c009ca0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.872+0000 7fef7e5ff700 1 -- 172.21.15.22:0/2310408907 shutdown_connections 2023-09-20T22:04:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.872+0000 7fef7e5ff700 1 --2- 172.21.15.22:0/2310408907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef7810abd0 0x7fef78113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.872+0000 7fef7e5ff700 1 --2- 172.21.15.22:0/2310408907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef78107de0 0x7fef7810a210 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.872+0000 7fef7e5ff700 1 --2- 172.21.15.22:0/2310408907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef7810a750 0x7fef78111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.872+0000 7fef7e5ff700 1 -- 172.21.15.22:0/2310408907 >> 172.21.15.22:0/2310408907 conn(0x7fef7806db90 msgr2=0x7fef7806ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:00.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.874+0000 7fef7e5ff700 1 -- 172.21.15.22:0/2310408907 shutdown_connections 2023-09-20T22:04:00.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.875+0000 7fef7e5ff700 1 -- 172.21.15.22:0/2310408907 wait complete. 2023-09-20T22:04:00.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.876+0000 7fef7e5ff700 1 Processor -- start 2023-09-20T22:04:00.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.876+0000 7fef7e5ff700 1 -- start start 2023-09-20T22:04:00.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.877+0000 7fef7e5ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef78107de0 0x7fef781a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.877+0000 7fef7e5ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef7810a750 0x7fef781a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.877+0000 7fef7e5ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef7810abd0 0x7fef781acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.877+0000 7fef7e5ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef781ad4b0 con 0x7fef7810abd0 2023-09-20T22:04:00.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.877+0000 7fef7e5ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef781ad5f0 con 0x7fef78107de0 2023-09-20T22:04:00.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.877+0000 7fef7e5ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef781ad780 con 0x7fef7810a750 2023-09-20T22:04:00.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.880+0000 7fef77fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef78107de0 0x7fef781a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:00.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.880+0000 7fef77fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef78107de0 0x7fef781a5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42926/0 (socket says 172.21.15.22:42926) 2023-09-20T22:04:00.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.880+0000 7fef77fff700 1 -- 172.21.15.22:0/164406379 learned_addr learned my addr 172.21.15.22:0/164406379 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:04:00.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.881+0000 7fef77fff700 1 -- 172.21.15.22:0/164406379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef7810a750 msgr2=0x7fef781a5e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-20T22:04:00.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.881+0000 7fef77fff700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef7810a750 0x7fef781a5e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.881+0000 7fef77fff700 1 -- 172.21.15.22:0/164406379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef7810abd0 msgr2=0x7fef781acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:00.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.881+0000 7fef77fff700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef7810abd0 0x7fef781acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.881+0000 7fef77fff700 1 -- 172.21.15.22:0/164406379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c0095f0 con 0x7fef78107de0 2023-09-20T22:04:00.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.882+0000 7fef77fff700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef78107de0 0x7fef781a5930 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fef6c004800 tx=0x7fef6c00f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:00.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.882+0000 7fef757fa700 1 -- 172.21.15.22:0/164406379 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c01d070 con 0x7fef78107de0 2023-09-20T22:04:00.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.883+0000 7fef757fa700 1 -- 172.21.15.22:0/164406379 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef6c00be60 con 0x7fef78107de0 2023-09-20T22:04:00.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.883+0000 7fef757fa700 1 -- 172.21.15.22:0/164406379 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c017650 con 0x7fef78107de0 2023-09-20T22:04:00.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.883+0000 7fef7e5ff700 1 -- 172.21.15.22:0/164406379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef781ada00 con 0x7fef78107de0 2023-09-20T22:04:00.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.883+0000 7fef7e5ff700 1 -- 172.21.15.22:0/164406379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef781ade50 con 0x7fef78107de0 2023-09-20T22:04:00.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.889+0000 7fef757fa700 1 -- 172.21.15.22:0/164406379 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fef6c021410 con 0x7fef78107de0 2023-09-20T22:04:00.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.890+0000 7fef91e3fb80 1 Processor -- start 2023-09-20T22:04:00.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.890+0000 7fef91e3fb80 1 -- start start 2023-09-20T22:04:00.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.891+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd76f0 0x558840cd7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.891+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 0x558840cd85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.891+0000 7fef7cb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd76f0 0x558840cd7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:00.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.891+0000 7fef7cb9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd76f0 0x558840cd7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42930/0 (socket says 172.21.15.22:42930) 2023-09-20T22:04:00.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.893+0000 7fef757fa700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef600a1480 0x7fef600a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.894+0000 7fef757fa700 1 -- 172.21.15.22:0/164406379 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fef6c0ce480 con 0x7fef78107de0 2023-09-20T22:04:00.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.895+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840cd9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.895+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840c4bca0 con 0x558840cd8be0 2023-09-20T22:04:00.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.895+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840c2fc70 con 0x558840cd76f0 2023-09-20T22:04:00.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.895+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840c2d590 con 0x558840cd8050 2023-09-20T22:04:00.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.895+0000 7fef7cb9c700 1 -- 172.21.15.22:0/1452213450 learned_addr learned my addr 172.21.15.22:0/1452213450 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:04:00.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.895+0000 7fef77fff700 1 --2- 172.21.15.22:0/1452213450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 0x558840cd85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:00.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.896+0000 7fef77fff700 1 -- 172.21.15.22:0/1452213450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd76f0 msgr2=0x558840cd7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:00.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.896+0000 7fef77fff700 1 --2- 172.21.15.22:0/1452213450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd76f0 0x558840cd7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.896+0000 7fef77fff700 1 -- 172.21.15.22:0/1452213450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 msgr2=0x558840cd9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:00.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.896+0000 7fef77fff700 1 --2- 172.21.15.22:0/1452213450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840cd9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.896+0000 7fef77fff700 1 -- 172.21.15.22:0/1452213450 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558840a53de0 con 0x558840cd8050 2023-09-20T22:04:00.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.896+0000 7fef777fe700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef600a1480 0x7fef600a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:00.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.896+0000 7fef77fff700 1 --2- 172.21.15.22:0/1452213450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 0x558840cd85e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fef6c09b1b0 tx=0x7fef6c00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3991fe5e5c3d54 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:00.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.897+0000 7fef5e7fc700 1 -- 172.21.15.22:0/1452213450 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c09b4b0 con 0x558840cd8050 2023-09-20T22:04:00.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.897+0000 7fef5e7fc700 1 -- 172.21.15.22:0/1452213450 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef6c09f2e0 con 0x558840cd8050 2023-09-20T22:04:00.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.897+0000 7fef5e7fc700 1 -- 172.21.15.22:0/1452213450 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c09a9a0 con 0x558840cd8050 2023-09-20T22:04:00.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.897+0000 7fef777fe700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef600a1480 0x7fef600a3940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fef7000ab90 tx=0x7fef70009250 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:00.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.898+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1452213450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 msgr2=0x558840cd85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:00.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.898+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/1452213450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 0x558840cd85e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fef6c09b1b0 tx=0x7fef6c00bfd0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.898+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1452213450 shutdown_connections 2023-09-20T22:04:00.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.898+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/1452213450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840cd9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.898+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/1452213450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd76f0 0x558840cd7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.898+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/1452213450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 0x558840cd85e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.898+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1452213450 >> 172.21.15.22:0/1452213450 conn(0x558840c04000 msgr2=0x558840cd21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:00.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.898+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1452213450 shutdown_connections 2023-09-20T22:04:00.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.898+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1452213450 wait complete. 2023-09-20T22:04:00.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.899+0000 7fef91e3fb80 1 -- start start 2023-09-20T22:04:00.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.899+0000 7fef91e3fb80 1 Processor -- start 2023-09-20T22:04:00.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.900+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd76f0 0x558840ce0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.900+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 0x558840ce1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.900+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840ce84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.900+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840c28e80 con 0x558840cd8be0 2023-09-20T22:04:00.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.900+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840bc5740 con 0x558840cd8050 2023-09-20T22:04:00.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.900+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840bc53e0 con 0x558840cd76f0 2023-09-20T22:04:00.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.900+0000 7fef91e3fb80 10 client.0 _ll_register_callbacks cb 0x558840b0fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:04:00.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef777fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 0x558840ce1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:00.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef77fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd76f0 0x558840ce0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:00.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef777fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 0x558840ce1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:42934/0 (socket says 172.21.15.22:42934) 2023-09-20T22:04:00.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef777fe700 1 -- 172.21.15.22:0/439667795 learned_addr learned my addr 172.21.15.22:0/439667795 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:04:00.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef7cb9c700 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840ce84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:00.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef77fff700 1 -- 172.21.15.22:0/439667795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 msgr2=0x558840ce1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:00.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef77fff700 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 0x558840ce1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef77fff700 1 -- 172.21.15.22:0/439667795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 msgr2=0x558840ce84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:00.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef77fff700 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840ce84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef77fff700 1 -- 172.21.15.22:0/439667795 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef781ade50 con 0x558840cd76f0 2023-09-20T22:04:00.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef777fe700 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 0x558840ce1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:04:00.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.901+0000 7fef7cb9c700 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840ce84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:04:00.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.902+0000 7fef77fff700 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd76f0 0x558840ce0f20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x558840cd3010 tx=0x7fef6c09aff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa44f0be9f232710 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:00.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.902+0000 7fef577fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T22:04:00.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.902+0000 7fef577fe700 1 -- 172.21.15.22:0/439667795 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0a3200 con 0x558840cd76f0 2023-09-20T22:04:00.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.902+0000 7fef91e3fb80 1 -- 172.21.15.22:0/439667795 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558840b99bb0 con 0x558840cd76f0 2023-09-20T22:04:00.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.902+0000 7fef577fe700 1 -- 172.21.15.22:0/439667795 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fef6c09b4b0 con 0x558840cd76f0 2023-09-20T22:04:00.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.902+0000 7fef577fe700 1 -- 172.21.15.22:0/439667795 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef6c0a3490 con 0x558840cd76f0 2023-09-20T22:04:00.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.902+0000 7fef91e3fb80 10 client.26188 fetch_fsmap learned FSMap version 32 2023-09-20T22:04:00.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.902+0000 7fef91e3fb80 1 -- 172.21.15.22:0/439667795 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558840a3cc60 con 0x558840cd76f0 2023-09-20T22:04:00.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.903+0000 7fef577fe700 1 -- 172.21.15.22:0/439667795 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef6c0a9080 con 0x558840cd76f0 2023-09-20T22:04:00.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.903+0000 7fef577fe700 1 -- 172.21.15.22:0/439667795 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fef6c09a410 con 0x558840cd76f0 2023-09-20T22:04:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.903+0000 7fef91e3fb80 10 client.26188 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:04:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.903+0000 7fef91e3fb80 10 client.26188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:04:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.903+0000 7fef91e3fb80 20 client.26188 populate_metadata read hostname 'smithi022' 2023-09-20T22:04:00.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.903+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x558840cec680 0x558840ceeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.903+0000 7fef91e3fb80 4 client.26188 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T22:04:00.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.903+0000 7fef91e3fb80 1 -- 172.21.15.22:0/439667795 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558840ca6020 con 0x558840cec680 2023-09-20T22:04:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.904+0000 7fef777fe700 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x558840cec680 0x558840ceeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.908+0000 7fef777fe700 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x558840cec680 0x558840ceeaa0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab7c92905104cab7 server_cookie=28c88b4d69419a09 in_seq=0 out_seq=0 2023-09-20T22:04:00.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.908+0000 7fef577fe700 10 client.26188 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:04:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.910+0000 7fef577fe700 1 -- 172.21.15.22:0/439667795 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558840ca6020 con 0x558840cec680 2023-09-20T22:04:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.910+0000 7fef577fe700 10 client.26188 handle_command_reply: tid=0 2023-09-20T22:04:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.911+0000 7fef91e3fb80 1 client.26188 shutdown 2023-09-20T22:04:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef91e3fb80 1 -- 172.21.15.22:0/439667795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd76f0 msgr2=0x558840ce0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:00.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd76f0 0x558840ce0f20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x558840cd3010 tx=0x7fef6c09aff0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef91e3fb80 1 -- 172.21.15.22:0/439667795 shutdown_connections 2023-09-20T22:04:00.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x558840cec680 0x558840ceeaa0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840ce84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd76f0 0x558840ce0f20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/439667795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 0x558840ce1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef91e3fb80 1 -- 172.21.15.22:0/439667795 >> 172.21.15.22:0/439667795 conn(0x558840c04000 msgr2=0x558840cd1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:00.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef577fe700 0 client.26188 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:04:00.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef91e3fb80 1 -- 172.21.15.22:0/439667795 shutdown_connections 2023-09-20T22:04:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.912+0000 7fef91e3fb80 1 -- 172.21.15.22:0/439667795 wait complete. 2023-09-20T22:04:00.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.913+0000 7fef91e3fb80 20 client.26188 trim_cache size 0 max 16384 2023-09-20T22:04:00.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.945+0000 7fef91e3fb80 1 Processor -- start 2023-09-20T22:04:00.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.951+0000 7fef91e3fb80 1 -- start start 2023-09-20T22:04:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.952+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd76f0 0x558840cee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.952+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 0x558840ce8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.952+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840cdbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.952+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840aa9920 con 0x558840cd8be0 2023-09-20T22:04:00.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.952+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840c2d590 con 0x558840cd76f0 2023-09-20T22:04:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.952+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840c28e80 con 0x558840cd8050 2023-09-20T22:04:00.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.953+0000 7fef7cb9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840cdbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.953+0000 7fef7cb9c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840cdbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41512/0 (socket says 172.21.15.22:41512) 2023-09-20T22:04:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.953+0000 7fef7cb9c700 1 -- 172.21.15.22:0/689944223 learned_addr learned my addr 172.21.15.22:0/689944223 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:04:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.953+0000 7fef777fe700 1 --2- 172.21.15.22:0/689944223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 0x558840ce8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.954+0000 7fef7cb9c700 1 -- 172.21.15.22:0/689944223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 msgr2=0x558840ce8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:00.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.954+0000 7fef7cb9c700 1 --2- 172.21.15.22:0/689944223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 0x558840ce8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.954+0000 7fef7cb9c700 1 -- 172.21.15.22:0/689944223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd76f0 msgr2=0x558840cee690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-20T22:04:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.954+0000 7fef7cb9c700 1 --2- 172.21.15.22:0/689944223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd76f0 0x558840cee690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.954+0000 7fef7cb9c700 1 -- 172.21.15.22:0/689944223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558840a7a250 con 0x558840cd8be0 2023-09-20T22:04:00.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.954+0000 7fef7cb9c700 1 --2- 172.21.15.22:0/689944223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840cdbbc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fef68009cc0 tx=0x7fef680123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5103d78ce26acf75 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:00.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.956+0000 7fef5effd700 1 -- 172.21.15.22:0/689944223 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68017070 con 0x558840cd8be0 2023-09-20T22:04:00.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.956+0000 7fef5effd700 1 -- 172.21.15.22:0/689944223 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef68004030 con 0x558840cd8be0 2023-09-20T22:04:00.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.956+0000 7fef5effd700 1 -- 172.21.15.22:0/689944223 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68004e90 con 0x558840cd8be0 2023-09-20T22:04:00.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.957+0000 7fef91e3fb80 1 -- 172.21.15.22:0/689944223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 msgr2=0x558840cdbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:00.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.957+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/689944223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840cdbbc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fef68009cc0 tx=0x7fef680123f0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.973+0000 7fef91e3fb80 1 -- 172.21.15.22:0/689944223 shutdown_connections 2023-09-20T22:04:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.973+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/689944223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd8be0 0x558840cdbbc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.973+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/689944223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd76f0 0x558840cee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.973+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/689944223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8050 0x558840ce8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:00.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.973+0000 7fef91e3fb80 1 -- 172.21.15.22:0/689944223 >> 172.21.15.22:0/689944223 conn(0x558840c04000 msgr2=0x558840cdf080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:00.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.981+0000 7fef91e3fb80 1 -- 172.21.15.22:0/689944223 shutdown_connections 2023-09-20T22:04:01.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.990+0000 7fef91e3fb80 1 -- 172.21.15.22:0/689944223 wait complete. 2023-09-20T22:04:01.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.991+0000 7fef91e3fb80 1 -- start start 2023-09-20T22:04:01.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.991+0000 7fef91e3fb80 1 Processor -- start 2023-09-20T22:04:01.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.992+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd76f0 0x558840ce8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.992+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 0x558840ce6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.992+0000 7fef91e3fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8be0 0x558840ce6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.992+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840ce8830 con 0x558840cd76f0 2023-09-20T22:04:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.992+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840ce70a0 con 0x558840cd8050 2023-09-20T22:04:01.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.992+0000 7fef91e3fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x558840ce7260 con 0x558840cd8be0 2023-09-20T22:04:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.992+0000 7fef91e3fb80 10 client.0 _ll_register_callbacks cb 0x558840b0fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:04:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.993+0000 7fef7cb9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8be0 0x558840ce6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.993+0000 7fef7cb9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8be0 0x558840ce6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:37756/0 (socket says 172.21.15.22:37756) 2023-09-20T22:04:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.993+0000 7fef7cb9c700 1 -- 172.21.15.22:0/1427404603 learned_addr learned my addr 172.21.15.22:0/1427404603 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:04:01.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.994+0000 7fef777fe700 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 0x558840ce6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.999+0000 7fef7cb9c700 1 -- 172.21.15.22:0/1427404603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 msgr2=0x558840ce6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.999+0000 7fef7cb9c700 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 0x558840ce6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.999+0000 7fef7cb9c700 1 -- 172.21.15.22:0/1427404603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd76f0 msgr2=0x558840ce8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.999+0000 7fef7cb9c700 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd76f0 0x558840ce8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:00.999+0000 7fef7cb9c700 1 -- 172.21.15.22:0/1427404603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef680099c0 con 0x558840cd8be0 2023-09-20T22:04:01.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.000+0000 7fef7cb9c700 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8be0 0x558840ce6b60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef680092a0 tx=0x7fef68012e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a75b85d2365c8c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:01.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.000+0000 7fef577fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T22:04:01.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.006+0000 7fef577fe700 1 -- 172.21.15.22:0/1427404603 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68020040 con 0x558840cd8be0 2023-09-20T22:04:01.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.006+0000 7fef577fe700 1 -- 172.21.15.22:0/1427404603 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fef68005220 con 0x558840cd8be0 2023-09-20T22:04:01.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.007+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1427404603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef6c09f2e0 con 0x558840cd8be0 2023-09-20T22:04:01.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.014+0000 7fef577fe700 1 -- 172.21.15.22:0/1427404603 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef68017040 con 0x558840cd8be0 2023-09-20T22:04:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.014+0000 7fef91e3fb80 10 client.26191 fetch_fsmap learned FSMap version 32 2023-09-20T22:04:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.014+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1427404603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558840a522b0 con 0x558840cd8be0 2023-09-20T22:04:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.021+0000 7fef577fe700 1 -- 172.21.15.22:0/1427404603 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef6c09f2e0 con 0x558840cd8be0 2023-09-20T22:04:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.021+0000 7fef577fe700 1 -- 172.21.15.22:0/1427404603 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fef6802f470 con 0x558840cd8be0 2023-09-20T22:04:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.023+0000 7fef91e3fb80 10 client.26191 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:04:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.023+0000 7fef91e3fb80 10 client.26191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:04:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.023+0000 7fef91e3fb80 20 client.26191 populate_metadata read hostname 'smithi022' 2023-09-20T22:04:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.024+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x558840d63930 0x558840d65d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.024+0000 7fef91e3fb80 4 client.26191 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T22:04:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.024+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1427404603 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558840c21e60 con 0x558840d63930 2023-09-20T22:04:01.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.041+0000 7fef77fff700 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x558840d63930 0x558840d65d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:01.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.075+0000 7fef77fff700 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x558840d63930 0x558840d65d50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffc671a2fb871a9d server_cookie=eb2cbc94df728570 in_seq=0 out_seq=0 2023-09-20T22:04:01.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.075+0000 7fef577fe700 10 client.26191 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:04:01.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.075+0000 7fef577fe700 1 -- 172.21.15.22:0/1427404603 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558840c21e60 con 0x558840d63930 2023-09-20T22:04:01.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.075+0000 7fef577fe700 10 client.26191 handle_command_reply: tid=0 2023-09-20T22:04:01.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.076+0000 7fef91e3fb80 1 client.26191 shutdown 2023-09-20T22:04:01.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.083+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1427404603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8be0 msgr2=0x558840ce6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:01.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.083+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8be0 0x558840ce6b60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef680092a0 tx=0x7fef68012e70 comp rx=0 tx=0).stop 2023-09-20T22:04:01.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.093+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1427404603 shutdown_connections 2023-09-20T22:04:01.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.093+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x558840d63930 0x558840d65d50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:01.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.093+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558840cd8be0 0x558840ce6b60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:01.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.093+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558840cd76f0 0x558840ce8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:01.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.093+0000 7fef91e3fb80 1 --2- 172.21.15.22:0/1427404603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558840cd8050 0x558840ce6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:01.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.093+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1427404603 >> 172.21.15.22:0/1427404603 conn(0x558840c04000 msgr2=0x558840cdf080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:01.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.093+0000 7fef577fe700 0 client.26191 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:04:01.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.100+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1427404603 shutdown_connections 2023-09-20T22:04:01.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.101+0000 7fef91e3fb80 1 -- 172.21.15.22:0/1427404603 wait complete. 2023-09-20T22:04:01.109 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:04:01.110 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T22:04:01.110 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T22:04:01.110 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T22:04:01.110 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T22:04:01.111 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T22:04:01.111 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T22:04:01.111 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:04:01.111 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:04:01.111 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:04:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.108+0000 7fef91e3fb80 20 client.26191 trim_cache size 0 max 16384 2023-09-20T22:04:01.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.112+0000 7fef7e5ff700 1 -- 172.21.15.22:0/164406379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef600a1480 msgr2=0x7fef600a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:01.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.112+0000 7fef7e5ff700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef600a1480 0x7fef600a3940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fef7000ab90 tx=0x7fef70009250 comp rx=0 tx=0).stop 2023-09-20T22:04:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.112+0000 7fef7e5ff700 1 -- 172.21.15.22:0/164406379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef78107de0 msgr2=0x7fef781a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:01.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.112+0000 7fef7e5ff700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef78107de0 0x7fef781a5930 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fef6c004800 tx=0x7fef6c00f740 comp rx=0 tx=0).stop 2023-09-20T22:04:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.116+0000 7fef7e5ff700 1 -- 172.21.15.22:0/164406379 shutdown_connections 2023-09-20T22:04:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.116+0000 7fef7e5ff700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef7810abd0 0x7fef781acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:01.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.116+0000 7fef7e5ff700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fef600a1480 0x7fef600a3940 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.116+0000 7fef7e5ff700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef78107de0 0x7fef781a5930 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.116+0000 7fef7e5ff700 1 --2- 172.21.15.22:0/164406379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef7810a750 0x7fef781a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:01.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.116+0000 7fef7e5ff700 1 -- 172.21.15.22:0/164406379 >> 172.21.15.22:0/164406379 conn(0x7fef7806db90 msgr2=0x7fef7806e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:01.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.126+0000 7fef7e5ff700 1 -- 172.21.15.22:0/164406379 shutdown_connections 2023-09-20T22:04:01.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:01.130+0000 7fef7e5ff700 1 -- 172.21.15.22:0/164406379 wait complete. 2023-09-20T22:04:01.181 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T22:04:01.998 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:01 smithi022 ceph-mon[97651]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.7 MiB/s wr, 454 op/s 2023-09-20T22:04:02.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:01 smithi123 ceph-mon[134433]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.7 MiB/s wr, 454 op/s 2023-09-20T22:04:02.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:01 smithi089 ceph-mon[134090]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.7 MiB/s wr, 454 op/s 2023-09-20T22:04:03.567 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 880.0 s, 447.9 tps, lat 214.694 ms stddev 227.625 2023-09-20T22:04:03.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:03 smithi022 ceph-mon[97651]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.8 MiB/s wr, 527 op/s 2023-09-20T22:04:04.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:03 smithi123 ceph-mon[134433]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.8 MiB/s wr, 527 op/s 2023-09-20T22:04:04.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:03 smithi089 ceph-mon[134090]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.8 MiB/s wr, 527 op/s 2023-09-20T22:04:06.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:05 smithi123 ceph-mon[134433]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.3 MiB/s wr, 432 op/s 2023-09-20T22:04:06.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:05 smithi089 ceph-mon[134090]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.3 MiB/s wr, 432 op/s 2023-09-20T22:04:06.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:05 smithi022 ceph-mon[97651]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.3 MiB/s wr, 432 op/s 2023-09-20T22:04:08.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:07 smithi123 ceph-mon[134433]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.2 MiB/s wr, 469 op/s 2023-09-20T22:04:08.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:07 smithi089 ceph-mon[134090]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.2 MiB/s wr, 469 op/s 2023-09-20T22:04:08.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:07 smithi022 ceph-mon[97651]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.2 MiB/s wr, 469 op/s 2023-09-20T22:04:08.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 885.0 s, 454.0 tps, lat 225.724 ms stddev 280.534 2023-09-20T22:04:10.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:09 smithi123 ceph-mon[134433]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.6 MiB/s wr, 539 op/s 2023-09-20T22:04:10.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:09 smithi089 ceph-mon[134090]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.6 MiB/s wr, 539 op/s 2023-09-20T22:04:10.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:09 smithi022 ceph-mon[97651]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.6 MiB/s wr, 539 op/s 2023-09-20T22:04:12.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:11 smithi022 ceph-mon[97651]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.5 MiB/s wr, 488 op/s 2023-09-20T22:04:12.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:11 smithi123 ceph-mon[134433]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.5 MiB/s wr, 488 op/s 2023-09-20T22:04:12.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:11 smithi089 ceph-mon[134090]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.5 MiB/s wr, 488 op/s 2023-09-20T22:04:13.566 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 890.0 s, 417.8 tps, lat 239.832 ms stddev 277.418 2023-09-20T22:04:14.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:13 smithi022 ceph-mon[97651]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.3 MiB/s wr, 546 op/s 2023-09-20T22:04:14.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:13 smithi123 ceph-mon[134433]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.3 MiB/s wr, 546 op/s 2023-09-20T22:04:14.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:13 smithi089 ceph-mon[134090]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.3 MiB/s wr, 546 op/s 2023-09-20T22:04:15.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:14 smithi022 ceph-mon[97651]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 424 op/s 2023-09-20T22:04:15.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:14 smithi123 ceph-mon[134433]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 424 op/s 2023-09-20T22:04:15.364 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:14 smithi089 ceph-mon[134090]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 424 op/s 2023-09-20T22:04:17.594 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:17 smithi123 ceph-mon[134433]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 452 op/s 2023-09-20T22:04:17.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:17 smithi089 ceph-mon[134090]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 452 op/s 2023-09-20T22:04:17.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:17 smithi022 ceph-mon[97651]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 452 op/s 2023-09-20T22:04:18.571 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 895.0 s, 419.1 tps, lat 233.660 ms stddev 257.765 2023-09-20T22:04:20.004 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:19 smithi022 ceph-mon[97651]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.1 MiB/s wr, 513 op/s 2023-09-20T22:04:20.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:19 smithi123 ceph-mon[134433]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.1 MiB/s wr, 513 op/s 2023-09-20T22:04:20.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:19 smithi089 ceph-mon[134090]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.1 MiB/s wr, 513 op/s 2023-09-20T22:04:21.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:21 smithi022 ceph-mon[97651]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 437 op/s 2023-09-20T22:04:22.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:21 smithi123 ceph-mon[134433]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 437 op/s 2023-09-20T22:04:22.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:21 smithi089 ceph-mon[134090]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 437 op/s 2023-09-20T22:04:23.575 INFO:teuthology.orchestra.run.smithi022.stderr:progress: 900.0 s, 490.7 tps, lat 204.814 ms stddev 244.182 2023-09-20T22:04:23.991 INFO:teuthology.orchestra.run.smithi022.stdout:transaction type: 2023-09-20T22:04:23.991 INFO:teuthology.orchestra.run.smithi022.stdout:scaling factor: 32 2023-09-20T22:04:23.991 INFO:teuthology.orchestra.run.smithi022.stdout:query mode: simple 2023-09-20T22:04:23.992 INFO:teuthology.orchestra.run.smithi022.stdout:number of clients: 100 2023-09-20T22:04:23.992 INFO:teuthology.orchestra.run.smithi022.stdout:number of threads: 4 2023-09-20T22:04:23.992 INFO:teuthology.orchestra.run.smithi022.stdout:duration: 900 s 2023-09-20T22:04:23.992 INFO:teuthology.orchestra.run.smithi022.stdout:number of transactions actually processed: 647974 2023-09-20T22:04:23.992 INFO:teuthology.orchestra.run.smithi022.stdout:latency average = 138.898 ms 2023-09-20T22:04:23.993 INFO:teuthology.orchestra.run.smithi022.stdout:latency stddev = 188.178 ms 2023-09-20T22:04:23.993 INFO:teuthology.orchestra.run.smithi022.stdout:tps = 719.631684 (including connections establishing) 2023-09-20T22:04:23.993 INFO:teuthology.orchestra.run.smithi022.stdout:tps = 719.637443 (excluding connections establishing) 2023-09-20T22:04:24.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-09-20T22:04:24.073 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:23 smithi022 ceph-mon[97651]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 6.3 MiB/s wr, 525 op/s 2023-09-20T22:04:24.092 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:23 smithi123 ceph-mon[134433]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 6.3 MiB/s wr, 525 op/s 2023-09-20T22:04:24.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:23 smithi089 ceph-mon[134090]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 6.3 MiB/s wr, 525 op/s 2023-09-20T22:04:26.250 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:25 smithi022 ceph-mon[97651]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 5.0 MiB/s wr, 429 op/s 2023-09-20T22:04:26.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:25 smithi123 ceph-mon[134433]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 5.0 MiB/s wr, 429 op/s 2023-09-20T22:04:26.362 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:25 smithi089 ceph-mon[134090]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 5.0 MiB/s wr, 429 op/s 2023-09-20T22:04:28.251 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:27 smithi022 ceph-mon[97651]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 5.6 MiB/s wr, 463 op/s 2023-09-20T22:04:28.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:27 smithi123 ceph-mon[134433]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 5.6 MiB/s wr, 463 op/s 2023-09-20T22:04:28.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:27 smithi089 ceph-mon[134090]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 5.6 MiB/s wr, 463 op/s 2023-09-20T22:04:30.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:29 smithi123 ceph-mon[134433]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 6.4 MiB/s wr, 591 op/s 2023-09-20T22:04:30.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:29 smithi089 ceph-mon[134090]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 6.4 MiB/s wr, 591 op/s 2023-09-20T22:04:30.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:29 smithi022 ceph-mon[97651]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 6.4 MiB/s wr, 591 op/s 2023-09-20T22:04:31.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T22:04:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.846+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/502704867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6a8107de0 msgr2=0x7ff6a810a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:31.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.846+0000 7ff6ae1ff700 1 --2- 172.21.15.22:0/502704867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6a8107de0 0x7ff6a810a210 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff698009990 tx=0x7ff698009ca0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.852+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/502704867 shutdown_connections 2023-09-20T22:04:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.852+0000 7ff6ae1ff700 1 --2- 172.21.15.22:0/502704867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6a810d120 0x7ff6a810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.852+0000 7ff6ae1ff700 1 --2- 172.21.15.22:0/502704867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6a8107de0 0x7ff6a810a210 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.852+0000 7ff6ae1ff700 1 --2- 172.21.15.22:0/502704867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6a810a750 0x7ff6a810cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.852+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/502704867 >> 172.21.15.22:0/502704867 conn(0x7ff6a806db90 msgr2=0x7ff6a806ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:31.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.871+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/502704867 shutdown_connections 2023-09-20T22:04:31.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.878+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/502704867 wait complete. 2023-09-20T22:04:31.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.879+0000 7ff6ae1ff700 1 Processor -- start 2023-09-20T22:04:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.891+0000 7ff6ae1ff700 1 -- start start 2023-09-20T22:04:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.891+0000 7ff6ae1ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6a8107de0 0x7ff6a8114320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.891+0000 7ff6ae1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6a810a750 0x7ff6a8114860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.891+0000 7ff6ae1ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6a810d120 0x7ff6a81b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.891+0000 7ff6ae1ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a8117940 con 0x7ff6a8107de0 2023-09-20T22:04:31.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.891+0000 7ff6ae1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a8117ab0 con 0x7ff6a810a750 2023-09-20T22:04:31.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.891+0000 7ff6ae1ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a81b60b0 con 0x7ff6a810d120 2023-09-20T22:04:31.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.895+0000 7ff6a6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6a810a750 0x7ff6a8114860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:31.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.895+0000 7ff6a6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6a810a750 0x7ff6a8114860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:44582/0 (socket says 172.21.15.22:44582) 2023-09-20T22:04:31.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.895+0000 7ff6a6ffd700 1 -- 172.21.15.22:0/2649422293 learned_addr learned my addr 172.21.15.22:0/2649422293 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:04:31.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.896+0000 7ff6a7fff700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6a810d120 0x7ff6a81b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:31.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.897+0000 7ff6a7fff700 1 -- 172.21.15.22:0/2649422293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6a810a750 msgr2=0x7ff6a8114860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:31.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.897+0000 7ff6a7fff700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6a810a750 0x7ff6a8114860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.897+0000 7ff6a7fff700 1 -- 172.21.15.22:0/2649422293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6a8107de0 msgr2=0x7ff6a8114320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:31.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.897+0000 7ff6a7fff700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6a8107de0 0x7ff6a8114320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.897+0000 7ff6a7fff700 1 -- 172.21.15.22:0/2649422293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6980095f0 con 0x7ff6a810d120 2023-09-20T22:04:31.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.897+0000 7ff6a7fff700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6a810d120 0x7ff6a81b5b70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00e970 tx=0x7ff69c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:31.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.901+0000 7ff6a4ff9700 1 -- 172.21.15.22:0/2649422293 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00cbd0 con 0x7ff6a810d120 2023-09-20T22:04:31.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.901+0000 7ff6a4ff9700 1 -- 172.21.15.22:0/2649422293 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff69c00ee60 con 0x7ff6a810d120 2023-09-20T22:04:31.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.901+0000 7ff6a4ff9700 1 -- 172.21.15.22:0/2649422293 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c005020 con 0x7ff6a810d120 2023-09-20T22:04:31.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.903+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/2649422293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a81b6250 con 0x7ff6a810d120 2023-09-20T22:04:31.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.903+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/2649422293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a81b6790 con 0x7ff6a810d120 2023-09-20T22:04:31.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.911+0000 7ff6c1b4ab80 1 Processor -- start 2023-09-20T22:04:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.913+0000 7ff6c1b4ab80 1 -- start start 2023-09-20T22:04:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.913+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f686f0 0x562848f68b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.913+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 0x562848f695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.913+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f6a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.913+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848edcca0 con 0x562848f686f0 2023-09-20T22:04:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.914+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848ec0c70 con 0x562848f69050 2023-09-20T22:04:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.914+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848ebe590 con 0x562848f69be0 2023-09-20T22:04:31.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.918+0000 7ff6a4ff9700 1 -- 172.21.15.22:0/2649422293 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ff69c004750 con 0x7ff6a810d120 2023-09-20T22:04:31.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.919+0000 7ff6a77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 0x562848f695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:31.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.919+0000 7ff6a6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f6a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:31.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.923+0000 7ff6a6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f6a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:51864/0 (socket says 172.21.15.22:51864) 2023-09-20T22:04:31.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.923+0000 7ff6a6ffd700 1 -- 172.21.15.22:0/936075252 learned_addr learned my addr 172.21.15.22:0/936075252 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:04:31.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.923+0000 7ff6a6ffd700 1 --2- 172.21.15.22:0/936075252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f686f0 0x562848f68b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:31.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.923+0000 7ff6a6ffd700 1 -- 172.21.15.22:0/936075252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 msgr2=0x562848f695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:31.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.923+0000 7ff6a6ffd700 1 --2- 172.21.15.22:0/936075252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 0x562848f695e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.923+0000 7ff6a6ffd700 1 -- 172.21.15.22:0/936075252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f686f0 msgr2=0x562848f68b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:31.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.923+0000 7ff6a6ffd700 1 --2- 172.21.15.22:0/936075252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f686f0 0x562848f68b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.923+0000 7ff6a6ffd700 1 -- 172.21.15.22:0/936075252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562848ce4de0 con 0x562848f69be0 2023-09-20T22:04:31.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.924+0000 7ff6a6ffd700 1 --2- 172.21.15.22:0/936075252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f6a060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0008920 tx=0x7ff6a0009cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d6d7339239b31f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:31.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.924+0000 7ff6a4ff9700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff6900a1500 0x7ff6900a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.925+0000 7ff6a4ff9700 1 -- 172.21.15.22:0/2649422293 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ff69c014070 con 0x7ff6a810d120 2023-09-20T22:04:31.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.935+0000 7ff68dffb700 1 -- 172.21.15.22:0/936075252 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0015670 con 0x562848f69be0 2023-09-20T22:04:31.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.935+0000 7ff68dffb700 1 -- 172.21.15.22:0/936075252 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6a0015c90 con 0x562848f69be0 2023-09-20T22:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.935+0000 7ff68dffb700 1 -- 172.21.15.22:0/936075252 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0018cb0 con 0x562848f69be0 2023-09-20T22:04:31.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.936+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/936075252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 msgr2=0x562848f6a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:31.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.936+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/936075252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f6a060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0008920 tx=0x7ff6a0009cf0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.937+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/936075252 shutdown_connections 2023-09-20T22:04:31.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.937+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/936075252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f6a060 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.937+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/936075252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f686f0 0x562848f68b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.937+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/936075252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 0x562848f695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.937+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/936075252 >> 172.21.15.22:0/936075252 conn(0x562848e95000 msgr2=0x562848f631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:31.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.945+0000 7ff6a77fe700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff6900a1500 0x7ff6900a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:31.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.958+0000 7ff6a77fe700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff6900a1500 0x7ff6900a39c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff698006010 tx=0x7ff69800bb10 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:31.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.959+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/936075252 shutdown_connections 2023-09-20T22:04:31.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.959+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/936075252 wait complete. 2023-09-20T22:04:31.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.959+0000 7ff6c1b4ab80 1 -- start start 2023-09-20T22:04:31.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.960+0000 7ff6c1b4ab80 1 Processor -- start 2023-09-20T22:04:31.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.961+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f72260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.961+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f69050 0x562848f727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.961+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 0x562848f797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.961+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848eb9e80 con 0x562848f69050 2023-09-20T22:04:31.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.961+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848e56740 con 0x562848f69be0 2023-09-20T22:04:31.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.961+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848e563e0 con 0x562848f686f0 2023-09-20T22:04:31.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.961+0000 7ff6c1b4ab80 10 client.0 _ll_register_callbacks cb 0x562848da0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:04:31.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.962+0000 7ff6a77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f72260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:31.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.963+0000 7ff6a77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f72260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:51880/0 (socket says 172.21.15.22:51880) 2023-09-20T22:04:31.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.963+0000 7ff6a77fe700 1 -- 172.21.15.22:0/197495583 learned_addr learned my addr 172.21.15.22:0/197495583 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:04:31.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.963+0000 7ff6a7fff700 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 0x562848f797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:31.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.963+0000 7ff6a77fe700 1 -- 172.21.15.22:0/197495583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 msgr2=0x562848f797e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:31.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.963+0000 7ff6a77fe700 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 0x562848f797e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.964+0000 7ff6a77fe700 1 -- 172.21.15.22:0/197495583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f69050 msgr2=0x562848f727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:31.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.964+0000 7ff6a77fe700 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f69050 0x562848f727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:31.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.964+0000 7ff6a77fe700 1 -- 172.21.15.22:0/197495583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a0011040 con 0x562848f686f0 2023-09-20T22:04:31.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.964+0000 7ff6a77fe700 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f72260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff698009990 tx=0x7ff698000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9547a204d0451e server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:31.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.965+0000 7ff686ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T22:04:31.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.965+0000 7ff686ffd700 1 -- 172.21.15.22:0/197495583 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69801e070 con 0x562848f686f0 2023-09-20T22:04:31.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.965+0000 7ff686ffd700 1 -- 172.21.15.22:0/197495583 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ff69800be60 con 0x562848f686f0 2023-09-20T22:04:31.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.965+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/197495583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562848e2abb0 con 0x562848f686f0 2023-09-20T22:04:31.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.966+0000 7ff686ffd700 1 -- 172.21.15.22:0/197495583 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff698004c70 con 0x562848f686f0 2023-09-20T22:04:31.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.967+0000 7ff6c1b4ab80 10 client.26203 fetch_fsmap learned FSMap version 32 2023-09-20T22:04:31.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.967+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/197495583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562848ccdc60 con 0x562848f686f0 2023-09-20T22:04:31.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.978+0000 7ff686ffd700 1 -- 172.21.15.22:0/197495583 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562848e2abb0 con 0x562848f686f0 2023-09-20T22:04:31.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.978+0000 7ff686ffd700 1 -- 172.21.15.22:0/197495583 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff698026450 con 0x562848f686f0 2023-09-20T22:04:31.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.978+0000 7ff6c1b4ab80 10 client.26203 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:04:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.978+0000 7ff6c1b4ab80 10 client.26203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:04:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.979+0000 7ff6c1b4ab80 20 client.26203 populate_metadata read hostname 'smithi022' 2023-09-20T22:04:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.979+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x562848f7e140 0x562848f80560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.979+0000 7ff6c1b4ab80 4 client.26203 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T22:04:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.979+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/197495583 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562848f37020 con 0x562848f7e140 2023-09-20T22:04:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.980+0000 7ff6a6ffd700 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x562848f7e140 0x562848f80560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:32.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.995+0000 7ff6a6ffd700 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x562848f7e140 0x562848f80560 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd6a4415c19211e8 server_cookie=5771a4626df94c44 in_seq=0 out_seq=0 2023-09-20T22:04:32.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:31.996+0000 7ff686ffd700 10 client.26203 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:04:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.007+0000 7ff686ffd700 1 -- 172.21.15.22:0/197495583 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562848f37020 con 0x562848f7e140 2023-09-20T22:04:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.007+0000 7ff686ffd700 10 client.26203 handle_command_reply: tid=0 2023-09-20T22:04:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.007+0000 7ff6c1b4ab80 1 client.26203 shutdown 2023-09-20T22:04:32.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.026+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/197495583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 msgr2=0x562848f72260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:32.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.026+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f72260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff698009990 tx=0x7ff698000bc0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.027+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/197495583 shutdown_connections 2023-09-20T22:04:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.027+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x562848f7e140 0x562848f80560 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.027+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 0x562848f797e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.027+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f72260 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.027+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/197495583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f69050 0x562848f727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.027+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/197495583 >> 172.21.15.22:0/197495583 conn(0x562848e95000 msgr2=0x562848f62880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:32.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.032+0000 7ff686ffd700 0 client.26203 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:04:32.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.032+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/197495583 shutdown_connections 2023-09-20T22:04:32.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.032+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/197495583 wait complete. 2023-09-20T22:04:32.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.044+0000 7ff6c1b4ab80 20 client.26203 trim_cache size 0 max 16384 2023-09-20T22:04:32.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.087+0000 7ff6c1b4ab80 1 Processor -- start 2023-09-20T22:04:32.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.087+0000 7ff6c1b4ab80 1 -- start start 2023-09-20T22:04:32.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.087+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:32.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.087+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f69050 0x562848f79770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:32.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.087+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 0x562848f6cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:32.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.087+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848d3a920 con 0x562848f69050 2023-09-20T22:04:32.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.087+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848f69430 con 0x562848f69be0 2023-09-20T22:04:32.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.087+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848f5b1a0 con 0x562848f686f0 2023-09-20T22:04:32.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.088+0000 7ff6a77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f70f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:32.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.088+0000 7ff6a77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f70f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:51894/0 (socket says 172.21.15.22:51894) 2023-09-20T22:04:32.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.088+0000 7ff6a77fe700 1 -- 172.21.15.22:0/195805403 learned_addr learned my addr 172.21.15.22:0/195805403 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:04:32.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.088+0000 7ff6a7fff700 1 --2- 172.21.15.22:0/195805403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 0x562848f6cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:32.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.089+0000 7ff6a77fe700 1 -- 172.21.15.22:0/195805403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 msgr2=0x562848f6cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:32.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.089+0000 7ff6a77fe700 1 --2- 172.21.15.22:0/195805403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 0x562848f6cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.089+0000 7ff6a77fe700 1 -- 172.21.15.22:0/195805403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f69050 msgr2=0x562848f79770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:32.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.089+0000 7ff6a77fe700 1 --2- 172.21.15.22:0/195805403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f69050 0x562848f79770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.089+0000 7ff6a77fe700 1 -- 172.21.15.22:0/195805403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562848d0b250 con 0x562848f686f0 2023-09-20T22:04:32.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.089+0000 7ff6a77fe700 1 --2- 172.21.15.22:0/195805403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f70f20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff6980267b0 tx=0x7ff698005d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c6ad38ac123611c server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:32.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.089+0000 7ff6a7fff700 1 --2- 172.21.15.22:0/195805403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 0x562848f6cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:04:32.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:31 smithi123 ceph-mon[134433]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 6.5 MiB/s wr, 591 op/s 2023-09-20T22:04:32.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.093+0000 7ff68e7fc700 1 -- 172.21.15.22:0/195805403 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698026450 con 0x562848f686f0 2023-09-20T22:04:32.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.093+0000 7ff68e7fc700 1 -- 172.21.15.22:0/195805403 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6980265b0 con 0x562848f686f0 2023-09-20T22:04:32.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.093+0000 7ff68e7fc700 1 -- 172.21.15.22:0/195805403 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800fb80 con 0x562848f686f0 2023-09-20T22:04:32.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.094+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/195805403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 msgr2=0x562848f70f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:32.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.094+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/195805403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f70f20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff6980267b0 tx=0x7ff698005d00 comp rx=0 tx=0).stop 2023-09-20T22:04:32.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.095+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/195805403 shutdown_connections 2023-09-20T22:04:32.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.095+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/195805403 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69be0 0x562848f6cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.095+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/195805403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f686f0 0x562848f70f20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.095+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/195805403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f69050 0x562848f79770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.095+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/195805403 >> 172.21.15.22:0/195805403 conn(0x562848e95000 msgr2=0x562848f80150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.095+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/195805403 shutdown_connections 2023-09-20T22:04:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.095+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/195805403 wait complete. 2023-09-20T22:04:32.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.096+0000 7ff6c1b4ab80 1 -- start start 2023-09-20T22:04:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.096+0000 7ff6c1b4ab80 1 Processor -- start 2023-09-20T22:04:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.097+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f686f0 0x562848f773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.097+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 0x562848f77900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.097+0000 7ff6c1b4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f78910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:32.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.097+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848f78e50 con 0x562848f686f0 2023-09-20T22:04:32.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.097+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848f78fc0 con 0x562848f69050 2023-09-20T22:04:32.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.097+0000 7ff6c1b4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x562848f79180 con 0x562848f69be0 2023-09-20T22:04:32.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.097+0000 7ff6c1b4ab80 10 client.0 _ll_register_callbacks cb 0x562848da0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:04:32.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.097+0000 7ff6a6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 0x562848f77900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:32.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.098+0000 7ff6a7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f78910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:32.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.098+0000 7ff6a6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 0x562848f77900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:44632/0 (socket says 172.21.15.22:44632) 2023-09-20T22:04:32.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.098+0000 7ff6a6ffd700 1 -- 172.21.15.22:0/2067705243 learned_addr learned my addr 172.21.15.22:0/2067705243 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:04:32.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.101+0000 7ff6a7fff700 1 -- 172.21.15.22:0/2067705243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 msgr2=0x562848f77900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:32.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.101+0000 7ff6a7fff700 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 0x562848f77900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.101+0000 7ff6a7fff700 1 -- 172.21.15.22:0/2067705243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f686f0 msgr2=0x562848f773c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:04:32.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.101+0000 7ff6a7fff700 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f686f0 0x562848f773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.101+0000 7ff6a7fff700 1 -- 172.21.15.22:0/2067705243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562848ccdc60 con 0x562848f69be0 2023-09-20T22:04:32.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.102+0000 7ff6a7fff700 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f78910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00c5b0 tx=0x7ff69c09a470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98694ef571dadb58 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:04:32.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.102+0000 7ff686ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-09-20T22:04:32.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.102+0000 7ff686ffd700 1 -- 172.21.15.22:0/2067705243 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c099f30 con 0x562848f69be0 2023-09-20T22:04:32.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.102+0000 7ff686ffd700 1 -- 172.21.15.22:0/2067705243 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ff69c09dd60 con 0x562848f69be0 2023-09-20T22:04:32.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.102+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/2067705243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6a0015c90 con 0x562848f69be0 2023-09-20T22:04:32.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:31 smithi089 ceph-mon[134090]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 6.5 MiB/s wr, 591 op/s 2023-09-20T22:04:32.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.109+0000 7ff686ffd700 1 -- 172.21.15.22:0/2067705243 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff69c0a1fa0 con 0x562848f69be0 2023-09-20T22:04:32.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.111+0000 7ff6c1b4ab80 10 client.26209 fetch_fsmap learned FSMap version 32 2023-09-20T22:04:32.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.111+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/2067705243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562848ce32b0 con 0x562848f69be0 2023-09-20T22:04:32.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.111+0000 7ff686ffd700 1 -- 172.21.15.22:0/2067705243 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6a0015c90 con 0x562848f69be0 2023-09-20T22:04:32.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.112+0000 7ff686ffd700 1 -- 172.21.15.22:0/2067705243 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff69c0a1b10 con 0x562848f69be0 2023-09-20T22:04:32.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.113+0000 7ff6c1b4ab80 10 client.26209 fetch_fsmap finished waiting for FSMap version 32 2023-09-20T22:04:32.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.113+0000 7ff6c1b4ab80 10 client.26209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:04:32.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.113+0000 7ff6c1b4ab80 20 client.26209 populate_metadata read hostname 'smithi022' 2023-09-20T22:04:32.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.113+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x562848fd2e90 0x562848fd52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:04:32.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.113+0000 7ff6c1b4ab80 4 client.26209 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T22:04:32.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.113+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/2067705243 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562848eb2e60 con 0x562848fd2e90 2023-09-20T22:04:32.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.124+0000 7ff6a77fe700 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x562848fd2e90 0x562848fd52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:04:32.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.139+0000 7ff6a77fe700 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x562848fd2e90 0x562848fd52b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f14c5859d14b61aa server_cookie=6fb21506ea3e63d7 in_seq=0 out_seq=0 2023-09-20T22:04:32.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.140+0000 7ff686ffd700 10 client.26209 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:04:32.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.151+0000 7ff686ffd700 1 -- 172.21.15.22:0/2067705243 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562848eb2e60 con 0x562848fd2e90 2023-09-20T22:04:32.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.151+0000 7ff686ffd700 10 client.26209 handle_command_reply: tid=0 2023-09-20T22:04:32.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.151+0000 7ff6c1b4ab80 1 client.26209 shutdown 2023-09-20T22:04:32.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.159+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/2067705243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 msgr2=0x562848f78910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:32.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.159+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f78910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00c5b0 tx=0x7ff69c09a470 comp rx=0 tx=0).stop 2023-09-20T22:04:32.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.159+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/2067705243 shutdown_connections 2023-09-20T22:04:32.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.159+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x562848fd2e90 0x562848fd52b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.159+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562848f69be0 0x562848f78910 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.159+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562848f686f0 0x562848f773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.159+0000 7ff6c1b4ab80 1 --2- 172.21.15.22:0/2067705243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562848f69050 0x562848f77900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.159+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/2067705243 >> 172.21.15.22:0/2067705243 conn(0x562848e95000 msgr2=0x562848f70f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:32.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.159+0000 7ff686ffd700 0 client.26209 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:04:32.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.161+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/2067705243 shutdown_connections 2023-09-20T22:04:32.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.167+0000 7ff6c1b4ab80 1 -- 172.21.15.22:0/2067705243 wait complete. 2023-09-20T22:04:32.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.180+0000 7ff6c1b4ab80 20 client.26209 trim_cache size 0 max 16384 2023-09-20T22:04:32.184 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:04:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T22:04:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T22:04:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T22:04:32.184 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T22:04:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T22:04:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T22:04:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:04:32.185 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:04:32.185 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:04:32.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.184+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/2649422293 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff6900a1500 msgr2=0x7ff6900a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:32.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.184+0000 7ff6ae1ff700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff6900a1500 0x7ff6900a39c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff698006010 tx=0x7ff69800bb10 comp rx=0 tx=0).stop 2023-09-20T22:04:32.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.184+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/2649422293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6a810d120 msgr2=0x7ff6a81b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:04:32.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.184+0000 7ff6ae1ff700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6a810d120 0x7ff6a81b5b70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00e970 tx=0x7ff69c00ec80 comp rx=0 tx=0).stop 2023-09-20T22:04:32.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.191+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/2649422293 shutdown_connections 2023-09-20T22:04:32.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.191+0000 7ff6ae1ff700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6a810d120 0x7ff6a81b5b70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.191+0000 7ff6ae1ff700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6a8107de0 0x7ff6a8114320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.191+0000 7ff6ae1ff700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff6900a1500 0x7ff6900a39c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.191+0000 7ff6ae1ff700 1 --2- 172.21.15.22:0/2649422293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff6a810a750 0x7ff6a8114860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:04:32.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.192+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/2649422293 >> 172.21.15.22:0/2649422293 conn(0x7ff6a806db90 msgr2=0x7ff6a810b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:04:32.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.200+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/2649422293 shutdown_connections 2023-09-20T22:04:32.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:04:32.209+0000 7ff6ae1ff700 1 -- 172.21.15.22:0/2649422293 wait complete. 2023-09-20T22:04:32.246 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T22:04:32.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:31 smithi022 ceph-mon[97651]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 6.5 MiB/s wr, 591 op/s 2023-09-20T22:04:34.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:33 smithi022 ceph-mon[97651]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 6.8 MiB/s wr, 611 op/s 2023-09-20T22:04:34.342 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:33 smithi123 ceph-mon[134433]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 6.8 MiB/s wr, 611 op/s 2023-09-20T22:04:34.363 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:33 smithi089 ceph-mon[134090]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 6.8 MiB/s wr, 611 op/s 2023-09-20T22:04:35.250 DEBUG:teuthology.orchestra.run.smithi022:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-09-20T22:04:35.464 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/: 2023-09-20T22:04:35.465 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.465 INFO:teuthology.orchestra.run.smithi022.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:48 . 2023-09-20T22:04:35.465 INFO:teuthology.orchestra.run.smithi022.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:48 .. 2023-09-20T22:04:35.465 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 data 2023-09-20T22:04:35.465 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.466 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data: 2023-09-20T22:04:35.466 INFO:teuthology.orchestra.run.smithi022.stdout:total 31 2023-09-20T22:04:35.466 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 . 2023-09-20T22:04:35.466 INFO:teuthology.orchestra.run.smithi022.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:48 .. 2023-09-20T22:04:35.466 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 20 21:49 base 2023-09-20T22:04:35.467 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 20 21:49 current_logfiles 2023-09-20T22:04:35.467 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Sep 20 21:49 global 2023-09-20T22:04:35.467 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 log 2023-09-20T22:04:35.467 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_commit_ts 2023-09-20T22:04:35.467 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_dynshmem 2023-09-20T22:04:35.468 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Sep 20 21:49 pg_hba.conf 2023-09-20T22:04:35.468 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 20 21:49 pg_ident.conf 2023-09-20T22:04:35.468 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 22:04 pg_logical 2023-09-20T22:04:35.468 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 pg_multixact 2023-09-20T22:04:35.468 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 pg_notify 2023-09-20T22:04:35.469 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_replslot 2023-09-20T22:04:35.469 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_serial 2023-09-20T22:04:35.469 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_snapshots 2023-09-20T22:04:35.469 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 22:04 pg_stat 2023-09-20T22:04:35.469 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 22:04 pg_stat_tmp 2023-09-20T22:04:35.470 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 22:04 pg_subtrans 2023-09-20T22:04:35.470 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_tblspc 2023-09-20T22:04:35.470 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pg_twophase 2023-09-20T22:04:35.470 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 PG_VERSION 2023-09-20T22:04:35.470 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Sep 20 22:04 pg_wal 2023-09-20T22:04:35.471 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 pg_xact 2023-09-20T22:04:35.471 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 20 21:49 postgresql.auto.conf 2023-09-20T22:04:35.471 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Sep 20 21:49 postgresql.conf 2023-09-20T22:04:35.471 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 20 21:49 postmaster.opts 2023-09-20T22:04:35.471 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.472 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/base: 2023-09-20T22:04:35.472 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.472 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 20 21:49 . 2023-09-20T22:04:35.472 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.472 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 20 21:49 1 2023-09-20T22:04:35.473 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 20 21:49 12346 2023-09-20T22:04:35.473 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Sep 20 21:49 12347 2023-09-20T22:04:35.473 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 pgsql_tmp 2023-09-20T22:04:35.473 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.473 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/base/1: 2023-09-20T22:04:35.473 INFO:teuthology.orchestra.run.smithi022.stdout:total 7233 2023-09-20T22:04:35.474 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 20 21:49 . 2023-09-20T22:04:35.474 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 20 21:49 .. 2023-09-20T22:04:35.474 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 112 2023-09-20T22:04:35.474 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 113 2023-09-20T22:04:35.474 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 20 21:49 12182 2023-09-20T22:04:35.475 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12182_fsm 2023-09-20T22:04:35.475 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12182_vm 2023-09-20T22:04:35.475 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12184 2023-09-20T22:04:35.475 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12186 2023-09-20T22:04:35.475 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187 2023-09-20T22:04:35.476 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12187_fsm 2023-09-20T22:04:35.476 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187_vm 2023-09-20T22:04:35.476 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12189 2023-09-20T22:04:35.476 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12191 2023-09-20T22:04:35.476 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192 2023-09-20T22:04:35.477 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12192_fsm 2023-09-20T22:04:35.477 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192_vm 2023-09-20T22:04:35.477 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12194 2023-09-20T22:04:35.477 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12196 2023-09-20T22:04:35.477 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197 2023-09-20T22:04:35.477 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12197_fsm 2023-09-20T22:04:35.478 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197_vm 2023-09-20T22:04:35.478 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12199 2023-09-20T22:04:35.478 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12201 2023-09-20T22:04:35.478 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202 2023-09-20T22:04:35.478 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12202_fsm 2023-09-20T22:04:35.479 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202_vm 2023-09-20T22:04:35.479 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12204 2023-09-20T22:04:35.479 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12206 2023-09-20T22:04:35.479 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207 2023-09-20T22:04:35.479 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12207_fsm 2023-09-20T22:04:35.479 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207_vm 2023-09-20T22:04:35.480 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12209 2023-09-20T22:04:35.480 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12211 2023-09-20T22:04:35.480 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12212 2023-09-20T22:04:35.480 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12214 2023-09-20T22:04:35.480 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12216 2023-09-20T22:04:35.480 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 20 21:49 1247 2023-09-20T22:04:35.481 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1247_fsm 2023-09-20T22:04:35.481 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1247_vm 2023-09-20T22:04:35.481 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 20 21:49 1249 2023-09-20T22:04:35.481 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1249_fsm 2023-09-20T22:04:35.481 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1249_vm 2023-09-20T22:04:35.481 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 20 21:49 1255 2023-09-20T22:04:35.482 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1255_fsm 2023-09-20T22:04:35.482 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1255_vm 2023-09-20T22:04:35.482 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 20 21:49 1259 2023-09-20T22:04:35.482 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1259_fsm 2023-09-20T22:04:35.482 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1259_vm 2023-09-20T22:04:35.483 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417 2023-09-20T22:04:35.483 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417_vm 2023-09-20T22:04:35.483 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418 2023-09-20T22:04:35.483 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418_vm 2023-09-20T22:04:35.483 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 174 2023-09-20T22:04:35.483 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 175 2023-09-20T22:04:35.484 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2187 2023-09-20T22:04:35.484 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224 2023-09-20T22:04:35.484 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224_vm 2023-09-20T22:04:35.484 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328 2023-09-20T22:04:35.484 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328_vm 2023-09-20T22:04:35.484 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336 2023-09-20T22:04:35.485 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336_vm 2023-09-20T22:04:35.485 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2337 2023-09-20T22:04:35.485 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2600 2023-09-20T22:04:35.485 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2600_fsm 2023-09-20T22:04:35.485 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2600_vm 2023-09-20T22:04:35.485 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601 2023-09-20T22:04:35.486 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2601_fsm 2023-09-20T22:04:35.486 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601_vm 2023-09-20T22:04:35.486 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2602 2023-09-20T22:04:35.486 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2602_fsm 2023-09-20T22:04:35.486 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2602_vm 2023-09-20T22:04:35.486 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2603 2023-09-20T22:04:35.487 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2603_fsm 2023-09-20T22:04:35.487 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2603_vm 2023-09-20T22:04:35.487 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604 2023-09-20T22:04:35.487 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604_vm 2023-09-20T22:04:35.487 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2605 2023-09-20T22:04:35.487 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2605_fsm 2023-09-20T22:04:35.488 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2605_vm 2023-09-20T22:04:35.488 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606 2023-09-20T22:04:35.488 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2606_fsm 2023-09-20T22:04:35.488 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606_vm 2023-09-20T22:04:35.488 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607 2023-09-20T22:04:35.489 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607_fsm 2023-09-20T22:04:35.489 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2607_vm 2023-09-20T22:04:35.489 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 20 21:49 2608 2023-09-20T22:04:35.489 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2608_fsm 2023-09-20T22:04:35.489 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2608_vm 2023-09-20T22:04:35.489 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 20 21:49 2609 2023-09-20T22:04:35.490 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2609_fsm 2023-09-20T22:04:35.490 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2609_vm 2023-09-20T22:04:35.490 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610 2023-09-20T22:04:35.490 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610_fsm 2023-09-20T22:04:35.490 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2610_vm 2023-09-20T22:04:35.491 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611 2023-09-20T22:04:35.491 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611_vm 2023-09-20T22:04:35.491 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612 2023-09-20T22:04:35.491 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2612_fsm 2023-09-20T22:04:35.491 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612_vm 2023-09-20T22:04:35.492 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613 2023-09-20T22:04:35.492 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613_vm 2023-09-20T22:04:35.492 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615 2023-09-20T22:04:35.492 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2615_fsm 2023-09-20T22:04:35.492 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615_vm 2023-09-20T22:04:35.492 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616 2023-09-20T22:04:35.493 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616_fsm 2023-09-20T22:04:35.493 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2616_vm 2023-09-20T22:04:35.493 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 20 21:49 2617 2023-09-20T22:04:35.493 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2617_fsm 2023-09-20T22:04:35.493 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2617_vm 2023-09-20T22:04:35.493 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 20 21:49 2618 2023-09-20T22:04:35.494 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2618_fsm 2023-09-20T22:04:35.494 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2618_vm 2023-09-20T22:04:35.494 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 20 21:49 2619 2023-09-20T22:04:35.494 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2619_fsm 2023-09-20T22:04:35.494 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2619_vm 2023-09-20T22:04:35.494 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620 2023-09-20T22:04:35.495 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620_vm 2023-09-20T22:04:35.495 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2650 2023-09-20T22:04:35.495 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2651 2023-09-20T22:04:35.495 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2652 2023-09-20T22:04:35.495 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2653 2023-09-20T22:04:35.495 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2654 2023-09-20T22:04:35.496 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2655 2023-09-20T22:04:35.496 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2656 2023-09-20T22:04:35.496 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2657 2023-09-20T22:04:35.496 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 20 21:49 2658 2023-09-20T22:04:35.496 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2659 2023-09-20T22:04:35.497 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2660 2023-09-20T22:04:35.497 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2661 2023-09-20T22:04:35.497 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2662 2023-09-20T22:04:35.497 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2663 2023-09-20T22:04:35.497 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2664 2023-09-20T22:04:35.497 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2665 2023-09-20T22:04:35.498 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2666 2023-09-20T22:04:35.498 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2667 2023-09-20T22:04:35.498 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2668 2023-09-20T22:04:35.498 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2669 2023-09-20T22:04:35.498 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2670 2023-09-20T22:04:35.498 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 20 21:49 2673 2023-09-20T22:04:35.499 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Sep 20 21:49 2674 2023-09-20T22:04:35.499 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 20 21:49 2675 2023-09-20T22:04:35.499 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2678 2023-09-20T22:04:35.499 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2679 2023-09-20T22:04:35.499 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2680 2023-09-20T22:04:35.499 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2681 2023-09-20T22:04:35.500 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2682 2023-09-20T22:04:35.500 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2683 2023-09-20T22:04:35.500 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2684 2023-09-20T22:04:35.500 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2685 2023-09-20T22:04:35.500 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2686 2023-09-20T22:04:35.500 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2687 2023-09-20T22:04:35.501 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2688 2023-09-20T22:04:35.501 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2689 2023-09-20T22:04:35.501 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2690 2023-09-20T22:04:35.501 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 20 21:49 2691 2023-09-20T22:04:35.501 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2692 2023-09-20T22:04:35.501 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2693 2023-09-20T22:04:35.502 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2696 2023-09-20T22:04:35.502 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2699 2023-09-20T22:04:35.502 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2701 2023-09-20T22:04:35.502 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2702 2023-09-20T22:04:35.502 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2703 2023-09-20T22:04:35.503 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2704 2023-09-20T22:04:35.503 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2753 2023-09-20T22:04:35.503 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2753_fsm 2023-09-20T22:04:35.503 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2753_vm 2023-09-20T22:04:35.503 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2754 2023-09-20T22:04:35.503 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2755 2023-09-20T22:04:35.504 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2756 2023-09-20T22:04:35.504 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2757 2023-09-20T22:04:35.504 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830 2023-09-20T22:04:35.504 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830_vm 2023-09-20T22:04:35.504 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2831 2023-09-20T22:04:35.504 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832 2023-09-20T22:04:35.505 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832_vm 2023-09-20T22:04:35.505 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2833 2023-09-20T22:04:35.505 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834 2023-09-20T22:04:35.505 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834_vm 2023-09-20T22:04:35.505 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2835 2023-09-20T22:04:35.505 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836 2023-09-20T22:04:35.506 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836_vm 2023-09-20T22:04:35.506 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2837 2023-09-20T22:04:35.506 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 20 21:49 2838 2023-09-20T22:04:35.506 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2838_fsm 2023-09-20T22:04:35.506 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2838_vm 2023-09-20T22:04:35.506 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2839 2023-09-20T22:04:35.507 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840 2023-09-20T22:04:35.507 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840_fsm 2023-09-20T22:04:35.507 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2840_vm 2023-09-20T22:04:35.507 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2841 2023-09-20T22:04:35.507 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995 2023-09-20T22:04:35.508 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995_vm 2023-09-20T22:04:35.508 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2996 2023-09-20T22:04:35.508 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079 2023-09-20T22:04:35.508 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3079_fsm 2023-09-20T22:04:35.508 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079_vm 2023-09-20T22:04:35.508 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3080 2023-09-20T22:04:35.509 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3081 2023-09-20T22:04:35.509 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3085 2023-09-20T22:04:35.509 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118 2023-09-20T22:04:35.509 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118_vm 2023-09-20T22:04:35.509 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3119 2023-09-20T22:04:35.509 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3164 2023-09-20T22:04:35.510 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256 2023-09-20T22:04:35.510 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256_vm 2023-09-20T22:04:35.510 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3257 2023-09-20T22:04:35.510 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3258 2023-09-20T22:04:35.510 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350 2023-09-20T22:04:35.511 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350_vm 2023-09-20T22:04:35.511 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3351 2023-09-20T22:04:35.511 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3379 2023-09-20T22:04:35.511 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3380 2023-09-20T22:04:35.511 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381 2023-09-20T22:04:35.512 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381_vm 2023-09-20T22:04:35.512 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3394 2023-09-20T22:04:35.512 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3394_fsm 2023-09-20T22:04:35.512 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3394_vm 2023-09-20T22:04:35.512 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3395 2023-09-20T22:04:35.512 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439 2023-09-20T22:04:35.513 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439_vm 2023-09-20T22:04:35.513 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3440 2023-09-20T22:04:35.513 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 3455 2023-09-20T22:04:35.513 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3456 2023-09-20T22:04:35.513 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3456_fsm 2023-09-20T22:04:35.513 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3456_vm 2023-09-20T22:04:35.514 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466 2023-09-20T22:04:35.514 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466_vm 2023-09-20T22:04:35.514 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3467 2023-09-20T22:04:35.514 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3468 2023-09-20T22:04:35.514 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501 2023-09-20T22:04:35.515 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501_vm 2023-09-20T22:04:35.515 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3502 2023-09-20T22:04:35.515 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3503 2023-09-20T22:04:35.515 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3534 2023-09-20T22:04:35.515 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541 2023-09-20T22:04:35.515 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3541_fsm 2023-09-20T22:04:35.516 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541_vm 2023-09-20T22:04:35.516 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3542 2023-09-20T22:04:35.516 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3574 2023-09-20T22:04:35.516 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3575 2023-09-20T22:04:35.516 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576 2023-09-20T22:04:35.516 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576_vm 2023-09-20T22:04:35.517 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596 2023-09-20T22:04:35.517 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596_vm 2023-09-20T22:04:35.517 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3597 2023-09-20T22:04:35.517 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598 2023-09-20T22:04:35.517 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598_vm 2023-09-20T22:04:35.517 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3599 2023-09-20T22:04:35.518 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600 2023-09-20T22:04:35.518 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3600_fsm 2023-09-20T22:04:35.518 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600_vm 2023-09-20T22:04:35.518 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601 2023-09-20T22:04:35.518 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3601_fsm 2023-09-20T22:04:35.519 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601_vm 2023-09-20T22:04:35.519 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602 2023-09-20T22:04:35.519 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3602_fsm 2023-09-20T22:04:35.519 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602_vm 2023-09-20T22:04:35.519 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3603 2023-09-20T22:04:35.519 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3603_fsm 2023-09-20T22:04:35.520 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3603_vm 2023-09-20T22:04:35.521 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3604 2023-09-20T22:04:35.521 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3605 2023-09-20T22:04:35.521 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3606 2023-09-20T22:04:35.522 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3607 2023-09-20T22:04:35.522 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3608 2023-09-20T22:04:35.522 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 3609 2023-09-20T22:04:35.522 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3712 2023-09-20T22:04:35.522 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764 2023-09-20T22:04:35.523 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3764_fsm 2023-09-20T22:04:35.523 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764_vm 2023-09-20T22:04:35.523 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3766 2023-09-20T22:04:35.523 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3767 2023-09-20T22:04:35.523 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3997 2023-09-20T22:04:35.523 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 5002 2023-09-20T22:04:35.524 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 548 2023-09-20T22:04:35.524 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 549 2023-09-20T22:04:35.524 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102 2023-09-20T22:04:35.524 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102_vm 2023-09-20T22:04:35.524 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104 2023-09-20T22:04:35.524 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104_vm 2023-09-20T22:04:35.525 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106 2023-09-20T22:04:35.525 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106_vm 2023-09-20T22:04:35.525 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6110 2023-09-20T22:04:35.525 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6111 2023-09-20T22:04:35.525 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6112 2023-09-20T22:04:35.526 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6113 2023-09-20T22:04:35.526 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6117 2023-09-20T22:04:35.526 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826 2023-09-20T22:04:35.526 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826_vm 2023-09-20T22:04:35.526 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 827 2023-09-20T22:04:35.527 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 828 2023-09-20T22:04:35.527 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 20 21:49 pg_filenode.map 2023-09-20T22:04:35.527 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 PG_VERSION 2023-09-20T22:04:35.527 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.527 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-09-20T22:04:35.527 INFO:teuthology.orchestra.run.smithi022.stdout:total 7233 2023-09-20T22:04:35.528 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Sep 20 21:49 . 2023-09-20T22:04:35.528 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 20 21:49 .. 2023-09-20T22:04:35.528 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 112 2023-09-20T22:04:35.528 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 113 2023-09-20T22:04:35.528 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 20 21:49 12182 2023-09-20T22:04:35.528 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12182_fsm 2023-09-20T22:04:35.529 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12182_vm 2023-09-20T22:04:35.529 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12184 2023-09-20T22:04:35.529 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12186 2023-09-20T22:04:35.529 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187 2023-09-20T22:04:35.529 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12187_fsm 2023-09-20T22:04:35.529 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187_vm 2023-09-20T22:04:35.530 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12189 2023-09-20T22:04:35.530 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12191 2023-09-20T22:04:35.530 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192 2023-09-20T22:04:35.530 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12192_fsm 2023-09-20T22:04:35.530 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192_vm 2023-09-20T22:04:35.530 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12194 2023-09-20T22:04:35.531 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12196 2023-09-20T22:04:35.531 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197 2023-09-20T22:04:35.531 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12197_fsm 2023-09-20T22:04:35.531 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197_vm 2023-09-20T22:04:35.531 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12199 2023-09-20T22:04:35.531 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12201 2023-09-20T22:04:35.532 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202 2023-09-20T22:04:35.532 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12202_fsm 2023-09-20T22:04:35.532 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202_vm 2023-09-20T22:04:35.532 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12204 2023-09-20T22:04:35.532 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12206 2023-09-20T22:04:35.532 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207 2023-09-20T22:04:35.533 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12207_fsm 2023-09-20T22:04:35.533 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207_vm 2023-09-20T22:04:35.533 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12209 2023-09-20T22:04:35.533 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12211 2023-09-20T22:04:35.533 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12212 2023-09-20T22:04:35.533 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12214 2023-09-20T22:04:35.534 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12216 2023-09-20T22:04:35.534 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 20 21:49 1247 2023-09-20T22:04:35.534 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1247_fsm 2023-09-20T22:04:35.534 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1247_vm 2023-09-20T22:04:35.534 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 20 21:49 1249 2023-09-20T22:04:35.534 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1249_fsm 2023-09-20T22:04:35.535 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1249_vm 2023-09-20T22:04:35.535 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 20 21:49 1255 2023-09-20T22:04:35.535 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1255_fsm 2023-09-20T22:04:35.535 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1255_vm 2023-09-20T22:04:35.535 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 20 21:49 1259 2023-09-20T22:04:35.536 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1259_fsm 2023-09-20T22:04:35.536 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1259_vm 2023-09-20T22:04:35.536 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417 2023-09-20T22:04:35.536 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417_vm 2023-09-20T22:04:35.536 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418 2023-09-20T22:04:35.536 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418_vm 2023-09-20T22:04:35.537 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 174 2023-09-20T22:04:35.537 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 175 2023-09-20T22:04:35.537 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2187 2023-09-20T22:04:35.537 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224 2023-09-20T22:04:35.537 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224_vm 2023-09-20T22:04:35.537 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328 2023-09-20T22:04:35.538 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328_vm 2023-09-20T22:04:35.538 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336 2023-09-20T22:04:35.538 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336_vm 2023-09-20T22:04:35.538 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2337 2023-09-20T22:04:35.538 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2600 2023-09-20T22:04:35.539 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2600_fsm 2023-09-20T22:04:35.539 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2600_vm 2023-09-20T22:04:35.539 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601 2023-09-20T22:04:35.539 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2601_fsm 2023-09-20T22:04:35.539 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601_vm 2023-09-20T22:04:35.540 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2602 2023-09-20T22:04:35.540 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2602_fsm 2023-09-20T22:04:35.540 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2602_vm 2023-09-20T22:04:35.540 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2603 2023-09-20T22:04:35.540 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2603_fsm 2023-09-20T22:04:35.540 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2603_vm 2023-09-20T22:04:35.541 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604 2023-09-20T22:04:35.541 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604_vm 2023-09-20T22:04:35.541 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2605 2023-09-20T22:04:35.541 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2605_fsm 2023-09-20T22:04:35.541 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2605_vm 2023-09-20T22:04:35.542 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606 2023-09-20T22:04:35.542 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2606_fsm 2023-09-20T22:04:35.542 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606_vm 2023-09-20T22:04:35.542 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607 2023-09-20T22:04:35.542 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607_fsm 2023-09-20T22:04:35.543 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2607_vm 2023-09-20T22:04:35.543 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Sep 20 21:49 2608 2023-09-20T22:04:35.543 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2608_fsm 2023-09-20T22:04:35.543 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2608_vm 2023-09-20T22:04:35.543 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 20 21:49 2609 2023-09-20T22:04:35.544 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2609_fsm 2023-09-20T22:04:35.544 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2609_vm 2023-09-20T22:04:35.544 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610 2023-09-20T22:04:35.544 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610_fsm 2023-09-20T22:04:35.544 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2610_vm 2023-09-20T22:04:35.544 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611 2023-09-20T22:04:35.545 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611_vm 2023-09-20T22:04:35.545 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612 2023-09-20T22:04:35.545 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2612_fsm 2023-09-20T22:04:35.545 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612_vm 2023-09-20T22:04:35.545 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613 2023-09-20T22:04:35.546 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613_vm 2023-09-20T22:04:35.546 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615 2023-09-20T22:04:35.546 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2615_fsm 2023-09-20T22:04:35.546 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615_vm 2023-09-20T22:04:35.546 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616 2023-09-20T22:04:35.546 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616_fsm 2023-09-20T22:04:35.547 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2616_vm 2023-09-20T22:04:35.547 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 20 21:49 2617 2023-09-20T22:04:35.547 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2617_fsm 2023-09-20T22:04:35.547 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2617_vm 2023-09-20T22:04:35.547 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 20 21:49 2618 2023-09-20T22:04:35.547 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2618_fsm 2023-09-20T22:04:35.548 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2618_vm 2023-09-20T22:04:35.548 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 20 21:49 2619 2023-09-20T22:04:35.548 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2619_fsm 2023-09-20T22:04:35.548 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2619_vm 2023-09-20T22:04:35.548 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620 2023-09-20T22:04:35.549 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620_vm 2023-09-20T22:04:35.549 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2650 2023-09-20T22:04:35.549 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2651 2023-09-20T22:04:35.549 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2652 2023-09-20T22:04:35.549 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2653 2023-09-20T22:04:35.549 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2654 2023-09-20T22:04:35.550 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2655 2023-09-20T22:04:35.550 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2656 2023-09-20T22:04:35.550 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2657 2023-09-20T22:04:35.550 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 20 21:49 2658 2023-09-20T22:04:35.550 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2659 2023-09-20T22:04:35.551 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2660 2023-09-20T22:04:35.551 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2661 2023-09-20T22:04:35.551 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2662 2023-09-20T22:04:35.551 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2663 2023-09-20T22:04:35.551 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2664 2023-09-20T22:04:35.551 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2665 2023-09-20T22:04:35.552 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2666 2023-09-20T22:04:35.552 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2667 2023-09-20T22:04:35.552 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2668 2023-09-20T22:04:35.552 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2669 2023-09-20T22:04:35.552 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2670 2023-09-20T22:04:35.553 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 20 21:49 2673 2023-09-20T22:04:35.553 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Sep 20 21:49 2674 2023-09-20T22:04:35.553 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 20 21:49 2675 2023-09-20T22:04:35.553 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2678 2023-09-20T22:04:35.553 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2679 2023-09-20T22:04:35.553 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2680 2023-09-20T22:04:35.554 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2681 2023-09-20T22:04:35.554 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2682 2023-09-20T22:04:35.554 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2683 2023-09-20T22:04:35.554 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2684 2023-09-20T22:04:35.554 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2685 2023-09-20T22:04:35.555 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2686 2023-09-20T22:04:35.555 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2687 2023-09-20T22:04:35.555 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2688 2023-09-20T22:04:35.555 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2689 2023-09-20T22:04:35.555 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2690 2023-09-20T22:04:35.556 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 20 21:49 2691 2023-09-20T22:04:35.556 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2692 2023-09-20T22:04:35.556 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2693 2023-09-20T22:04:35.556 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2696 2023-09-20T22:04:35.556 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2699 2023-09-20T22:04:35.556 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2701 2023-09-20T22:04:35.557 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2702 2023-09-20T22:04:35.557 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2703 2023-09-20T22:04:35.557 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2704 2023-09-20T22:04:35.557 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2753 2023-09-20T22:04:35.557 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2753_fsm 2023-09-20T22:04:35.558 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2753_vm 2023-09-20T22:04:35.558 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2754 2023-09-20T22:04:35.558 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2755 2023-09-20T22:04:35.558 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2756 2023-09-20T22:04:35.558 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2757 2023-09-20T22:04:35.558 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830 2023-09-20T22:04:35.559 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830_vm 2023-09-20T22:04:35.559 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2831 2023-09-20T22:04:35.559 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832 2023-09-20T22:04:35.559 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832_vm 2023-09-20T22:04:35.559 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2833 2023-09-20T22:04:35.560 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834 2023-09-20T22:04:35.560 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834_vm 2023-09-20T22:04:35.560 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2835 2023-09-20T22:04:35.560 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836 2023-09-20T22:04:35.560 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836_vm 2023-09-20T22:04:35.560 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2837 2023-09-20T22:04:35.561 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 20 21:49 2838 2023-09-20T22:04:35.561 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2838_fsm 2023-09-20T22:04:35.561 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2838_vm 2023-09-20T22:04:35.561 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2839 2023-09-20T22:04:35.561 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840 2023-09-20T22:04:35.562 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840_fsm 2023-09-20T22:04:35.562 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2840_vm 2023-09-20T22:04:35.562 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2841 2023-09-20T22:04:35.562 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995 2023-09-20T22:04:35.562 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995_vm 2023-09-20T22:04:35.563 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2996 2023-09-20T22:04:35.563 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079 2023-09-20T22:04:35.563 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3079_fsm 2023-09-20T22:04:35.563 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079_vm 2023-09-20T22:04:35.563 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3080 2023-09-20T22:04:35.563 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3081 2023-09-20T22:04:35.564 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3085 2023-09-20T22:04:35.564 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118 2023-09-20T22:04:35.564 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118_vm 2023-09-20T22:04:35.564 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3119 2023-09-20T22:04:35.564 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3164 2023-09-20T22:04:35.565 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256 2023-09-20T22:04:35.565 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256_vm 2023-09-20T22:04:35.565 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3257 2023-09-20T22:04:35.565 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3258 2023-09-20T22:04:35.565 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350 2023-09-20T22:04:35.565 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350_vm 2023-09-20T22:04:35.566 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3351 2023-09-20T22:04:35.566 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3379 2023-09-20T22:04:35.566 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3380 2023-09-20T22:04:35.566 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381 2023-09-20T22:04:35.566 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381_vm 2023-09-20T22:04:35.567 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3394 2023-09-20T22:04:35.567 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3394_fsm 2023-09-20T22:04:35.567 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3394_vm 2023-09-20T22:04:35.567 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3395 2023-09-20T22:04:35.567 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439 2023-09-20T22:04:35.568 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439_vm 2023-09-20T22:04:35.568 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3440 2023-09-20T22:04:35.568 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 3455 2023-09-20T22:04:35.568 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3456 2023-09-20T22:04:35.568 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3456_fsm 2023-09-20T22:04:35.568 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3456_vm 2023-09-20T22:04:35.569 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466 2023-09-20T22:04:35.569 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466_vm 2023-09-20T22:04:35.569 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3467 2023-09-20T22:04:35.569 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3468 2023-09-20T22:04:35.569 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501 2023-09-20T22:04:35.569 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501_vm 2023-09-20T22:04:35.570 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3502 2023-09-20T22:04:35.570 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3503 2023-09-20T22:04:35.570 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3534 2023-09-20T22:04:35.570 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541 2023-09-20T22:04:35.570 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3541_fsm 2023-09-20T22:04:35.571 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541_vm 2023-09-20T22:04:35.571 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3542 2023-09-20T22:04:35.571 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3574 2023-09-20T22:04:35.571 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3575 2023-09-20T22:04:35.571 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576 2023-09-20T22:04:35.571 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576_vm 2023-09-20T22:04:35.572 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596 2023-09-20T22:04:35.572 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596_vm 2023-09-20T22:04:35.572 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3597 2023-09-20T22:04:35.572 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598 2023-09-20T22:04:35.572 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598_vm 2023-09-20T22:04:35.573 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3599 2023-09-20T22:04:35.573 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600 2023-09-20T22:04:35.573 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3600_fsm 2023-09-20T22:04:35.573 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600_vm 2023-09-20T22:04:35.573 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601 2023-09-20T22:04:35.573 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3601_fsm 2023-09-20T22:04:35.574 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601_vm 2023-09-20T22:04:35.574 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602 2023-09-20T22:04:35.574 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3602_fsm 2023-09-20T22:04:35.574 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602_vm 2023-09-20T22:04:35.574 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3603 2023-09-20T22:04:35.575 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3603_fsm 2023-09-20T22:04:35.575 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3603_vm 2023-09-20T22:04:35.575 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3604 2023-09-20T22:04:35.575 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3605 2023-09-20T22:04:35.575 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3606 2023-09-20T22:04:35.575 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3607 2023-09-20T22:04:35.576 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3608 2023-09-20T22:04:35.578 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 3609 2023-09-20T22:04:35.578 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3712 2023-09-20T22:04:35.579 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764 2023-09-20T22:04:35.579 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3764_fsm 2023-09-20T22:04:35.579 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764_vm 2023-09-20T22:04:35.579 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3766 2023-09-20T22:04:35.579 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3767 2023-09-20T22:04:35.579 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3997 2023-09-20T22:04:35.580 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 5002 2023-09-20T22:04:35.580 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 548 2023-09-20T22:04:35.580 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 549 2023-09-20T22:04:35.580 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102 2023-09-20T22:04:35.580 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102_vm 2023-09-20T22:04:35.580 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104 2023-09-20T22:04:35.581 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104_vm 2023-09-20T22:04:35.581 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106 2023-09-20T22:04:35.581 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106_vm 2023-09-20T22:04:35.581 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6110 2023-09-20T22:04:35.581 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6111 2023-09-20T22:04:35.582 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6112 2023-09-20T22:04:35.582 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6113 2023-09-20T22:04:35.582 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6117 2023-09-20T22:04:35.582 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826 2023-09-20T22:04:35.582 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826_vm 2023-09-20T22:04:35.583 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 827 2023-09-20T22:04:35.583 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 828 2023-09-20T22:04:35.583 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 20 21:49 pg_filenode.map 2023-09-20T22:04:35.583 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 PG_VERSION 2023-09-20T22:04:35.583 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.583 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-09-20T22:04:35.584 INFO:teuthology.orchestra.run.smithi022.stdout:total 546712 2023-09-20T22:04:35.584 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Sep 20 21:49 . 2023-09-20T22:04:35.584 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 20 21:49 .. 2023-09-20T22:04:35.584 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 112 2023-09-20T22:04:35.584 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 113 2023-09-20T22:04:35.584 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 20 21:49 12182 2023-09-20T22:04:35.585 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12182_fsm 2023-09-20T22:04:35.585 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12182_vm 2023-09-20T22:04:35.585 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12184 2023-09-20T22:04:35.585 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12186 2023-09-20T22:04:35.585 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187 2023-09-20T22:04:35.586 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12187_fsm 2023-09-20T22:04:35.586 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12187_vm 2023-09-20T22:04:35.586 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12189 2023-09-20T22:04:35.586 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12191 2023-09-20T22:04:35.586 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192 2023-09-20T22:04:35.587 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12192_fsm 2023-09-20T22:04:35.587 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12192_vm 2023-09-20T22:04:35.587 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12194 2023-09-20T22:04:35.587 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12196 2023-09-20T22:04:35.587 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197 2023-09-20T22:04:35.587 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12197_fsm 2023-09-20T22:04:35.588 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12197_vm 2023-09-20T22:04:35.588 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12199 2023-09-20T22:04:35.588 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12201 2023-09-20T22:04:35.588 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202 2023-09-20T22:04:35.588 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12202_fsm 2023-09-20T22:04:35.588 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12202_vm 2023-09-20T22:04:35.589 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12204 2023-09-20T22:04:35.589 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12206 2023-09-20T22:04:35.589 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207 2023-09-20T22:04:35.589 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 12207_fsm 2023-09-20T22:04:35.589 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12207_vm 2023-09-20T22:04:35.589 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12209 2023-09-20T22:04:35.590 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12211 2023-09-20T22:04:35.590 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12212 2023-09-20T22:04:35.590 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 12214 2023-09-20T22:04:35.590 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 12216 2023-09-20T22:04:35.590 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Sep 20 21:49 1247 2023-09-20T22:04:35.590 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1247_fsm 2023-09-20T22:04:35.591 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1247_vm 2023-09-20T22:04:35.591 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Sep 20 21:49 1249 2023-09-20T22:04:35.591 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1249_fsm 2023-09-20T22:04:35.591 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1249_vm 2023-09-20T22:04:35.591 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Sep 20 21:49 1255 2023-09-20T22:04:35.591 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1255_fsm 2023-09-20T22:04:35.592 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1255_vm 2023-09-20T22:04:35.592 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 20 22:04 1259 2023-09-20T22:04:35.592 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1259_fsm 2023-09-20T22:04:35.592 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1259_vm 2023-09-20T22:04:35.592 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417 2023-09-20T22:04:35.592 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1417_vm 2023-09-20T22:04:35.593 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418 2023-09-20T22:04:35.593 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 1418_vm 2023-09-20T22:04:35.593 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2392064 Sep 20 22:04 16387 2023-09-20T22:04:35.593 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 22:04 16387_fsm 2023-09-20T22:04:35.593 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 22:03 16387_vm 2023-09-20T22:04:35.593 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1236992 Sep 20 22:04 16393 2023-09-20T22:04:35.594 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 22:04 16393_fsm 2023-09-20T22:04:35.594 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 22:04 16393_vm 2023-09-20T22:04:35.594 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 441016320 Sep 20 22:04 16396 2023-09-20T22:04:35.594 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 20 22:02 16396_fsm 2023-09-20T22:04:35.594 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 22:01 16396_vm 2023-09-20T22:04:35.595 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 22:04 16397 2023-09-20T22:04:35.595 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Sep 20 22:04 16399 2023-09-20T22:04:35.595 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 20 22:04 16401 2023-09-20T22:04:35.595 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 35209216 Sep 20 22:04 16403 2023-09-20T22:04:35.595 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 22:04 16403_fsm 2023-09-20T22:04:35.595 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 174 2023-09-20T22:04:35.596 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 175 2023-09-20T22:04:35.596 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2187 2023-09-20T22:04:35.596 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224 2023-09-20T22:04:35.596 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2224_vm 2023-09-20T22:04:35.596 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328 2023-09-20T22:04:35.597 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2328_vm 2023-09-20T22:04:35.597 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336 2023-09-20T22:04:35.597 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2336_vm 2023-09-20T22:04:35.597 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2337 2023-09-20T22:04:35.597 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2600 2023-09-20T22:04:35.597 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2600_fsm 2023-09-20T22:04:35.598 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2600_vm 2023-09-20T22:04:35.598 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601 2023-09-20T22:04:35.598 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2601_fsm 2023-09-20T22:04:35.598 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2601_vm 2023-09-20T22:04:35.598 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2602 2023-09-20T22:04:35.599 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2602_fsm 2023-09-20T22:04:35.599 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2602_vm 2023-09-20T22:04:35.599 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2603 2023-09-20T22:04:35.599 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2603_fsm 2023-09-20T22:04:35.599 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2603_vm 2023-09-20T22:04:35.599 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604 2023-09-20T22:04:35.600 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2604_vm 2023-09-20T22:04:35.600 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2605 2023-09-20T22:04:35.600 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2605_fsm 2023-09-20T22:04:35.600 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2605_vm 2023-09-20T22:04:35.600 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606 2023-09-20T22:04:35.600 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2606_fsm 2023-09-20T22:04:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2606_vm 2023-09-20T22:04:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607 2023-09-20T22:04:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2607_fsm 2023-09-20T22:04:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2607_vm 2023-09-20T22:04:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Sep 20 21:49 2608 2023-09-20T22:04:35.601 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2608_fsm 2023-09-20T22:04:35.602 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2608_vm 2023-09-20T22:04:35.602 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Sep 20 21:49 2609 2023-09-20T22:04:35.602 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2609_fsm 2023-09-20T22:04:35.602 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2609_vm 2023-09-20T22:04:35.602 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2610 2023-09-20T22:04:35.603 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2610_fsm 2023-09-20T22:04:35.603 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2610_vm 2023-09-20T22:04:35.603 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611 2023-09-20T22:04:35.603 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2611_vm 2023-09-20T22:04:35.603 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612 2023-09-20T22:04:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2612_fsm 2023-09-20T22:04:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2612_vm 2023-09-20T22:04:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613 2023-09-20T22:04:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2613_vm 2023-09-20T22:04:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615 2023-09-20T22:04:35.604 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2615_fsm 2023-09-20T22:04:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2615_vm 2023-09-20T22:04:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616 2023-09-20T22:04:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2616_fsm 2023-09-20T22:04:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2616_vm 2023-09-20T22:04:35.605 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 20 21:49 2617 2023-09-20T22:04:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2617_fsm 2023-09-20T22:04:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2617_vm 2023-09-20T22:04:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 20 21:49 2618 2023-09-20T22:04:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2618_fsm 2023-09-20T22:04:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2618_vm 2023-09-20T22:04:35.606 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 20 22:04 2619 2023-09-20T22:04:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:59 2619_fsm 2023-09-20T22:04:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2619_vm 2023-09-20T22:04:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620 2023-09-20T22:04:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2620_vm 2023-09-20T22:04:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2650 2023-09-20T22:04:35.607 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2651 2023-09-20T22:04:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2652 2023-09-20T22:04:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2653 2023-09-20T22:04:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2654 2023-09-20T22:04:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2655 2023-09-20T22:04:35.608 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2656 2023-09-20T22:04:35.609 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2657 2023-09-20T22:04:35.609 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 20 21:49 2658 2023-09-20T22:04:35.609 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2659 2023-09-20T22:04:35.609 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2660 2023-09-20T22:04:35.609 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2661 2023-09-20T22:04:35.609 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2662 2023-09-20T22:04:35.610 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2663 2023-09-20T22:04:35.610 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2664 2023-09-20T22:04:35.610 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2665 2023-09-20T22:04:35.610 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2666 2023-09-20T22:04:35.610 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2667 2023-09-20T22:04:35.611 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2668 2023-09-20T22:04:35.611 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2669 2023-09-20T22:04:35.611 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2670 2023-09-20T22:04:35.611 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 20 21:49 2673 2023-09-20T22:04:35.611 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Sep 20 21:49 2674 2023-09-20T22:04:35.611 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 20 21:49 2675 2023-09-20T22:04:35.611 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2678 2023-09-20T22:04:35.612 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2679 2023-09-20T22:04:35.612 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2680 2023-09-20T22:04:35.612 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2681 2023-09-20T22:04:35.612 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2682 2023-09-20T22:04:35.612 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2683 2023-09-20T22:04:35.613 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2684 2023-09-20T22:04:35.613 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2685 2023-09-20T22:04:35.613 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2686 2023-09-20T22:04:35.613 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2687 2023-09-20T22:04:35.613 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2688 2023-09-20T22:04:35.613 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 20 21:49 2689 2023-09-20T22:04:35.614 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 20 21:49 2690 2023-09-20T22:04:35.614 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Sep 20 21:49 2691 2023-09-20T22:04:35.614 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2692 2023-09-20T22:04:35.614 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2693 2023-09-20T22:04:35.614 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:59 2696 2023-09-20T22:04:35.615 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2699 2023-09-20T22:04:35.615 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2701 2023-09-20T22:04:35.615 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2702 2023-09-20T22:04:35.615 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2703 2023-09-20T22:04:35.615 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 2704 2023-09-20T22:04:35.615 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2753 2023-09-20T22:04:35.616 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2753_fsm 2023-09-20T22:04:35.616 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2753_vm 2023-09-20T22:04:35.616 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2754 2023-09-20T22:04:35.616 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2755 2023-09-20T22:04:35.616 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2756 2023-09-20T22:04:35.617 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 2757 2023-09-20T22:04:35.617 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830 2023-09-20T22:04:35.617 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2830_vm 2023-09-20T22:04:35.617 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2831 2023-09-20T22:04:35.617 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832 2023-09-20T22:04:35.617 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2832_vm 2023-09-20T22:04:35.618 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2833 2023-09-20T22:04:35.618 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834 2023-09-20T22:04:35.618 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2834_vm 2023-09-20T22:04:35.618 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2835 2023-09-20T22:04:35.618 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836 2023-09-20T22:04:35.619 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2836_vm 2023-09-20T22:04:35.619 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2837 2023-09-20T22:04:35.619 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Sep 20 21:49 2838 2023-09-20T22:04:35.619 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2838_fsm 2023-09-20T22:04:35.619 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2838_vm 2023-09-20T22:04:35.619 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2839 2023-09-20T22:04:35.620 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840 2023-09-20T22:04:35.620 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2840_fsm 2023-09-20T22:04:35.620 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2840_vm 2023-09-20T22:04:35.620 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2841 2023-09-20T22:04:35.620 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995 2023-09-20T22:04:35.620 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2995_vm 2023-09-20T22:04:35.621 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2996 2023-09-20T22:04:35.621 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079 2023-09-20T22:04:35.621 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3079_fsm 2023-09-20T22:04:35.621 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3079_vm 2023-09-20T22:04:35.621 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3080 2023-09-20T22:04:35.621 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3081 2023-09-20T22:04:35.622 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3085 2023-09-20T22:04:35.622 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118 2023-09-20T22:04:35.622 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3118_vm 2023-09-20T22:04:35.622 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3119 2023-09-20T22:04:35.622 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3164 2023-09-20T22:04:35.623 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256 2023-09-20T22:04:35.623 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3256_vm 2023-09-20T22:04:35.623 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3257 2023-09-20T22:04:35.623 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3258 2023-09-20T22:04:35.623 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350 2023-09-20T22:04:35.623 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3350_vm 2023-09-20T22:04:35.624 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3351 2023-09-20T22:04:35.624 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3379 2023-09-20T22:04:35.624 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3380 2023-09-20T22:04:35.624 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381 2023-09-20T22:04:35.624 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3381_vm 2023-09-20T22:04:35.625 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3394 2023-09-20T22:04:35.625 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3394_fsm 2023-09-20T22:04:35.625 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3394_vm 2023-09-20T22:04:35.625 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3395 2023-09-20T22:04:35.625 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439 2023-09-20T22:04:35.625 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3439_vm 2023-09-20T22:04:35.626 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3440 2023-09-20T22:04:35.626 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 20 21:49 3455 2023-09-20T22:04:35.626 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3456 2023-09-20T22:04:35.626 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3456_fsm 2023-09-20T22:04:35.626 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3456_vm 2023-09-20T22:04:35.627 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466 2023-09-20T22:04:35.627 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3466_vm 2023-09-20T22:04:35.627 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3467 2023-09-20T22:04:35.627 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3468 2023-09-20T22:04:35.627 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501 2023-09-20T22:04:35.627 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3501_vm 2023-09-20T22:04:35.628 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3502 2023-09-20T22:04:35.628 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3503 2023-09-20T22:04:35.628 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3534 2023-09-20T22:04:35.628 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541 2023-09-20T22:04:35.628 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3541_fsm 2023-09-20T22:04:35.628 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3541_vm 2023-09-20T22:04:35.629 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3542 2023-09-20T22:04:35.629 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3574 2023-09-20T22:04:35.629 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3575 2023-09-20T22:04:35.629 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576 2023-09-20T22:04:35.629 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3576_vm 2023-09-20T22:04:35.629 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596 2023-09-20T22:04:35.630 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3596_vm 2023-09-20T22:04:35.630 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3597 2023-09-20T22:04:35.630 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598 2023-09-20T22:04:35.630 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3598_vm 2023-09-20T22:04:35.630 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3599 2023-09-20T22:04:35.631 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600 2023-09-20T22:04:35.631 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3600_fsm 2023-09-20T22:04:35.631 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3600_vm 2023-09-20T22:04:35.631 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601 2023-09-20T22:04:35.631 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3601_fsm 2023-09-20T22:04:35.632 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3601_vm 2023-09-20T22:04:35.632 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602 2023-09-20T22:04:35.632 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3602_fsm 2023-09-20T22:04:35.632 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3602_vm 2023-09-20T22:04:35.632 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3603 2023-09-20T22:04:35.632 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3603_fsm 2023-09-20T22:04:35.633 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3603_vm 2023-09-20T22:04:35.633 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3604 2023-09-20T22:04:35.633 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3605 2023-09-20T22:04:35.633 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3606 2023-09-20T22:04:35.633 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3607 2023-09-20T22:04:35.634 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3608 2023-09-20T22:04:35.634 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 20 21:49 3609 2023-09-20T22:04:35.634 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3712 2023-09-20T22:04:35.634 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764 2023-09-20T22:04:35.634 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 3764_fsm 2023-09-20T22:04:35.635 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3764_vm 2023-09-20T22:04:35.635 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3766 2023-09-20T22:04:35.635 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 3767 2023-09-20T22:04:35.635 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3997 2023-09-20T22:04:35.635 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 5002 2023-09-20T22:04:35.635 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 548 2023-09-20T22:04:35.636 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 549 2023-09-20T22:04:35.636 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102 2023-09-20T22:04:35.636 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6102_vm 2023-09-20T22:04:35.636 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104 2023-09-20T22:04:35.636 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6104_vm 2023-09-20T22:04:35.637 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106 2023-09-20T22:04:35.637 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6106_vm 2023-09-20T22:04:35.637 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6110 2023-09-20T22:04:35.637 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6111 2023-09-20T22:04:35.637 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6112 2023-09-20T22:04:35.637 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6113 2023-09-20T22:04:35.638 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6117 2023-09-20T22:04:35.638 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826 2023-09-20T22:04:35.638 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 826_vm 2023-09-20T22:04:35.638 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 827 2023-09-20T22:04:35.638 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 828 2023-09-20T22:04:35.638 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 20 21:49 pg_filenode.map 2023-09-20T22:04:35.639 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Sep 20 21:49 pg_internal.init 2023-09-20T22:04:35.639 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 21:49 PG_VERSION 2023-09-20T22:04:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.639 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-09-20T22:04:35.639 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.639 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.640 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 20 21:49 .. 2023-09-20T22:04:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.640 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/global: 2023-09-20T22:04:35.640 INFO:teuthology.orchestra.run.smithi022.stdout:total 574 2023-09-20T22:04:35.640 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Sep 20 21:49 . 2023-09-20T22:04:35.641 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.641 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1136 2023-09-20T22:04:35.641 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1136_fsm 2023-09-20T22:04:35.641 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1136_vm 2023-09-20T22:04:35.641 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 1137 2023-09-20T22:04:35.641 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1213 2023-09-20T22:04:35.642 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1213_fsm 2023-09-20T22:04:35.642 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1213_vm 2023-09-20T22:04:35.642 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1214 2023-09-20T22:04:35.642 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1214_fsm 2023-09-20T22:04:35.642 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1214_vm 2023-09-20T22:04:35.642 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 1232 2023-09-20T22:04:35.643 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 1233 2023-09-20T22:04:35.643 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1260 2023-09-20T22:04:35.643 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1260_fsm 2023-09-20T22:04:35.643 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1260_vm 2023-09-20T22:04:35.643 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1261 2023-09-20T22:04:35.644 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1261_fsm 2023-09-20T22:04:35.644 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1261_vm 2023-09-20T22:04:35.644 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1262 2023-09-20T22:04:35.644 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 1262_fsm 2023-09-20T22:04:35.644 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 1262_vm 2023-09-20T22:04:35.644 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2396 2023-09-20T22:04:35.645 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 20 21:49 2396_fsm 2023-09-20T22:04:35.645 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2396_vm 2023-09-20T22:04:35.645 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2397 2023-09-20T22:04:35.645 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2671 2023-09-20T22:04:35.645 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2672 2023-09-20T22:04:35.646 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2676 2023-09-20T22:04:35.646 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2677 2023-09-20T22:04:35.646 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2694 2023-09-20T22:04:35.646 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2695 2023-09-20T22:04:35.646 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2697 2023-09-20T22:04:35.647 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 20 21:49 2698 2023-09-20T22:04:35.647 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2846 2023-09-20T22:04:35.647 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2846_vm 2023-09-20T22:04:35.647 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2847 2023-09-20T22:04:35.647 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2964 2023-09-20T22:04:35.647 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2964_vm 2023-09-20T22:04:35.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2965 2023-09-20T22:04:35.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2966 2023-09-20T22:04:35.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 2966_vm 2023-09-20T22:04:35.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 2967 2023-09-20T22:04:35.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3592 2023-09-20T22:04:35.648 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 3592_vm 2023-09-20T22:04:35.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 3593 2023-09-20T22:04:35.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 4060 2023-09-20T22:04:35.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 4060_vm 2023-09-20T22:04:35.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 4061 2023-09-20T22:04:35.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6000 2023-09-20T22:04:35.649 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6000_vm 2023-09-20T22:04:35.650 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6001 2023-09-20T22:04:35.650 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6002 2023-09-20T22:04:35.650 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6100 2023-09-20T22:04:35.650 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 6100_vm 2023-09-20T22:04:35.650 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6114 2023-09-20T22:04:35.651 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 6115 2023-09-20T22:04:35.651 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 22:04 pg_control 2023-09-20T22:04:35.651 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 20 21:49 pg_filenode.map 2023-09-20T22:04:35.651 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Sep 20 21:49 pg_internal.init 2023-09-20T22:04:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.651 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/log: 2023-09-20T22:04:35.652 INFO:teuthology.orchestra.run.smithi022.stdout:total 1 2023-09-20T22:04:35.652 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 . 2023-09-20T22:04:35.652 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.652 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 805 Sep 20 22:04 postgresql-Wed.log 2023-09-20T22:04:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.653 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-09-20T22:04:35.653 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.653 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.653 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.653 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-09-20T22:04:35.654 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.654 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.654 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.654 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-09-20T22:04:35.655 INFO:teuthology.orchestra.run.smithi022.stdout:total 1 2023-09-20T22:04:35.655 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 22:04 . 2023-09-20T22:04:35.655 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.655 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 mappings 2023-09-20T22:04:35.655 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 20 22:04 replorigin_checkpoint 2023-09-20T22:04:35.656 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 snapshots 2023-09-20T22:04:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.656 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-09-20T22:04:35.656 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.656 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.657 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 22:04 .. 2023-09-20T22:04:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.657 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-09-20T22:04:35.657 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.657 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.658 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 22:04 .. 2023-09-20T22:04:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.658 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-09-20T22:04:35.658 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.658 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 . 2023-09-20T22:04:35.658 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.659 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 members 2023-09-20T22:04:35.659 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 offsets 2023-09-20T22:04:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.659 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-09-20T22:04:35.659 INFO:teuthology.orchestra.run.smithi022.stdout:total 8 2023-09-20T22:04:35.659 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 . 2023-09-20T22:04:35.660 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 .. 2023-09-20T22:04:35.660 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 0000 2023-09-20T22:04:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.660 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-09-20T22:04:35.660 INFO:teuthology.orchestra.run.smithi022.stdout:total 8 2023-09-20T22:04:35.661 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 . 2023-09-20T22:04:35.661 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 20 21:49 .. 2023-09-20T22:04:35.661 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 0000 2023-09-20T22:04:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.661 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-09-20T22:04:35.661 INFO:teuthology.orchestra.run.smithi022.stdout:total 8 2023-09-20T22:04:35.662 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 . 2023-09-20T22:04:35.662 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.662 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 20 21:49 0000 2023-09-20T22:04:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.662 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-09-20T22:04:35.663 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.663 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.663 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.663 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-09-20T22:04:35.663 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.664 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.664 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.664 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-09-20T22:04:35.664 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.665 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.665 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.665 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-09-20T22:04:35.665 INFO:teuthology.orchestra.run.smithi022.stdout:total 13 2023-09-20T22:04:35.665 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 20 22:04 . 2023-09-20T22:04:35.666 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.666 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Sep 20 22:04 db_0.stat 2023-09-20T22:04:35.666 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Sep 20 22:04 db_12347.stat 2023-09-20T22:04:35.666 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Sep 20 22:04 global.stat 2023-09-20T22:04:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.667 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-09-20T22:04:35.667 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.667 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 22:04 . 2023-09-20T22:04:35.667 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.667 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-09-20T22:04:35.668 INFO:teuthology.orchestra.run.smithi022.stdout:total 232 2023-09-20T22:04:35.668 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 22:04 . 2023-09-20T22:04:35.668 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.668 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 237568 Sep 20 22:04 0009 2023-09-20T22:04:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.668 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-09-20T22:04:35.669 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.669 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.669 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.669 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-09-20T22:04:35.669 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.670 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.670 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.670 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-09-20T22:04:35.670 INFO:teuthology.orchestra.run.smithi022.stdout:total 1032192 2023-09-20T22:04:35.670 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Sep 20 22:04 . 2023-09-20T22:04:35.671 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.671 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000C9 2023-09-20T22:04:35.671 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000CA 2023-09-20T22:04:35.671 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000CB 2023-09-20T22:04:35.671 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000CC 2023-09-20T22:04:35.671 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000CD 2023-09-20T22:04:35.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000CE 2023-09-20T22:04:35.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000CF 2023-09-20T22:04:35.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000D0 2023-09-20T22:04:35.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000D1 2023-09-20T22:04:35.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000D2 2023-09-20T22:04:35.672 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:04 0000000100000000000000D3 2023-09-20T22:04:35.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:04 0000000100000000000000D4 2023-09-20T22:04:35.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:04 0000000100000000000000D5 2023-09-20T22:04:35.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:04 0000000100000000000000D6 2023-09-20T22:04:35.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:57 0000000100000000000000D7 2023-09-20T22:04:35.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:57 0000000100000000000000D8 2023-09-20T22:04:35.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:57 0000000100000000000000D9 2023-09-20T22:04:35.673 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:56 0000000100000000000000DA 2023-09-20T22:04:35.674 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:57 0000000100000000000000DB 2023-09-20T22:04:35.674 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:57 0000000100000000000000DC 2023-09-20T22:04:35.674 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:57 0000000100000000000000DD 2023-09-20T22:04:35.674 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:57 0000000100000000000000DE 2023-09-20T22:04:35.674 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:59 0000000100000000000000DF 2023-09-20T22:04:35.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:56 0000000100000000000000E0 2023-09-20T22:04:35.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:59 0000000100000000000000E1 2023-09-20T22:04:35.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:58 0000000100000000000000E2 2023-09-20T22:04:35.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:56 0000000100000000000000E3 2023-09-20T22:04:35.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:57 0000000100000000000000E4 2023-09-20T22:04:35.675 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:59 0000000100000000000000E5 2023-09-20T22:04:35.676 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:57 0000000100000000000000E6 2023-09-20T22:04:35.676 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:59 0000000100000000000000E7 2023-09-20T22:04:35.676 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:58 0000000100000000000000E8 2023-09-20T22:04:35.676 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:56 0000000100000000000000E9 2023-09-20T22:04:35.676 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:56 0000000100000000000000EA 2023-09-20T22:04:35.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:58 0000000100000000000000EB 2023-09-20T22:04:35.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:57 0000000100000000000000EC 2023-09-20T22:04:35.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:58 0000000100000000000000ED 2023-09-20T22:04:35.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:58 0000000100000000000000EE 2023-09-20T22:04:35.677 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:02 0000000100000000000000EF 2023-09-20T22:04:35.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:59 0000000100000000000000F0 2023-09-20T22:04:35.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:01 0000000100000000000000F1 2023-09-20T22:04:35.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 0000000100000000000000F2 2023-09-20T22:04:35.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 0000000100000000000000F3 2023-09-20T22:04:35.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:01 0000000100000000000000F4 2023-09-20T22:04:35.678 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 0000000100000000000000F5 2023-09-20T22:04:35.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:02 0000000100000000000000F6 2023-09-20T22:04:35.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 21:59 0000000100000000000000F7 2023-09-20T22:04:35.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 0000000100000000000000F8 2023-09-20T22:04:35.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:03 0000000100000000000000F9 2023-09-20T22:04:35.679 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 0000000100000000000000FA 2023-09-20T22:04:35.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 0000000100000000000000FB 2023-09-20T22:04:35.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:01 0000000100000000000000FC 2023-09-20T22:04:35.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 0000000100000000000000FD 2023-09-20T22:04:35.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:02 0000000100000000000000FE 2023-09-20T22:04:35.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 0000000100000000000000FF 2023-09-20T22:04:35.680 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:02 000000010000000100000000 2023-09-20T22:04:35.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 000000010000000100000001 2023-09-20T22:04:35.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 000000010000000100000002 2023-09-20T22:04:35.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:01 000000010000000100000003 2023-09-20T22:04:35.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:01 000000010000000100000004 2023-09-20T22:04:35.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:01 000000010000000100000005 2023-09-20T22:04:35.681 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:01 000000010000000100000006 2023-09-20T22:04:35.682 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 20 22:00 000000010000000100000007 2023-09-20T22:04:35.682 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 archive_status 2023-09-20T22:04:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.682 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-09-20T22:04:35.682 INFO:teuthology.orchestra.run.smithi022.stdout:total 0 2023-09-20T22:04:35.683 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 20 21:49 . 2023-09-20T22:04:35.683 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Sep 20 22:04 .. 2023-09-20T22:04:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:04:35.683 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-09-20T22:04:35.683 INFO:teuthology.orchestra.run.smithi022.stdout:total 160 2023-09-20T22:04:35.684 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 20 21:49 . 2023-09-20T22:04:35.684 INFO:teuthology.orchestra.run.smithi022.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 20 22:04 .. 2023-09-20T22:04:35.684 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 20 22:04 0000 2023-09-20T22:04:35.684 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-09-20T22:04:35.695 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-09-20T22:04:36.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:35 smithi022 ceph-mon[97651]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 668 op/s 2023-09-20T22:04:36.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:35 smithi123 ceph-mon[134433]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 668 op/s 2023-09-20T22:04:36.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:35 smithi089 ceph-mon[134090]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 668 op/s 2023-09-20T22:04:38.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:37 smithi022 ceph-mon[97651]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 700 op/s 2023-09-20T22:04:38.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:37 smithi123 ceph-mon[134433]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 700 op/s 2023-09-20T22:04:38.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:37 smithi089 ceph-mon[134090]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 700 op/s 2023-09-20T22:04:40.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:39 smithi022 ceph-mon[97651]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 701 op/s 2023-09-20T22:04:40.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:39 smithi123 ceph-mon[134433]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 701 op/s 2023-09-20T22:04:40.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:39 smithi089 ceph-mon[134090]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.7 MiB/s wr, 701 op/s 2023-09-20T22:04:41.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:40 smithi022 ceph-mon[97651]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T22:04:41.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:40 smithi123 ceph-mon[134433]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T22:04:41.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:40 smithi089 ceph-mon[134090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-09-20T22:04:42.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:41 smithi022 ceph-mon[97651]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2023-09-20T22:04:42.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:41 smithi123 ceph-mon[134433]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2023-09-20T22:04:42.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:41 smithi089 ceph-mon[134090]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2023-09-20T22:04:44.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:43 smithi022 ceph-mon[97651]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.3 MiB/s wr, 441 op/s 2023-09-20T22:04:44.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:43 smithi123 ceph-mon[134433]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.3 MiB/s wr, 441 op/s 2023-09-20T22:04:44.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:43 smithi089 ceph-mon[134090]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.3 MiB/s wr, 441 op/s 2023-09-20T22:04:46.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:45 smithi022 ceph-mon[97651]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.7 MiB/s wr, 372 op/s 2023-09-20T22:04:46.345 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:45 smithi123 ceph-mon[134433]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.7 MiB/s wr, 372 op/s 2023-09-20T22:04:46.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:45 smithi089 ceph-mon[134090]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.7 MiB/s wr, 372 op/s 2023-09-20T22:04:48.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:47 smithi022 ceph-mon[97651]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2023-09-20T22:04:48.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:47 smithi123 ceph-mon[134433]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2023-09-20T22:04:48.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:47 smithi089 ceph-mon[134090]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2023-09-20T22:04:49.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:48 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:04:49.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:48 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:04:49.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:48 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:04:50.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:49 smithi022 ceph-mon[97651]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-09-20T22:04:50.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:04:50.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:04:50.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:04:50.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:49 smithi123 ceph-mon[134433]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-09-20T22:04:50.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:04:50.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:04:50.344 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:04:50.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:49 smithi089 ceph-mon[134090]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2023-09-20T22:04:50.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:04:50.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:04:50.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:04:52.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:51 smithi022 ceph-mon[97651]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 499 KiB/s wr, 53 op/s 2023-09-20T22:04:52.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:51 smithi123 ceph-mon[134433]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 499 KiB/s wr, 53 op/s 2023-09-20T22:04:52.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:51 smithi089 ceph-mon[134090]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 499 KiB/s wr, 53 op/s 2023-09-20T22:04:54.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:53 smithi022 ceph-mon[97651]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2023-09-20T22:04:54.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:53 smithi123 ceph-mon[134433]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2023-09-20T22:04:54.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:53 smithi089 ceph-mon[134090]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2023-09-20T22:04:56.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:55 smithi022 ceph-mon[97651]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-20T22:04:56.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:55 smithi123 ceph-mon[134433]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-20T22:04:56.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:55 smithi089 ceph-mon[134090]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 17 KiB/s wr, 14 op/s 2023-09-20T22:04:58.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:57 smithi022 ceph-mon[97651]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-20T22:04:58.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:57 smithi123 ceph-mon[134433]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-20T22:04:58.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:57 smithi089 ceph-mon[134090]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 14 KiB/s wr, 12 op/s 2023-09-20T22:05:00.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:04:59 smithi022 ceph-mon[97651]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-20T22:05:00.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:04:59 smithi123 ceph-mon[134433]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-20T22:05:00.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:04:59 smithi089 ceph-mon[134090]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-09-20T22:05:02.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:01 smithi022 ceph-mon[97651]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-20T22:05:02.248 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-09-20T22:05:02.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:01 smithi123 ceph-mon[134433]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-20T22:05:02.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:01 smithi089 ceph-mon[134090]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-20T22:05:02.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.490+0000 7f89261ff700 1 -- 172.21.15.22:0/700682085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8920101850 msgr2=0x7f8920103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.490+0000 7f89261ff700 1 --2- 172.21.15.22:0/700682085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8920101850 0x7f8920103c40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8910009990 tx=0x7f8910009ca0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.490+0000 7f89261ff700 1 -- 172.21.15.22:0/700682085 shutdown_connections 2023-09-20T22:05:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.490+0000 7f89261ff700 1 --2- 172.21.15.22:0/700682085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8920106ab0 0x7f8920108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.490+0000 7f89261ff700 1 --2- 172.21.15.22:0/700682085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8920101850 0x7f8920103c40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.490+0000 7f89261ff700 1 --2- 172.21.15.22:0/700682085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8920104180 0x7f8920106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.490+0000 7f89261ff700 1 -- 172.21.15.22:0/700682085 >> 172.21.15.22:0/700682085 conn(0x7f89200fb240 msgr2=0x7f89200fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:02.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.490+0000 7f89261ff700 1 -- 172.21.15.22:0/700682085 shutdown_connections 2023-09-20T22:05:02.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.490+0000 7f89261ff700 1 -- 172.21.15.22:0/700682085 wait complete. 2023-09-20T22:05:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.491+0000 7f89261ff700 1 Processor -- start 2023-09-20T22:05:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.491+0000 7f89261ff700 1 -- start start 2023-09-20T22:05:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f89261ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8920101850 0x7f8920196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f89261ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8920104180 0x7f8920197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f89261ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8920106ab0 0x7f892019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f89261ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892019e650 con 0x7f8920104180 2023-09-20T22:05:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f89261ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892019e790 con 0x7f8920106ab0 2023-09-20T22:05:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f89261ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892019e920 con 0x7f8920101850 2023-09-20T22:05:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8920101850 0x7f8920196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891ffff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8920106ab0 0x7f892019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8920104180 0x7f8920197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8920101850 0x7f8920196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:45112/0 (socket says 172.21.15.22:45112) 2023-09-20T22:05:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891f7fe700 1 -- 172.21.15.22:0/3800709921 learned_addr learned my addr 172.21.15.22:0/3800709921 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891ffff700 1 -- 172.21.15.22:0/3800709921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8920101850 msgr2=0x7f8920196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891ffff700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8920101850 0x7f8920196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891ffff700 1 -- 172.21.15.22:0/3800709921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8920104180 msgr2=0x7f8920197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891ffff700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8920104180 0x7f8920197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891ffff700 1 -- 172.21.15.22:0/3800709921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100095f0 con 0x7f8920106ab0 2023-09-20T22:05:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.492+0000 7f891f7fe700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8920101850 0x7f8920196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:05:02.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.493+0000 7f891ffff700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8920106ab0 0x7f892019e080 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f891800efc0 tx=0x7f891800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:02.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.493+0000 7f891cff9700 1 -- 172.21.15.22:0/3800709921 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918018070 con 0x7f8920106ab0 2023-09-20T22:05:02.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.493+0000 7f891cff9700 1 -- 172.21.15.22:0/3800709921 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891800f040 con 0x7f8920106ab0 2023-09-20T22:05:02.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.493+0000 7f891cff9700 1 -- 172.21.15.22:0/3800709921 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891800cd40 con 0x7f8920106ab0 2023-09-20T22:05:02.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.493+0000 7f89261ff700 1 -- 172.21.15.22:0/3800709921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892019eba0 con 0x7f8920106ab0 2023-09-20T22:05:02.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.493+0000 7f89261ff700 1 -- 172.21.15.22:0/3800709921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892019f060 con 0x7f8920106ab0 2023-09-20T22:05:02.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.497+0000 7f891cff9700 1 -- 172.21.15.22:0/3800709921 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f8918003a00 con 0x7f8920106ab0 2023-09-20T22:05:02.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.498+0000 7f891cff9700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89080a1380 0x7f89080a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.498+0000 7f891cff9700 1 -- 172.21.15.22:0/3800709921 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f89180ccb20 con 0x7f8920106ab0 2023-09-20T22:05:02.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.499+0000 7f891f7fe700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89080a1380 0x7f89080a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.499+0000 7f891f7fe700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89080a1380 0x7f89080a3840 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f891000b2e0 tx=0x7f8910005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:02.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.501+0000 7f8939b88b80 1 Processor -- start 2023-09-20T22:05:02.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.501+0000 7f8939b88b80 1 -- start start 2023-09-20T22:05:02.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.501+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 0x55f92a5c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.501+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 0x55f92a5ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.501+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5cb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.501+0000 7f8939b88b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a53dca0 con 0x55f92a5cabe0 2023-09-20T22:05:02.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.501+0000 7f8939b88b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a521c70 con 0x55f92a5c96f0 2023-09-20T22:05:02.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.501+0000 7f8939b88b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a51f590 con 0x55f92a5ca050 2023-09-20T22:05:02.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.501+0000 7f891ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.501+0000 7f891effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 0x55f92a5ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.502+0000 7f891effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 0x55f92a5ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:45118/0 (socket says 172.21.15.22:45118) 2023-09-20T22:05:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.502+0000 7f891effd700 1 -- 172.21.15.22:0/795371908 learned_addr learned my addr 172.21.15.22:0/795371908 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.502+0000 7f891f7fe700 1 --2- 172.21.15.22:0/795371908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 0x55f92a5c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.502+0000 7f891ffff700 1 -- 172.21.15.22:0/795371908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 msgr2=0x55f92a5ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:05:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.502+0000 7f891ffff700 1 --2- 172.21.15.22:0/795371908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 0x55f92a5ca5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.502+0000 7f891ffff700 1 -- 172.21.15.22:0/795371908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 msgr2=0x55f92a5c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:05:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.502+0000 7f891ffff700 1 --2- 172.21.15.22:0/795371908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 0x55f92a5c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.502+0000 7f891ffff700 1 -- 172.21.15.22:0/795371908 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f92a345de0 con 0x55f92a5cabe0 2023-09-20T22:05:02.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.502+0000 7f891ffff700 1 --2- 172.21.15.22:0/795371908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5cb060 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f891800ef90 tx=0x7f89180092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1541dee0da4d801 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:02.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.502+0000 7f8905ffb700 1 -- 172.21.15.22:0/795371908 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891801b320 con 0x55f92a5cabe0 2023-09-20T22:05:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.503+0000 7f8905ffb700 1 -- 172.21.15.22:0/795371908 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8918004ad0 con 0x55f92a5cabe0 2023-09-20T22:05:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.503+0000 7f8905ffb700 1 -- 172.21.15.22:0/795371908 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918096260 con 0x55f92a5cabe0 2023-09-20T22:05:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.503+0000 7f8939b88b80 1 -- 172.21.15.22:0/795371908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 msgr2=0x55f92a5cb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.503+0000 7f8939b88b80 1 --2- 172.21.15.22:0/795371908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5cb060 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f891800ef90 tx=0x7f89180092e0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.503+0000 7f8939b88b80 1 -- 172.21.15.22:0/795371908 shutdown_connections 2023-09-20T22:05:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.503+0000 7f8939b88b80 1 --2- 172.21.15.22:0/795371908 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5cb060 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.503+0000 7f8939b88b80 1 --2- 172.21.15.22:0/795371908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 0x55f92a5c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.503+0000 7f8939b88b80 1 --2- 172.21.15.22:0/795371908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 0x55f92a5ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.503+0000 7f8939b88b80 1 -- 172.21.15.22:0/795371908 >> 172.21.15.22:0/795371908 conn(0x55f92a4f6000 msgr2=0x55f92a5c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.504+0000 7f8939b88b80 1 -- 172.21.15.22:0/795371908 shutdown_connections 2023-09-20T22:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.504+0000 7f8939b88b80 1 -- 172.21.15.22:0/795371908 wait complete. 2023-09-20T22:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.504+0000 7f8939b88b80 1 -- start start 2023-09-20T22:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.504+0000 7f8939b88b80 1 Processor -- start 2023-09-20T22:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5c96f0 0x55f92a5d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5cabe0 0x55f92a5da780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f8939b88b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a51ae80 con 0x55f92a5c96f0 2023-09-20T22:05:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f8939b88b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a4b7740 con 0x55f92a5ca050 2023-09-20T22:05:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f8939b88b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a4b73e0 con 0x55f92a5cabe0 2023-09-20T22:05:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f8939b88b80 10 client.0 _ll_register_callbacks cb 0x55f92a401a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:05:02.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f891f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5c96f0 0x55f92a5d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f891effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f891effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:36014/0 (socket says 172.21.15.22:36014) 2023-09-20T22:05:02.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f891effd700 1 -- 172.21.15.22:0/2739300926 learned_addr learned my addr 172.21.15.22:0/2739300926 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:02.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f891ffff700 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5cabe0 0x55f92a5da780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.505+0000 7f891effd700 1 -- 172.21.15.22:0/2739300926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5cabe0 msgr2=0x55f92a5da780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f891effd700 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5cabe0 0x55f92a5da780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f891effd700 1 -- 172.21.15.22:0/2739300926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5c96f0 msgr2=0x55f92a5d3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f891effd700 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5c96f0 0x55f92a5d3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f891effd700 1 -- 172.21.15.22:0/2739300926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892019f060 con 0x55f92a5ca050 2023-09-20T22:05:02.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f891effd700 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5d3740 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8914012840 tx=0x7f8914012b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4221069e03f00e1a server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f88feffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T22:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f88feffd700 1 -- 172.21.15.22:0/2739300926 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914017070 con 0x55f92a5ca050 2023-09-20T22:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f8939b88b80 1 -- 172.21.15.22:0/2739300926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f92a48bbb0 con 0x55f92a5ca050 2023-09-20T22:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f88feffd700 1 -- 172.21.15.22:0/2739300926 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f8914004050 con 0x55f92a5ca050 2023-09-20T22:05:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f88feffd700 1 -- 172.21.15.22:0/2739300926 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8914004210 con 0x55f92a5ca050 2023-09-20T22:05:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f8939b88b80 10 client.26180 fetch_fsmap learned FSMap version 33 2023-09-20T22:05:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.506+0000 7f8939b88b80 1 -- 172.21.15.22:0/2739300926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f92a32ec60 con 0x55f92a5ca050 2023-09-20T22:05:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.507+0000 7f88feffd700 1 -- 172.21.15.22:0/2739300926 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f92a48bbb0 con 0x55f92a5ca050 2023-09-20T22:05:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.507+0000 7f88feffd700 1 -- 172.21.15.22:0/2739300926 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 33) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f891400f510 con 0x55f92a5ca050 2023-09-20T22:05:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.507+0000 7f8939b88b80 10 client.26180 fetch_fsmap finished waiting for FSMap version 33 2023-09-20T22:05:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.507+0000 7f8939b88b80 10 client.26180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:05:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.507+0000 7f8939b88b80 20 client.26180 populate_metadata read hostname 'smithi022' 2023-09-20T22:05:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.507+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f92a5e33f0 0x55f92a5e5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.507+0000 7f8939b88b80 4 client.26180 mds_command: new command op to 24578 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-20T22:05:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.507+0000 7f8939b88b80 1 -- 172.21.15.22:0/2739300926 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f92a598020 con 0x55f92a5e33f0 2023-09-20T22:05:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.507+0000 7f891f7fe700 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f92a5e33f0 0x55f92a5e5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.508+0000 7f891f7fe700 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f92a5e33f0 0x55f92a5e5810 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94d0d6c3e27d640 server_cookie=8dcbf6c9dc411162 in_seq=0 out_seq=0 2023-09-20T22:05:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.508+0000 7f88feffd700 10 client.26180 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:05:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f88feffd700 1 -- 172.21.15.22:0/2739300926 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f92a598020 con 0x55f92a5e33f0 2023-09-20T22:05:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f88feffd700 10 client.26180 handle_command_reply: tid=0 2023-09-20T22:05:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 client.26180 shutdown 2023-09-20T22:05:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 -- 172.21.15.22:0/2739300926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 msgr2=0x55f92a5d3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5d3740 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8914012840 tx=0x7f8914012b50 comp rx=0 tx=0).stop 2023-09-20T22:05:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 -- 172.21.15.22:0/2739300926 shutdown_connections 2023-09-20T22:05:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f92a5e33f0 0x55f92a5e5810 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5cabe0 0x55f92a5da780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5c96f0 0x55f92a5d3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2739300926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5d3740 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f88feffd700 0 client.26180 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:05:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 -- 172.21.15.22:0/2739300926 >> 172.21.15.22:0/2739300926 conn(0x55f92a4f6000 msgr2=0x55f92a5c3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 -- 172.21.15.22:0/2739300926 shutdown_connections 2023-09-20T22:05:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 1 -- 172.21.15.22:0/2739300926 wait complete. 2023-09-20T22:05:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.509+0000 7f8939b88b80 20 client.26180 trim_cache size 0 max 16384 2023-09-20T22:05:02.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.530+0000 7f8939b88b80 1 Processor -- start 2023-09-20T22:05:02.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.530+0000 7f8939b88b80 1 -- start start 2023-09-20T22:05:02.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.530+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5c96f0 0x55f92a5cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.530+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.530+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.530+0000 7f8939b88b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a39b920 con 0x55f92a5cabe0 2023-09-20T22:05:02.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.530+0000 7f8939b88b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a5d2d70 con 0x55f92a5ca050 2023-09-20T22:05:02.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.530+0000 7f8939b88b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a5caa40 con 0x55f92a5c96f0 2023-09-20T22:05:02.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:36026/0 (socket says 172.21.15.22:36026) 2023-09-20T22:05:02.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891effd700 1 -- 172.21.15.22:0/3200063346 learned_addr learned my addr 172.21.15.22:0/3200063346 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:02.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891ffff700 1 --2- 172.21.15.22:0/3200063346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891f7fe700 1 --2- 172.21.15.22:0/3200063346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5c96f0 0x55f92a5cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891effd700 1 -- 172.21.15.22:0/3200063346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5c96f0 msgr2=0x55f92a5cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:05:02.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891effd700 1 --2- 172.21.15.22:0/3200063346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5c96f0 0x55f92a5cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891effd700 1 -- 172.21.15.22:0/3200063346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 msgr2=0x55f92a5da740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:05:02.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891effd700 1 --2- 172.21.15.22:0/3200063346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5da740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891effd700 1 -- 172.21.15.22:0/3200063346 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f92a36c250 con 0x55f92a5ca050 2023-09-20T22:05:02.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.531+0000 7f891effd700 1 --2- 172.21.15.22:0/3200063346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5ce020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8914009340 tx=0x7f8914005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fac310304dcc0b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:02.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f89067fc700 1 -- 172.21.15.22:0/3200063346 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914017040 con 0x55f92a5ca050 2023-09-20T22:05:02.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f89067fc700 1 -- 172.21.15.22:0/3200063346 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891400f680 con 0x55f92a5ca050 2023-09-20T22:05:02.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f89067fc700 1 -- 172.21.15.22:0/3200063346 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914003ae0 con 0x55f92a5ca050 2023-09-20T22:05:02.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f8939b88b80 1 -- 172.21.15.22:0/3200063346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 msgr2=0x55f92a5ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f8939b88b80 1 --2- 172.21.15.22:0/3200063346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5ce020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8914009340 tx=0x7f8914005900 comp rx=0 tx=0).stop 2023-09-20T22:05:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f8939b88b80 1 -- 172.21.15.22:0/3200063346 shutdown_connections 2023-09-20T22:05:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f8939b88b80 1 --2- 172.21.15.22:0/3200063346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f8939b88b80 1 --2- 172.21.15.22:0/3200063346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5c96f0 0x55f92a5cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f8939b88b80 1 --2- 172.21.15.22:0/3200063346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5ca050 0x55f92a5ce020 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f8939b88b80 1 -- 172.21.15.22:0/3200063346 >> 172.21.15.22:0/3200063346 conn(0x55f92a4f6000 msgr2=0x55f92a5d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:02.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.532+0000 7f8939b88b80 1 -- 172.21.15.22:0/3200063346 shutdown_connections 2023-09-20T22:05:02.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.533+0000 7f8939b88b80 1 -- 172.21.15.22:0/3200063346 wait complete. 2023-09-20T22:05:02.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.533+0000 7f8939b88b80 1 -- start start 2023-09-20T22:05:02.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.533+0000 7f8939b88b80 1 Processor -- start 2023-09-20T22:05:02.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 0x55f92a5da340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 0x55f92a5d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f8939b88b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5d7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f8939b88b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a5d7860 con 0x55f92a5cabe0 2023-09-20T22:05:02.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f8939b88b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a5d79d0 con 0x55f92a5c96f0 2023-09-20T22:05:02.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f8939b88b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92a637230 con 0x55f92a5ca050 2023-09-20T22:05:02.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f8939b88b80 10 client.0 _ll_register_callbacks cb 0x55f92a401a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-20T22:05:02.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 0x55f92a5da340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891f7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 0x55f92a5da340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:36038/0 (socket says 172.21.15.22:36038) 2023-09-20T22:05:02.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891f7fe700 1 -- 172.21.15.22:0/2860512918 learned_addr learned my addr 172.21.15.22:0/2860512918 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:02.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891effd700 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 0x55f92a5d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891ffff700 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5d7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891f7fe700 1 -- 172.21.15.22:0/2860512918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 msgr2=0x55f92a5d6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:05:02.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891f7fe700 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 0x55f92a5d6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891f7fe700 1 -- 172.21.15.22:0/2860512918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 msgr2=0x55f92a5d7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-20T22:05:02.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891f7fe700 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5d7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891f7fe700 1 -- 172.21.15.22:0/2860512918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f92a32ec60 con 0x55f92a5c96f0 2023-09-20T22:05:02.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891ffff700 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5d7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-20T22:05:02.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f891f7fe700 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 0x55f92a5da340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8910005850 tx=0x7f8910009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4af86df4cc1539ac server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:02.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.534+0000 7f88feffd700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2023-09-20T22:05:02.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f88feffd700 1 -- 172.21.15.22:0/2860512918 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891002b040 con 0x55f92a5c96f0 2023-09-20T22:05:02.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f8939b88b80 1 -- 172.21.15.22:0/2860512918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8918004ad0 con 0x55f92a5c96f0 2023-09-20T22:05:02.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f88feffd700 1 -- 172.21.15.22:0/2860512918 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f8910036150 con 0x55f92a5c96f0 2023-09-20T22:05:02.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f88feffd700 1 -- 172.21.15.22:0/2860512918 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8910036310 con 0x55f92a5c96f0 2023-09-20T22:05:02.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f8939b88b80 10 client.26186 fetch_fsmap learned FSMap version 33 2023-09-20T22:05:02.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f8939b88b80 1 -- 172.21.15.22:0/2860512918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f92a3442b0 con 0x55f92a5c96f0 2023-09-20T22:05:02.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f88feffd700 1 -- 172.21.15.22:0/2860512918 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8918004ad0 con 0x55f92a5c96f0 2023-09-20T22:05:02.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f88feffd700 1 -- 172.21.15.22:0/2860512918 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 33) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8910024c90 con 0x55f92a5c96f0 2023-09-20T22:05:02.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f8939b88b80 10 client.26186 fetch_fsmap finished waiting for FSMap version 33 2023-09-20T22:05:02.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f8939b88b80 10 client.26186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-09-20T22:05:02.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.535+0000 7f8939b88b80 20 client.26186 populate_metadata read hostname 'smithi022' 2023-09-20T22:05:02.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.536+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f92a638b70 0x55f92a63af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.536+0000 7f8939b88b80 4 client.26186 mds_command: new command op to 24578 tid=0[{"prefix": "scrub status"}] 2023-09-20T22:05:02.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.536+0000 7f8939b88b80 1 -- 172.21.15.22:0/2860512918 --> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f92a513e60 con 0x55f92a638b70 2023-09-20T22:05:02.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.536+0000 7f891effd700 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f92a638b70 0x55f92a63af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.536+0000 7f88feffd700 10 client.26186 ms_handle_connect on v2:172.21.15.89:6838/2165264535 2023-09-20T22:05:02.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.536+0000 7f891effd700 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f92a638b70 0x55f92a63af90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5de9ae0dbcf65d server_cookie=e3c1385ab5b4556a in_seq=0 out_seq=0 2023-09-20T22:05:02.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f88feffd700 1 -- 172.21.15.22:0/2860512918 <== mds.0 v2:172.21.15.89:6838/2165264535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f92a513e60 con 0x55f92a638b70 2023-09-20T22:05:02.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f88feffd700 10 client.26186 handle_command_reply: tid=0 2023-09-20T22:05:02.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 client.26186 shutdown 2023-09-20T22:05:02.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 -- 172.21.15.22:0/2860512918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 msgr2=0x55f92a5da340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 0x55f92a5da340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8910005850 tx=0x7f8910009ca0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 -- 172.21.15.22:0/2860512918 shutdown_connections 2023-09-20T22:05:02.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.89:6838/2165264535,v1:172.21.15.89:6839/2165264535] conn(0x55f92a638b70 0x55f92a63af90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92a5cabe0 0x55f92a5d7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f92a5c96f0 0x55f92a5da340 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 --2- 172.21.15.22:0/2860512918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92a5ca050 0x55f92a5d6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f88feffd700 0 client.26186 ms_handle_reset on v2:172.21.15.89:6838/2165264535 2023-09-20T22:05:02.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 -- 172.21.15.22:0/2860512918 >> 172.21.15.22:0/2860512918 conn(0x55f92a4f6000 msgr2=0x55f92a5cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:02.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 -- 172.21.15.22:0/2860512918 shutdown_connections 2023-09-20T22:05:02.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 1 -- 172.21.15.22:0/2860512918 wait complete. 2023-09-20T22:05:02.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.537+0000 7f8939b88b80 20 client.26186 trim_cache size 0 max 16384 2023-09-20T22:05:02.551 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:05:02.551 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (1 inodes in the stack)", 2023-09-20T22:05:02.551 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2023-09-20T22:05:02.551 INFO:teuthology.orchestra.run.smithi022.stdout: "35c75b6f-9e35-4397-a1b5-d2795c12eb42": { 2023-09-20T22:05:02.551 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2023-09-20T22:05:02.552 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "35c75b6f-9e35-4397-a1b5-d2795c12eb42", 2023-09-20T22:05:02.552 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,force" 2023-09-20T22:05:02.552 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:02.552 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:02.552 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:05:02.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 -- 172.21.15.22:0/3800709921 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89080a1380 msgr2=0x7f89080a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89080a1380 0x7f89080a3840 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f891000b2e0 tx=0x7f8910005fb0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 -- 172.21.15.22:0/3800709921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8920106ab0 msgr2=0x7f892019e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8920106ab0 0x7f892019e080 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f891800efc0 tx=0x7f891800c3a0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 -- 172.21.15.22:0/3800709921 shutdown_connections 2023-09-20T22:05:02.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f89080a1380 0x7f89080a3840 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8920106ab0 0x7f892019e080 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8920101850 0x7f8920196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 --2- 172.21.15.22:0/3800709921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8920104180 0x7f8920197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 -- 172.21.15.22:0/3800709921 >> 172.21.15.22:0/3800709921 conn(0x7f89200fb240 msgr2=0x7f89201024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 -- 172.21.15.22:0/3800709921 shutdown_connections 2023-09-20T22:05:02.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.538+0000 7f89261ff700 1 -- 172.21.15.22:0/3800709921 wait complete. 2023-09-20T22:05:02.559 INFO:tasks.cephfs.filesystem:scrub status for tag:35c75b6f-9e35-4397-a1b5-d2795c12eb42 - {'path': '/', 'tag': '35c75b6f-9e35-4397-a1b5-d2795c12eb42', 'options': 'recursive,force'} 2023-09-20T22:05:02.559 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_bf8a061ff16ac1d393cf108f740d92dfa49f2e54/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_bf8a061ff16ac1d393cf108f740d92dfa49f2e54/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_bf8a061ff16ac1d393cf108f740d92dfa49f2e54/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_bf8a061ff16ac1d393cf108f740d92dfa49f2e54/qa/tasks/cephfs/filesystem.py", line 1730, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-09-20T22:05:02.560 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bf8a061ff16ac1d393cf108f740d92dfa49f2e54/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_bf8a061ff16ac1d393cf108f740d92dfa49f2e54/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-09-20T22:05:02.560 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-09-20T22:05:02.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T22:05:02.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.809+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/1897891270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e141018e0 msgr2=0x7f5e14103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.809+0000 7f5e1b9ff700 1 --2- 172.21.15.22:0/1897891270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e141018e0 0x7f5e14103cd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009990 tx=0x7f5e10009ca0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.809+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/1897891270 shutdown_connections 2023-09-20T22:05:02.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.809+0000 7f5e1b9ff700 1 --2- 172.21.15.22:0/1897891270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e14106b40 0x7f5e14108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.809+0000 7f5e1b9ff700 1 --2- 172.21.15.22:0/1897891270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e141018e0 0x7f5e14103cd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.809+0000 7f5e1b9ff700 1 --2- 172.21.15.22:0/1897891270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e14104210 0x7f5e14106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.809+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/1897891270 >> 172.21.15.22:0/1897891270 conn(0x7f5e140fb210 msgr2=0x7f5e140fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:02.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.810+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/1897891270 shutdown_connections 2023-09-20T22:05:02.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.810+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/1897891270 wait complete. 2023-09-20T22:05:02.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.811+0000 7f5e1b9ff700 1 Processor -- start 2023-09-20T22:05:02.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.811+0000 7f5e1b9ff700 1 -- start start 2023-09-20T22:05:02.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.811+0000 7f5e1b9ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e141018e0 0x7f5e14198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.811+0000 7f5e1b9ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e14104210 0x7f5e141991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.811+0000 7f5e1b9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e14106b40 0x7f5e141a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.811+0000 7f5e1b9ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e141a07b0 con 0x7f5e141018e0 2023-09-20T22:05:02.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.811+0000 7f5e1b9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e141a0920 con 0x7f5e14106b40 2023-09-20T22:05:02.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.811+0000 7f5e1b9ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e141a0ae0 con 0x7f5e14104210 2023-09-20T22:05:02.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.811+0000 7f5e1979b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e141018e0 0x7f5e14198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e19f9c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e14106b40 0x7f5e141a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e1979b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e141018e0 0x7f5e14198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51226/0 (socket says 172.21.15.22:51226) 2023-09-20T22:05:02.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e1979b700 1 -- 172.21.15.22:0/2679699365 learned_addr learned my addr 172.21.15.22:0/2679699365 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:02.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e18f9a700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e14104210 0x7f5e141991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e19f9c700 1 -- 172.21.15.22:0/2679699365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e14104210 msgr2=0x7f5e141991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e19f9c700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e14104210 0x7f5e141991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e19f9c700 1 -- 172.21.15.22:0/2679699365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e141018e0 msgr2=0x7f5e14198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:02.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e19f9c700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e141018e0 0x7f5e14198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:02.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e19f9c700 1 -- 172.21.15.22:0/2679699365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e100095f0 con 0x7f5e14106b40 2023-09-20T22:05:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e19f9c700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e14106b40 0x7f5e141a01e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00efc0 tx=0x7f5e0c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e0a7fc700 1 -- 172.21.15.22:0/2679699365 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c018070 con 0x7f5e14106b40 2023-09-20T22:05:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e0a7fc700 1 -- 172.21.15.22:0/2679699365 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e0c00f040 con 0x7f5e14106b40 2023-09-20T22:05:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e0a7fc700 1 -- 172.21.15.22:0/2679699365 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c00cd40 con 0x7f5e14106b40 2023-09-20T22:05:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.812+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/2679699365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e141a0d60 con 0x7f5e14106b40 2023-09-20T22:05:02.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.813+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/2679699365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e141a11c0 con 0x7f5e14106b40 2023-09-20T22:05:02.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.814+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/2679699365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e141086c0 con 0x7f5e14106b40 2023-09-20T22:05:02.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.816+0000 7f5e0a7fc700 1 -- 172.21.15.22:0/2679699365 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5e0c003a00 con 0x7f5e14106b40 2023-09-20T22:05:02.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.817+0000 7f5e0a7fc700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5e000a13d0 0x7f5e000a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:02.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.817+0000 7f5e0a7fc700 1 -- 172.21.15.22:0/2679699365 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f5e0c0ccaa0 con 0x7f5e14106b40 2023-09-20T22:05:02.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.817+0000 7f5e1979b700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5e000a13d0 0x7f5e000a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:02.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.818+0000 7f5e1979b700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5e000a13d0 0x7f5e000a3890 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5e1419a140 tx=0x7f5e1000b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:02.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:02.822+0000 7f5e0a7fc700 1 -- 172.21.15.22:0/2679699365 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5e0c095e10 con 0x7f5e14106b40 2023-09-20T22:05:03.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.093+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/2679699365 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e1410c5f0 con 0x7f5e14106b40 2023-09-20T22:05:03.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.095+0000 7f5e0a7fc700 1 -- 172.21.15.22:0/2679699365 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7f5e0c01b320 con 0x7f5e14106b40 2023-09-20T22:05:03.095 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:05:03.096 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T22:05:03.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/2679699365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5e000a13d0 msgr2=0x7f5e000a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:03.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5e000a13d0 0x7f5e000a3890 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5e1419a140 tx=0x7f5e1000b260 comp rx=0 tx=0).stop 2023-09-20T22:05:03.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/2679699365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e14106b40 msgr2=0x7f5e141a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:03.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e14106b40 0x7f5e141a01e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00efc0 tx=0x7f5e0c00c3a0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/2679699365 shutdown_connections 2023-09-20T22:05:03.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5e000a13d0 0x7f5e000a3890 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e14106b40 0x7f5e141a01e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e141018e0 0x7f5e14198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 --2- 172.21.15.22:0/2679699365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e14104210 0x7f5e141991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/2679699365 >> 172.21.15.22:0/2679699365 conn(0x7f5e140fb210 msgr2=0x7f5e140fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:03.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/2679699365 shutdown_connections 2023-09-20T22:05:03.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.098+0000 7f5e1b9ff700 1 -- 172.21.15.22:0/2679699365 wait complete. 2023-09-20T22:05:03.118 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T22:05:03.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T22:05:03.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.355+0000 7f1d40fff700 1 -- 172.21.15.22:0/3175104890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c101850 msgr2=0x7f1d3c103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.355+0000 7f1d40fff700 1 --2- 172.21.15.22:0/3175104890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c101850 0x7f1d3c103c40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0098f0 tx=0x7f1d2c009c00 comp rx=0 tx=0).stop 2023-09-20T22:05:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.355+0000 7f1d40fff700 1 -- 172.21.15.22:0/3175104890 shutdown_connections 2023-09-20T22:05:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.355+0000 7f1d40fff700 1 --2- 172.21.15.22:0/3175104890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d3c106ab0 0x7f1d3c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.355+0000 7f1d40fff700 1 --2- 172.21.15.22:0/3175104890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c101850 0x7f1d3c103c40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.355+0000 7f1d40fff700 1 --2- 172.21.15.22:0/3175104890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d3c104180 0x7f1d3c106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.355+0000 7f1d40fff700 1 -- 172.21.15.22:0/3175104890 >> 172.21.15.22:0/3175104890 conn(0x7f1d3c0fb240 msgr2=0x7f1d3c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:03.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.355+0000 7f1d40fff700 1 -- 172.21.15.22:0/3175104890 shutdown_connections 2023-09-20T22:05:03.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.355+0000 7f1d40fff700 1 -- 172.21.15.22:0/3175104890 wait complete. 2023-09-20T22:05:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.357+0000 7f1d40fff700 1 Processor -- start 2023-09-20T22:05:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.357+0000 7f1d40fff700 1 -- start start 2023-09-20T22:05:03.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.357+0000 7f1d40fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d3c101850 0x7f1d3c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.357+0000 7f1d40fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c104180 0x7f1d3c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.357+0000 7f1d40fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d3c106ab0 0x7f1d3c1a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d40fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1a08d0 con 0x7f1d3c104180 2023-09-20T22:05:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d40fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1a0a10 con 0x7f1d3c106ab0 2023-09-20T22:05:03.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d40fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1a0ba0 con 0x7f1d3c101850 2023-09-20T22:05:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d3a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d3c101850 0x7f1d3c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d3a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d3c101850 0x7f1d3c198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:45192/0 (socket says 172.21.15.22:45192) 2023-09-20T22:05:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d3a59c700 1 -- 172.21.15.22:0/166206015 learned_addr learned my addr 172.21.15.22:0/166206015 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d3ad9d700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d3c106ab0 0x7f1d3c1a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:03.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d39d9b700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c104180 0x7f1d3c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d3ad9d700 1 -- 172.21.15.22:0/166206015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d3c101850 msgr2=0x7f1d3c198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d3ad9d700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d3c101850 0x7f1d3c198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d3ad9d700 1 -- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c104180 msgr2=0x7f1d3c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d3ad9d700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c104180 0x7f1d3c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d3ad9d700 1 -- 172.21.15.22:0/166206015 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34009d00 con 0x7f1d3c106ab0 2023-09-20T22:05:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.358+0000 7f1d39d9b700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c104180 0x7f1d3c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:05:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.359+0000 7f1d3ad9d700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d3c106ab0 0x7f1d3c1a0300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009cd0 tx=0x7f1d3400c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.359+0000 7f1d2b7fe700 1 -- 172.21.15.22:0/166206015 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34017070 con 0x7f1d3c106ab0 2023-09-20T22:05:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.359+0000 7f1d2b7fe700 1 -- 172.21.15.22:0/166206015 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d3400f040 con 0x7f1d3c106ab0 2023-09-20T22:05:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.359+0000 7f1d40fff700 1 -- 172.21.15.22:0/166206015 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2c009520 con 0x7f1d3c106ab0 2023-09-20T22:05:03.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.359+0000 7f1d2b7fe700 1 -- 172.21.15.22:0/166206015 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400e050 con 0x7f1d3c106ab0 2023-09-20T22:05:03.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.359+0000 7f1d40fff700 1 -- 172.21.15.22:0/166206015 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c1a11a0 con 0x7f1d3c106ab0 2023-09-20T22:05:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.360+0000 7f1d40fff700 1 -- 172.21.15.22:0/166206015 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c108680 con 0x7f1d3c106ab0 2023-09-20T22:05:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.363+0000 7f1d2b7fe700 1 -- 172.21.15.22:0/166206015 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f1d34003a00 con 0x7f1d3c106ab0 2023-09-20T22:05:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.364+0000 7f1d2b7fe700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d240a1300 0x7f1d240a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:03.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.364+0000 7f1d2b7fe700 1 -- 172.21.15.22:0/166206015 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f1d340cc9a0 con 0x7f1d3c106ab0 2023-09-20T22:05:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.364+0000 7f1d3a59c700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d240a1300 0x7f1d240a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:03.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.365+0000 7f1d3a59c700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d240a1300 0x7f1d240a37c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00c010 tx=0x7f1d2c019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.368+0000 7f1d2b7fe700 1 -- 172.21.15.22:0/166206015 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f1d34095ce0 con 0x7f1d3c106ab0 2023-09-20T22:05:03.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.587+0000 7f1d40fff700 1 -- 172.21.15.22:0/166206015 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d3c02d1f0 con 0x7f1d3c106ab0 2023-09-20T22:05:03.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.587+0000 7f1d2b7fe700 1 -- 172.21.15.22:0/166206015 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d34099300 con 0x7f1d3c106ab0 2023-09-20T22:05:03.588 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T22:05:03.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.589+0000 7f1d40fff700 1 -- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d240a1300 msgr2=0x7f1d240a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:03.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d240a1300 0x7f1d240a37c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00c010 tx=0x7f1d2c019040 comp rx=0 tx=0).stop 2023-09-20T22:05:03.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 -- 172.21.15.22:0/166206015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d3c106ab0 msgr2=0x7f1d3c1a0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:03.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d3c106ab0 0x7f1d3c1a0300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009cd0 tx=0x7f1d3400c620 comp rx=0 tx=0).stop 2023-09-20T22:05:03.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 -- 172.21.15.22:0/166206015 shutdown_connections 2023-09-20T22:05:03.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f1d240a1300 0x7f1d240a37c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d3c106ab0 0x7f1d3c1a0300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d3c101850 0x7f1d3c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 --2- 172.21.15.22:0/166206015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d3c104180 0x7f1d3c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 -- 172.21.15.22:0/166206015 >> 172.21.15.22:0/166206015 conn(0x7f1d3c0fb240 msgr2=0x7f1d3c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:03.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 -- 172.21.15.22:0/166206015 shutdown_connections 2023-09-20T22:05:03.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.590+0000 7f1d40fff700 1 -- 172.21.15.22:0/166206015 wait complete. 2023-09-20T22:05:03.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T22:05:03.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.880+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/2745076863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c74100dd0 msgr2=0x7f8c741011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:03.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.880+0000 7f8c7c7ff700 1 --2- 172.21.15.22:0/2745076863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c74100dd0 0x7f8c741011f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8c700099c0 tx=0x7f8c70009cd0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.880+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/2745076863 shutdown_connections 2023-09-20T22:05:03.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.880+0000 7f8c7c7ff700 1 --2- 172.21.15.22:0/2745076863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c74102960 0x7f8c74109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.880+0000 7f8c7c7ff700 1 --2- 172.21.15.22:0/2745076863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c74100dd0 0x7f8c741011f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.880+0000 7f8c7c7ff700 1 --2- 172.21.15.22:0/2745076863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c74101fc0 0x7f8c74102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.880+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/2745076863 >> 172.21.15.22:0/2745076863 conn(0x7f8c740fc320 msgr2=0x7f8c740fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:03.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.880+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/2745076863 shutdown_connections 2023-09-20T22:05:03.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.880+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/2745076863 wait complete. 2023-09-20T22:05:03.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.881+0000 7f8c7c7ff700 1 Processor -- start 2023-09-20T22:05:03.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.881+0000 7f8c7c7ff700 1 -- start start 2023-09-20T22:05:03.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c7c7ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c74100dd0 0x7f8c74198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c7c7ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c74101fc0 0x7f8c74199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c7c7ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c74102960 0x7f8c741a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c7c7ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c741a08d0 con 0x7f8c74102960 2023-09-20T22:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c7c7ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c741a0a10 con 0x7f8c74100dd0 2023-09-20T22:05:03.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c7c7ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c741a0ba0 con 0x7f8c74101fc0 2023-09-20T22:05:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c7a59b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c74100dd0 0x7f8c74198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c7a59b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c74100dd0 0x7f8c74198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:36088/0 (socket says 172.21.15.22:36088) 2023-09-20T22:05:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c7a59b700 1 -- 172.21.15.22:0/4163480263 learned_addr learned my addr 172.21.15.22:0/4163480263 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c79d9a700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c74101fc0 0x7f8c74199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:03.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.882+0000 7f8c7ad9c700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c74102960 0x7f8c741a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:03.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c7a59b700 1 -- 172.21.15.22:0/4163480263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c74101fc0 msgr2=0x7f8c74199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:03.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c7a59b700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c74101fc0 0x7f8c74199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c7a59b700 1 -- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c74102960 msgr2=0x7f8c741a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:03.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c7a59b700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c74102960 0x7f8c741a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:03.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c7a59b700 1 -- 172.21.15.22:0/4163480263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c700095f0 con 0x7f8c74100dd0 2023-09-20T22:05:03.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c7ad9c700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c74102960 0x7f8c741a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:05:03.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c7a59b700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c74100dd0 0x7f8c74198d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009fd0 tx=0x7f8c7000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:03.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c6b7fe700 1 -- 172.21.15.22:0/4163480263 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7001d070 con 0x7f8c74100dd0 2023-09-20T22:05:03.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c6b7fe700 1 -- 172.21.15.22:0/4163480263 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c70005510 con 0x7f8c74100dd0 2023-09-20T22:05:03.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c6b7fe700 1 -- 172.21.15.22:0/4163480263 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70004b00 con 0x7f8c74100dd0 2023-09-20T22:05:03.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/4163480263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c741a0e20 con 0x7f8c74100dd0 2023-09-20T22:05:03.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.883+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/4163480263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c741a1190 con 0x7f8c74100dd0 2023-09-20T22:05:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.885+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/4163480263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c74193420 con 0x7f8c74100dd0 2023-09-20T22:05:03.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.887+0000 7f8c6b7fe700 1 -- 172.21.15.22:0/4163480263 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f8c70004ca0 con 0x7f8c74100dd0 2023-09-20T22:05:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.889+0000 7f8c6b7fe700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8c600a1380 0x7f8c600a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.889+0000 7f8c6b7fe700 1 -- 172.21.15.22:0/4163480263 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f8c700cddc0 con 0x7f8c74100dd0 2023-09-20T22:05:03.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.889+0000 7f8c79d9a700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8c600a1380 0x7f8c600a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:03.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.890+0000 7f8c6b7fe700 1 -- 172.21.15.22:0/4163480263 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f8c70097130 con 0x7f8c74100dd0 2023-09-20T22:05:03.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:03.890+0000 7f8c79d9a700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8c600a1380 0x7f8c600a3840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009c60 tx=0x7f8c64009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:04.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.173+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/4163480263 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c740008d0 con 0x7f8c74100dd0 2023-09-20T22:05:04.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.173+0000 7f8c6b7fe700 1 -- 172.21.15.22:0/4163480263 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c7009a750 con 0x7f8c74100dd0 2023-09-20T22:05:04.174 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T22:05:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8c600a1380 msgr2=0x7f8c600a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:04.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8c600a1380 0x7f8c600a3840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009c60 tx=0x7f8c64009260 comp rx=0 tx=0).stop 2023-09-20T22:05:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/4163480263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c74100dd0 msgr2=0x7f8c74198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c74100dd0 0x7f8c74198d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009fd0 tx=0x7f8c7000f690 comp rx=0 tx=0).stop 2023-09-20T22:05:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/4163480263 shutdown_connections 2023-09-20T22:05:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8c74102960 0x7f8c741a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f8c600a1380 0x7f8c600a3840 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c74100dd0 0x7f8c74198d50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 --2- 172.21.15.22:0/4163480263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c74101fc0 0x7f8c74199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/4163480263 >> 172.21.15.22:0/4163480263 conn(0x7f8c740fc320 msgr2=0x7f8c74107940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:04.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/4163480263 shutdown_connections 2023-09-20T22:05:04.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.176+0000 7f8c7c7ff700 1 -- 172.21.15.22:0/4163480263 wait complete. 2023-09-20T22:05:04.196 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:03 smithi022 ceph-mon[97651]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-20T22:05:04.196 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:03 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2679699365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T22:05:04.196 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:03 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/166206015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T22:05:04.198 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T22:05:04.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:03 smithi123 ceph-mon[134433]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-20T22:05:04.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:03 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2679699365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T22:05:04.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:03 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/166206015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T22:05:04.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:03 smithi089 ceph-mon[134090]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-09-20T22:05:04.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:03 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2679699365' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T22:05:04.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:03 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/166206015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T22:05:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.464+0000 7f2d639ff700 1 -- 172.21.15.22:0/851421812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c101fe0 msgr2=0x7f2d5c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.464+0000 7f2d639ff700 1 --2- 172.21.15.22:0/851421812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c101fe0 0x7f2d5c102440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0099c0 tx=0x7f2d4c009cd0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.464+0000 7f2d639ff700 1 -- 172.21.15.22:0/851421812 shutdown_connections 2023-09-20T22:05:04.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.465+0000 7f2d639ff700 1 --2- 172.21.15.22:0/851421812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d5c102980 0x7f2d5c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.465+0000 7f2d639ff700 1 --2- 172.21.15.22:0/851421812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d5c100df0 0x7f2d5c101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.465+0000 7f2d639ff700 1 --2- 172.21.15.22:0/851421812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c101fe0 0x7f2d5c102440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.465+0000 7f2d639ff700 1 -- 172.21.15.22:0/851421812 >> 172.21.15.22:0/851421812 conn(0x7f2d5c0fc360 msgr2=0x7f2d5c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:04.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.465+0000 7f2d639ff700 1 -- 172.21.15.22:0/851421812 shutdown_connections 2023-09-20T22:05:04.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.465+0000 7f2d639ff700 1 -- 172.21.15.22:0/851421812 wait complete. 2023-09-20T22:05:04.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.466+0000 7f2d639ff700 1 Processor -- start 2023-09-20T22:05:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.466+0000 7f2d639ff700 1 -- start start 2023-09-20T22:05:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.466+0000 7f2d639ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d5c100df0 0x7f2d5c074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.466+0000 7f2d639ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c101fe0 0x7f2d5c0731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.466+0000 7f2d639ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d5c102980 0x7f2d5c073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:04.469 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.466+0000 7f2d639ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c073eb0 con 0x7f2d5c100df0 2023-09-20T22:05:04.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.466+0000 7f2d639ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c074020 con 0x7f2d5c101fe0 2023-09-20T22:05:04.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.466+0000 7f2d639ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c1a0b40 con 0x7f2d5c102980 2023-09-20T22:05:04.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d6179b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d5c100df0 0x7f2d5c074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:04.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d61f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d5c102980 0x7f2d5c073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:04.470 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d60f9a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c101fe0 0x7f2d5c0731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:04.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d61f9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d5c102980 0x7f2d5c073710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:45222/0 (socket says 172.21.15.22:45222) 2023-09-20T22:05:04.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d61f9c700 1 -- 172.21.15.22:0/51723836 learned_addr learned my addr 172.21.15.22:0/51723836 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:04.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d60f9a700 1 -- 172.21.15.22:0/51723836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d5c102980 msgr2=0x7f2d5c073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:04.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d60f9a700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d5c102980 0x7f2d5c073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.471 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d60f9a700 1 -- 172.21.15.22:0/51723836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d5c100df0 msgr2=0x7f2d5c074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d60f9a700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d5c100df0 0x7f2d5c074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d60f9a700 1 -- 172.21.15.22:0/51723836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0095f0 con 0x7f2d5c101fe0 2023-09-20T22:05:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d60f9a700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c101fe0 0x7f2d5c0731d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0098e0 tx=0x7f2d4c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.467+0000 7f2d527fc700 1 -- 172.21.15.22:0/51723836 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c01d070 con 0x7f2d5c101fe0 2023-09-20T22:05:04.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.468+0000 7f2d527fc700 1 -- 172.21.15.22:0/51723836 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d4c005020 con 0x7f2d5c101fe0 2023-09-20T22:05:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.468+0000 7f2d527fc700 1 -- 172.21.15.22:0/51723836 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c004a70 con 0x7f2d5c101fe0 2023-09-20T22:05:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.468+0000 7f2d639ff700 1 -- 172.21.15.22:0/51723836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5c1a0dc0 con 0x7f2d5c101fe0 2023-09-20T22:05:04.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.468+0000 7f2d639ff700 1 -- 172.21.15.22:0/51723836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5c1a1130 con 0x7f2d5c101fe0 2023-09-20T22:05:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.469+0000 7f2d639ff700 1 -- 172.21.15.22:0/51723836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5c10c6c0 con 0x7f2d5c101fe0 2023-09-20T22:05:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.471+0000 7f2d527fc700 1 -- 172.21.15.22:0/51723836 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2d4c005400 con 0x7f2d5c101fe0 2023-09-20T22:05:04.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.473+0000 7f2d527fc700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d480a1380 0x7f2d480a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:04.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.473+0000 7f2d527fc700 1 -- 172.21.15.22:0/51723836 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f2d4c0cd970 con 0x7f2d5c101fe0 2023-09-20T22:05:04.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.473+0000 7f2d6179b700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d480a1380 0x7f2d480a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:04.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.474+0000 7f2d6179b700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d480a1380 0x7f2d480a3840 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c101e40 tx=0x7f2d58009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:04.475 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.476+0000 7f2d527fc700 1 -- 172.21.15.22:0/51723836 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2d4c096ce0 con 0x7f2d5c101fe0 2023-09-20T22:05:04.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.723+0000 7f2d639ff700 1 -- 172.21.15.22:0/51723836 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d5c107cf0 con 0x7f2d5c101fe0 2023-09-20T22:05:04.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.723+0000 7f2d527fc700 1 -- 172.21.15.22:0/51723836 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2d4c005190 con 0x7f2d5c101fe0 2023-09-20T22:05:04.724 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T22:05:04.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 -- 172.21.15.22:0/51723836 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d480a1380 msgr2=0x7f2d480a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:04.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d480a1380 0x7f2d480a3840 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c101e40 tx=0x7f2d58009260 comp rx=0 tx=0).stop 2023-09-20T22:05:04.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 -- 172.21.15.22:0/51723836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c101fe0 msgr2=0x7f2d5c0731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:04.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c101fe0 0x7f2d5c0731d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0098e0 tx=0x7f2d4c00f690 comp rx=0 tx=0).stop 2023-09-20T22:05:04.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 -- 172.21.15.22:0/51723836 shutdown_connections 2023-09-20T22:05:04.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d5c102980 0x7f2d5c073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d5c100df0 0x7f2d5c074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2d480a1380 0x7f2d480a3840 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 --2- 172.21.15.22:0/51723836 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c101fe0 0x7f2d5c0731d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 -- 172.21.15.22:0/51723836 >> 172.21.15.22:0/51723836 conn(0x7f2d5c0fc360 msgr2=0x7f2d5c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:04.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 -- 172.21.15.22:0/51723836 shutdown_connections 2023-09-20T22:05:04.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.726+0000 7f2d639ff700 1 -- 172.21.15.22:0/51723836 wait complete. 2023-09-20T22:05:04.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T22:05:04.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.996+0000 7fe2875ff700 1 -- 172.21.15.22:0/1776036352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe280101850 msgr2=0x7fe280103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:04.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.996+0000 7fe2875ff700 1 --2- 172.21.15.22:0/1776036352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe280101850 0x7fe280103c40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009990 tx=0x7fe27c009ca0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.996+0000 7fe2875ff700 1 -- 172.21.15.22:0/1776036352 shutdown_connections 2023-09-20T22:05:04.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.996+0000 7fe2875ff700 1 --2- 172.21.15.22:0/1776036352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe280106ab0 0x7fe280108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.996+0000 7fe2875ff700 1 --2- 172.21.15.22:0/1776036352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe280101850 0x7fe280103c40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.996+0000 7fe2875ff700 1 --2- 172.21.15.22:0/1776036352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe280104180 0x7fe280106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:04.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.996+0000 7fe2875ff700 1 -- 172.21.15.22:0/1776036352 >> 172.21.15.22:0/1776036352 conn(0x7fe2800fb240 msgr2=0x7fe2800fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:04.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.996+0000 7fe2875ff700 1 -- 172.21.15.22:0/1776036352 shutdown_connections 2023-09-20T22:05:04.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.996+0000 7fe2875ff700 1 -- 172.21.15.22:0/1776036352 wait complete. 2023-09-20T22:05:04.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:04 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4163480263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T22:05:04.999 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:04 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/51723836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T22:05:05.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.997+0000 7fe2875ff700 1 Processor -- start 2023-09-20T22:05:05.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe2875ff700 1 -- start start 2023-09-20T22:05:05.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe2875ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe280101850 0x7fe280198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:05.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe2875ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe280104180 0x7fe280199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:05.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe2875ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe280106ab0 0x7fe2801a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:05.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe2875ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2801a0860 con 0x7fe280104180 2023-09-20T22:05:05.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe2875ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2801a09d0 con 0x7fe280101850 2023-09-20T22:05:05.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe2875ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2801a0b90 con 0x7fe280106ab0 2023-09-20T22:05:05.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe28539b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe280101850 0x7fe280198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:05.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe284b9a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe280104180 0x7fe280199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe28539b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe280101850 0x7fe280198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:36106/0 (socket says 172.21.15.22:36106) 2023-09-20T22:05:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe28539b700 1 -- 172.21.15.22:0/1393655927 learned_addr learned my addr 172.21.15.22:0/1393655927 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.998+0000 7fe285b9c700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe280106ab0 0x7fe2801a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:05.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe28539b700 1 -- 172.21.15.22:0/1393655927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe280106ab0 msgr2=0x7fe2801a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:05.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe28539b700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe280106ab0 0x7fe2801a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe28539b700 1 -- 172.21.15.22:0/1393655927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe280104180 msgr2=0x7fe280199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:05.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe28539b700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe280104180 0x7fe280199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe28539b700 1 -- 172.21.15.22:0/1393655927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c0095f0 con 0x7fe280101850 2023-09-20T22:05:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe285b9c700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe280106ab0 0x7fe2801a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:05:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe28539b700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe280101850 0x7fe280198d20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b680 tx=0x7fe27c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe2767fc700 1 -- 172.21.15.22:0/1393655927 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c01d070 con 0x7fe280101850 2023-09-20T22:05:05.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe2875ff700 1 -- 172.21.15.22:0/1393655927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2801a0e10 con 0x7fe280101850 2023-09-20T22:05:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe2767fc700 1 -- 172.21.15.22:0/1393655927 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe27c022470 con 0x7fe280101850 2023-09-20T22:05:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:04.999+0000 7fe2767fc700 1 -- 172.21.15.22:0/1393655927 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c017650 con 0x7fe280101850 2023-09-20T22:05:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.000+0000 7fe2875ff700 1 -- 172.21.15.22:0/1393655927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2801a1180 con 0x7fe280101850 2023-09-20T22:05:05.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.001+0000 7fe2875ff700 1 -- 172.21.15.22:0/1393655927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe280108680 con 0x7fe280101850 2023-09-20T22:05:05.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.003+0000 7fe2767fc700 1 -- 172.21.15.22:0/1393655927 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fe27c0225e0 con 0x7fe280101850 2023-09-20T22:05:05.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.004+0000 7fe2767fc700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe26c0a1300 0x7fe26c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:05.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.005+0000 7fe2767fc700 1 -- 172.21.15.22:0/1393655927 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe27c0ce5a0 con 0x7fe280101850 2023-09-20T22:05:05.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.005+0000 7fe284b9a700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe26c0a1300 0x7fe26c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:05.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.005+0000 7fe284b9a700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe26c0a1300 0x7fe26c0a37c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe270009c60 tx=0x7fe270009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:05.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.007+0000 7fe2767fc700 1 -- 172.21.15.22:0/1393655927 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fe27c097970 con 0x7fe280101850 2023-09-20T22:05:05.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.277+0000 7fe2875ff700 1 -- 172.21.15.22:0/1393655927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe28010c850 con 0x7fe280101850 2023-09-20T22:05:05.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.278+0000 7fe2767fc700 1 -- 172.21.15.22:0/1393655927 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe27c027020 con 0x7fe280101850 2023-09-20T22:05:05.278 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T22:05:05.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 -- 172.21.15.22:0/1393655927 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe26c0a1300 msgr2=0x7fe26c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:05.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe26c0a1300 0x7fe26c0a37c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe270009c60 tx=0x7fe270009260 comp rx=0 tx=0).stop 2023-09-20T22:05:05.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 -- 172.21.15.22:0/1393655927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe280101850 msgr2=0x7fe280198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:05.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe280101850 0x7fe280198d20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b680 tx=0x7fe27c005740 comp rx=0 tx=0).stop 2023-09-20T22:05:05.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 -- 172.21.15.22:0/1393655927 shutdown_connections 2023-09-20T22:05:05.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe280106ab0 0x7fe2801a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fe26c0a1300 0x7fe26c0a37c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe280101850 0x7fe280198d20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 --2- 172.21.15.22:0/1393655927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe280104180 0x7fe280199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 -- 172.21.15.22:0/1393655927 >> 172.21.15.22:0/1393655927 conn(0x7fe2800fb240 msgr2=0x7fe2800ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 -- 172.21.15.22:0/1393655927 shutdown_connections 2023-09-20T22:05:05.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.281+0000 7fe2875ff700 1 -- 172.21.15.22:0/1393655927 wait complete. 2023-09-20T22:05:05.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-09-20T22:05:05.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:04 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4163480263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T22:05:05.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:04 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/51723836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T22:05:05.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:04 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4163480263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T22:05:05.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:04 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/51723836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T22:05:05.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.534+0000 7fdc7b59e700 1 -- 172.21.15.22:0/2516002981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7c103fb0 msgr2=0x7fdc7c104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:05.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.534+0000 7fdc7b59e700 1 --2- 172.21.15.22:0/2516002981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7c103fb0 0x7fdc7c104430 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009990 tx=0x7fdc70009ca0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.534+0000 7fdc7b59e700 1 -- 172.21.15.22:0/2516002981 shutdown_connections 2023-09-20T22:05:05.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.534+0000 7fdc7b59e700 1 --2- 172.21.15.22:0/2516002981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc7c104970 0x7fdc7c10b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.534+0000 7fdc7b59e700 1 --2- 172.21.15.22:0/2516002981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc7c102e60 0x7fdc7c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.534+0000 7fdc7b59e700 1 --2- 172.21.15.22:0/2516002981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7c103fb0 0x7fdc7c104430 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.534+0000 7fdc7b59e700 1 -- 172.21.15.22:0/2516002981 >> 172.21.15.22:0/2516002981 conn(0x7fdc7c0fe3b0 msgr2=0x7fdc7c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:05.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.534+0000 7fdc7b59e700 1 -- 172.21.15.22:0/2516002981 shutdown_connections 2023-09-20T22:05:05.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.534+0000 7fdc7b59e700 1 -- 172.21.15.22:0/2516002981 wait complete. 2023-09-20T22:05:05.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.536+0000 7fdc7b59e700 1 Processor -- start 2023-09-20T22:05:05.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.536+0000 7fdc7b59e700 1 -- start start 2023-09-20T22:05:05.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.537+0000 7fdc7b59e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc7c102e60 0x7fdc7c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:05.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.537+0000 7fdc7b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc7c103fb0 0x7fdc7c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:05.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.537+0000 7fdc7b59e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7c104970 0x7fdc7c19ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.537+0000 7fdc7b59e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c1a0550 con 0x7fdc7c102e60 2023-09-20T22:05:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.537+0000 7fdc7b59e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c1a0690 con 0x7fdc7c104970 2023-09-20T22:05:05.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.537+0000 7fdc7b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c1a0820 con 0x7fdc7c103fb0 2023-09-20T22:05:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.537+0000 7fdc7a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc7c102e60 0x7fdc7c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.538+0000 7fdc7a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc7c102e60 0x7fdc7c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:51316/0 (socket says 172.21.15.22:51316) 2023-09-20T22:05:05.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.538+0000 7fdc7a59c700 1 -- 172.21.15.22:0/171257051 learned_addr learned my addr 172.21.15.22:0/171257051 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:05.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.538+0000 7fdc7ad9d700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7c104970 0x7fdc7c19ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:05.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.538+0000 7fdc79d9b700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc7c103fb0 0x7fdc7c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:05.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.538+0000 7fdc7a59c700 1 -- 172.21.15.22:0/171257051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc7c103fb0 msgr2=0x7fdc7c198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:05.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.538+0000 7fdc7a59c700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc7c103fb0 0x7fdc7c198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.538+0000 7fdc7a59c700 1 -- 172.21.15.22:0/171257051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7c104970 msgr2=0x7fdc7c19ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:05.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.538+0000 7fdc7a59c700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7c104970 0x7fdc7c19ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:05.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.538+0000 7fdc7a59c700 1 -- 172.21.15.22:0/171257051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc700095f0 con 0x7fdc7c102e60 2023-09-20T22:05:05.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.538+0000 7fdc7a59c700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc7c102e60 0x7fdc7c1989d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00e9a0 tx=0x7fdc6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:05.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.539+0000 7fdc6b7fe700 1 -- 172.21.15.22:0/171257051 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c009790 con 0x7fdc7c102e60 2023-09-20T22:05:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.539+0000 7fdc6b7fe700 1 -- 172.21.15.22:0/171257051 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc6c010430 con 0x7fdc7c102e60 2023-09-20T22:05:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.539+0000 7fdc7b59e700 1 -- 172.21.15.22:0/171257051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c1a0b00 con 0x7fdc7c102e60 2023-09-20T22:05:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.539+0000 7fdc6b7fe700 1 -- 172.21.15.22:0/171257051 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c018680 con 0x7fdc7c102e60 2023-09-20T22:05:05.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.539+0000 7fdc7b59e700 1 -- 172.21.15.22:0/171257051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c1a1060 con 0x7fdc7c102e60 2023-09-20T22:05:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.540+0000 7fdc7b59e700 1 -- 172.21.15.22:0/171257051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c109c90 con 0x7fdc7c102e60 2023-09-20T22:05:05.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.542+0000 7fdc6b7fe700 1 -- 172.21.15.22:0/171257051 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fdc6c0105a0 con 0x7fdc7c102e60 2023-09-20T22:05:05.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.544+0000 7fdc6b7fe700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdc640a1470 0x7fdc640a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:05.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.544+0000 7fdc6b7fe700 1 -- 172.21.15.22:0/171257051 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fdc6c014070 con 0x7fdc7c102e60 2023-09-20T22:05:05.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.544+0000 7fdc79d9b700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdc640a1470 0x7fdc640a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:05.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.545+0000 7fdc79d9b700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdc640a1470 0x7fdc640a3930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000b2e0 tx=0x7fdc70005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:05.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.549+0000 7fdc6b7fe700 1 -- 172.21.15.22:0/171257051 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fdc6c051e10 con 0x7fdc7c102e60 2023-09-20T22:05:05.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.935+0000 7fdc7b59e700 1 -- 172.21.15.22:0/171257051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fdc7c10e6b0 con 0x7fdc7c102e60 2023-09-20T22:05:05.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.947+0000 7fdc6b7fe700 1 -- 172.21.15.22:0/171257051 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v16) v1 ==== 81+0+733654 (secure 0 0 0) 0x7fdc6c096430 con 0x7fdc7c102e60 2023-09-20T22:05:05.957 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:05:05.957 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:05:05.957 INFO:teuthology.orchestra.run.smithi022.stdout: "epoch": 16, 2023-09-20T22:05:05.957 INFO:teuthology.orchestra.run.smithi022.stdout: "active_gid": 14154, 2023-09-20T22:05:05.958 INFO:teuthology.orchestra.run.smithi022.stdout: "active_name": "x", 2023-09-20T22:05:05.958 INFO:teuthology.orchestra.run.smithi022.stdout: "active_addrs": { 2023-09-20T22:05:05.959 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2023-09-20T22:05:05.959 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:05.959 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2023-09-20T22:05:05.960 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6800", 2023-09-20T22:05:05.960 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 718010003 2023-09-20T22:05:05.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:05.960 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v1", 2023-09-20T22:05:05.961 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:6801", 2023-09-20T22:05:05.961 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 718010003 2023-09-20T22:05:05.961 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:05.961 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-09-20T22:05:05.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.961 INFO:teuthology.orchestra.run.smithi022.stdout: "active_addr": "172.21.15.22:6801/718010003", 2023-09-20T22:05:05.962 INFO:teuthology.orchestra.run.smithi022.stdout: "active_change": "2023-09-20T21:40:07.017297+0000", 2023-09-20T22:05:05.962 INFO:teuthology.orchestra.run.smithi022.stdout: "active_mgr_features": 4540138322906710015, 2023-09-20T22:05:05.962 INFO:teuthology.orchestra.run.smithi022.stdout: "available": true, 2023-09-20T22:05:05.962 INFO:teuthology.orchestra.run.smithi022.stdout: "standbys": [ 2023-09-20T22:05:05.962 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:05.963 INFO:teuthology.orchestra.run.smithi022.stdout: "gid": 24107, 2023-09-20T22:05:05.963 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "y", 2023-09-20T22:05:05.963 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_features": 4540138322906710015, 2023-09-20T22:05:05.963 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2023-09-20T22:05:05.963 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:05.964 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2023-09-20T22:05:05.964 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:05.964 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:05.964 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:05.964 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:05.965 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:05.965 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:05.965 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.965 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.965 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:05.965 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.966 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.966 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:05.966 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2023-09-20T22:05:05.966 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:05.966 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:05.967 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:05.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.967 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:05.967 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:05.968 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:05.968 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.968 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.968 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:05.968 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.969 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.969 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:05.969 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:05.969 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:05.969 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:05.970 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:05.970 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:05.970 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:05.970 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:05.970 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:05.971 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:05.971 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:05.971 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:05.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.971 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:05.972 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:05.972 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:05.972 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.972 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.972 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:05.973 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.973 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.973 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:05.973 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:05.973 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:05.974 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:05.974 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:05.974 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.974 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:05.974 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:05.975 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:05.975 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.975 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.975 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:05.975 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.976 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.976 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:05.976 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:05.976 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:05.976 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:05.977 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:05.977 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:05.977 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:05.977 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:05.978 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:05.978 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:05.978 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:05.978 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:05.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.979 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:05.979 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:05.979 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:05.979 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.980 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.980 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:05.980 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.980 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.980 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:05.980 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:05.981 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:05.981 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:05.981 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:05.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.981 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2023-09-20T22:05:05.982 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2023-09-20T22:05:05.982 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:05.982 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.982 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.982 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:05.983 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.983 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.984 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:05.985 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2023-09-20T22:05:05.985 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:05.985 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:05.986 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:05.986 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.986 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2023-09-20T22:05:05.986 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2023-09-20T22:05:05.987 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:05.987 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.987 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.987 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2023-09-20T22:05:05.987 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.988 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.988 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:05.988 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2023-09-20T22:05:05.988 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:05.988 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:05.989 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:05.989 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.989 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2023-09-20T22:05:05.989 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2023-09-20T22:05:05.989 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:05.989 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.990 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.990 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:05.990 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.990 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.990 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:05.991 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2023-09-20T22:05:05.991 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:05.991 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:05.991 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:05.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.991 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2023-09-20T22:05:05.992 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2023-09-20T22:05:05.992 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:05.992 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.992 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.992 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:05.993 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.993 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.993 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:05.993 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2023-09-20T22:05:05.993 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:05.993 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:05.994 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:05.994 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.994 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2023-09-20T22:05:05.994 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2023-09-20T22:05:05.995 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:05.995 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.995 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.995 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2023-09-20T22:05:05.995 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.995 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.996 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:05.996 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2023-09-20T22:05:05.996 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:05.996 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:05.996 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:05.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:05.997 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2023-09-20T22:05:05.997 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2023-09-20T22:05:05.997 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:05.997 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:05.998 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:05.998 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:05.998 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:05.998 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:05.998 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:05.999 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2023-09-20T22:05:05.999 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.000 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.000 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.000 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2023-09-20T22:05:06.001 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2023-09-20T22:05:06.001 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.001 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.001 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.001 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.002 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.002 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.002 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.002 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-20T22:05:06.002 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.003 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.003 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.003 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2023-09-20T22:05:06.003 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2023-09-20T22:05:06.003 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.004 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.004 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.004 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.004 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.004 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.005 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.005 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2023-09-20T22:05:06.005 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.005 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.005 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.006 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.006 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.006 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2023-09-20T22:05:06.007 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.007 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.007 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.007 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2023-09-20T22:05:06.007 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2023-09-20T22:05:06.008 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.008 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.008 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.008 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.009 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.009 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.009 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.009 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2023-09-20T22:05:06.009 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.010 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.010 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.011 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2023-09-20T22:05:06.011 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2023-09-20T22:05:06.011 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.011 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.011 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.012 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2023-09-20T22:05:06.012 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.012 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.012 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.013 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2023-09-20T22:05:06.013 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-20T22:05:06.013 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.013 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.014 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2023-09-20T22:05:06.014 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2023-09-20T22:05:06.015 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:06.015 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.015 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.015 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:06.015 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2023-09-20T22:05:06.016 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2023-09-20T22:05:06.016 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.016 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-20T22:05:06.016 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-20T22:05:06.017 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.017 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.017 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2023-09-20T22:05:06.018 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2023-09-20T22:05:06.018 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:06.018 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.018 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.019 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2023-09-20T22:05:06.019 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:06.019 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2023-09-20T22:05:06.019 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.020 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-20T22:05:06.020 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.021 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.021 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.021 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2023-09-20T22:05:06.021 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2023-09-20T22:05:06.021 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.022 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.022 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.022 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2023-09-20T22:05:06.023 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.023 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.023 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.023 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-20T22:05:06.023 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-20T22:05:06.023 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.024 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.024 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2023-09-20T22:05:06.024 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2023-09-20T22:05:06.024 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:06.025 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.025 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.025 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2023-09-20T22:05:06.025 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2023-09-20T22:05:06.025 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2023-09-20T22:05:06.026 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.026 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2023-09-20T22:05:06.026 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-20T22:05:06.026 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.026 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.027 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2023-09-20T22:05:06.027 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2023-09-20T22:05:06.027 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.027 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.028 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.028 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2023-09-20T22:05:06.028 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.028 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.028 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.029 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2023-09-20T22:05:06.029 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-20T22:05:06.029 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.029 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.030 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2023-09-20T22:05:06.030 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2023-09-20T22:05:06.030 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:06.030 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.031 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.031 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:06.031 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2023-09-20T22:05:06.031 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2023-09-20T22:05:06.032 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.032 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-20T22:05:06.032 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-20T22:05:06.032 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.032 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.033 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.033 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.033 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.033 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.034 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.034 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.035 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.035 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.035 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.035 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.035 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.036 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.036 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.036 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.036 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.036 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.037 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.037 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.037 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.037 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.037 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.038 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.038 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.038 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.038 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.038 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.039 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.039 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.039 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.039 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.040 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.040 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.040 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.040 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.041 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.041 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.041 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.041 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.042 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.042 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.042 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.043 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.043 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.043 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.043 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.043 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.044 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.044 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.044 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.044 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.044 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.045 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.045 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.045 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.045 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.046 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.046 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.046 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.046 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.047 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.047 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.047 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.047 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.047 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.048 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.048 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.048 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.049 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2023-09-20T22:05:06.049 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2023-09-20T22:05:06.049 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:06.049 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.050 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.050 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:06.050 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.050 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.051 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.051 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-20T22:05:06.051 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.051 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.051 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.052 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2023-09-20T22:05:06.052 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2023-09-20T22:05:06.052 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.053 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.053 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.053 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2023-09-20T22:05:06.053 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.054 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.054 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.054 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2023-09-20T22:05:06.054 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2023-09-20T22:05:06.055 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap" 2023-09-20T22:05:06.055 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.055 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2023-09-20T22:05:06.055 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.056 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.056 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.056 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2023-09-20T22:05:06.056 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2023-09-20T22:05:06.057 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.057 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.057 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.057 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.057 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.058 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.058 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.058 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-20T22:05:06.058 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.058 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.059 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.059 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:06.059 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:06.059 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.060 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.060 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.060 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:06.060 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.060 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.061 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.061 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-20T22:05:06.061 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.061 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.061 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.061 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.062 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2023-09-20T22:05:06.062 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2023-09-20T22:05:06.062 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.062 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.062 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.063 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:06.063 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:06.063 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.063 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.063 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-20T22:05:06.064 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-20T22:05:06.064 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.064 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.065 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2023-09-20T22:05:06.065 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2023-09-20T22:05:06.065 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:06.065 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.065 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.066 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:06.066 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.066 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.066 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.066 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-20T22:05:06.066 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.067 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.067 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.067 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.067 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.068 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.068 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.068 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2023-09-20T22:05:06.068 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.068 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.068 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.069 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2023-09-20T22:05:06.069 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2023-09-20T22:05:06.069 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:06.069 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.069 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.070 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2023-09-20T22:05:06.070 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.070 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.070 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.070 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-20T22:05:06.070 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.071 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.071 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.071 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2023-09-20T22:05:06.071 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2023-09-20T22:05:06.072 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.072 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.072 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.072 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2023-09-20T22:05:06.072 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.073 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.073 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.073 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-20T22:05:06.073 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.073 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.073 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.074 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2023-09-20T22:05:06.074 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2023-09-20T22:05:06.074 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.075 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.075 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.075 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2023-09-20T22:05:06.075 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.076 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.076 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.076 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-20T22:05:06.076 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.076 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.077 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.077 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2023-09-20T22:05:06.078 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2023-09-20T22:05:06.078 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.078 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.081 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.081 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.081 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.081 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.081 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.082 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-20T22:05:06.082 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-20T22:05:06.082 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.082 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.082 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.083 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2023-09-20T22:05:06.083 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2023-09-20T22:05:06.083 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.084 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.084 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.084 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:06.084 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.084 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.085 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.085 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-20T22:05:06.085 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.085 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.086 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.086 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2023-09-20T22:05:06.086 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2023-09-20T22:05:06.087 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:06.087 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.087 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.087 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2023-09-20T22:05:06.088 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.088 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.088 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.088 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-20T22:05:06.089 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.089 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.089 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.090 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2023-09-20T22:05:06.090 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2023-09-20T22:05:06.090 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.090 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.090 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.091 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.091 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.091 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.091 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.091 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-20T22:05:06.092 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.092 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.092 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.092 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2023-09-20T22:05:06.093 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2023-09-20T22:05:06.093 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.093 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.093 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.093 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.094 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.094 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.094 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.094 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-20T22:05:06.094 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.095 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.095 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.095 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2023-09-20T22:05:06.095 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2023-09-20T22:05:06.096 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.096 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.096 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.096 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.096 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.097 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.097 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.097 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-20T22:05:06.097 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.097 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.098 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.098 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2023-09-20T22:05:06.098 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2023-09-20T22:05:06.098 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.099 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.099 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.099 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-20T22:05:06.099 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.099 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.100 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.100 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:06.100 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.100 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.100 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.101 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2023-09-20T22:05:06.101 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2023-09-20T22:05:06.101 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.102 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.102 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.102 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-20T22:05:06.102 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.102 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.103 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.103 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2023-09-20T22:05:06.103 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.103 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.103 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.104 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2023-09-20T22:05:06.104 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2023-09-20T22:05:06.104 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.105 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.105 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.105 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-20T22:05:06.105 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.105 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.106 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.106 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2023-09-20T22:05:06.106 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.106 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.106 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.107 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2023-09-20T22:05:06.107 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2023-09-20T22:05:06.107 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.107 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.108 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.108 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-20T22:05:06.108 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.108 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.108 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.109 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:06.109 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.109 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.109 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.110 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.110 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2023-09-20T22:05:06.110 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2023-09-20T22:05:06.110 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.110 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.111 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.111 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-20T22:05:06.111 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.111 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.112 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.112 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2023-09-20T22:05:06.112 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.112 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.112 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.113 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.113 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2023-09-20T22:05:06.113 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2023-09-20T22:05:06.113 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.113 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.114 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.114 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-20T22:05:06.114 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.114 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.114 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2023-09-20T22:05:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.115 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2023-09-20T22:05:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2023-09-20T22:05:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.116 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-20T22:05:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.117 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2023-09-20T22:05:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.118 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2023-09-20T22:05:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2023-09-20T22:05:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.119 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-20T22:05:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2023-09-20T22:05:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.120 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2023-09-20T22:05:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2023-09-20T22:05:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.121 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-20T22:05:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.122 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2023-09-20T22:05:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2023-09-20T22:05:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2023-09-20T22:05:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.124 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-20T22:05:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.125 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2023-09-20T22:05:06.126 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.126 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.126 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.126 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2023-09-20T22:05:06.127 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2023-09-20T22:05:06.127 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.127 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.127 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.127 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-20T22:05:06.127 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.128 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.128 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.128 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:06.128 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.128 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.129 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.129 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.129 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2023-09-20T22:05:06.129 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2023-09-20T22:05:06.129 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.129 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.130 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.130 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-20T22:05:06.130 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.130 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.130 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.131 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2023-09-20T22:05:06.131 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.131 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.131 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.131 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.132 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2023-09-20T22:05:06.132 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2023-09-20T22:05:06.132 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.132 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.132 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.132 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-20T22:05:06.133 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.133 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.133 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.133 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:06.133 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.134 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.134 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.134 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2023-09-20T22:05:06.134 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2023-09-20T22:05:06.134 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.135 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.135 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.135 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-20T22:05:06.135 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.135 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.136 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.136 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2023-09-20T22:05:06.136 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.136 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.136 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.137 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2023-09-20T22:05:06.137 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2023-09-20T22:05:06.137 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.137 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.138 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.138 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-20T22:05:06.138 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.138 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.138 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.139 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2023-09-20T22:05:06.139 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.139 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.139 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.139 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2023-09-20T22:05:06.140 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2023-09-20T22:05:06.140 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.140 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.140 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.140 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.141 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.141 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.141 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.141 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2023-09-20T22:05:06.141 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.142 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.142 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.142 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2023-09-20T22:05:06.142 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2023-09-20T22:05:06.143 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.143 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.143 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.143 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:06.143 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.143 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.144 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.144 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-20T22:05:06.144 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.144 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.144 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.145 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2023-09-20T22:05:06.145 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2023-09-20T22:05:06.145 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.145 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.146 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.146 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2023-09-20T22:05:06.146 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.146 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.146 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.147 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-20T22:05:06.147 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.147 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.147 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.148 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2023-09-20T22:05:06.148 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2023-09-20T22:05:06.148 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.148 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.148 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.149 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2023-09-20T22:05:06.149 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.149 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.149 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.149 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-20T22:05:06.149 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.150 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.150 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.150 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2023-09-20T22:05:06.150 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2023-09-20T22:05:06.151 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.151 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.151 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.151 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2023-09-20T22:05:06.151 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.151 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.152 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.152 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2023-09-20T22:05:06.152 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.152 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.152 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.153 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2023-09-20T22:05:06.153 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2023-09-20T22:05:06.153 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.153 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.153 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.154 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.154 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.154 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.154 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.154 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-20T22:05:06.155 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.155 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.155 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.155 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.155 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2023-09-20T22:05:06.156 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2023-09-20T22:05:06.156 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.156 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.156 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.156 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:06.156 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.157 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.157 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.157 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2023-09-20T22:05:06.157 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.157 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.158 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.158 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.158 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2023-09-20T22:05:06.158 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2023-09-20T22:05:06.158 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.159 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.159 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.159 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:06.159 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.159 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.159 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.160 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2023-09-20T22:05:06.160 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.160 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.160 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.161 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2023-09-20T22:05:06.161 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2023-09-20T22:05:06.161 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.161 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.161 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.162 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.162 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.162 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.162 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.162 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-20T22:05:06.163 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.163 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.163 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.163 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.163 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.164 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.164 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.164 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.164 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.164 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.165 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.165 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.165 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.165 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.165 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.166 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.166 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.166 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.166 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.166 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.167 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.167 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.167 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.167 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2023-09-20T22:05:06.167 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2023-09-20T22:05:06.168 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.168 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.168 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.168 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.168 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.169 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.169 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.169 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-20T22:05:06.169 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.169 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.170 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.170 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.170 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.170 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.171 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.171 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.171 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.171 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.171 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.172 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.172 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-20T22:05:06.172 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.172 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.172 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.173 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.174 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.174 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.174 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.174 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.174 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.175 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.175 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.175 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.175 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.175 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.176 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.176 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.176 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.176 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.176 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.177 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.177 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.177 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.177 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.177 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.177 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.178 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.178 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.178 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.178 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.178 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.179 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.179 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.179 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.179 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-20T22:05:06.180 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-20T22:05:06.180 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.180 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.180 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.180 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.181 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.181 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.181 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.181 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-20T22:05:06.182 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.182 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.182 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.182 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-20T22:05:06.183 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-20T22:05:06.183 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.183 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.183 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.183 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2023-09-20T22:05:06.183 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.184 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.184 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.184 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-20T22:05:06.184 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.184 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.185 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.185 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2023-09-20T22:05:06.185 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2023-09-20T22:05:06.185 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.186 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.186 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.186 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:06.186 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.186 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.187 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.187 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2023-09-20T22:05:06.187 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.187 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.187 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.188 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2023-09-20T22:05:06.188 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2023-09-20T22:05:06.188 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.188 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.188 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.189 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:06.189 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.189 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.189 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.189 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-20T22:05:06.190 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.190 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.190 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.190 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2023-09-20T22:05:06.191 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2023-09-20T22:05:06.191 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.191 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.191 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.191 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.192 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.192 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.192 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.192 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2023-09-20T22:05:06.192 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.192 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.193 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.193 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2023-09-20T22:05:06.193 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2023-09-20T22:05:06.193 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.194 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.194 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.194 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2023-09-20T22:05:06.194 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.194 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.195 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.195 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2023-09-20T22:05:06.195 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2023-09-20T22:05:06.195 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.195 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2023-09-20T22:05:06.195 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.196 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.196 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.196 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2023-09-20T22:05:06.197 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2023-09-20T22:05:06.197 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.197 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.197 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.197 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-20T22:05:06.197 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.198 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.198 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.198 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-20T22:05:06.198 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.198 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.199 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.199 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2023-09-20T22:05:06.199 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2023-09-20T22:05:06.199 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.200 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.200 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.200 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.200 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.200 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.200 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.201 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-20T22:05:06.201 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.201 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.201 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.202 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2023-09-20T22:05:06.202 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2023-09-20T22:05:06.202 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.202 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.202 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.203 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.203 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.203 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.203 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.203 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-20T22:05:06.204 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.204 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.204 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.204 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2023-09-20T22:05:06.204 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2023-09-20T22:05:06.205 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.205 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.205 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.205 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.205 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.206 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.206 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.206 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-20T22:05:06.206 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.206 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.207 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.207 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.207 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2023-09-20T22:05:06.207 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2023-09-20T22:05:06.207 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.208 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.208 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.208 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.208 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.208 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.209 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.209 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-20T22:05:06.209 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.209 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.209 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.210 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2023-09-20T22:05:06.210 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2023-09-20T22:05:06.210 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.210 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.210 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.211 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.211 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.211 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.211 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.211 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-20T22:05:06.212 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.212 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.212 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.212 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2023-09-20T22:05:06.213 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2023-09-20T22:05:06.213 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.213 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.213 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.213 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2023-09-20T22:05:06.214 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.214 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.214 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.214 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2023-09-20T22:05:06.214 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.215 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.215 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.215 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.215 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2023-09-20T22:05:06.215 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2023-09-20T22:05:06.216 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.216 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.216 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.216 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.216 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.217 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.217 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.217 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-20T22:05:06.217 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.217 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.217 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.218 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.218 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2023-09-20T22:05:06.218 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2023-09-20T22:05:06.218 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.218 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.219 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.219 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.219 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.219 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.219 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.220 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-20T22:05:06.220 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.220 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.220 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.221 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2023-09-20T22:05:06.221 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2023-09-20T22:05:06.221 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.221 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.221 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.222 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.222 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.222 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.222 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.222 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-20T22:05:06.223 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.223 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.223 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.223 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2023-09-20T22:05:06.223 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2023-09-20T22:05:06.224 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.224 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.224 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.224 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.224 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.224 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.225 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.225 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2023-09-20T22:05:06.225 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.225 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.225 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.226 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.226 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2023-09-20T22:05:06.226 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2023-09-20T22:05:06.226 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.226 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.227 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.227 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.227 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.227 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.227 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.228 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-20T22:05:06.228 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.228 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.228 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.229 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2023-09-20T22:05:06.229 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2023-09-20T22:05:06.229 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.229 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.229 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.229 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.230 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.230 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.230 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.230 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-20T22:05:06.230 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.231 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.231 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.231 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.231 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.231 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.232 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.232 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2023-09-20T22:05:06.232 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.232 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.232 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.232 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.233 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.233 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.233 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.233 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.233 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.234 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.234 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.234 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.234 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.234 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.235 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.235 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.235 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.235 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.235 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.236 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.236 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.236 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.236 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.236 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.237 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.237 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.237 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.237 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.237 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.238 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.238 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.238 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.238 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.238 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.238 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.239 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.239 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.239 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.239 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.239 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.240 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.240 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.240 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.240 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.240 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.241 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.241 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.241 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.241 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.241 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.241 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.242 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.242 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.242 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.242 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.242 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.243 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.243 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.243 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.243 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.244 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.244 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.244 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.244 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.244 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.244 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.245 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.245 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.245 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.245 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.245 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.246 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2023-09-20T22:05:06.246 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2023-09-20T22:05:06.246 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.246 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.247 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.247 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2023-09-20T22:05:06.247 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.247 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.247 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.247 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-20T22:05:06.248 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.248 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.248 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.248 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.249 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2023-09-20T22:05:06.249 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2023-09-20T22:05:06.249 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.249 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.249 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.249 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2023-09-20T22:05:06.250 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.250 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.250 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.250 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-20T22:05:06.250 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.251 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.251 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.251 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.251 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.251 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.252 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.252 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2023-09-20T22:05:06.252 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.252 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.252 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.253 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-20T22:05:06.253 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-20T22:05:06.253 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.253 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.253 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.253 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:06.254 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.254 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.254 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.254 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.254 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.255 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.255 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.255 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2023-09-20T22:05:06.255 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-20T22:05:06.256 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.256 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.256 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.256 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.256 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.257 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.257 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.257 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.257 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.257 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.258 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.258 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-20T22:05:06.258 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-20T22:05:06.258 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.259 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.259 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.259 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.259 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.259 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.259 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.260 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.260 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.260 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.260 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.260 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.261 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2023-09-20T22:05:06.261 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2023-09-20T22:05:06.261 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.261 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.261 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.262 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.262 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.262 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.262 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.262 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.263 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.263 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.263 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.263 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-20T22:05:06.264 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-20T22:05:06.264 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.264 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.264 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.264 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.265 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.265 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.265 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.265 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.265 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.265 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.266 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.266 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.266 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2023-09-20T22:05:06.266 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-20T22:05:06.267 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.267 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.267 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.267 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.267 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.267 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.268 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.268 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.268 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.268 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.268 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.269 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-20T22:05:06.269 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-20T22:05:06.269 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.269 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.270 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.270 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.270 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.270 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.270 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.270 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.271 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.271 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.271 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.271 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-20T22:05:06.272 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-20T22:05:06.272 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.272 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.272 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.272 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.272 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.273 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.273 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.273 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.273 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.273 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.274 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.274 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.274 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-20T22:05:06.274 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-20T22:05:06.274 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.275 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.275 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.275 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.275 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.275 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.276 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.276 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.276 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.276 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.276 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.276 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.277 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-20T22:05:06.277 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-20T22:05:06.277 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.277 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.277 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.278 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.278 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.278 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.278 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.278 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.279 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.279 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.279 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.279 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-20T22:05:06.279 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-20T22:05:06.280 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.280 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.280 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.280 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.280 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.280 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.281 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.281 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.281 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.281 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.281 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.282 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-20T22:05:06.282 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-20T22:05:06.282 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.285 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.285 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.285 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.285 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.285 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.286 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.286 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.286 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.286 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.286 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.287 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-20T22:05:06.287 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-20T22:05:06.287 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.287 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.288 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.288 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.288 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.288 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.288 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.289 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.289 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.289 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.289 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.289 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-20T22:05:06.290 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-20T22:05:06.290 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.290 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.290 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.290 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.291 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.291 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.291 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.291 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.291 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.292 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.292 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.292 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2023-09-20T22:05:06.292 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-20T22:05:06.292 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.293 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.293 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.293 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2023-09-20T22:05:06.293 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.293 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.294 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.294 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.294 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.294 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.294 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.295 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-20T22:05:06.295 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-20T22:05:06.295 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.295 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.295 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.296 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.296 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.296 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.296 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.296 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.297 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.297 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.297 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.297 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.297 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2023-09-20T22:05:06.297 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2023-09-20T22:05:06.298 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.298 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.298 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.298 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.298 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.299 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.299 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.299 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.299 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.299 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.300 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.300 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.300 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2023-09-20T22:05:06.300 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-20T22:05:06.300 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.300 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.301 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.301 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2023-09-20T22:05:06.301 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.301 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.301 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.302 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.302 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.302 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.302 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.303 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-20T22:05:06.303 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-20T22:05:06.303 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.303 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.303 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.303 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.304 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.304 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.304 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.304 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.304 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.305 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.305 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.305 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.305 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-20T22:05:06.305 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-20T22:05:06.306 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.306 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.306 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.306 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.306 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.307 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.307 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.307 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.307 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.307 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.307 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.308 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.308 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-20T22:05:06.308 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-20T22:05:06.308 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.308 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.309 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.309 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.309 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.309 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.309 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.310 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.310 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.310 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.310 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.310 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.310 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-20T22:05:06.311 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-20T22:05:06.311 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.311 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.311 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.311 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.312 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.312 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.312 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.312 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.312 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.313 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.313 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.313 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2023-09-20T22:05:06.313 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-20T22:05:06.313 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.314 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.314 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.314 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.314 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.314 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.315 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.315 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.315 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.315 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.315 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.316 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-20T22:05:06.316 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-20T22:05:06.316 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.316 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.317 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.317 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.317 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.317 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.317 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.318 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.318 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.318 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.318 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.319 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-20T22:05:06.319 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-20T22:05:06.319 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.319 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.319 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.319 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.320 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.320 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.320 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.320 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.320 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.321 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.321 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.321 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.321 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-20T22:05:06.321 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-20T22:05:06.322 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.322 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.322 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.322 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.322 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.323 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.323 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.323 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.323 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.323 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.323 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.324 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-20T22:05:06.324 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-20T22:05:06.324 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.324 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.325 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.325 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.325 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.325 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.325 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.325 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.326 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.326 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.326 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.326 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.326 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-20T22:05:06.327 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-20T22:05:06.327 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.327 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.327 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.327 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.328 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.328 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.328 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.328 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.328 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.329 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.329 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.329 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.329 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-20T22:05:06.329 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-20T22:05:06.330 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.330 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.330 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.330 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.330 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.331 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.331 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.331 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.331 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.331 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.331 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.332 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-20T22:05:06.332 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-20T22:05:06.332 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.332 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.333 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.333 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.333 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.333 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.333 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.334 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.334 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.334 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.334 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.335 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-20T22:05:06.335 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-20T22:05:06.335 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.335 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.335 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.335 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.336 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.336 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.336 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.336 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.336 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.337 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.337 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.337 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2023-09-20T22:05:06.337 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-20T22:05:06.338 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.338 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.338 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.338 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.338 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.339 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.339 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.339 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.339 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.339 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.340 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.340 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.340 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-20T22:05:06.340 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-20T22:05:06.340 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.340 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.341 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.341 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-20T22:05:06.341 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.341 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.341 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.342 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.342 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.342 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.342 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.343 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-20T22:05:06.343 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-20T22:05:06.343 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.343 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.343 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.344 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:06.344 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.344 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.344 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.344 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.345 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.345 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.345 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.345 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.345 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-20T22:05:06.346 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-20T22:05:06.346 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.346 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.346 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.346 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2023-09-20T22:05:06.347 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.347 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.347 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.347 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.347 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.347 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.348 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.348 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.348 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-20T22:05:06.348 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-20T22:05:06.348 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.349 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.349 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.349 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2023-09-20T22:05:06.349 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.349 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.350 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.350 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.350 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.350 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.350 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.351 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2023-09-20T22:05:06.351 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-20T22:05:06.351 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.351 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.351 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.352 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.352 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.352 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.352 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.352 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.352 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.353 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.353 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.353 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.353 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-20T22:05:06.353 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-20T22:05:06.354 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.354 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.354 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.354 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2023-09-20T22:05:06.354 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.355 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.355 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.355 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.355 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.355 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.356 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.356 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2023-09-20T22:05:06.356 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-20T22:05:06.356 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.357 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.357 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.357 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.357 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.357 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.357 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.358 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.358 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.358 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.358 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.358 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.359 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2023-09-20T22:05:06.359 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-20T22:05:06.359 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.359 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.359 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.360 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.360 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.360 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.360 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.360 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.361 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.361 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.361 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.361 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-20T22:05:06.361 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-20T22:05:06.362 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.362 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.362 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.362 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:06.362 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.363 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.363 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.363 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.363 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.363 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.363 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.364 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.364 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-20T22:05:06.364 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-20T22:05:06.364 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.364 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.365 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.365 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:06.365 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.365 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.365 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.366 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.366 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.366 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.366 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.366 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.366 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-20T22:05:06.367 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-20T22:05:06.367 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.367 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.367 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.367 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:06.368 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.368 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.368 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.368 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.368 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.368 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.369 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.369 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.369 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2023-09-20T22:05:06.369 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2023-09-20T22:05:06.369 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.370 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.370 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.370 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.370 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.370 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.371 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.371 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.371 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.371 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.371 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.372 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.372 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2023-09-20T22:05:06.372 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2023-09-20T22:05:06.372 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.372 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.372 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.373 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.373 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.373 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.373 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.373 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.374 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.374 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.374 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.374 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.374 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2023-09-20T22:05:06.374 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2023-09-20T22:05:06.375 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.375 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.375 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.375 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.375 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.376 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.376 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.376 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2023-09-20T22:05:06.376 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.376 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.377 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.377 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2023-09-20T22:05:06.377 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2023-09-20T22:05:06.377 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.377 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.378 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.378 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2023-09-20T22:05:06.378 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.378 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.378 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.379 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.379 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.379 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.379 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.379 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.380 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2023-09-20T22:05:06.380 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2023-09-20T22:05:06.380 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.380 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.380 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.381 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.381 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.381 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.381 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.381 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.381 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.382 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.382 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.382 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.382 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.382 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.383 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.383 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.383 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.383 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.383 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.384 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.384 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.384 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.384 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.384 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.384 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.385 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.385 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.385 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.385 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.385 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.386 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.386 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.386 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.386 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.386 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.387 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.387 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.387 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.387 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.387 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.387 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.388 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.388 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.388 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.388 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.388 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.389 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.389 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.389 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.389 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.389 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.389 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.390 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.390 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.390 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.390 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.390 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.391 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.391 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.391 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.391 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.391 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.392 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.392 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.392 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.392 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.392 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.393 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.393 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.393 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.393 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.393 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.394 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.394 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.394 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.394 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.394 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.394 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.395 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.395 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.395 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.395 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.395 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2023-09-20T22:05:06.396 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2023-09-20T22:05:06.396 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.396 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.396 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.396 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.397 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.397 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.397 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.397 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2023-09-20T22:05:06.397 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.398 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.398 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.398 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.398 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2023-09-20T22:05:06.398 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2023-09-20T22:05:06.399 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.399 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.399 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.399 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.399 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.399 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.400 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.400 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.400 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.400 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.400 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.401 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.401 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2023-09-20T22:05:06.401 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2023-09-20T22:05:06.401 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.401 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.402 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.402 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2023-09-20T22:05:06.402 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.402 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.402 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.403 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.403 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.403 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.403 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.403 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.403 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2023-09-20T22:05:06.404 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2023-09-20T22:05:06.404 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.404 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.404 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.404 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2023-09-20T22:05:06.405 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.405 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.405 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.405 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.405 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.406 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.406 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.406 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.406 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2023-09-20T22:05:06.406 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2023-09-20T22:05:06.407 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.407 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.407 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.407 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.407 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.408 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.408 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.408 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.408 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.408 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.409 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.409 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.409 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2023-09-20T22:05:06.409 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2023-09-20T22:05:06.409 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.410 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.410 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.410 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2023-09-20T22:05:06.410 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.410 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.411 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.411 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.411 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.411 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.411 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.412 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2023-09-20T22:05:06.412 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2023-09-20T22:05:06.412 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.412 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.412 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.413 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2023-09-20T22:05:06.413 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.413 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.413 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.413 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.414 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2023-09-20T22:05:06.414 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.414 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.414 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.414 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.415 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.415 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.415 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2023-09-20T22:05:06.415 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2023-09-20T22:05:06.415 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.416 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.416 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.416 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2023-09-20T22:05:06.416 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2023-09-20T22:05:06.416 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2023-09-20T22:05:06.417 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.417 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.417 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.417 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.417 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.417 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.418 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2023-09-20T22:05:06.418 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2023-09-20T22:05:06.418 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.418 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.418 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.419 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.419 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.419 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.419 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.419 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.420 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.420 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.420 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.420 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.420 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.421 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.421 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2023-09-20T22:05:06.421 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.421 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.421 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.422 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2023-09-20T22:05:06.422 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2023-09-20T22:05:06.422 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.422 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.422 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.423 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.423 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.423 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.423 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.423 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2023-09-20T22:05:06.423 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.424 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.424 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.424 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.424 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.424 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.425 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.425 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.425 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.425 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.425 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.426 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.426 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.426 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.426 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.426 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.427 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.427 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.427 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.427 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.427 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.428 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.428 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.428 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.428 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.428 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.429 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.429 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.429 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.429 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.429 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.430 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.430 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.430 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.430 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.430 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.431 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.431 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.431 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.431 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.431 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.432 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.432 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.432 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.432 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.432 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.433 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.433 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.433 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.433 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.433 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.433 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.434 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.434 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.434 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.434 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.434 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.435 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.435 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.435 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.435 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.436 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.436 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.436 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.436 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.436 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.436 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.437 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.437 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.437 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.437 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2023-09-20T22:05:06.438 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2023-09-20T22:05:06.438 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.438 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.438 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.438 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2023-09-20T22:05:06.438 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.439 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.439 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.439 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-20T22:05:06.439 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.439 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.440 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.440 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2023-09-20T22:05:06.440 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2023-09-20T22:05:06.440 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.441 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.441 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.441 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2023-09-20T22:05:06.441 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.441 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.441 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.442 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-20T22:05:06.442 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.442 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.442 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.442 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.443 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2023-09-20T22:05:06.443 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2023-09-20T22:05:06.443 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.443 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.443 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.444 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2023-09-20T22:05:06.444 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.444 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.444 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.444 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2023-09-20T22:05:06.444 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.445 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.445 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.445 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2023-09-20T22:05:06.445 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2023-09-20T22:05:06.446 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.446 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.446 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.446 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2023-09-20T22:05:06.446 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.447 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.447 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.447 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-20T22:05:06.447 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.447 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.448 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.448 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.448 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2023-09-20T22:05:06.448 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2023-09-20T22:05:06.448 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.448 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.449 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.449 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.449 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.449 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.449 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.450 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-20T22:05:06.450 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.450 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.450 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.450 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.451 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:06.451 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:06.451 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.451 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.451 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.451 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:06.452 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.452 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.452 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.452 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2023-09-20T22:05:06.452 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.453 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.453 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.453 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2023-09-20T22:05:06.453 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2023-09-20T22:05:06.454 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.454 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.454 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.454 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2023-09-20T22:05:06.454 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.455 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.455 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.455 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-20T22:05:06.455 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.455 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.455 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.456 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.456 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.456 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.456 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2023-09-20T22:05:06.457 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.457 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.457 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.457 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.457 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.458 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.458 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.458 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.458 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.458 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.459 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.459 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.459 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.459 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.459 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.459 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.460 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.460 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.460 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.460 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.460 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.461 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.461 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.461 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.461 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.461 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.462 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.462 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.462 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.462 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.463 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.463 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.463 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.463 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.463 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.463 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.464 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.464 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.464 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.464 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.465 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.465 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.465 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.465 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.465 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.466 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.466 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.466 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.466 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.466 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.467 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.467 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.467 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.467 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.467 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.468 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.468 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.468 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.468 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.468 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.468 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.469 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.469 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.469 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.469 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.469 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.470 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.470 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.470 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.470 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.470 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.471 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2023-09-20T22:05:06.471 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2023-09-20T22:05:06.471 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.471 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.472 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.472 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2023-09-20T22:05:06.472 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.472 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.472 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.472 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.473 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.473 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.473 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.473 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.473 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2023-09-20T22:05:06.474 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2023-09-20T22:05:06.474 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.474 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.474 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.474 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2023-09-20T22:05:06.475 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.475 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.475 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.475 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.475 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.476 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.476 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.476 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.476 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:06.476 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:06.477 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.477 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.477 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.477 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:06.477 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.477 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.478 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.478 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.478 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.478 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.478 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.479 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.479 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.479 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.479 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.479 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2023-09-20T22:05:06.480 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2023-09-20T22:05:06.480 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2023-09-20T22:05:06.480 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.480 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2023-09-20T22:05:06.480 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2023-09-20T22:05:06.480 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.481 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.481 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.481 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2023-09-20T22:05:06.481 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.481 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.482 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.482 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-20T22:05:06.482 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.482 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.482 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.483 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.483 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2023-09-20T22:05:06.483 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2023-09-20T22:05:06.483 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.483 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.484 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.484 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2023-09-20T22:05:06.484 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.484 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.484 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.485 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-20T22:05:06.485 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.485 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.485 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.485 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.486 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2023-09-20T22:05:06.486 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2023-09-20T22:05:06.486 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.486 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.486 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.486 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.487 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.487 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.487 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.487 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2023-09-20T22:05:06.488 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.488 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.488 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.488 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.488 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:06.488 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:06.489 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.489 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.489 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.489 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2023-09-20T22:05:06.489 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2023-09-20T22:05:06.490 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.490 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.490 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-20T22:05:06.490 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.490 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.493 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.493 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.493 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.493 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.493 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.494 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.494 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.494 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.494 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.494 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.495 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.495 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.495 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.495 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.495 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.496 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.496 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.496 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.496 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.496 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.497 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.497 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.497 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.497 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.497 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.497 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.498 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.498 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.498 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.498 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.498 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.499 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.499 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.499 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.499 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.499 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.500 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.500 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.500 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.500 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.500 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.501 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.501 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.501 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.501 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.501 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.502 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.502 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.502 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.502 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.502 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.503 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.503 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.503 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.503 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.503 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.504 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.504 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.504 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.504 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.504 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.504 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.505 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.505 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.505 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.505 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.505 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.506 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.506 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.506 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.506 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.506 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.507 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2023-09-20T22:05:06.507 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2023-09-20T22:05:06.507 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.507 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.507 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.508 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.508 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.508 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.508 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.508 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2023-09-20T22:05:06.509 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.509 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.509 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.509 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.509 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2023-09-20T22:05:06.509 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2023-09-20T22:05:06.510 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.510 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.510 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.510 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2023-09-20T22:05:06.511 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.511 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.511 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.511 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2023-09-20T22:05:06.511 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.511 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.512 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.512 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.512 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2023-09-20T22:05:06.512 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2023-09-20T22:05:06.512 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.513 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.513 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.513 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2023-09-20T22:05:06.513 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.513 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.514 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.514 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-20T22:05:06.514 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.514 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.514 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.515 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.515 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2023-09-20T22:05:06.515 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2023-09-20T22:05:06.515 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.515 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.515 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.516 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:06.516 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:06.516 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2023-09-20T22:05:06.516 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.516 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-20T22:05:06.517 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.517 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.517 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.517 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.517 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2023-09-20T22:05:06.518 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2023-09-20T22:05:06.518 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.518 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.518 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.518 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.519 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.519 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.519 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.519 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2023-09-20T22:05:06.519 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.519 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.520 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.520 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.520 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2023-09-20T22:05:06.520 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2023-09-20T22:05:06.520 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.521 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.521 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.521 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2023-09-20T22:05:06.521 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.521 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.522 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.522 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-20T22:05:06.522 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.522 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.522 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.523 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.523 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.523 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.523 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.523 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2023-09-20T22:05:06.523 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.524 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.524 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.524 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.524 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.524 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.525 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.525 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.525 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.525 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.525 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.526 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.526 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.526 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.526 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.526 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.527 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.527 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.527 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.527 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.527 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.527 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.528 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.528 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.528 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.528 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.528 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.529 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.529 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.529 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.529 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.529 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.530 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.530 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.530 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.530 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.530 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.531 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.531 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.531 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.531 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.531 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.531 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.532 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.532 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.532 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.532 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.532 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.533 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.533 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.533 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.533 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.533 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.534 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.534 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.534 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.534 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.534 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.534 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.535 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.535 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.535 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.535 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.535 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.536 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.536 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.536 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.536 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.536 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.537 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.537 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.537 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.537 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.537 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.538 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.538 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.538 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2023-09-20T22:05:06.538 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.538 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.539 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.539 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.539 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.539 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.539 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.540 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.540 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.540 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.540 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.540 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.540 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.541 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.541 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.541 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.541 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.541 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.542 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.542 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.542 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.542 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.542 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.543 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.543 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.543 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.543 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.543 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.543 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.544 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.544 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.544 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.544 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.544 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.545 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.545 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.545 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.545 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.545 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.546 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.546 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.546 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.546 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.546 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.547 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.547 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.547 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.547 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.547 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.547 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.548 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.548 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.548 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.548 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.548 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.549 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.549 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.549 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.549 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.549 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.550 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.550 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.550 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.550 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.550 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.551 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.551 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.551 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.551 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.551 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.552 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.552 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.552 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.552 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.552 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.552 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.553 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2023-09-20T22:05:06.553 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.553 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.553 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.553 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2023-09-20T22:05:06.554 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2023-09-20T22:05:06.554 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.554 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.554 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.554 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2023-09-20T22:05:06.555 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.555 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.555 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.555 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-20T22:05:06.555 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.556 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.556 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.556 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.556 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2023-09-20T22:05:06.556 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2023-09-20T22:05:06.557 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.557 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.557 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.557 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:06.557 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2023-09-20T22:05:06.558 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.558 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.558 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-20T22:05:06.558 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.558 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.559 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.559 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.559 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.559 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.559 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.560 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.560 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.560 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.560 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.560 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.560 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.561 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.561 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.561 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.561 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.561 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.562 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.562 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.562 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.562 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.563 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.563 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.563 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.563 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.563 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.563 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.564 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.564 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.564 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.564 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.564 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.565 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.565 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.565 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.565 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.565 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.566 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.566 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.566 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.566 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.566 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.566 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.567 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.567 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.567 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.567 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.567 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.568 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.568 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.568 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.568 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.568 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.569 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.569 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.569 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.569 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.569 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.569 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.570 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.570 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.570 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.570 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.570 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.571 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.571 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.571 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.571 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.571 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.572 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.572 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.572 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.572 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.572 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.573 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.573 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2023-09-20T22:05:06.573 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.573 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.573 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.574 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2023-09-20T22:05:06.574 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2023-09-20T22:05:06.574 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.574 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.574 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.575 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2023-09-20T22:05:06.575 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.575 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.575 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.575 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2023-09-20T22:05:06.576 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-20T22:05:06.576 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.576 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.576 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.577 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.577 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.577 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.577 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.577 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.578 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.579 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.579 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.579 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.579 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.579 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.580 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.580 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.580 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.580 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.580 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.581 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.581 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.581 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.581 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.581 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.582 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.582 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.582 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.582 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.582 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.583 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.583 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.583 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.583 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.584 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.584 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.584 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.584 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.584 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.585 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.585 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.585 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.585 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.585 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.585 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.586 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.586 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.586 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.586 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.586 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.587 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.587 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.587 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.587 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.588 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.588 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.588 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.588 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.588 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.588 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.589 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.589 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.589 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.589 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.589 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.590 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2023-09-20T22:05:06.590 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2023-09-20T22:05:06.590 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.590 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.591 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.591 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.591 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.591 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.591 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.592 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2023-09-20T22:05:06.592 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-20T22:05:06.592 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.592 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.592 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2023-09-20T22:05:06.593 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2023-09-20T22:05:06.593 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.593 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.593 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.593 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2023-09-20T22:05:06.594 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.594 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.594 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.594 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2023-09-20T22:05:06.594 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.594 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.595 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.595 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2023-09-20T22:05:06.595 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2023-09-20T22:05:06.595 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.596 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.596 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.596 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2023-09-20T22:05:06.596 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.596 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.597 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.597 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2023-09-20T22:05:06.597 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.597 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.597 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.598 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2023-09-20T22:05:06.598 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2023-09-20T22:05:06.598 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.598 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.598 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.599 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.599 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.599 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.599 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.599 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2023-09-20T22:05:06.600 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.600 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.600 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.600 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2023-09-20T22:05:06.601 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2023-09-20T22:05:06.601 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.601 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.601 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.601 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2023-09-20T22:05:06.601 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.602 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.602 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.602 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-20T22:05:06.602 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-20T22:05:06.602 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.603 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.603 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.603 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.603 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.604 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2023-09-20T22:05:06.604 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.604 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.604 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.604 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.604 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.605 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.605 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.605 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.605 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.605 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.606 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.606 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.606 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.606 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.606 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.607 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.607 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.607 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.607 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.607 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.607 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.608 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.608 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.608 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.608 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.609 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.609 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.609 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.609 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.610 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.610 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.610 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.610 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.610 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.611 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.611 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.611 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.611 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.611 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.612 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.612 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.612 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.612 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.612 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.613 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.613 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.613 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.613 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.613 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.613 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.614 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.614 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.614 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.614 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.614 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.615 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.615 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.615 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.615 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.616 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.616 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.616 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.616 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.616 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.616 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.617 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.617 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.617 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.617 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.617 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.618 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.618 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.618 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.618 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2023-09-20T22:05:06.619 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.619 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.619 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.619 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.619 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.619 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.620 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.620 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.620 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.620 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.620 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.621 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.621 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.621 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.621 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.621 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.621 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.622 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.622 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.622 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.622 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.622 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.623 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.623 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.623 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.623 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.624 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.624 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.624 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.624 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.624 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.624 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.625 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.625 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.625 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.625 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.625 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.626 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.626 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.626 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.626 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.626 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.627 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.627 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.627 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.627 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.627 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.627 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.628 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.628 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.628 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.628 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.628 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.629 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.629 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.629 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.629 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.629 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.630 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.630 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.630 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.630 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.630 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.631 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.631 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.631 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.631 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.631 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.632 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.632 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.632 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.632 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.633 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2023-09-20T22:05:06.633 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.633 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.633 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.633 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.634 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.634 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.634 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.634 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.634 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.634 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.635 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.635 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.635 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.635 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.635 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.636 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.636 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.636 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.636 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.636 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.636 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.637 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.637 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.637 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.637 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.638 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.638 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.638 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.638 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.638 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.639 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.639 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.639 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.639 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.639 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.639 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.640 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.640 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.640 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.640 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.641 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.641 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.641 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.641 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.641 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.642 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.642 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.642 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.642 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.642 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.642 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.643 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.643 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.643 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.643 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.643 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.644 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.644 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.644 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.644 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.644 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.645 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.645 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.645 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.645 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.645 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.646 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.646 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.646 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.646 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.646 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.647 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2023-09-20T22:05:06.647 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.647 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.648 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.648 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2023-09-20T22:05:06.648 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2023-09-20T22:05:06.648 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.648 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.649 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.649 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.649 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.649 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.649 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.649 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-20T22:05:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.651 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.651 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.651 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.651 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.651 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.652 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.652 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.652 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.652 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.652 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.652 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.653 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.653 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.653 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.653 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.653 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.654 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.654 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.654 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.654 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.655 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.655 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.655 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.655 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.655 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.655 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.656 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.656 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.656 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.656 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.656 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.657 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.657 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.657 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.657 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.657 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.658 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.658 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.658 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.658 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.658 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.659 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.659 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.659 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.659 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.659 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.660 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.660 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.660 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.660 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.660 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.660 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.661 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.661 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.661 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.661 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.662 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.662 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.662 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.662 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.662 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.663 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.663 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.663 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.663 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.663 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2023-09-20T22:05:06.664 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2023-09-20T22:05:06.664 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.664 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.664 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.664 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.665 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.665 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.665 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.665 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2023-09-20T22:05:06.665 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2023-09-20T22:05:06.665 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2023-09-20T22:05:06.666 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.666 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2023-09-20T22:05:06.666 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.666 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.666 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.667 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.667 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.667 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.667 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2023-09-20T22:05:06.667 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.668 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.668 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.668 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.668 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.668 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.669 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.669 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.669 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.669 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.669 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.669 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.670 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.670 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.670 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.670 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.670 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.671 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.671 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.671 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.671 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.671 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.672 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.672 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.672 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.672 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.672 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.673 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.673 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.673 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.673 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.673 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.674 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.674 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.674 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.674 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.675 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.675 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.675 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.675 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.675 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.676 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.676 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.676 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.676 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.676 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.677 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.677 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.677 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.677 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.677 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.677 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.678 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.678 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.678 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.678 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.679 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.679 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.679 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.679 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.679 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.680 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.680 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.680 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.680 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.680 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.680 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.681 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.681 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.681 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.681 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.682 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.682 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2023-09-20T22:05:06.682 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.682 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.682 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.682 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.683 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.683 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.683 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.683 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.683 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.684 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.684 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.684 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.684 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.685 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.685 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.685 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.685 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.685 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.685 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.686 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.686 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.686 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.686 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.687 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.687 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.687 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.687 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.687 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.688 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.688 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.688 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.688 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.688 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.689 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.689 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.689 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.689 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.689 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.690 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.690 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.690 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.690 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.690 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.691 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.691 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.691 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.691 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.691 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.692 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.692 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.692 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.692 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.692 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.692 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.693 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.693 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.693 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.693 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.694 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.694 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.694 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.694 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.694 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.695 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.695 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.695 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.695 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.695 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.696 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.696 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.696 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.696 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2023-09-20T22:05:06.697 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.697 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.697 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.697 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.697 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.698 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.698 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.698 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.698 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.698 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.698 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.699 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.699 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.700 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.701 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.701 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.701 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.701 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.701 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.702 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.702 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.702 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.702 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.702 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.703 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.703 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.703 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.703 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.703 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.704 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.704 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.704 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.704 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.704 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.705 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.705 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.705 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.705 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.705 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.705 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.706 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.706 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.706 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.706 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.706 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.707 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.707 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.707 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.707 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.707 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.708 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.708 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.708 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.708 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.708 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.709 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.709 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.709 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.709 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.709 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.710 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.710 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.710 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.710 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.710 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.711 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.711 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.711 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.711 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.711 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.712 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.712 INFO:teuthology.orchestra.run.smithi022.stdout: "noautoscale": { 2023-09-20T22:05:06.712 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "noautoscale", 2023-09-20T22:05:06.712 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.712 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.713 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.713 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.713 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.713 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.713 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.713 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "global autoscale flag", 2023-09-20T22:05:06.714 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-20T22:05:06.714 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.714 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.714 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.714 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:06.715 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:06.715 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.715 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.715 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.715 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:06.716 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.716 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.716 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.716 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.716 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.716 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.717 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.717 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2023-09-20T22:05:06.717 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2023-09-20T22:05:06.717 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:06.718 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.718 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.718 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2023-09-20T22:05:06.718 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2023-09-20T22:05:06.718 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.719 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.719 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2023-09-20T22:05:06.719 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-20T22:05:06.719 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.719 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.720 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.720 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.720 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2023-09-20T22:05:06.720 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.721 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.721 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.721 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2023-09-20T22:05:06.721 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2023-09-20T22:05:06.721 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.722 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.722 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.722 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.722 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.722 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.723 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.723 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-20T22:05:06.723 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.723 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.723 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.724 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2023-09-20T22:05:06.724 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2023-09-20T22:05:06.724 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.724 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.725 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.725 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.725 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.725 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.725 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.725 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.726 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.726 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.726 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.726 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.727 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.727 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.727 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.727 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.727 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.728 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.728 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.728 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.728 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.728 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.729 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.729 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.729 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.729 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.729 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.729 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.730 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.730 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.730 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.730 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.730 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.731 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.731 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.731 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.731 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.731 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.732 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.732 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.732 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.732 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.732 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.732 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.733 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.733 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.733 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.733 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.733 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.734 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.734 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.734 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.734 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.734 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.735 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.735 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.735 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.735 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.735 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.736 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.736 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.736 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.736 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.736 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.736 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.737 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.737 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.737 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.737 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.738 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.738 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.738 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.738 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.738 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.739 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.739 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.739 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.739 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.739 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.740 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2023-09-20T22:05:06.740 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2023-09-20T22:05:06.740 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.740 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.740 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.741 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2023-09-20T22:05:06.741 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.741 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.741 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.741 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2023-09-20T22:05:06.742 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.742 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.742 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.742 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:06.743 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:06.743 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:06.743 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.743 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.743 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:06.744 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.744 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.744 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.744 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2023-09-20T22:05:06.744 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.745 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.745 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.745 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.745 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.745 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.746 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.746 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2023-09-20T22:05:06.746 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.746 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.746 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.747 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2023-09-20T22:05:06.747 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2023-09-20T22:05:06.747 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.747 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.747 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.748 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.748 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.748 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.748 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.748 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.748 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.749 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.749 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.749 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2023-09-20T22:05:06.749 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2023-09-20T22:05:06.750 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.750 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.750 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.750 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.750 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.751 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.751 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.751 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-20T22:05:06.751 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-20T22:05:06.751 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.751 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.752 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.752 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.752 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.752 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.753 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.753 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.753 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.753 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.753 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.754 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.754 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.754 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.754 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.754 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.755 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.755 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.755 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.755 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.755 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.756 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.756 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.756 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.756 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.756 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.757 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.757 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.757 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.757 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.757 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.757 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.758 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.758 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.758 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.758 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.758 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.759 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.759 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.759 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.759 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.759 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.760 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.760 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.760 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.760 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.760 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.760 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.761 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.761 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.761 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.761 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.761 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.762 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.762 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.762 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.762 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.762 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.763 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.763 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.763 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.763 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.763 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.763 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.764 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.764 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.764 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.764 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.764 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.765 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.765 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.765 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.765 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2023-09-20T22:05:06.765 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2023-09-20T22:05:06.766 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.766 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.766 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.766 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.766 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.767 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.767 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.767 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.767 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.767 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.768 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.768 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-20T22:05:06.768 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-20T22:05:06.768 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.769 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.769 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.769 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2023-09-20T22:05:06.769 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.769 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.770 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.770 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.770 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.770 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.770 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.771 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2023-09-20T22:05:06.771 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2023-09-20T22:05:06.771 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:06.771 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.771 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.772 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2023-09-20T22:05:06.772 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.772 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.772 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.772 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.773 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.773 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.773 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.773 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.773 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2023-09-20T22:05:06.774 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2023-09-20T22:05:06.774 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.774 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.774 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.774 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2023-09-20T22:05:06.775 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.775 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.775 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.775 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-20T22:05:06.775 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.775 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.776 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.776 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2023-09-20T22:05:06.776 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2023-09-20T22:05:06.776 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.777 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.777 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.777 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2023-09-20T22:05:06.777 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.777 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.778 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.778 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-20T22:05:06.778 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.778 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.778 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.779 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2023-09-20T22:05:06.779 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2023-09-20T22:05:06.779 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.779 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.780 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.780 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2023-09-20T22:05:06.780 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.780 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.780 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.780 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.781 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.781 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.781 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.781 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2023-09-20T22:05:06.782 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2023-09-20T22:05:06.782 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.782 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.782 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.782 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2023-09-20T22:05:06.783 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.783 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.783 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.783 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2023-09-20T22:05:06.783 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2023-09-20T22:05:06.783 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.784 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.784 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.784 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.784 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.785 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2023-09-20T22:05:06.785 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2023-09-20T22:05:06.785 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.785 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.785 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.786 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2023-09-20T22:05:06.786 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2023-09-20T22:05:06.786 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2023-09-20T22:05:06.786 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.786 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.787 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.787 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.787 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.787 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.787 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.787 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.788 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.788 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2023-09-20T22:05:06.788 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.788 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.788 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.789 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.789 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.789 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.789 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.789 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.789 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.790 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.790 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.790 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.790 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.790 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.791 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.791 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.791 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.791 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.791 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.792 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.792 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.792 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.792 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.792 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.792 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.793 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.793 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.793 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.793 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.793 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.794 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.794 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.794 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.794 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.794 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.795 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.795 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.795 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.795 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.795 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.796 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.796 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.796 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.796 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.796 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.797 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.797 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.797 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.797 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.797 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.798 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.798 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.798 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.798 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.798 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.799 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.799 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.799 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.799 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.799 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.800 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.800 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.800 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.800 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.800 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.801 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.801 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.801 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.801 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.801 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.801 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.802 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.802 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.802 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2023-09-20T22:05:06.802 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2023-09-20T22:05:06.802 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.803 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.803 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.803 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:06.803 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.803 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.804 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.804 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.804 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.804 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.804 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.805 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2023-09-20T22:05:06.805 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2023-09-20T22:05:06.805 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.805 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.805 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.806 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.806 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.806 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.806 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.806 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.807 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.807 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.807 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.807 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2023-09-20T22:05:06.808 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2023-09-20T22:05:06.808 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.808 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.808 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.808 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.809 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.809 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.809 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.809 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.809 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.809 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.810 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.810 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.810 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.810 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2023-09-20T22:05:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2023-09-20T22:05:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2023-09-20T22:05:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.813 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.813 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.813 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.813 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.813 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.814 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.814 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2023-09-20T22:05:06.814 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2023-09-20T22:05:06.814 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.815 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.815 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.815 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.815 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.816 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.816 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.816 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.816 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.816 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.817 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.817 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.817 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.818 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.818 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.818 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.818 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.818 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.818 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.819 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.819 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.819 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.819 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.819 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.820 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.820 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.820 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.820 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.820 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.821 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.821 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.821 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.821 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.821 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.822 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.822 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.822 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.822 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.822 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.823 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.823 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.823 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.823 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.824 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.824 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.824 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.824 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.824 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.824 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.825 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.825 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.825 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.825 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.825 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.826 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.826 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.826 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.826 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.826 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.827 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.827 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.827 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.827 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.828 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.828 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.828 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.828 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.828 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.829 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.829 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.829 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.829 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.829 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.829 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.830 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.830 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.830 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.831 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2023-09-20T22:05:06.831 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2023-09-20T22:05:06.831 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.831 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.831 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.832 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.832 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.832 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.832 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.832 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.832 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.833 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.833 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.833 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.833 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2023-09-20T22:05:06.833 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2023-09-20T22:05:06.834 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.834 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.834 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.834 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.834 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.835 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.835 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.835 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.835 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.835 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.836 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.836 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.836 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.836 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.836 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.836 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2023-09-20T22:05:06.837 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.837 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.837 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.837 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.837 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.838 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.838 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.838 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.838 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.838 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.839 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.839 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.839 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.839 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.839 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.840 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.840 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.840 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.840 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.840 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.841 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.841 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.841 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.841 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.842 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.842 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.842 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.842 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.842 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.843 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.843 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.843 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.843 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.843 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.844 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.844 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.844 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.844 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.844 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.844 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.845 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.845 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.845 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.845 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.845 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.846 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.846 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.846 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.846 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.846 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.847 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.847 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.847 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.847 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.847 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.847 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.848 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.848 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.848 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.848 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.848 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.849 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.849 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.945 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.945 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.946 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.946 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.946 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.946 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.946 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.947 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.947 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.947 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.947 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.948 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2023-09-20T22:05:06.948 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.948 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.948 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.948 INFO:teuthology.orchestra.run.smithi022.stdout: "drive_group_interval": { 2023-09-20T22:05:06.948 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "drive_group_interval", 2023-09-20T22:05:06.949 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:06.949 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.949 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.949 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300.0", 2023-09-20T22:05:06.949 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.950 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.950 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.950 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-20T22:05:06.950 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.950 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.951 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.951 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.951 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.951 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.951 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.951 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.952 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.952 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.952 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.952 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.952 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.953 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.953 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.953 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.953 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.953 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.954 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.954 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.954 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.954 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.954 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.955 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.955 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.955 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.955 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.955 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.955 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.956 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.956 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.956 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.956 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.956 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.957 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.957 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.957 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.957 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.958 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.958 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.958 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.958 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.958 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.959 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.959 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.959 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.959 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.959 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.960 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.960 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.960 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.960 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.960 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.960 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.961 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.961 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.961 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.961 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.962 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.962 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.962 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.962 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.962 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.963 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.963 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.963 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.963 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.963 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.964 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.964 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.964 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.964 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2023-09-20T22:05:06.965 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2023-09-20T22:05:06.965 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.965 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.965 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.965 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2023-09-20T22:05:06.966 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.966 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.966 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.966 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-20T22:05:06.966 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.967 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.967 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.967 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.967 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:06.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.967 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:06.968 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2023-09-20T22:05:06.968 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:06.968 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:06.968 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:06.969 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:06.969 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:06.969 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.969 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.969 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.970 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.970 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.970 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.970 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.970 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.971 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.971 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.971 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.971 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.971 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.972 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.972 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.972 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.972 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.972 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.973 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:06.973 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:06.973 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.973 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.973 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.974 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.974 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.974 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.974 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.974 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.975 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.975 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.975 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.975 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:06.976 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:06.976 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.976 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.976 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.976 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:06.976 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.977 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.977 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:06.977 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:06.977 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:06.977 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:06.978 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:06.978 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:06.978 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:06.978 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:06.978 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.979 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.979 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.979 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.979 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:06.979 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:06.980 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.980 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.980 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:06.980 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:06.980 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.981 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.981 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.981 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.981 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.981 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.982 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.982 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.982 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2023-09-20T22:05:06.982 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2023-09-20T22:05:06.982 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.982 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.983 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.983 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.983 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.983 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.983 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.984 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.984 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.984 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.984 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.985 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2023-09-20T22:05:06.985 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2023-09-20T22:05:06.985 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.985 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.985 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.985 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2023-09-20T22:05:06.986 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.986 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.986 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.986 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.986 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.987 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.987 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.987 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.987 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2023-09-20T22:05:06.987 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2023-09-20T22:05:06.988 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.988 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.988 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.988 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.988 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.988 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.989 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.989 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.989 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.989 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.989 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.990 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.990 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2023-09-20T22:05:06.990 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2023-09-20T22:05:06.990 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:06.990 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.991 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.991 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.991 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.991 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.992 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.992 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.992 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.992 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.992 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.993 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2023-09-20T22:05:06.993 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2023-09-20T22:05:06.993 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:06.993 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.993 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.994 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.994 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.994 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.994 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.994 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.995 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.995 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.995 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.995 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2023-09-20T22:05:06.996 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2023-09-20T22:05:06.996 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:06.996 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.996 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.996 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.997 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.997 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:06.997 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:06.997 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:06.997 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:06.997 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:06.998 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:06.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:06.998 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2023-09-20T22:05:06.998 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2023-09-20T22:05:06.998 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:06.999 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:06.999 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:06.999 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:06.999 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:06.999 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.000 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.000 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.000 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.000 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.000 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.001 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2023-09-20T22:05:07.001 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2023-09-20T22:05:07.001 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.001 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.001 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.002 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.002 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.002 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.002 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.002 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.003 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.003 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.003 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.003 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2023-09-20T22:05:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2023-09-20T22:05:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2023-09-20T22:05:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.006 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.006 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2023-09-20T22:05:07.006 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2023-09-20T22:05:07.006 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.007 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.007 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.007 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.007 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.007 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.008 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.008 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.008 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.008 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.008 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.009 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2023-09-20T22:05:07.009 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2023-09-20T22:05:07.009 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.009 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.009 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.010 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.010 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.010 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.010 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.010 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.010 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.011 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.011 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.011 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2023-09-20T22:05:07.011 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2023-09-20T22:05:07.012 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.012 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.012 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.012 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.012 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.013 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.013 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.013 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.013 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.013 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.013 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.014 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.014 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.014 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.014 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2023-09-20T22:05:07.015 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.015 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.015 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.015 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2023-09-20T22:05:07.015 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2023-09-20T22:05:07.016 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.016 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.016 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.016 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.016 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.016 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.017 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.017 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2023-09-20T22:05:07.017 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.017 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.017 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.018 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2023-09-20T22:05:07.018 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2023-09-20T22:05:07.018 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.018 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.019 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.019 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.019 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.019 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.019 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.019 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2023-09-20T22:05:07.020 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.020 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.020 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.020 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.021 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.021 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.021 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.021 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.021 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.022 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.022 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.022 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.022 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.022 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.023 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.023 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.023 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.023 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.023 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.024 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.024 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.024 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.024 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.025 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.025 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.025 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.025 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.025 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.026 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.026 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.026 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.026 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.026 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.027 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.027 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.027 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.027 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.028 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.028 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.028 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.028 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.028 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.028 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.029 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.029 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.029 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.029 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.029 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.030 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.030 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.030 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.030 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.030 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.031 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.032 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.034 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2023-09-20T22:05:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.035 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.036 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.037 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.038 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.040 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.041 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.042 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.042 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.042 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.042 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.042 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.043 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.043 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.043 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.043 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.043 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.044 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.044 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.044 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.044 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.044 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.044 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.045 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.045 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.045 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.045 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.045 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.046 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.046 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.046 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.046 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.046 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.047 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.047 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.047 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.047 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.047 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.047 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.048 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.048 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.048 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.048 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.048 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.049 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.049 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.049 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.049 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2023-09-20T22:05:07.050 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.050 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.050 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.050 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.050 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.051 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.051 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.051 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.051 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.051 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.051 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.052 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.052 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.052 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.052 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.052 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.053 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.053 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.053 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.053 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.053 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.053 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.054 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.054 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.054 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.054 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.055 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.055 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.055 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.055 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.055 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.056 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.056 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.056 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.056 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.056 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.056 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.057 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.057 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.057 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.057 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.057 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.058 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.058 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.058 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.058 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.058 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.059 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.059 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.059 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.059 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.059 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.059 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.060 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.060 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.060 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.060 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.060 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.061 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.061 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.061 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.061 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.061 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.061 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.062 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.062 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.062 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.062 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.062 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.063 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.063 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.063 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.063 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.064 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.064 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2023-09-20T22:05:07.064 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.064 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.064 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.065 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2023-09-20T22:05:07.065 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2023-09-20T22:05:07.065 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.065 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.065 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.065 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-20T22:05:07.066 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.066 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.066 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.066 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.066 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.067 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.067 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.067 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.067 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:07.067 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:07.068 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.068 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.068 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.068 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2023-09-20T22:05:07.068 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.068 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.069 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.069 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.069 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.069 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.069 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.070 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.070 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.070 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.070 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.070 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.071 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.071 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.071 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.071 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.071 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.072 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.072 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.072 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.072 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.072 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.073 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.073 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.073 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.073 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.073 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.073 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.074 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.074 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.074 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.074 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.074 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.075 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.075 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.075 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.075 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.075 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.075 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.076 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.076 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.076 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.076 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.077 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.077 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.077 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.077 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.077 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.078 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.078 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.078 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.078 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.078 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.079 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.079 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.079 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.079 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.079 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.079 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.080 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.080 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.080 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.080 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.081 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.081 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.081 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.081 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.081 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.082 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.082 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.082 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.082 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.082 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.083 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.083 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.083 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.083 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.083 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.084 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.084 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.084 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2023-09-20T22:05:07.084 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.084 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.085 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.085 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2023-09-20T22:05:07.085 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2023-09-20T22:05:07.085 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.085 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.085 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.086 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.086 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.086 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.086 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.086 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-20T22:05:07.087 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.087 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.087 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.087 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2023-09-20T22:05:07.088 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2023-09-20T22:05:07.088 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.088 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.088 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.088 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.089 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.089 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.089 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.089 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-20T22:05:07.089 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.089 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.090 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.090 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.090 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2023-09-20T22:05:07.090 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2023-09-20T22:05:07.090 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.091 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.091 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.091 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.091 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.091 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.092 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.092 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-20T22:05:07.092 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.092 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.092 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.093 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2023-09-20T22:05:07.093 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2023-09-20T22:05:07.093 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.093 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.094 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.094 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.094 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.094 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.094 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.095 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-20T22:05:07.095 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.095 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.095 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.095 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2023-09-20T22:05:07.096 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2023-09-20T22:05:07.096 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.096 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.096 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.096 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.097 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.097 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.097 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.097 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-20T22:05:07.097 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.098 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.098 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.098 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.098 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2023-09-20T22:05:07.098 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2023-09-20T22:05:07.099 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.099 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.099 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.099 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.099 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.099 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.100 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.100 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.100 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.100 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.100 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.101 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.101 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2023-09-20T22:05:07.101 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2023-09-20T22:05:07.101 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.101 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.102 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.102 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.102 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.102 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.102 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.102 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.103 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.103 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.103 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.103 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.103 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2023-09-20T22:05:07.104 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2023-09-20T22:05:07.104 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.104 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.104 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.104 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-20T22:05:07.105 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.105 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.105 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.105 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.105 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.105 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.106 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.106 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.106 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2023-09-20T22:05:07.106 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2023-09-20T22:05:07.106 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.107 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.107 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.107 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.107 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.107 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.108 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.108 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.108 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.108 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.108 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.108 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.109 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:07.109 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:07.109 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.109 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.109 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.110 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2023-09-20T22:05:07.110 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2023-09-20T22:05:07.110 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.110 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.110 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.110 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.111 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.111 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.111 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.111 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2023-09-20T22:05:07.111 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2023-09-20T22:05:07.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.112 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.112 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.112 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2023-09-20T22:05:07.112 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.113 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.113 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.113 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.113 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.113 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.114 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.114 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2023-09-20T22:05:07.114 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2023-09-20T22:05:07.114 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.115 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.115 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.115 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.115 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.115 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.116 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.116 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.116 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.116 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.116 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.117 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2023-09-20T22:05:07.117 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2023-09-20T22:05:07.117 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.117 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.118 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.118 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.118 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.118 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.118 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.119 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.119 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.119 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.119 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.119 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.119 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.120 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.120 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.120 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.120 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.120 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.121 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.121 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.121 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.121 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.121 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.122 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.123 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.123 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.123 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.123 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.124 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.124 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.124 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.124 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.124 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.125 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.125 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.125 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.125 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.125 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.126 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.126 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.126 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.126 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.126 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.126 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.127 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.127 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.127 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.127 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.127 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.128 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.128 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.128 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.128 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.128 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.129 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.129 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.129 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.129 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.129 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.130 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.130 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.130 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.130 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.130 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.131 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.132 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2023-09-20T22:05:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2023-09-20T22:05:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.133 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.134 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.135 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.135 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.135 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.135 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.136 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2023-09-20T22:05:07.136 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2023-09-20T22:05:07.136 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.136 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.136 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.137 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.137 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.137 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.137 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.137 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.137 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.138 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.138 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.138 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.138 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2023-09-20T22:05:07.138 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2023-09-20T22:05:07.139 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.139 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.139 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.139 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-20T22:05:07.139 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.140 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.140 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.140 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.140 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.140 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.141 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.141 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.141 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.141 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2023-09-20T22:05:07.142 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.142 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.142 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.142 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.142 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.143 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.143 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.143 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.143 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.143 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.144 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.144 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.144 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.144 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.144 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.144 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.145 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.145 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.145 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.145 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.145 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.146 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.146 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.146 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.146 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.146 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.147 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.147 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.147 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.147 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.147 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.148 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.148 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.148 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.148 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.148 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.149 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.149 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.149 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.149 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.150 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.150 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.150 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.150 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.150 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.151 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.151 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.151 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.151 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.151 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.152 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.152 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.152 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.152 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.152 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.152 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.153 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.153 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.153 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.153 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.154 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.154 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.154 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.154 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.154 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.155 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.155 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.155 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.155 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.155 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.155 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.156 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.156 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.156 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2023-09-20T22:05:07.156 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.156 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.157 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.157 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.157 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.157 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.157 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.158 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.158 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.158 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.158 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.158 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.159 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.159 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.159 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.159 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.159 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.159 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.160 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.160 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.160 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.160 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.160 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.161 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.161 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.161 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.161 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.161 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.162 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.162 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.162 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.162 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.162 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.162 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.163 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.163 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.163 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.163 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.164 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.164 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.164 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.164 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.164 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.165 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.165 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.165 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.165 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.165 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.166 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.166 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.166 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.166 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.166 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.166 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.167 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.167 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.167 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.167 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.167 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.168 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.168 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.168 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.168 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.168 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.169 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.169 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.169 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.169 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.169 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.170 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.170 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.170 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.170 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2023-09-20T22:05:07.170 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2023-09-20T22:05:07.171 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.171 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.171 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.171 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2023-09-20T22:05:07.171 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.171 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.172 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.172 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-20T22:05:07.172 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.172 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.172 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.173 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.173 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2023-09-20T22:05:07.173 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2023-09-20T22:05:07.173 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.173 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.174 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.174 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.174 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.174 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.174 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.175 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2023-09-20T22:05:07.175 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.175 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.175 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.175 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2023-09-20T22:05:07.176 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2023-09-20T22:05:07.176 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.176 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.176 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.176 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:07.177 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.177 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.177 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.177 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-20T22:05:07.177 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.178 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.178 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.178 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2023-09-20T22:05:07.178 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2023-09-20T22:05:07.179 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.179 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.179 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.179 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.179 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.180 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.180 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.180 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2023-09-20T22:05:07.180 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.180 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.181 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.181 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.181 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.181 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.181 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2023-09-20T22:05:07.182 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.182 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.182 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.182 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2023-09-20T22:05:07.182 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2023-09-20T22:05:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2023-09-20T22:05:07.183 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.184 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.184 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.184 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.184 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.184 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.184 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.185 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2023-09-20T22:05:07.185 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2023-09-20T22:05:07.185 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.186 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.186 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.186 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.186 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.186 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.187 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.187 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.187 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.187 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.187 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.188 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:07.188 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:07.188 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.188 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.188 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.189 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:07.189 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.189 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.189 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.189 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.190 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.190 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.190 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.190 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.190 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.191 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.191 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.191 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.191 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.191 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.192 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.192 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.192 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.192 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.192 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.193 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.193 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.193 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.193 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.193 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.193 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.194 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.194 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.194 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.194 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.195 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.195 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.195 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.195 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.195 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.196 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.196 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.196 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.196 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.196 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.196 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.197 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.197 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.197 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.197 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.198 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.198 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.198 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.198 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.198 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.199 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.199 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.199 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.199 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.199 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.200 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.200 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.200 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.200 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.200 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.200 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.201 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.201 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.201 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.201 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.202 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.202 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.202 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.202 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.202 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.203 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.203 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.203 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.203 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.203 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.203 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2023-09-20T22:05:07.204 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2023-09-20T22:05:07.204 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.204 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.204 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.204 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.205 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.205 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.205 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.205 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.205 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.206 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.206 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.206 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2023-09-20T22:05:07.206 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2023-09-20T22:05:07.207 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.207 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.207 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.207 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2023-09-20T22:05:07.207 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.208 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.208 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.208 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.208 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.208 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.208 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.209 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.209 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2023-09-20T22:05:07.209 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2023-09-20T22:05:07.209 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.209 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.210 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.210 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-20T22:05:07.210 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.210 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.210 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.211 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.211 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.211 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.211 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.211 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.211 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.212 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.212 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-09-20T22:05:07.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.212 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.212 INFO:teuthology.orchestra.run.smithi022.stdout: "gid": 24109, 2023-09-20T22:05:07.213 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "z", 2023-09-20T22:05:07.213 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr_features": 4540138322906710015, 2023-09-20T22:05:07.213 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2023-09-20T22:05:07.213 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.213 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2023-09-20T22:05:07.213 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.214 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.214 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.214 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:07.214 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:07.214 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.215 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.215 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.215 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:07.215 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.215 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.216 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.216 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2023-09-20T22:05:07.216 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.216 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.216 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.217 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.217 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.217 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.217 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.217 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.218 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.218 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.218 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.218 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.218 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.219 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.219 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.219 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.219 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.219 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.219 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.220 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.220 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.220 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.220 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.220 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.221 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.221 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.221 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.221 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.221 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.222 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.222 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.222 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.222 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.222 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.223 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.223 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.223 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.223 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.223 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.224 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.224 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.224 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.224 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.224 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.225 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.225 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.225 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.225 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.225 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.225 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.226 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.226 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.226 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.226 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.226 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.227 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.227 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.227 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.227 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.228 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.228 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.228 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.228 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.228 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.228 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.229 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.229 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.229 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.229 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.229 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.230 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.230 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.230 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2023-09-20T22:05:07.230 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2023-09-20T22:05:07.230 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.231 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.231 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.231 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.231 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.231 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.231 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.232 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2023-09-20T22:05:07.232 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.232 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.232 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.233 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2023-09-20T22:05:07.233 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2023-09-20T22:05:07.233 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.233 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.233 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.234 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2023-09-20T22:05:07.234 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.234 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.234 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.234 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2023-09-20T22:05:07.235 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.235 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.235 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.235 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.235 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2023-09-20T22:05:07.235 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2023-09-20T22:05:07.236 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.236 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.236 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.236 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.236 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.237 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.237 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.237 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2023-09-20T22:05:07.237 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.237 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.238 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.238 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.238 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2023-09-20T22:05:07.238 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2023-09-20T22:05:07.238 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.238 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.239 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.239 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.239 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.239 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.239 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.240 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2023-09-20T22:05:07.240 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.240 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.240 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.241 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2023-09-20T22:05:07.241 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2023-09-20T22:05:07.241 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.241 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.241 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.242 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2023-09-20T22:05:07.242 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.242 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.242 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.242 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2023-09-20T22:05:07.243 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.243 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.243 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.243 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2023-09-20T22:05:07.243 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2023-09-20T22:05:07.244 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.244 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.244 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.244 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.244 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.245 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.245 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.245 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2023-09-20T22:05:07.245 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.245 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.246 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.246 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2023-09-20T22:05:07.246 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2023-09-20T22:05:07.246 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.247 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.247 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.247 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.247 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.247 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.248 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.248 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-20T22:05:07.248 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.248 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.248 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.248 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.249 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2023-09-20T22:05:07.249 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2023-09-20T22:05:07.249 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.249 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.249 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.250 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.250 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.250 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.250 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.250 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2023-09-20T22:05:07.251 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.251 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.251 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.251 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.251 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.252 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.252 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2023-09-20T22:05:07.252 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.252 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.252 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.253 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2023-09-20T22:05:07.253 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2023-09-20T22:05:07.253 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.253 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.253 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.254 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.254 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.254 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.254 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.254 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2023-09-20T22:05:07.255 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.255 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.255 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.255 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.255 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2023-09-20T22:05:07.255 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2023-09-20T22:05:07.256 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.256 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.256 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.256 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2023-09-20T22:05:07.256 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.257 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.257 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.257 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2023-09-20T22:05:07.257 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-20T22:05:07.257 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.258 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.258 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2023-09-20T22:05:07.258 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2023-09-20T22:05:07.258 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:07.259 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.259 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.259 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:07.259 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2023-09-20T22:05:07.259 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2023-09-20T22:05:07.260 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.260 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-20T22:05:07.260 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-20T22:05:07.260 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.260 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.261 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.261 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2023-09-20T22:05:07.261 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2023-09-20T22:05:07.261 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:07.261 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.261 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.262 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2023-09-20T22:05:07.262 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:07.262 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2023-09-20T22:05:07.262 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.262 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-20T22:05:07.263 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.263 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.263 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.263 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2023-09-20T22:05:07.264 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2023-09-20T22:05:07.264 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.264 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.264 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.264 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2023-09-20T22:05:07.264 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.265 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.265 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.265 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-20T22:05:07.265 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-20T22:05:07.265 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.266 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.266 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.266 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2023-09-20T22:05:07.266 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2023-09-20T22:05:07.266 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:07.267 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.267 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.267 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2023-09-20T22:05:07.267 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2023-09-20T22:05:07.267 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2023-09-20T22:05:07.268 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.268 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2023-09-20T22:05:07.268 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-20T22:05:07.268 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.268 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.269 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.269 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2023-09-20T22:05:07.269 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2023-09-20T22:05:07.269 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.269 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.269 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.270 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2023-09-20T22:05:07.270 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.270 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.270 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.270 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2023-09-20T22:05:07.271 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-20T22:05:07.271 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.271 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.271 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.271 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2023-09-20T22:05:07.271 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2023-09-20T22:05:07.272 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:07.272 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.272 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.272 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:07.272 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2023-09-20T22:05:07.273 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2023-09-20T22:05:07.273 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.273 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-20T22:05:07.273 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-20T22:05:07.273 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.274 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.274 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.274 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.274 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.274 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.275 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.275 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.275 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.275 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.275 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.275 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.276 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.276 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.276 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.276 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.276 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.277 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.277 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.277 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.277 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.277 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.278 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.278 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.278 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.278 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.279 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.279 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.279 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.279 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.279 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.279 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.280 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.280 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.280 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.280 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.280 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.281 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.281 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.281 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.281 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.281 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.281 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.282 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.282 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.282 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.282 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.282 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.283 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.283 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.283 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.283 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.283 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.283 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.284 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.284 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.284 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.284 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.284 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.285 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.285 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.285 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.285 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.285 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.286 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.286 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.286 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.286 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.286 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.286 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.287 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.287 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2023-09-20T22:05:07.287 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2023-09-20T22:05:07.287 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:07.288 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.288 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.288 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:07.288 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.288 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.288 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.289 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-20T22:05:07.289 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.289 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.289 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.290 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2023-09-20T22:05:07.290 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2023-09-20T22:05:07.290 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.290 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.290 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.290 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2023-09-20T22:05:07.291 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.291 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.291 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.291 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2023-09-20T22:05:07.291 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2023-09-20T22:05:07.292 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap" 2023-09-20T22:05:07.292 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.292 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2023-09-20T22:05:07.292 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.292 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.293 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.293 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2023-09-20T22:05:07.293 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2023-09-20T22:05:07.293 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.294 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.294 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.294 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.294 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.294 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.294 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.295 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-20T22:05:07.295 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.295 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.295 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.295 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.296 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:07.296 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:07.296 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.296 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.296 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.297 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:07.297 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.297 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.297 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.297 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-20T22:05:07.298 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.298 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.298 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.298 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2023-09-20T22:05:07.299 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2023-09-20T22:05:07.299 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.299 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.299 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.299 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:07.300 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:07.300 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.300 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.300 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-20T22:05:07.300 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-20T22:05:07.300 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.301 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.301 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2023-09-20T22:05:07.301 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2023-09-20T22:05:07.301 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:07.302 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.302 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.302 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:07.302 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.302 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.303 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.303 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-20T22:05:07.303 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.303 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.303 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.304 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.304 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.304 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.304 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2023-09-20T22:05:07.304 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.305 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.305 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.305 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2023-09-20T22:05:07.305 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2023-09-20T22:05:07.305 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:07.306 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.306 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.306 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2023-09-20T22:05:07.306 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.306 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.307 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.307 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-20T22:05:07.307 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.307 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.307 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.308 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2023-09-20T22:05:07.308 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2023-09-20T22:05:07.308 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.308 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.308 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.309 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2023-09-20T22:05:07.309 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.309 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.309 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.309 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-20T22:05:07.310 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.310 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.310 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.310 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.310 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2023-09-20T22:05:07.311 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2023-09-20T22:05:07.311 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.311 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.311 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.311 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2023-09-20T22:05:07.311 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.312 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.312 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.312 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-20T22:05:07.312 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.312 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.313 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.313 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2023-09-20T22:05:07.313 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2023-09-20T22:05:07.313 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.314 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.314 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.314 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.314 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.317 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.317 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.317 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-20T22:05:07.317 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-20T22:05:07.317 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.318 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.318 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2023-09-20T22:05:07.318 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2023-09-20T22:05:07.318 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.318 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.319 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.319 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:07.319 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.319 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.319 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.320 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-20T22:05:07.320 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.320 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.320 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.320 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2023-09-20T22:05:07.321 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2023-09-20T22:05:07.321 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:07.321 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.321 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.321 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2023-09-20T22:05:07.322 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.322 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.322 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.322 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-20T22:05:07.322 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.323 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.323 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.323 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2023-09-20T22:05:07.323 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2023-09-20T22:05:07.323 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.324 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.324 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.324 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.324 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.324 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.325 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.325 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-20T22:05:07.325 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.325 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.325 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.326 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.326 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2023-09-20T22:05:07.326 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2023-09-20T22:05:07.326 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.326 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.327 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.327 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.327 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.327 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.327 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-20T22:05:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.328 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2023-09-20T22:05:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2023-09-20T22:05:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.329 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.330 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.330 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.330 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.330 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-20T22:05:07.330 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.331 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.331 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.331 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.331 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2023-09-20T22:05:07.331 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2023-09-20T22:05:07.332 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.332 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.332 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.332 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-20T22:05:07.332 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.332 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.333 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.333 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:07.333 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.333 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.333 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.334 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2023-09-20T22:05:07.334 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2023-09-20T22:05:07.334 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.334 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.335 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.335 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-20T22:05:07.335 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.335 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.335 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.336 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2023-09-20T22:05:07.336 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.336 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.336 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.336 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.337 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2023-09-20T22:05:07.337 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2023-09-20T22:05:07.337 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.337 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.337 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.338 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-20T22:05:07.338 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.338 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.338 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.338 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2023-09-20T22:05:07.339 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.339 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.339 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.339 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2023-09-20T22:05:07.339 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2023-09-20T22:05:07.340 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.340 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.340 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.340 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-20T22:05:07.340 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.341 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.341 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.341 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:07.341 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.341 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.342 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.342 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2023-09-20T22:05:07.342 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2023-09-20T22:05:07.342 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.342 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.343 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.343 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-20T22:05:07.343 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.343 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.343 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.344 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2023-09-20T22:05:07.344 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.344 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.344 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.345 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2023-09-20T22:05:07.345 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2023-09-20T22:05:07.345 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.345 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.345 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.345 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-20T22:05:07.346 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.346 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.346 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.346 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2023-09-20T22:05:07.346 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.347 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.347 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.347 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2023-09-20T22:05:07.347 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2023-09-20T22:05:07.348 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.348 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.348 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.348 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-20T22:05:07.348 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.349 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.349 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.349 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2023-09-20T22:05:07.349 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.349 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.349 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.350 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2023-09-20T22:05:07.350 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2023-09-20T22:05:07.350 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.350 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.351 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.351 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-20T22:05:07.351 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.351 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.351 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.352 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2023-09-20T22:05:07.352 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.352 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.352 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.352 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2023-09-20T22:05:07.353 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2023-09-20T22:05:07.353 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.353 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.353 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.353 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-20T22:05:07.354 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.354 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.354 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.355 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2023-09-20T22:05:07.355 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.355 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.355 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.355 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.356 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2023-09-20T22:05:07.356 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2023-09-20T22:05:07.356 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.356 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.356 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.356 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-20T22:05:07.357 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.357 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.357 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.357 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2023-09-20T22:05:07.357 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.358 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.358 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.358 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.358 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2023-09-20T22:05:07.358 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2023-09-20T22:05:07.359 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.359 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.359 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.359 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-20T22:05:07.359 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.360 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.360 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.360 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:07.360 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.360 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.360 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.361 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2023-09-20T22:05:07.361 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2023-09-20T22:05:07.361 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.361 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.362 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.362 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-20T22:05:07.362 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.362 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.362 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.363 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2023-09-20T22:05:07.363 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.363 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.363 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.363 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.363 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2023-09-20T22:05:07.364 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2023-09-20T22:05:07.364 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.364 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.364 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.364 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-20T22:05:07.365 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.365 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.365 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.365 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:07.365 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.365 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.366 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.366 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.366 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2023-09-20T22:05:07.366 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2023-09-20T22:05:07.366 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.367 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.367 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.367 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-20T22:05:07.367 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.367 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.367 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.368 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2023-09-20T22:05:07.368 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.368 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.368 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.369 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2023-09-20T22:05:07.369 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2023-09-20T22:05:07.369 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.369 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.369 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.370 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-20T22:05:07.370 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.370 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.370 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.370 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2023-09-20T22:05:07.370 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.371 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.371 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.371 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.371 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2023-09-20T22:05:07.371 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2023-09-20T22:05:07.372 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.372 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.372 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.372 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.372 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.372 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.373 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.373 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2023-09-20T22:05:07.373 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.373 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.373 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.374 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.374 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2023-09-20T22:05:07.374 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2023-09-20T22:05:07.374 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.374 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.375 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.375 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:07.375 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.375 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.375 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.375 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-20T22:05:07.376 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.376 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.376 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.376 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.376 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2023-09-20T22:05:07.377 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2023-09-20T22:05:07.377 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.377 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.377 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.377 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2023-09-20T22:05:07.377 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.378 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.378 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.378 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-20T22:05:07.378 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.378 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.379 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.379 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.379 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2023-09-20T22:05:07.379 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2023-09-20T22:05:07.379 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.380 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.380 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.380 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2023-09-20T22:05:07.380 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.380 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.380 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.381 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-20T22:05:07.381 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.381 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.381 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.381 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.382 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2023-09-20T22:05:07.382 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2023-09-20T22:05:07.382 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.382 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.382 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.383 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2023-09-20T22:05:07.383 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.383 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.383 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.383 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2023-09-20T22:05:07.383 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.384 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.384 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.384 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.384 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2023-09-20T22:05:07.384 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2023-09-20T22:05:07.385 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.385 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.385 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.385 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.385 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.386 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.386 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.386 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-20T22:05:07.386 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.386 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.386 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.387 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2023-09-20T22:05:07.387 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2023-09-20T22:05:07.387 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.387 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.388 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.388 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:07.388 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.388 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.388 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.389 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2023-09-20T22:05:07.389 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.389 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.389 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.389 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.389 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2023-09-20T22:05:07.390 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2023-09-20T22:05:07.390 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.390 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.390 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.391 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:07.391 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.391 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.391 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.391 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2023-09-20T22:05:07.391 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.392 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.392 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.392 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2023-09-20T22:05:07.392 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2023-09-20T22:05:07.393 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.393 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.393 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.393 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.393 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.394 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.394 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.394 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-20T22:05:07.394 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.394 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.394 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.395 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.395 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.395 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.395 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.395 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.396 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.396 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.396 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.396 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.396 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.397 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.397 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.397 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.397 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.397 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.398 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.398 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.398 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.398 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.398 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.399 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.399 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2023-09-20T22:05:07.399 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2023-09-20T22:05:07.399 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.399 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.400 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.400 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.400 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.400 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.400 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.401 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-20T22:05:07.401 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.401 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.401 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.401 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.402 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.402 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.402 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.402 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.402 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.402 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.403 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.403 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.403 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.403 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-20T22:05:07.403 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.404 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.404 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.404 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.404 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.404 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.404 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.405 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.405 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.405 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.405 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.405 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.406 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.406 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.406 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.406 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.406 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.407 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.407 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.407 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.407 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.407 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.408 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.408 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.408 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.408 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.408 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.409 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.409 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.409 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.409 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.409 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.409 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.410 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.410 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.410 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.410 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.410 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.411 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.411 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-20T22:05:07.411 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-20T22:05:07.411 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.411 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.412 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.412 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.412 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.412 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.412 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.413 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-20T22:05:07.413 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.413 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.413 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.413 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.413 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-20T22:05:07.414 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-20T22:05:07.414 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.414 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.414 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.414 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2023-09-20T22:05:07.415 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.415 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.415 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.415 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-20T22:05:07.415 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.416 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.416 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.416 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.416 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2023-09-20T22:05:07.416 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2023-09-20T22:05:07.416 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.417 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.417 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.417 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:07.417 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.417 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.418 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.418 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2023-09-20T22:05:07.418 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.418 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.418 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.419 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.419 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2023-09-20T22:05:07.419 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2023-09-20T22:05:07.419 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.419 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.419 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.420 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:07.420 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.420 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.420 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.420 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-20T22:05:07.421 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.421 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.421 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.421 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2023-09-20T22:05:07.421 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2023-09-20T22:05:07.422 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.422 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.422 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.422 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.422 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.423 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.423 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.423 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2023-09-20T22:05:07.423 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.423 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.424 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.424 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2023-09-20T22:05:07.424 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2023-09-20T22:05:07.424 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.424 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.425 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.425 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2023-09-20T22:05:07.425 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.425 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.425 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.426 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2023-09-20T22:05:07.426 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2023-09-20T22:05:07.426 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.426 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2023-09-20T22:05:07.426 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.427 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.427 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.427 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.427 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2023-09-20T22:05:07.427 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2023-09-20T22:05:07.427 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.428 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.428 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.428 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-20T22:05:07.428 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.428 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.429 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.429 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-20T22:05:07.429 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.429 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.429 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.430 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.430 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2023-09-20T22:05:07.430 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2023-09-20T22:05:07.430 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.430 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.431 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.431 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.431 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.431 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.431 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.431 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-20T22:05:07.432 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.432 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.432 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.432 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2023-09-20T22:05:07.433 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2023-09-20T22:05:07.433 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.433 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.433 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.433 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.433 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.434 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.434 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.434 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-20T22:05:07.434 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.434 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.435 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.435 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2023-09-20T22:05:07.435 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2023-09-20T22:05:07.435 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.436 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.436 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.436 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.436 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.436 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.436 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.437 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-20T22:05:07.437 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.437 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.437 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.437 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.438 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2023-09-20T22:05:07.438 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2023-09-20T22:05:07.438 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.438 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.438 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.439 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.439 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.439 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.439 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.439 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-20T22:05:07.440 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.440 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.440 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.440 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2023-09-20T22:05:07.440 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2023-09-20T22:05:07.441 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.441 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.441 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.441 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.441 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.442 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.442 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.442 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-20T22:05:07.442 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.442 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.442 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.443 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2023-09-20T22:05:07.443 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2023-09-20T22:05:07.443 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.443 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.444 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.444 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2023-09-20T22:05:07.444 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.444 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.444 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.445 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2023-09-20T22:05:07.445 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.445 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.445 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.445 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2023-09-20T22:05:07.446 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2023-09-20T22:05:07.446 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.446 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.446 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.446 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.447 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.447 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.447 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.447 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-20T22:05:07.447 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.448 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.448 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.448 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.448 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2023-09-20T22:05:07.448 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2023-09-20T22:05:07.448 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.449 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.449 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.449 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.449 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.449 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.450 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.450 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-20T22:05:07.450 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.450 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.450 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.451 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2023-09-20T22:05:07.451 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2023-09-20T22:05:07.451 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.451 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.451 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.452 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.452 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.452 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.452 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.452 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-20T22:05:07.452 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.453 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.453 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.453 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2023-09-20T22:05:07.453 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2023-09-20T22:05:07.454 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.454 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.454 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.454 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.454 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.455 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.455 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.455 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2023-09-20T22:05:07.455 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.455 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.456 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.456 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2023-09-20T22:05:07.456 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2023-09-20T22:05:07.456 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.456 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.457 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.457 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.457 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.457 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.457 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.458 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-20T22:05:07.458 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.458 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.458 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.459 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2023-09-20T22:05:07.459 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2023-09-20T22:05:07.459 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.459 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.459 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.460 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.460 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.460 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.460 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.460 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-20T22:05:07.460 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.461 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.461 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.461 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.461 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.461 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.462 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.462 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2023-09-20T22:05:07.462 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.462 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.462 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.463 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.463 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.463 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.463 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.463 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.463 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.464 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.464 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.464 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.464 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.464 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.465 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.465 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.465 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.465 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.465 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.466 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.466 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.466 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.466 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.467 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.467 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.467 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.467 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.467 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.467 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.468 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.468 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.468 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.468 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.468 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.469 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.469 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.469 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.469 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.470 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.470 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.470 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.470 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.470 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.471 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.471 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.471 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.471 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.471 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.471 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.472 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.472 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.472 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.472 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.472 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.473 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.473 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.473 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.473 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.473 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.474 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.474 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.474 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.474 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.474 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.475 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.475 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.475 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.475 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.475 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.475 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.476 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.476 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2023-09-20T22:05:07.476 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2023-09-20T22:05:07.476 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.476 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.477 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.477 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2023-09-20T22:05:07.477 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.477 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.477 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.478 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-20T22:05:07.478 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.478 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.478 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.479 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2023-09-20T22:05:07.479 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2023-09-20T22:05:07.479 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.479 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.479 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.480 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2023-09-20T22:05:07.480 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.480 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.480 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.480 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-20T22:05:07.481 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.481 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.481 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.481 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.481 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.481 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.482 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.482 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2023-09-20T22:05:07.482 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.482 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.482 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.483 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-20T22:05:07.483 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-20T22:05:07.483 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.483 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.483 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.484 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:07.484 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.484 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.484 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.484 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.485 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.485 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.485 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.485 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.485 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2023-09-20T22:05:07.485 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-20T22:05:07.486 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.486 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.486 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.486 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.486 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.487 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.487 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.487 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.487 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.487 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.488 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.488 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.488 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-20T22:05:07.488 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-20T22:05:07.488 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.488 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.489 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.489 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.489 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.489 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.489 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.490 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.490 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.490 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.490 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.490 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.491 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2023-09-20T22:05:07.491 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2023-09-20T22:05:07.491 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.491 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.491 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.492 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.492 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.492 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.492 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.492 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.493 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.493 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.493 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.494 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.494 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-20T22:05:07.494 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-20T22:05:07.494 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.495 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.495 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.495 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.495 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.495 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.495 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.496 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.496 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.496 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.496 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.496 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.497 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2023-09-20T22:05:07.497 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-20T22:05:07.497 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.497 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.497 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.497 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.498 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.498 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.498 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.498 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.498 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.499 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.499 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.499 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.499 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-20T22:05:07.499 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-20T22:05:07.499 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.500 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.500 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.500 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.500 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.500 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.501 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.501 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.501 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.501 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.501 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.501 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.502 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-20T22:05:07.502 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-20T22:05:07.502 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.502 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.502 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.503 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.503 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.503 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.503 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.503 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.504 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.504 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.504 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.504 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.504 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-20T22:05:07.504 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-20T22:05:07.505 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.505 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.505 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.505 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.505 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.506 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.506 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.506 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.506 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.506 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.506 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.507 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.507 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-20T22:05:07.507 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-20T22:05:07.507 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.507 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.508 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.508 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.508 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.508 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.508 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.508 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.509 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.509 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.509 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.509 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.509 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-20T22:05:07.510 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-20T22:05:07.510 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.510 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.510 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.510 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.511 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.511 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.511 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.511 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.511 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.511 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.512 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.512 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.512 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-20T22:05:07.512 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-20T22:05:07.512 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.513 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.513 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.513 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.513 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.513 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.514 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.514 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.514 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.514 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.514 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.515 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.515 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-20T22:05:07.515 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-20T22:05:07.515 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.515 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.515 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.516 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.516 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.516 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.516 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.516 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.517 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.517 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.517 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.517 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.517 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-20T22:05:07.517 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-20T22:05:07.518 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.518 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.518 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.518 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.518 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.519 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.519 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.519 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.519 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.519 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.520 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.521 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.521 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2023-09-20T22:05:07.521 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-20T22:05:07.522 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.522 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.522 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.522 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2023-09-20T22:05:07.522 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.523 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.523 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.523 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.523 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.523 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.524 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.524 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.524 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-20T22:05:07.524 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-20T22:05:07.524 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.525 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.525 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.525 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.525 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.525 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.525 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.526 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.526 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.526 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.526 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.526 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.527 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2023-09-20T22:05:07.527 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2023-09-20T22:05:07.527 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.527 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.527 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.528 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.528 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.528 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.528 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.528 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.528 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.529 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.529 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.529 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.529 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2023-09-20T22:05:07.529 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-20T22:05:07.530 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.530 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.530 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.530 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2023-09-20T22:05:07.530 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.531 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.531 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.531 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.531 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.531 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.531 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.532 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.532 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-20T22:05:07.532 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-20T22:05:07.532 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.532 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.533 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.533 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.533 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.533 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.533 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.533 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.534 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.534 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.534 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.534 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.534 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-20T22:05:07.535 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-20T22:05:07.535 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.535 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.535 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.535 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.536 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.536 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.536 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.536 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.536 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.537 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.537 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.537 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.537 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-20T22:05:07.537 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-20T22:05:07.537 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.538 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.538 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.538 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.538 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.538 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.539 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.539 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.539 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.539 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.539 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.539 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.540 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-20T22:05:07.540 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-20T22:05:07.540 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.540 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.540 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.541 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.541 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.541 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.541 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.541 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.541 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.542 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.542 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.542 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.542 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2023-09-20T22:05:07.542 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-20T22:05:07.543 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.543 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.543 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.543 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.543 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.544 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.544 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.544 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.544 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.544 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.544 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.545 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.545 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-20T22:05:07.545 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-20T22:05:07.545 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.545 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.546 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.546 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.546 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.546 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.546 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.547 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.547 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.547 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.547 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.547 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.548 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-20T22:05:07.548 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-20T22:05:07.548 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.548 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.548 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.548 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.549 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.549 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.549 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.549 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.549 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.550 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.550 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.550 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.550 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-20T22:05:07.550 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-20T22:05:07.551 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.551 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.551 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.551 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.551 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.552 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.552 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.552 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.552 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.552 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.552 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.553 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.553 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-20T22:05:07.553 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-20T22:05:07.553 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.553 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.554 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.554 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.554 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.554 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.554 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.555 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.555 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.555 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.555 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.555 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.555 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-20T22:05:07.556 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-20T22:05:07.556 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.556 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.556 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.556 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.557 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.557 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.557 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.557 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.557 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.557 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.558 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.558 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.558 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-20T22:05:07.558 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-20T22:05:07.558 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.559 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.559 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.559 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.559 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.559 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.560 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.560 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.560 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.560 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.560 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.561 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.561 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-20T22:05:07.561 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-20T22:05:07.561 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.561 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.561 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.562 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.562 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.562 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.562 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.562 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.563 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.563 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.563 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.563 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.563 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-20T22:05:07.564 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-20T22:05:07.564 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.564 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.564 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.564 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.564 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.565 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.565 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.565 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.565 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.565 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.566 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.566 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.566 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2023-09-20T22:05:07.566 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-20T22:05:07.566 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.567 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.567 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.567 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.567 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.567 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.568 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.568 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.568 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.568 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.568 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.568 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.569 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-20T22:05:07.569 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-20T22:05:07.569 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.569 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.569 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.570 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-20T22:05:07.570 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.570 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.570 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.570 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.571 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.571 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.571 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.571 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.571 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-20T22:05:07.571 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-20T22:05:07.572 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.572 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.572 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.572 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:07.572 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.573 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.573 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.573 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.573 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.573 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.574 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.574 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-20T22:05:07.574 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-20T22:05:07.574 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.575 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.575 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.575 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2023-09-20T22:05:07.575 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.575 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.575 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.576 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.576 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.576 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.576 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.577 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-20T22:05:07.577 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-20T22:05:07.577 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.577 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.577 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.577 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2023-09-20T22:05:07.578 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.578 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.578 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.578 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.578 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.579 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.579 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.579 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2023-09-20T22:05:07.579 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-20T22:05:07.579 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.580 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.580 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.580 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.580 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.580 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.581 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.581 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.581 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.581 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.581 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.582 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-20T22:05:07.582 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-20T22:05:07.582 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.582 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.582 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.583 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2023-09-20T22:05:07.583 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.583 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.583 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.583 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.584 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.584 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.584 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.584 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.584 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2023-09-20T22:05:07.585 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-20T22:05:07.585 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.585 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.585 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.585 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.585 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.586 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.586 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.586 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.586 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.586 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.587 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.587 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2023-09-20T22:05:07.587 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-20T22:05:07.587 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.588 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.588 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.588 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.588 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.588 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.588 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.589 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.589 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.589 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.589 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.589 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.590 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-20T22:05:07.590 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-20T22:05:07.590 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.590 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.590 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.590 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:07.591 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.591 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.591 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.591 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.591 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.592 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.592 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.592 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-20T22:05:07.592 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-20T22:05:07.593 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.593 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.593 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.593 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:07.593 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.593 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.594 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.594 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.594 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.594 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.594 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.595 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-20T22:05:07.595 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-20T22:05:07.595 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.595 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.596 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.596 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:07.596 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.596 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.596 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.597 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.597 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.597 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.597 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.597 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.597 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2023-09-20T22:05:07.598 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2023-09-20T22:05:07.598 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.598 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.598 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.598 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.599 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.599 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.599 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.599 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.599 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2023-09-20T22:05:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2023-09-20T22:05:07.600 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.601 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.601 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.601 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.601 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.601 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.602 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.602 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.602 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.602 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.602 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.603 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.603 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2023-09-20T22:05:07.603 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2023-09-20T22:05:07.603 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.603 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.604 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.604 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.604 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.604 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.604 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.604 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2023-09-20T22:05:07.605 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.605 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.605 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.605 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.605 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2023-09-20T22:05:07.606 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2023-09-20T22:05:07.606 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.606 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.606 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.606 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2023-09-20T22:05:07.607 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.607 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.607 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.607 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.607 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.607 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.608 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.608 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.608 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2023-09-20T22:05:07.608 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2023-09-20T22:05:07.608 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.609 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.609 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.609 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.609 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.609 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.609 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.610 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.610 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.610 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.610 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.611 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.611 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.611 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.611 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.611 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.612 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.612 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.612 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.612 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.612 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.612 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.613 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.613 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.613 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.613 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.613 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.614 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.614 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.614 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.614 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.614 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.614 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.615 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.615 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.615 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.615 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.615 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.616 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.616 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.616 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.616 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.616 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.616 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.617 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.617 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.617 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.617 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.617 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.618 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.618 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.618 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.618 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.618 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.619 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.619 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.619 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.619 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.619 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.619 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.620 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.620 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.620 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.620 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.620 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.621 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.621 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.621 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.621 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.622 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.622 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.622 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.622 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.622 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.622 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.623 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.623 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.623 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.623 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.624 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2023-09-20T22:05:07.624 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2023-09-20T22:05:07.624 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.624 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.624 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.624 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.625 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.625 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.625 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.625 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2023-09-20T22:05:07.625 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.626 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.626 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.626 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2023-09-20T22:05:07.626 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2023-09-20T22:05:07.627 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.627 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.627 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.627 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.627 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.628 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.628 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.628 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.628 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.628 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.628 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.629 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.629 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2023-09-20T22:05:07.629 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2023-09-20T22:05:07.629 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.629 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.630 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.630 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2023-09-20T22:05:07.630 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.630 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.630 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.631 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.631 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.631 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.631 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.632 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2023-09-20T22:05:07.632 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2023-09-20T22:05:07.632 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.632 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.632 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.632 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2023-09-20T22:05:07.633 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.633 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.633 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.633 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.633 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.634 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.634 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.634 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2023-09-20T22:05:07.634 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2023-09-20T22:05:07.635 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.635 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.635 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.635 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.635 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.636 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.636 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.636 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.636 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.636 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.636 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.637 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2023-09-20T22:05:07.637 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2023-09-20T22:05:07.637 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.637 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.638 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.638 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2023-09-20T22:05:07.638 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.638 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.638 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.639 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.639 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.639 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.639 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.639 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2023-09-20T22:05:07.640 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2023-09-20T22:05:07.640 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.640 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.640 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.640 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2023-09-20T22:05:07.641 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.641 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.641 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.641 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.641 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2023-09-20T22:05:07.642 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.642 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.642 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.642 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.642 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.643 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2023-09-20T22:05:07.643 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2023-09-20T22:05:07.643 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.643 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.643 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.644 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2023-09-20T22:05:07.644 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2023-09-20T22:05:07.644 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2023-09-20T22:05:07.644 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.644 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.645 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.645 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.645 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.645 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2023-09-20T22:05:07.646 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2023-09-20T22:05:07.646 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.646 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.646 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.646 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.647 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.647 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.647 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.647 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.647 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.647 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.648 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.648 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.648 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.649 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2023-09-20T22:05:07.649 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.649 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.649 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.649 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2023-09-20T22:05:07.649 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2023-09-20T22:05:07.650 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.650 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.650 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.650 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.650 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.651 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.651 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.651 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2023-09-20T22:05:07.651 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.651 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.652 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.652 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.652 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.652 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.653 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.653 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.653 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.653 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.653 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.653 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.654 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.654 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.654 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.654 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.654 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.654 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.655 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.655 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.655 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.655 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.655 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.656 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.656 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.656 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.656 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.656 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.657 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.657 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.657 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.657 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.657 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.657 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.658 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.658 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.658 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.658 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.658 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.659 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.659 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.659 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.659 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.659 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.660 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.660 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.660 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.660 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.660 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.661 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.661 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.661 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.661 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.661 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.661 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.662 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.662 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.662 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.662 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.663 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.663 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.663 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.663 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.663 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.664 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.664 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.664 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.664 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.664 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.664 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.665 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.665 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2023-09-20T22:05:07.665 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2023-09-20T22:05:07.665 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.666 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.666 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.666 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2023-09-20T22:05:07.666 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.666 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.667 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.667 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-20T22:05:07.667 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.667 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.667 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.668 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2023-09-20T22:05:07.668 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2023-09-20T22:05:07.668 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.668 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.668 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.669 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2023-09-20T22:05:07.669 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.669 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.669 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.669 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-20T22:05:07.670 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.670 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.670 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.670 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2023-09-20T22:05:07.671 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2023-09-20T22:05:07.671 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.671 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.671 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.671 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2023-09-20T22:05:07.672 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.672 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.672 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.672 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2023-09-20T22:05:07.672 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.672 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.673 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.673 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2023-09-20T22:05:07.673 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2023-09-20T22:05:07.673 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.674 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.674 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.674 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2023-09-20T22:05:07.674 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.674 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.675 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.675 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-20T22:05:07.675 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.675 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.675 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.676 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2023-09-20T22:05:07.676 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2023-09-20T22:05:07.676 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.676 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.676 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.677 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.677 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.677 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.677 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.677 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-20T22:05:07.677 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.678 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.678 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.678 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:07.678 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:07.679 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.679 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.679 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.679 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:07.679 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.680 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.680 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.680 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2023-09-20T22:05:07.680 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.680 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.680 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.681 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2023-09-20T22:05:07.681 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2023-09-20T22:05:07.681 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.681 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.682 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.682 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2023-09-20T22:05:07.682 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.682 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.682 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.683 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-20T22:05:07.683 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.683 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.683 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.683 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.684 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.684 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.684 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2023-09-20T22:05:07.684 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.684 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.685 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.685 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.685 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.685 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.685 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.686 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.686 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.686 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.686 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.686 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.687 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.687 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.687 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.687 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.687 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.688 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.688 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.688 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.688 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.688 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.688 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.689 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.689 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.689 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.689 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.689 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.690 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.690 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.690 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.690 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.690 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.691 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.691 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.691 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.691 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.692 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.692 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.692 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.692 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.692 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.692 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.693 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.693 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.693 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.693 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.693 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.694 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.694 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.694 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.694 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.694 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.694 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.695 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.695 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.695 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.695 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.696 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.696 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.696 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.696 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.696 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.697 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.697 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.697 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.697 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.697 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.698 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.698 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2023-09-20T22:05:07.698 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2023-09-20T22:05:07.698 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.698 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.699 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.699 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2023-09-20T22:05:07.699 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.699 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.699 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.700 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.700 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.700 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.700 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.700 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2023-09-20T22:05:07.701 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2023-09-20T22:05:07.701 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.701 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.701 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.701 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2023-09-20T22:05:07.701 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.702 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.702 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.702 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.702 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.703 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.703 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.703 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:07.703 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:07.704 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.704 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.704 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.704 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:07.704 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.704 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.705 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.705 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.705 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.705 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.705 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.706 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.706 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.706 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.706 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2023-09-20T22:05:07.707 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2023-09-20T22:05:07.707 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2023-09-20T22:05:07.707 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.707 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2023-09-20T22:05:07.708 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2023-09-20T22:05:07.709 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.709 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.709 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.709 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2023-09-20T22:05:07.709 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.710 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.710 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.710 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-20T22:05:07.710 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.710 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.711 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.711 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2023-09-20T22:05:07.711 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2023-09-20T22:05:07.711 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.711 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.712 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.712 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2023-09-20T22:05:07.712 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.712 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.712 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.713 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-20T22:05:07.713 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.713 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.713 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.714 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2023-09-20T22:05:07.714 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2023-09-20T22:05:07.714 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.714 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.714 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.715 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.715 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.715 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.715 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.715 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2023-09-20T22:05:07.715 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.716 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.716 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.716 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:07.716 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:07.717 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.717 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.717 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.717 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2023-09-20T22:05:07.717 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2023-09-20T22:05:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-20T22:05:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.718 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.719 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.720 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.720 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.720 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.720 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.720 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.721 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.721 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.721 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.721 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.721 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.722 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.722 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.722 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.722 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.722 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.722 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.723 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.723 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.723 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.723 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.724 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.724 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.724 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.724 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.724 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.725 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.725 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.725 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.725 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.725 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.726 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.726 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.726 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.726 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.726 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.727 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.727 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.727 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.727 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.727 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.728 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.728 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.728 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.728 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.728 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.729 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.729 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.729 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.729 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.729 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.730 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.730 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.730 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.730 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.730 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.731 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.731 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.731 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.731 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.731 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.731 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.732 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.732 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.732 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.732 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2023-09-20T22:05:07.732 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2023-09-20T22:05:07.733 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.733 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.733 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.733 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.733 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.734 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.734 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.734 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2023-09-20T22:05:07.734 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.734 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.735 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.735 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2023-09-20T22:05:07.735 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2023-09-20T22:05:07.735 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.735 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.736 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.736 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2023-09-20T22:05:07.736 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.736 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.736 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.737 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2023-09-20T22:05:07.737 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.737 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.737 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.737 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.737 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2023-09-20T22:05:07.738 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2023-09-20T22:05:07.738 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.738 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.738 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.739 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2023-09-20T22:05:07.739 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.739 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.739 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.739 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-20T22:05:07.740 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.740 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.740 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.740 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.740 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2023-09-20T22:05:07.741 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2023-09-20T22:05:07.741 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.741 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.741 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.741 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:07.742 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:07.742 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2023-09-20T22:05:07.742 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.742 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-20T22:05:07.742 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.743 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.743 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.743 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.743 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2023-09-20T22:05:07.743 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2023-09-20T22:05:07.744 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.744 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.744 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.744 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.744 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.744 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.745 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.745 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2023-09-20T22:05:07.745 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.745 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.745 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.746 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.746 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2023-09-20T22:05:07.746 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2023-09-20T22:05:07.746 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.746 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.747 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.747 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2023-09-20T22:05:07.747 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.747 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.747 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.747 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-20T22:05:07.748 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.748 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.748 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.748 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.748 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.749 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.749 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2023-09-20T22:05:07.749 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.749 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.750 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.750 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.750 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.750 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.750 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.751 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.751 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.751 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.751 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.751 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.751 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.752 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.752 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.752 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.752 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.752 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.753 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.753 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.753 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.753 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.753 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.753 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.754 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.754 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.754 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.754 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.754 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.755 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.755 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.755 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.755 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.755 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.756 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.756 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.756 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.756 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.756 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.757 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.757 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.757 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.757 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.757 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.757 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.758 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.758 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.758 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.758 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.758 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.759 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.759 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.759 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.759 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.759 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.760 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.760 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.760 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.760 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.760 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.761 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.761 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.761 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.761 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.761 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.761 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.762 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.762 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.762 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.762 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.762 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.763 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.763 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.763 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.763 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.764 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2023-09-20T22:05:07.764 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.764 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.764 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.764 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.764 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.765 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.765 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.765 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.765 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.765 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.766 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.766 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.766 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.766 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.766 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.767 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.767 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.767 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.767 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.767 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.767 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.768 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.768 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.768 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.768 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.768 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.769 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.769 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.769 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.769 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.769 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.769 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.770 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.770 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.770 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.770 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.770 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.771 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.771 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.771 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.771 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.771 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.772 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.772 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.772 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.772 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.772 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.772 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.773 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.773 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.773 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.773 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.773 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.774 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.774 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.774 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.774 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.774 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.775 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.775 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.775 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.775 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.775 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.775 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.776 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.776 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.776 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.776 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.776 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.777 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.777 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.777 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.777 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.777 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.778 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.778 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.778 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2023-09-20T22:05:07.778 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.778 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.779 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.779 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2023-09-20T22:05:07.779 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2023-09-20T22:05:07.779 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.779 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.779 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.780 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2023-09-20T22:05:07.780 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.780 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.780 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.780 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-20T22:05:07.780 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.781 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.781 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.781 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.781 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2023-09-20T22:05:07.781 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2023-09-20T22:05:07.782 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.782 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.782 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.782 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:07.782 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2023-09-20T22:05:07.783 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.783 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.783 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-20T22:05:07.783 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.783 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.784 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.784 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.784 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.784 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.784 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.784 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.785 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.785 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.785 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.785 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.785 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.786 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.786 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.786 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.786 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.786 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.787 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.787 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.787 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.787 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.787 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.787 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.788 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.788 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.788 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.788 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.789 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.789 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.789 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.789 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.789 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.789 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.790 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.790 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.790 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.790 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.791 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.791 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.791 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.791 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.791 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.792 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.792 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.792 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.792 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.792 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.792 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.793 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.793 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.793 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.793 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.793 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.794 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.794 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.794 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.794 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.794 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.795 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.795 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.795 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.795 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.795 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.795 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.796 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.796 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.796 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.796 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.796 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.797 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.797 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.797 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.797 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.797 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.797 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2023-09-20T22:05:07.798 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.798 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.798 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.798 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2023-09-20T22:05:07.798 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2023-09-20T22:05:07.799 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.799 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.799 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.799 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2023-09-20T22:05:07.799 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.800 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.800 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.800 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2023-09-20T22:05:07.800 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-20T22:05:07.800 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.800 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.801 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.801 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.801 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.801 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.802 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.802 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.802 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.802 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.802 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.803 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.804 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.804 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.804 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.804 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.804 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.804 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.805 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.805 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.805 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.805 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.805 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.806 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.806 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.806 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.806 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.806 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.807 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.807 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.807 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.807 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.807 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.808 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.808 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.808 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.808 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.808 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.808 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.809 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.809 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.809 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.809 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.809 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.810 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.810 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.810 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.810 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.810 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.810 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.811 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.811 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.811 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.812 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.812 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.812 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.812 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.813 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.813 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.813 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.813 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.813 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.814 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.814 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.814 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.814 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2023-09-20T22:05:07.814 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2023-09-20T22:05:07.814 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.815 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.815 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.815 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.815 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.815 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.816 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.816 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2023-09-20T22:05:07.816 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-20T22:05:07.816 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.816 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.817 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2023-09-20T22:05:07.817 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2023-09-20T22:05:07.817 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.817 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.817 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.818 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2023-09-20T22:05:07.818 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.818 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.818 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.818 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2023-09-20T22:05:07.819 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.819 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.819 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.819 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.819 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2023-09-20T22:05:07.819 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2023-09-20T22:05:07.820 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.820 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.820 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.820 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2023-09-20T22:05:07.820 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.821 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.821 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.821 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2023-09-20T22:05:07.821 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.821 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.821 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.822 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.822 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2023-09-20T22:05:07.822 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2023-09-20T22:05:07.822 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.822 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.823 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.823 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.823 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.823 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.823 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.824 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2023-09-20T22:05:07.824 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.824 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.824 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.824 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.824 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2023-09-20T22:05:07.825 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2023-09-20T22:05:07.825 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.825 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.825 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.825 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2023-09-20T22:05:07.826 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.826 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.826 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.826 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-20T22:05:07.826 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-20T22:05:07.827 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.827 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.827 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.827 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.827 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.828 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2023-09-20T22:05:07.828 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.828 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.828 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.828 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.829 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.829 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.829 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.829 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.829 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.829 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.830 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.830 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.830 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.830 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.830 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.831 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.831 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.831 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.831 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.831 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.831 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.832 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.832 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.832 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.832 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.833 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.833 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.833 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.833 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.833 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.833 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.834 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.834 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.834 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.834 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.834 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.835 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.835 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.835 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.835 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.835 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.835 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.836 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.836 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.836 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.836 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.836 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.837 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.837 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.837 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.837 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.837 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.838 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.838 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.838 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.838 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.838 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.839 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.839 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.839 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.839 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.839 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.839 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.840 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.840 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.840 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.840 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.840 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.841 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.841 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.841 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.841 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.841 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.842 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.842 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.842 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2023-09-20T22:05:07.842 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.842 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.843 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.843 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.843 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.843 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.843 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.843 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.844 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.844 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.844 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.844 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.844 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.845 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.845 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.845 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.845 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.845 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.846 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.846 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.846 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.846 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.847 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.847 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.847 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.847 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.848 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.848 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.848 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.848 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.848 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.848 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.849 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.849 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.849 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.849 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.849 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.850 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.850 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.850 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.850 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.850 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.851 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.851 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.851 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.851 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.851 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.851 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.852 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.852 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.852 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.852 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.852 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.853 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.853 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.853 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.853 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.853 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.854 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.854 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.854 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.854 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.854 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.855 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.855 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.855 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.855 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.855 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.856 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.856 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.856 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.856 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.856 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.857 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.857 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2023-09-20T22:05:07.857 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.857 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.857 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.858 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.858 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.858 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.858 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.858 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.859 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.859 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.859 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.859 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.859 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.859 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.860 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.860 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.860 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.860 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.860 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.861 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.861 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.861 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.861 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.861 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.862 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.862 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.862 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.862 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.862 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.863 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.863 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.863 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.863 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.863 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.863 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.864 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.864 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.864 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.864 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.864 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.865 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.865 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.865 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.865 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.865 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.866 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.866 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.866 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.866 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.866 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.867 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.867 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.867 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.867 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.867 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.867 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.868 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.868 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.868 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.868 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.868 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.869 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.869 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.869 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.869 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.869 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.869 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.870 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.870 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.870 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.870 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.870 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.871 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.871 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.871 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2023-09-20T22:05:07.871 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.872 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.872 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.872 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2023-09-20T22:05:07.872 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2023-09-20T22:05:07.872 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.873 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.873 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.873 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.873 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.873 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.873 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.874 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-20T22:05:07.874 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.874 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.874 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.875 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.875 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.875 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.875 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.875 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.876 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.876 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.876 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.876 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.876 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.877 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.877 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.877 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.877 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.877 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.877 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.878 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.878 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.878 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.878 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.878 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.879 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.879 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.879 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.879 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.880 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.880 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.880 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.880 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.880 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.880 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.881 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.881 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.881 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.881 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.881 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.882 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.882 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.882 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.882 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.882 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.883 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.883 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.883 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.883 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.883 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.884 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.884 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.884 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.884 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.884 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.884 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.885 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.885 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.885 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.885 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.886 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.886 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.886 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.886 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.886 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.887 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.887 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.887 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.887 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.887 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.888 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.888 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.888 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2023-09-20T22:05:07.888 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2023-09-20T22:05:07.888 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.889 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.889 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.889 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.889 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.889 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.890 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.890 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2023-09-20T22:05:07.890 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2023-09-20T22:05:07.890 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2023-09-20T22:05:07.890 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.891 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2023-09-20T22:05:07.891 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.891 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.891 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.891 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.892 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.892 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.892 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.892 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2023-09-20T22:05:07.892 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.892 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.893 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.893 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.893 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.893 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.893 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.894 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.894 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.894 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.894 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.894 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.895 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.895 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.895 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.895 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.895 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.895 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.896 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.896 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.896 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.896 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.896 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.897 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.897 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.897 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.897 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.897 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.898 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.898 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.898 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.898 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.898 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.899 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.899 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.899 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.899 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.899 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.900 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.900 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.900 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.900 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.900 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.900 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.901 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.901 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.901 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.901 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.901 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.902 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.902 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.902 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.902 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.902 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.902 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.903 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.903 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.903 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.903 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.903 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.904 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.904 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.904 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.904 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.904 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.905 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.906 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.906 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.906 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.906 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.906 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.907 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2023-09-20T22:05:07.907 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.907 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.907 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.907 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.908 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.908 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.908 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.908 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.908 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.908 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.909 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.909 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.909 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.909 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.909 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.910 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.910 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.910 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.910 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.910 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.911 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.911 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.911 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.911 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.911 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.911 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.912 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.912 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.912 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.912 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.912 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.913 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.913 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.913 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.913 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.913 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.913 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.914 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.914 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.914 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.914 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.914 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.915 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.915 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.915 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.915 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.915 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.915 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.916 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.916 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.916 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.916 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.916 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.917 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.917 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.917 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.917 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.917 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.917 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.918 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.918 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.918 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.918 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.918 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.919 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.919 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.919 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.919 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.919 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.920 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.920 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.920 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.920 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.920 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.921 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2023-09-20T22:05:07.921 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.921 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.921 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.922 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.922 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.922 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.922 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.922 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.923 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.923 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.923 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.923 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.923 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.924 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.924 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.924 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.924 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.924 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.925 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.925 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.925 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.925 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.925 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.926 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.926 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.926 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.926 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.926 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.927 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.927 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.927 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.927 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.927 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.927 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.928 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.928 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.928 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.928 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.929 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.929 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.929 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.929 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.929 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.930 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.930 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.930 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.930 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.930 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.931 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.931 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.931 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.931 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.931 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.931 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.932 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.932 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.932 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.932 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.933 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.933 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.933 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.933 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.933 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.934 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.934 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.934 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.934 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.934 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.935 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.935 INFO:teuthology.orchestra.run.smithi022.stdout: "noautoscale": { 2023-09-20T22:05:07.935 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "noautoscale", 2023-09-20T22:05:07.935 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.936 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.936 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.936 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.936 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.936 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.936 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.937 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "global autoscale flag", 2023-09-20T22:05:07.937 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-20T22:05:07.937 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.937 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.937 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.938 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:07.938 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:07.938 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.938 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.938 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.939 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:07.939 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.939 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.939 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.939 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.940 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.940 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.940 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.940 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.940 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2023-09-20T22:05:07.940 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2023-09-20T22:05:07.941 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:07.941 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.941 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.941 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2023-09-20T22:05:07.941 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2023-09-20T22:05:07.942 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.942 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.942 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2023-09-20T22:05:07.942 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-20T22:05:07.942 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.943 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.943 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.943 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.943 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.944 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2023-09-20T22:05:07.944 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.944 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.944 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.944 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2023-09-20T22:05:07.945 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2023-09-20T22:05:07.945 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.945 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.945 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.945 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.945 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.946 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.946 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.946 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-20T22:05:07.946 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.946 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.947 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.947 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.947 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2023-09-20T22:05:07.947 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2023-09-20T22:05:07.947 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.948 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.948 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.948 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.948 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.948 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.948 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.949 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.949 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.949 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.949 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.949 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.950 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.950 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.950 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.950 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.950 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.951 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.951 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.951 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.951 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.951 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.951 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.952 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.952 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.952 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.952 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.952 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.953 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.953 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.953 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.953 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.954 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.954 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.954 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.954 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.954 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.955 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.955 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.955 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.955 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.955 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.956 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.956 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.956 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.956 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.956 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.957 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.957 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.957 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.957 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.958 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.958 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.958 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.958 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.958 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.958 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.959 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.959 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.959 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.959 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.959 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.960 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.960 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.960 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.960 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.960 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.961 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.961 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.961 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.961 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.961 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.962 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.962 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.962 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.962 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.962 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.963 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.963 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2023-09-20T22:05:07.963 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2023-09-20T22:05:07.963 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.964 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.964 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.964 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2023-09-20T22:05:07.964 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.964 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.964 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.965 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2023-09-20T22:05:07.965 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.965 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.965 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.965 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.966 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:07.966 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:07.966 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:07.966 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.966 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.967 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:07.967 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.967 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.967 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.967 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2023-09-20T22:05:07.968 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.968 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.968 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.968 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.968 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:07.968 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.969 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:07.969 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2023-09-20T22:05:07.969 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:07.969 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:07.969 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:07.970 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2023-09-20T22:05:07.970 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2023-09-20T22:05:07.970 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.970 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.970 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.971 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.971 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.971 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.971 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.971 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.972 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.972 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.972 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.972 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.972 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2023-09-20T22:05:07.972 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2023-09-20T22:05:07.973 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.973 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.973 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.973 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:07.973 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.974 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.974 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.974 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-20T22:05:07.974 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-20T22:05:07.974 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.975 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.975 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:07.975 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:07.975 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.975 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.976 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.976 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.976 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.976 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.976 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.977 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.977 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.977 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.977 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.977 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.978 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.978 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.978 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.978 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.978 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.979 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.979 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.979 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:07.979 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:07.979 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.979 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.980 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.980 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.980 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.980 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.980 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.981 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.981 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.981 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.981 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.982 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:07.982 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:07.982 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.982 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.982 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.983 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:07.983 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.983 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.983 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:07.983 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:07.983 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:07.984 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:07.984 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:07.984 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:07.984 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:07.984 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:07.984 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.985 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.985 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.985 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.985 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:07.986 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:07.986 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:07.986 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.986 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:07.986 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:07.987 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.987 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.987 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.987 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.987 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.987 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.988 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.988 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2023-09-20T22:05:07.988 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2023-09-20T22:05:07.988 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.989 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.989 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.989 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:07.989 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.989 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.990 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.990 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.990 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.990 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.991 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.991 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-20T22:05:07.991 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-20T22:05:07.991 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.991 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.992 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.992 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2023-09-20T22:05:07.992 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.992 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.992 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.993 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.993 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.993 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.993 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.994 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2023-09-20T22:05:07.994 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2023-09-20T22:05:07.994 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:07.994 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.994 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.995 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2023-09-20T22:05:07.995 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.995 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.995 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.995 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:07.995 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.996 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.996 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.996 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2023-09-20T22:05:07.996 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2023-09-20T22:05:07.997 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:07.997 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:07.997 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:07.997 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2023-09-20T22:05:07.997 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:07.998 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:07.998 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:07.998 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-20T22:05:07.998 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:07.998 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:07.999 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:07.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:07.999 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2023-09-20T22:05:07.999 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2023-09-20T22:05:07.999 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:07.999 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.000 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.000 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2023-09-20T22:05:08.000 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.000 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.000 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.001 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-20T22:05:08.001 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.001 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.001 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.001 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2023-09-20T22:05:08.002 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2023-09-20T22:05:08.002 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.002 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.002 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.002 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2023-09-20T22:05:08.003 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.003 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.003 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.003 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.003 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.003 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.004 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.004 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2023-09-20T22:05:08.004 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2023-09-20T22:05:08.004 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.005 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.005 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.005 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2023-09-20T22:05:08.005 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.005 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.006 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.006 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2023-09-20T22:05:08.006 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2023-09-20T22:05:08.006 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.006 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.007 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.007 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.007 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.007 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2023-09-20T22:05:08.007 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2023-09-20T22:05:08.008 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.008 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.008 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.008 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2023-09-20T22:05:08.008 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2023-09-20T22:05:08.009 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2023-09-20T22:05:08.009 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.009 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.009 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.009 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.010 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.010 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.010 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.010 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.010 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2023-09-20T22:05:08.011 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.011 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.011 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.011 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.011 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.012 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.012 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.012 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.012 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.012 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.012 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.013 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.013 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.013 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.013 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.013 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.013 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.014 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.014 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.014 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.014 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.014 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.015 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.015 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.015 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.015 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.016 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.016 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.016 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.016 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.016 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.017 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.017 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.017 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.017 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.017 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.018 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.018 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.018 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.018 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.018 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.019 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.019 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.019 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.019 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.019 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.020 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.020 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.020 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.020 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.020 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.020 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.021 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.021 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.021 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.021 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.022 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.022 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.022 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.022 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.022 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.023 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.023 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.023 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.023 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.023 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.023 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.024 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.024 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.024 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2023-09-20T22:05:08.024 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2023-09-20T22:05:08.025 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.025 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.025 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.025 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:08.025 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.026 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.026 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.026 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.026 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.026 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.027 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.027 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2023-09-20T22:05:08.027 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2023-09-20T22:05:08.027 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.028 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.028 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.028 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.028 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.028 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.029 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.029 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.029 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.029 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.029 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.030 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2023-09-20T22:05:08.030 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2023-09-20T22:05:08.030 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.030 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.030 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.031 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.031 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.031 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.031 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.031 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.032 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.032 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.032 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.032 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.032 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.033 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.033 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2023-09-20T22:05:08.033 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.033 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.033 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.034 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2023-09-20T22:05:08.034 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2023-09-20T22:05:08.034 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.034 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.034 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.035 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.035 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.035 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.035 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.035 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.035 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.036 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.036 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.036 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2023-09-20T22:05:08.036 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2023-09-20T22:05:08.037 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.037 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.037 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.037 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.037 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.038 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.038 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.038 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.038 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.038 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.039 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.039 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.039 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.039 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.039 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.039 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.040 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.040 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.040 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.040 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.040 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.041 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.041 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.041 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.041 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.041 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.041 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.042 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.042 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.042 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.042 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.042 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.043 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.043 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.043 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.043 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.043 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.044 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.044 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.044 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.044 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.044 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.044 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.045 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.045 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.045 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.045 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.046 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.046 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.046 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.046 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.046 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.047 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.047 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.047 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.047 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.047 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.047 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.048 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.048 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.048 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.048 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.048 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.049 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.049 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.049 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.049 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.049 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.050 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.050 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.050 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.050 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.050 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.051 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.051 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.051 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.051 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.051 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.052 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.052 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2023-09-20T22:05:08.052 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2023-09-20T22:05:08.052 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.053 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.053 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.053 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.053 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.053 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.053 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.054 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.054 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.054 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.054 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.054 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.055 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2023-09-20T22:05:08.055 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2023-09-20T22:05:08.055 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.055 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.055 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.056 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.056 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.056 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.056 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.056 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.056 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.057 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.057 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.057 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.057 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.057 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.058 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.058 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2023-09-20T22:05:08.058 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.058 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.058 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.059 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.059 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.059 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.059 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.059 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.060 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.060 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.060 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.060 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.060 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.061 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.061 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.061 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.061 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.061 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.062 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.062 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.062 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.062 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.062 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.063 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.063 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.063 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.063 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.063 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.064 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.064 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.064 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.064 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.064 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.065 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.065 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.065 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.065 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.065 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.066 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.066 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.066 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.066 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.066 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.067 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.067 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.067 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.067 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.067 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.067 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.068 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.068 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.068 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.068 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.068 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.069 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.069 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.069 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.069 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.069 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.069 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.070 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.070 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.070 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.070 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.071 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.071 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.071 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.071 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.071 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.071 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.072 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.072 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.072 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.072 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.072 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.073 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.073 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2023-09-20T22:05:08.073 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.073 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.073 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.074 INFO:teuthology.orchestra.run.smithi022.stdout: "drive_group_interval": { 2023-09-20T22:05:08.074 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "drive_group_interval", 2023-09-20T22:05:08.074 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:08.074 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.074 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.074 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300.0", 2023-09-20T22:05:08.075 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.075 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.075 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.075 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-20T22:05:08.075 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.076 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.076 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.076 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.076 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.076 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.077 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.077 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.077 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.077 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.077 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.078 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.078 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.078 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.078 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.078 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.078 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.079 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.079 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.079 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.079 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.079 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.080 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.080 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.080 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.080 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.081 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.081 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.081 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.081 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.081 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.081 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.082 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.082 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.082 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.082 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.082 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.083 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.083 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.083 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.083 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.083 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.084 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.084 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.084 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.084 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.084 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.084 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.085 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.085 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.085 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.085 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.085 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.086 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.086 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.086 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.086 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.086 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.087 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.087 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.087 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.087 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.087 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.088 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.088 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.088 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.088 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.088 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.088 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.089 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.089 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.089 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.089 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2023-09-20T22:05:08.090 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2023-09-20T22:05:08.090 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.090 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.090 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.090 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2023-09-20T22:05:08.091 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.091 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.091 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.091 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-20T22:05:08.091 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.091 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.092 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.092 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.092 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.092 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.092 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.093 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2023-09-20T22:05:08.093 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.093 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.093 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.093 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.094 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.094 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.094 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.094 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.094 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.094 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.095 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.095 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.095 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.095 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.095 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.096 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.096 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.096 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.096 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.096 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.097 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.097 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.097 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.097 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.097 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.098 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.098 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.098 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.098 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.098 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.099 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.099 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.099 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.099 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.099 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.099 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.100 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.100 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.100 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.100 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.100 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.101 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.101 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.101 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.101 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.101 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.102 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.102 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.102 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.102 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.102 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.103 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.103 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.103 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.103 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.103 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.103 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.104 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.104 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.104 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.104 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.105 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.105 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.105 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.105 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.105 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.106 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.106 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.106 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.106 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.106 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.107 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2023-09-20T22:05:08.107 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2023-09-20T22:05:08.107 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.107 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.108 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.108 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.108 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.108 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.108 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.108 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.109 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.109 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.109 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.109 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2023-09-20T22:05:08.110 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2023-09-20T22:05:08.110 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.110 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.110 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.110 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2023-09-20T22:05:08.111 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.111 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.111 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.111 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.111 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.111 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.112 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.112 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2023-09-20T22:05:08.112 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2023-09-20T22:05:08.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.113 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.113 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.113 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.113 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.113 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.113 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.114 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.114 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.114 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.114 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.115 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2023-09-20T22:05:08.115 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2023-09-20T22:05:08.115 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.115 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.115 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.116 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.116 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.116 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.116 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.116 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.117 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.117 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.117 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.117 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2023-09-20T22:05:08.117 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2023-09-20T22:05:08.118 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:08.118 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.118 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.118 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.118 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.119 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.119 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.119 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.119 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.119 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.120 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.120 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2023-09-20T22:05:08.120 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2023-09-20T22:05:08.120 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.120 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.121 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.121 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.121 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.121 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.121 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.122 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.122 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.122 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.122 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.122 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.123 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2023-09-20T22:05:08.123 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2023-09-20T22:05:08.123 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.123 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.123 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.123 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.124 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.124 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.124 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.124 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.124 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.125 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.125 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.125 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.125 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2023-09-20T22:05:08.125 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2023-09-20T22:05:08.126 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.126 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.126 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.126 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.126 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.127 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.127 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.127 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.127 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.127 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.127 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.128 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2023-09-20T22:05:08.128 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2023-09-20T22:05:08.128 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.128 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.129 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.129 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.129 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:08.129 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2023-09-20T22:05:08.129 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.129 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.130 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.130 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.130 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.130 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2023-09-20T22:05:08.131 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2023-09-20T22:05:08.131 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.131 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.131 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.131 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.133 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.134 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.134 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.134 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.134 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.134 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.135 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.135 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2023-09-20T22:05:08.135 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2023-09-20T22:05:08.135 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.135 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.136 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.136 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.136 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.136 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.136 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.137 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.137 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.137 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.137 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.138 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2023-09-20T22:05:08.138 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2023-09-20T22:05:08.138 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.138 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.138 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.139 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.139 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.139 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.139 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.139 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.140 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.140 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.140 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.140 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.140 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.141 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.141 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.141 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2023-09-20T22:05:08.141 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.141 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.142 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.142 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2023-09-20T22:05:08.142 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2023-09-20T22:05:08.142 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.142 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.143 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.143 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.143 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.143 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.143 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.143 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2023-09-20T22:05:08.144 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.144 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.144 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.144 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.144 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2023-09-20T22:05:08.145 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2023-09-20T22:05:08.145 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.145 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.145 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.145 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.145 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.146 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.146 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.146 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2023-09-20T22:05:08.146 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.147 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.147 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.147 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.147 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.147 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.148 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.148 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.148 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.148 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.148 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.149 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.149 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.149 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.149 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.149 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.149 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.150 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.150 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.150 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.150 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.150 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.151 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.151 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.151 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.151 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.152 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.152 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.152 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.152 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.152 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.152 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.153 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.153 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.153 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.153 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.154 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.154 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.154 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.154 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.155 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.155 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.155 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.155 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.155 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.155 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.156 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.156 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.156 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.156 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.156 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.157 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.157 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.157 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.157 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.157 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.158 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.158 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.158 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.158 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.158 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.159 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.159 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.159 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.159 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.159 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.160 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.160 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.160 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.160 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.160 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.161 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.161 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2023-09-20T22:05:08.161 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.161 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.161 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.162 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.162 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.162 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.162 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.162 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.163 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.163 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.163 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.163 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.163 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.164 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.164 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.164 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.164 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.164 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.164 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.165 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.165 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.165 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.165 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.165 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.166 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.166 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.166 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.166 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.166 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.167 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.167 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.167 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.167 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.167 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.168 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.168 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.168 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.168 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.169 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.169 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.169 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.169 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.169 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.169 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.170 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.170 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.170 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.170 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.170 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.171 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.171 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.171 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.171 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.171 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.172 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.172 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.172 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.172 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.172 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.173 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.173 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.173 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.173 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.173 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.174 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.174 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.174 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.174 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.174 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.175 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.175 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.175 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.175 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.175 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.176 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2023-09-20T22:05:08.176 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.176 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.176 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.176 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.177 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.177 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.177 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.177 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.177 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.178 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.178 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.178 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.178 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.178 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.179 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.179 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.179 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.179 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.179 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.180 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.180 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.180 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.180 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.180 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.181 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.181 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.181 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.181 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.181 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.182 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.182 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.182 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.182 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.182 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.183 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.183 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.183 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.183 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.183 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.184 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.184 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.184 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.184 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.184 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.185 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.185 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.185 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.185 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.185 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.186 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.186 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.186 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.186 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.186 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.187 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.187 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.187 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.187 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.187 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.187 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.188 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.188 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.188 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.188 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.188 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.189 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.189 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.189 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.189 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.189 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.189 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.190 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.190 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.190 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2023-09-20T22:05:08.190 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.191 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.191 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.191 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2023-09-20T22:05:08.191 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2023-09-20T22:05:08.191 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.192 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.192 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.192 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-20T22:05:08.192 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.192 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.192 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.193 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.193 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.193 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.193 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.194 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:08.194 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:08.194 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.194 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.194 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.195 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2023-09-20T22:05:08.195 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.195 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.195 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.195 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.196 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.196 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.196 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.196 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.197 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.197 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.197 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.197 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.197 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.198 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.198 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.198 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.198 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.198 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.198 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.199 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.199 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.199 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.199 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.199 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.200 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.200 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.200 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.200 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.201 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.201 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.201 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.201 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.201 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.201 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.202 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.202 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.202 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.202 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.202 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.203 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.203 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.203 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.203 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.203 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.204 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.204 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.204 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.204 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.204 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.204 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.205 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.205 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.205 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.205 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.205 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.206 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.206 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.206 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.206 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.206 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.207 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.207 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.207 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.207 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.207 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.208 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.208 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.208 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.208 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.208 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.208 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.209 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.209 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.209 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.209 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.209 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.210 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.210 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.210 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2023-09-20T22:05:08.210 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.211 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.211 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.211 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2023-09-20T22:05:08.211 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2023-09-20T22:05:08.211 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.212 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.212 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.212 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.212 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.212 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.212 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.213 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-20T22:05:08.213 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.213 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.213 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.213 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.213 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2023-09-20T22:05:08.214 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2023-09-20T22:05:08.214 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.214 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.214 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.214 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.215 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.215 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.215 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.215 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-20T22:05:08.215 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.216 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.216 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.216 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2023-09-20T22:05:08.216 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2023-09-20T22:05:08.217 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.217 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.217 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.217 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.217 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.218 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.218 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.218 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-20T22:05:08.218 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.218 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.219 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.219 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2023-09-20T22:05:08.219 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2023-09-20T22:05:08.219 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.219 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.220 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.220 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.220 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.220 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.220 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.221 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-20T22:05:08.221 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.221 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.221 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.222 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2023-09-20T22:05:08.222 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2023-09-20T22:05:08.222 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.222 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.222 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.223 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.223 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.223 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.223 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.223 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-20T22:05:08.223 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.224 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.224 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.224 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2023-09-20T22:05:08.224 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2023-09-20T22:05:08.225 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.225 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.225 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.225 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.225 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.226 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.226 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.226 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.226 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.226 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.226 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.227 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2023-09-20T22:05:08.227 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2023-09-20T22:05:08.227 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.227 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.228 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.228 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.228 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.228 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.228 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.228 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.229 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.229 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.229 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.229 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2023-09-20T22:05:08.230 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2023-09-20T22:05:08.230 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.230 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.230 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.230 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-20T22:05:08.231 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.231 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.231 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.231 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.231 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.232 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.232 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.232 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2023-09-20T22:05:08.232 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2023-09-20T22:05:08.232 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.233 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.233 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.233 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.233 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.233 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.233 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.234 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.234 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.234 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.234 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.235 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:08.235 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:08.235 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.235 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.235 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.236 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2023-09-20T22:05:08.236 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2023-09-20T22:05:08.236 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.236 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.236 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.237 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.237 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.237 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.237 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2023-09-20T22:05:08.238 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2023-09-20T22:05:08.238 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.238 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.238 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.238 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2023-09-20T22:05:08.239 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.239 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.239 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.239 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.239 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.239 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.240 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.240 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2023-09-20T22:05:08.240 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2023-09-20T22:05:08.240 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.241 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.241 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.241 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.241 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.241 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.242 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.242 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.242 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.242 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.242 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.243 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.243 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2023-09-20T22:05:08.243 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2023-09-20T22:05:08.243 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.243 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.243 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.244 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.244 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.244 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.244 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.244 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.245 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.245 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.245 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.245 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.246 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.246 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.246 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.246 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.246 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.246 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.247 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.247 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.247 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.247 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.247 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.248 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.248 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.248 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.248 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.248 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.248 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.249 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.249 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.249 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.249 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.249 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.250 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.250 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.250 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.250 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.250 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.251 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.251 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.251 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.251 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.251 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.252 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.252 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.252 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.252 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.252 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.253 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.253 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.253 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.253 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.253 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.253 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.254 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.254 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.254 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.254 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.254 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.255 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.255 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.255 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.255 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.255 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.256 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.256 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.256 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.256 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.257 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.257 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.257 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.257 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.257 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.257 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.258 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.258 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.258 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.258 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.258 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.259 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2023-09-20T22:05:08.259 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2023-09-20T22:05:08.259 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.259 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.259 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.260 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.260 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.260 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.260 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.260 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.260 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.261 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.261 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.261 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.261 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2023-09-20T22:05:08.261 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2023-09-20T22:05:08.262 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.262 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.262 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.262 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.262 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.263 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.263 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.263 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.263 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.263 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.264 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.264 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.264 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2023-09-20T22:05:08.264 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2023-09-20T22:05:08.264 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.265 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.265 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.265 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-20T22:05:08.265 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.265 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.266 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.266 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.266 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.266 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.266 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.267 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.267 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.267 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.267 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.267 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2023-09-20T22:05:08.268 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.268 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.268 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.268 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.268 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.268 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.269 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.269 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.269 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.269 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.269 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.270 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.270 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.270 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.270 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.270 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.271 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.271 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.271 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.271 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.271 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.272 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.272 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.272 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.272 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.272 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.272 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.273 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.273 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.273 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.273 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.273 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.274 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.274 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.274 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.274 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.274 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.275 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.275 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.275 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.275 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.275 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.276 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.276 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.276 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.276 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.276 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.277 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.277 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.277 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.277 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.277 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.278 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.278 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.278 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.278 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.278 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.279 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.279 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.279 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.279 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.279 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.279 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.280 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.280 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.280 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.280 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.280 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.281 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.281 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.281 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.281 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.281 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.282 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.282 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.282 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2023-09-20T22:05:08.282 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.282 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.283 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.283 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.283 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.283 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.283 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.283 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.284 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.284 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.284 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.284 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.284 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.285 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.286 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.287 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.288 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.288 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.288 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.288 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.288 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.288 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.289 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.289 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.289 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.289 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.290 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.290 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.290 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.290 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.290 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.290 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.291 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.291 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.291 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.291 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.291 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.292 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.292 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.292 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.292 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.292 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.293 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.293 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.293 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.293 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.293 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.294 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.294 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.294 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.294 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.294 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.295 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.295 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.295 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.295 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.295 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.296 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.296 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2023-09-20T22:05:08.296 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2023-09-20T22:05:08.296 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.297 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.297 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.297 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2023-09-20T22:05:08.297 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.297 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.298 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.298 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-20T22:05:08.298 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.298 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.298 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.299 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2023-09-20T22:05:08.299 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2023-09-20T22:05:08.299 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.299 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.300 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.300 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.300 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.300 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.300 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.301 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2023-09-20T22:05:08.301 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.301 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.301 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.301 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.301 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2023-09-20T22:05:08.302 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2023-09-20T22:05:08.302 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.302 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.302 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.302 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:08.303 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.303 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.303 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.303 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-20T22:05:08.303 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.304 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.304 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.304 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.304 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2023-09-20T22:05:08.304 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2023-09-20T22:05:08.305 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.305 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.305 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.305 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.305 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.305 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.306 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.306 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2023-09-20T22:05:08.306 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.306 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.306 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.307 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.307 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.307 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.307 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.307 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2023-09-20T22:05:08.308 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.308 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.308 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.308 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2023-09-20T22:05:08.308 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2023-09-20T22:05:08.309 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:08.309 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.309 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.309 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2023-09-20T22:05:08.309 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.309 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.310 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.310 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.310 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.310 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.310 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.311 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2023-09-20T22:05:08.311 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2023-09-20T22:05:08.311 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.311 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.312 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.312 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.312 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.312 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.312 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.313 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.313 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.313 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.313 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.313 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.314 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:08.314 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:08.314 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.314 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.314 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.315 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:08.315 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.315 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.315 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.315 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.315 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.316 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.316 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.316 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.316 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.316 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.317 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.317 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.317 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.317 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.317 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.318 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.318 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.318 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.318 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.318 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.319 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.319 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.319 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.319 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.319 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.320 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.320 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.320 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.320 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.320 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.320 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.321 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.321 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.321 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.321 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.321 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.322 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.322 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.322 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.322 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.322 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.323 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.323 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.323 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.323 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.323 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.324 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.324 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.324 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.324 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.324 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.324 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.325 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.325 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.325 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.325 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.325 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.326 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.326 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.326 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.326 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.326 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.327 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.327 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.327 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.327 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.327 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.328 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.328 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.328 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.328 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.328 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.328 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.329 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.329 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.329 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.329 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.329 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.329 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2023-09-20T22:05:08.330 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2023-09-20T22:05:08.330 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.330 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.330 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.330 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.331 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.331 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.331 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.331 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.331 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.331 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.332 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.332 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2023-09-20T22:05:08.332 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2023-09-20T22:05:08.332 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.333 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.333 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.333 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2023-09-20T22:05:08.333 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.333 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.333 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.334 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.334 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.334 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.334 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.337 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2023-09-20T22:05:08.337 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2023-09-20T22:05:08.338 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.338 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.338 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.338 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-20T22:05:08.338 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.339 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.339 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.339 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.339 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.340 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.340 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.340 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.340 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.340 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.340 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-09-20T22:05:08.341 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.341 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.341 INFO:teuthology.orchestra.run.smithi022.stdout: "modules": [ 2023-09-20T22:05:08.341 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2023-09-20T22:05:08.341 INFO:teuthology.orchestra.run.smithi022.stdout: "dashboard", 2023-09-20T22:05:08.342 INFO:teuthology.orchestra.run.smithi022.stdout: "iostat", 2023-09-20T22:05:08.342 INFO:teuthology.orchestra.run.smithi022.stdout: "nfs", 2023-09-20T22:05:08.342 INFO:teuthology.orchestra.run.smithi022.stdout: "restful" 2023-09-20T22:05:08.342 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.342 INFO:teuthology.orchestra.run.smithi022.stdout: "available_modules": [ 2023-09-20T22:05:08.343 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.343 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "alerts", 2023-09-20T22:05:08.343 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.343 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.343 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.343 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:08.344 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:08.344 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.344 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.344 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.344 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:08.345 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.345 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.345 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.345 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How frequently to reexamine health status", 2023-09-20T22:05:08.345 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.346 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.346 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.346 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.346 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.346 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.347 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.347 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.347 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.347 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.347 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.347 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.348 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.348 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.348 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.348 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.348 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.349 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.349 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.349 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.349 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.349 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.349 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.350 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.350 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.350 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.350 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.350 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.351 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.351 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.351 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.351 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.351 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.351 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.352 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.352 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.352 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.352 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.353 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.353 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.353 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.353 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.353 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.354 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.354 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.354 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.354 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.354 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.355 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.355 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.355 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.355 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.355 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.355 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.356 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.356 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.356 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.356 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.357 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.357 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.357 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.357 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.357 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.357 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.358 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.358 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.358 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.358 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.358 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.359 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.359 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.359 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.359 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_destination": { 2023-09-20T22:05:08.359 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_destination", 2023-09-20T22:05:08.360 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.360 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.360 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.360 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.360 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.360 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.361 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.361 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email address to send alerts to", 2023-09-20T22:05:08.361 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.361 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.361 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.362 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_from_name": { 2023-09-20T22:05:08.362 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_from_name", 2023-09-20T22:05:08.362 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.362 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.362 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.363 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "Ceph", 2023-09-20T22:05:08.363 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.363 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.363 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.363 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Email From: name", 2023-09-20T22:05:08.363 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.364 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.364 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.364 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.364 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_host": { 2023-09-20T22:05:08.364 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_host", 2023-09-20T22:05:08.365 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.365 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.365 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.365 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.365 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.365 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.366 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.366 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP server", 2023-09-20T22:05:08.366 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.366 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.366 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.367 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.367 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_password": { 2023-09-20T22:05:08.367 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_password", 2023-09-20T22:05:08.367 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.367 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.368 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.368 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.368 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.368 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.368 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.369 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Password to authenticate with", 2023-09-20T22:05:08.369 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.369 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.369 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.369 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.369 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_port": { 2023-09-20T22:05:08.370 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_port", 2023-09-20T22:05:08.370 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.370 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.370 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.370 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "465", 2023-09-20T22:05:08.371 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.371 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.371 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.371 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP port", 2023-09-20T22:05:08.371 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.372 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.372 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.372 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.372 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_sender": { 2023-09-20T22:05:08.372 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_sender", 2023-09-20T22:05:08.372 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.373 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.373 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.373 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.373 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.373 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.374 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.374 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SMTP envelope sender", 2023-09-20T22:05:08.374 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.374 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.374 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.374 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.375 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_ssl": { 2023-09-20T22:05:08.375 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_ssl", 2023-09-20T22:05:08.375 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.375 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.375 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.376 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.376 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.376 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.376 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.376 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use SSL to connect to SMTP server", 2023-09-20T22:05:08.377 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.377 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.377 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.377 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.377 INFO:teuthology.orchestra.run.smithi022.stdout: "smtp_user": { 2023-09-20T22:05:08.377 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "smtp_user", 2023-09-20T22:05:08.378 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.378 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.378 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.378 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.378 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.379 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.379 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.379 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "User to authenticate as", 2023-09-20T22:05:08.379 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.379 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.380 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.380 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.380 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.380 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.380 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.381 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "balancer", 2023-09-20T22:05:08.381 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.381 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.381 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.381 INFO:teuthology.orchestra.run.smithi022.stdout: "active": { 2023-09-20T22:05:08.381 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "active", 2023-09-20T22:05:08.382 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.382 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.382 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.382 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.382 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.383 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.383 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.383 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically balance PGs across cluster", 2023-09-20T22:05:08.383 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.383 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.384 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.384 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.384 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_time": { 2023-09-20T22:05:08.384 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_time", 2023-09-20T22:05:08.384 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.384 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.385 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.385 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0000", 2023-09-20T22:05:08.385 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.385 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.385 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.386 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "beginning time of day to automatically balance", 2023-09-20T22:05:08.386 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-20T22:05:08.386 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.386 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.386 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.387 INFO:teuthology.orchestra.run.smithi022.stdout: "begin_weekday": { 2023-09-20T22:05:08.387 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "begin_weekday", 2023-09-20T22:05:08.387 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:08.387 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.387 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.388 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:08.388 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2023-09-20T22:05:08.388 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2023-09-20T22:05:08.388 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.388 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-09-20T22:05:08.388 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-20T22:05:08.389 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.389 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.389 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.389 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_max_iterations": { 2023-09-20T22:05:08.389 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_max_iterations", 2023-09-20T22:05:08.390 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:08.390 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.390 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.390 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "25", 2023-09-20T22:05:08.390 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:08.391 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "250", 2023-09-20T22:05:08.391 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.391 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-09-20T22:05:08.391 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.391 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.391 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.392 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.392 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_metrics": { 2023-09-20T22:05:08.392 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_metrics", 2023-09-20T22:05:08.392 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.392 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.392 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.393 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "pgs,objects,bytes", 2023-09-20T22:05:08.393 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.393 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.393 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.393 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-09-20T22:05:08.394 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-09-20T22:05:08.394 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.394 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.394 INFO:teuthology.orchestra.run.smithi022.stdout: "crush_compat_step": { 2023-09-20T22:05:08.395 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crush_compat_step", 2023-09-20T22:05:08.395 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:08.395 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.395 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.395 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.5", 2023-09-20T22:05:08.395 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0.001", 2023-09-20T22:05:08.396 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "0.999", 2023-09-20T22:05:08.396 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.396 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "aggressiveness of optimization", 2023-09-20T22:05:08.396 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-09-20T22:05:08.397 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.397 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.397 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.397 INFO:teuthology.orchestra.run.smithi022.stdout: "end_time": { 2023-09-20T22:05:08.397 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_time", 2023-09-20T22:05:08.397 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.398 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.398 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.398 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2359", 2023-09-20T22:05:08.398 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.398 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.399 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.399 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ending time of day to automatically balance", 2023-09-20T22:05:08.399 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-09-20T22:05:08.399 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.399 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.399 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.400 INFO:teuthology.orchestra.run.smithi022.stdout: "end_weekday": { 2023-09-20T22:05:08.400 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "end_weekday", 2023-09-20T22:05:08.400 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:08.400 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.400 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.400 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:08.401 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "0", 2023-09-20T22:05:08.401 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "6", 2023-09-20T22:05:08.401 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.401 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-09-20T22:05:08.401 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-09-20T22:05:08.402 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.402 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.402 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.402 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.402 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.403 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.403 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.403 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.403 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.403 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.403 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.404 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.404 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.404 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.404 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.404 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.404 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.405 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.405 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.405 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.405 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.405 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.406 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.406 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.406 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.406 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.406 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.407 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.407 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.407 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.407 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.407 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.407 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.408 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.408 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.408 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.408 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.408 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.409 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.409 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.409 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.409 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.409 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.409 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.410 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.410 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.410 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.410 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.410 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.411 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.411 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.411 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.411 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.411 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.412 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.412 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.412 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.412 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.412 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.412 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.413 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.413 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.413 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.413 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.413 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.414 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.414 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.414 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.414 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.414 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.415 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.415 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.415 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.415 INFO:teuthology.orchestra.run.smithi022.stdout: "min_score": { 2023-09-20T22:05:08.415 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_score", 2023-09-20T22:05:08.416 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:08.416 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.416 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.416 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:08.416 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.417 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.417 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.417 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-09-20T22:05:08.417 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.417 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.417 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.418 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.418 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2023-09-20T22:05:08.418 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2023-09-20T22:05:08.418 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.418 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.419 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.419 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "upmap", 2023-09-20T22:05:08.419 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.419 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.419 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.420 INFO:teuthology.orchestra.run.smithi022.stdout: "crush-compat", 2023-09-20T22:05:08.420 INFO:teuthology.orchestra.run.smithi022.stdout: "none", 2023-09-20T22:05:08.420 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap" 2023-09-20T22:05:08.420 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.420 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Balancer mode", 2023-09-20T22:05:08.420 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.421 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.421 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.421 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_ids": { 2023-09-20T22:05:08.421 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_ids", 2023-09-20T22:05:08.421 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.422 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.422 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.422 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.422 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.422 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.423 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.423 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-09-20T22:05:08.423 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.423 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.423 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.424 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.424 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:08.424 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:08.424 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.424 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.424 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.425 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:08.425 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.425 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.425 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.425 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-09-20T22:05:08.426 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.426 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.426 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.426 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_deviation": { 2023-09-20T22:05:08.427 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_deviation", 2023-09-20T22:05:08.427 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.427 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.427 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.427 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:08.427 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:08.428 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.428 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.428 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "deviation below which no optimization is attempted", 2023-09-20T22:05:08.428 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-09-20T22:05:08.428 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.429 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.429 INFO:teuthology.orchestra.run.smithi022.stdout: "upmap_max_optimizations": { 2023-09-20T22:05:08.429 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "upmap_max_optimizations", 2023-09-20T22:05:08.429 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:08.429 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.430 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.430 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:08.430 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.430 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.430 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.431 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-09-20T22:05:08.431 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.431 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.431 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.431 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.431 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.432 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.432 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cephadm", 2023-09-20T22:05:08.432 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.432 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.433 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.433 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_down_multiplier": { 2023-09-20T22:05:08.433 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_down_multiplier", 2023-09-20T22:05:08.433 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:08.433 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.433 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.434 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2023-09-20T22:05:08.434 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.434 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.434 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.434 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-09-20T22:05:08.435 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.435 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.435 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.435 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.435 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_refresh_rate": { 2023-09-20T22:05:08.436 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_refresh_rate", 2023-09-20T22:05:08.436 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.436 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.436 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.436 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "20", 2023-09-20T22:05:08.436 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.437 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.437 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.437 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-09-20T22:05:08.437 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.437 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.438 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.438 INFO:teuthology.orchestra.run.smithi022.stdout: "agent_starting_port": { 2023-09-20T22:05:08.438 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "agent_starting_port", 2023-09-20T22:05:08.438 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.439 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.439 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.439 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4721", 2023-09-20T22:05:08.439 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.439 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.439 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.440 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-09-20T22:05:08.440 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.440 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.440 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.440 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.441 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_ptrace": { 2023-09-20T22:05:08.441 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_ptrace", 2023-09-20T22:05:08.441 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.441 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.441 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.441 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.442 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.442 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.442 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.442 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-09-20T22:05:08.442 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-09-20T22:05:08.443 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.443 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.443 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.443 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_interval": { 2023-09-20T22:05:08.443 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_interval", 2023-09-20T22:05:08.444 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.444 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.444 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.444 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:08.444 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.444 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.445 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.445 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to autotune daemon memory", 2023-09-20T22:05:08.445 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.445 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.445 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.446 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.446 INFO:teuthology.orchestra.run.smithi022.stdout: "autotune_memory_target_ratio": { 2023-09-20T22:05:08.446 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "autotune_memory_target_ratio", 2023-09-20T22:05:08.446 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:08.446 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.446 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.447 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0.7", 2023-09-20T22:05:08.447 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.447 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.447 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.447 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-09-20T22:05:08.448 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.448 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.448 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.448 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.448 INFO:teuthology.orchestra.run.smithi022.stdout: "cgroups_split": { 2023-09-20T22:05:08.449 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cgroups_split", 2023-09-20T22:05:08.449 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.449 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.449 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.449 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.449 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.450 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.450 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.450 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-09-20T22:05:08.450 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.450 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.451 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.451 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.451 INFO:teuthology.orchestra.run.smithi022.stdout: "config_checks_enabled": { 2023-09-20T22:05:08.451 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_checks_enabled", 2023-09-20T22:05:08.451 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.451 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.452 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.452 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.452 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.452 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.452 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.453 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-09-20T22:05:08.453 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.453 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.453 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.453 INFO:teuthology.orchestra.run.smithi022.stdout: "config_dashboard": { 2023-09-20T22:05:08.454 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_dashboard", 2023-09-20T22:05:08.454 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.454 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.454 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.454 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.455 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.455 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.455 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.455 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-09-20T22:05:08.455 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.455 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.456 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.456 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_alertmanager": { 2023-09-20T22:05:08.456 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_alertmanager", 2023-09-20T22:05:08.456 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.457 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.457 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.457 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-09-20T22:05:08.457 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.457 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.458 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.458 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:08.458 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.458 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.458 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.459 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_base": { 2023-09-20T22:05:08.459 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_base", 2023-09-20T22:05:08.459 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.459 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.459 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.460 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph", 2023-09-20T22:05:08.460 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.460 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.460 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.460 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Container image name, without the tag", 2023-09-20T22:05:08.460 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.461 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.461 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.461 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.461 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_elasticsearch": { 2023-09-20T22:05:08.461 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_elasticsearch", 2023-09-20T22:05:08.462 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.462 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.462 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.462 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-09-20T22:05:08.462 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.463 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.463 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.463 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "elasticsearch container image", 2023-09-20T22:05:08.463 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.463 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.463 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.464 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_grafana": { 2023-09-20T22:05:08.464 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_grafana", 2023-09-20T22:05:08.464 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.464 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.465 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.465 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-09-20T22:05:08.465 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.465 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.465 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.465 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:08.466 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.466 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.466 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.466 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_haproxy": { 2023-09-20T22:05:08.467 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_haproxy", 2023-09-20T22:05:08.467 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.467 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.467 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.467 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-09-20T22:05:08.468 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.468 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.468 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.468 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "HAproxy container image", 2023-09-20T22:05:08.468 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.469 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.469 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.469 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_agent": { 2023-09-20T22:05:08.469 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_agent", 2023-09-20T22:05:08.469 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.470 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.470 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.470 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-09-20T22:05:08.470 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.470 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.471 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.471 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger agent container image", 2023-09-20T22:05:08.471 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.471 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.471 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.472 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_collector": { 2023-09-20T22:05:08.472 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_collector", 2023-09-20T22:05:08.472 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.472 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.472 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.473 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-09-20T22:05:08.473 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.473 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.473 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.473 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger collector container image", 2023-09-20T22:05:08.474 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.474 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.474 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.474 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.474 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_jaeger_query": { 2023-09-20T22:05:08.474 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_jaeger_query", 2023-09-20T22:05:08.475 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.475 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.475 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.475 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-09-20T22:05:08.475 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Jaeger query container image", 2023-09-20T22:05:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.476 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_keepalived": { 2023-09-20T22:05:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_keepalived", 2023-09-20T22:05:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.477 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.478 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.478 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-09-20T22:05:08.478 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.478 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.478 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.479 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Keepalived container image", 2023-09-20T22:05:08.479 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.479 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.479 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.479 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.479 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_loki": { 2023-09-20T22:05:08.480 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_loki", 2023-09-20T22:05:08.480 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.480 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.480 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.480 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-09-20T22:05:08.481 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.481 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.481 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.481 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Loki container image", 2023-09-20T22:05:08.481 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.481 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.482 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.482 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.482 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_node_exporter": { 2023-09-20T22:05:08.482 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_node_exporter", 2023-09-20T22:05:08.482 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.483 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.483 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.483 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-09-20T22:05:08.484 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.484 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.484 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.485 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:08.485 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.485 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.485 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.486 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.486 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_nvmeof": { 2023-09-20T22:05:08.486 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_nvmeof", 2023-09-20T22:05:08.486 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.487 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.487 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.487 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-09-20T22:05:08.487 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.487 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.488 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.488 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Nvme-of container image", 2023-09-20T22:05:08.488 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.488 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.488 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.488 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.489 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_prometheus": { 2023-09-20T22:05:08.489 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_prometheus", 2023-09-20T22:05:08.489 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.489 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.489 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.490 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-09-20T22:05:08.490 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.490 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.490 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.490 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Prometheus container image", 2023-09-20T22:05:08.490 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.491 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.491 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.491 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.491 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_promtail": { 2023-09-20T22:05:08.491 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_promtail", 2023-09-20T22:05:08.492 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.492 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.492 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.492 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-09-20T22:05:08.492 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.492 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.493 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.493 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Promtail container image", 2023-09-20T22:05:08.493 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.493 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.493 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.494 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.494 INFO:teuthology.orchestra.run.smithi022.stdout: "container_image_snmp_gateway": { 2023-09-20T22:05:08.494 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_image_snmp_gateway", 2023-09-20T22:05:08.494 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.494 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.494 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.495 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-09-20T22:05:08.495 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.495 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.495 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.495 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "SNMP Gateway container image", 2023-09-20T22:05:08.496 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.496 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.496 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.496 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.496 INFO:teuthology.orchestra.run.smithi022.stdout: "container_init": { 2023-09-20T22:05:08.496 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "container_init", 2023-09-20T22:05:08.497 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.497 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.497 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.497 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.497 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.498 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.498 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.498 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Run podman/docker with `--init`", 2023-09-20T22:05:08.498 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.498 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.499 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.499 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.499 INFO:teuthology.orchestra.run.smithi022.stdout: "daemon_cache_timeout": { 2023-09-20T22:05:08.499 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "daemon_cache_timeout", 2023-09-20T22:05:08.499 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.499 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.500 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.500 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:08.500 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.500 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.500 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.501 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-09-20T22:05:08.501 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.501 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.501 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.501 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.501 INFO:teuthology.orchestra.run.smithi022.stdout: "default_cephadm_command_timeout": { 2023-09-20T22:05:08.502 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_cephadm_command_timeout", 2023-09-20T22:05:08.502 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.502 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.502 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.502 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "900", 2023-09-20T22:05:08.503 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.503 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.503 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.503 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-09-20T22:05:08.503 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.504 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.504 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.504 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.504 INFO:teuthology.orchestra.run.smithi022.stdout: "default_registry": { 2023-09-20T22:05:08.504 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "default_registry", 2023-09-20T22:05:08.504 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.505 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.505 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.505 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "docker.io", 2023-09-20T22:05:08.505 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.505 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.506 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.506 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-09-20T22:05:08.506 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.506 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.506 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.506 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.507 INFO:teuthology.orchestra.run.smithi022.stdout: "device_cache_timeout": { 2023-09-20T22:05:08.507 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_cache_timeout", 2023-09-20T22:05:08.507 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.507 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.507 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.508 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1800", 2023-09-20T22:05:08.508 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.508 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.508 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.508 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache device inventory", 2023-09-20T22:05:08.508 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.509 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.509 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.509 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.509 INFO:teuthology.orchestra.run.smithi022.stdout: "device_enhanced_scan": { 2023-09-20T22:05:08.509 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_enhanced_scan", 2023-09-20T22:05:08.510 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.510 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.510 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.510 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.510 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.511 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.511 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.511 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use libstoragemgmt during device scans", 2023-09-20T22:05:08.511 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.511 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.512 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.512 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.512 INFO:teuthology.orchestra.run.smithi022.stdout: "facts_cache_timeout": { 2023-09-20T22:05:08.512 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "facts_cache_timeout", 2023-09-20T22:05:08.512 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.512 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.513 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.513 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:08.513 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.513 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.513 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.514 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "seconds to cache host facts data", 2023-09-20T22:05:08.514 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.514 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.514 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.514 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.515 INFO:teuthology.orchestra.run.smithi022.stdout: "host_check_interval": { 2023-09-20T22:05:08.515 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "host_check_interval", 2023-09-20T22:05:08.515 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.515 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.515 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.515 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:08.516 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.516 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.516 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.516 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to perform a host check", 2023-09-20T22:05:08.516 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.517 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.517 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.517 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.517 INFO:teuthology.orchestra.run.smithi022.stdout: "inventory_list_all": { 2023-09-20T22:05:08.517 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "inventory_list_all", 2023-09-20T22:05:08.518 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.518 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.518 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.518 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.518 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.519 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.519 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.519 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-09-20T22:05:08.519 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.519 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.519 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.520 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.520 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.520 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.520 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.520 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.521 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.521 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.521 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.521 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.521 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.521 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.522 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.522 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.522 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.522 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.522 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.523 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.523 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.523 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.523 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.523 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.523 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.524 INFO:teuthology.orchestra.run.smithi022.stdout: "log_refresh_metadata": { 2023-09-20T22:05:08.524 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_refresh_metadata", 2023-09-20T22:05:08.524 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.524 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.524 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.525 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.525 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.525 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.525 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.525 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-09-20T22:05:08.526 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.526 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.526 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.526 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.526 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.527 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.527 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.527 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.527 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.527 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.527 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.528 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.528 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.528 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-09-20T22:05:08.528 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.528 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.529 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.529 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.529 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.529 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.529 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.529 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.530 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.530 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.530 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.530 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.530 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.530 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.531 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.531 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.531 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.531 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.531 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.532 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.532 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.532 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.532 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.532 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.532 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.533 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.533 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.533 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.533 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.533 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.533 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.534 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.534 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.534 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.534 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.534 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.535 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.535 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.535 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.535 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf": { 2023-09-20T22:05:08.535 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-09-20T22:05:08.535 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.536 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.536 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.536 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.536 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.536 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.537 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.537 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-09-20T22:05:08.537 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.537 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.537 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.538 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.538 INFO:teuthology.orchestra.run.smithi022.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-09-20T22:05:08.538 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-09-20T22:05:08.538 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.538 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.539 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.539 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "*", 2023-09-20T22:05:08.539 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.539 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.539 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.539 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-09-20T22:05:08.540 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.540 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.540 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.540 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.540 INFO:teuthology.orchestra.run.smithi022.stdout: "max_count_per_host": { 2023-09-20T22:05:08.541 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_count_per_host", 2023-09-20T22:05:08.541 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.541 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.541 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.541 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:08.541 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.542 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.542 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.542 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of daemons per service per host", 2023-09-20T22:05:08.542 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.545 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.545 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.545 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.545 INFO:teuthology.orchestra.run.smithi022.stdout: "max_osd_draining_count": { 2023-09-20T22:05:08.545 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_osd_draining_count", 2023-09-20T22:05:08.546 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.546 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.546 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.546 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:08.546 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.547 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.547 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.547 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-09-20T22:05:08.547 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.547 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.547 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.548 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.548 INFO:teuthology.orchestra.run.smithi022.stdout: "migration_current": { 2023-09-20T22:05:08.548 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "migration_current", 2023-09-20T22:05:08.548 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.548 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.549 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.549 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.549 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.549 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.549 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.550 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "internal - do not modify", 2023-09-20T22:05:08.550 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.550 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.550 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.550 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.550 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": { 2023-09-20T22:05:08.551 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mode", 2023-09-20T22:05:08.551 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.551 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.551 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.551 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "root", 2023-09-20T22:05:08.552 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.552 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.552 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.552 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm-package", 2023-09-20T22:05:08.552 INFO:teuthology.orchestra.run.smithi022.stdout: "root" 2023-09-20T22:05:08.552 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.553 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "mode for remote execution of cephadm", 2023-09-20T22:05:08.553 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.553 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.553 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.553 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.554 INFO:teuthology.orchestra.run.smithi022.stdout: "prometheus_alerts_path": { 2023-09-20T22:05:08.554 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus_alerts_path", 2023-09-20T22:05:08.554 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.554 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.554 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.555 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-09-20T22:05:08.555 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.555 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.555 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.555 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-09-20T22:05:08.555 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.556 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.556 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.556 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.556 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_insecure": { 2023-09-20T22:05:08.556 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_insecure", 2023-09-20T22:05:08.557 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.557 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.557 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.557 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.557 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.557 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.558 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.558 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-09-20T22:05:08.558 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.558 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.558 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.559 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.559 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_password": { 2023-09-20T22:05:08.559 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_password", 2023-09-20T22:05:08.559 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.559 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.559 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.560 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.560 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.560 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.560 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.560 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-09-20T22:05:08.561 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.561 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.561 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.561 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.561 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_url": { 2023-09-20T22:05:08.561 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_url", 2023-09-20T22:05:08.562 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.562 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.562 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.562 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.562 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.563 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.563 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.563 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-09-20T22:05:08.563 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.563 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.564 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.564 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.564 INFO:teuthology.orchestra.run.smithi022.stdout: "registry_username": { 2023-09-20T22:05:08.564 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "registry_username", 2023-09-20T22:05:08.564 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.564 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.565 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.565 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.565 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.565 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.565 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.566 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-09-20T22:05:08.566 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.566 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.566 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.566 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.567 INFO:teuthology.orchestra.run.smithi022.stdout: "secure_monitoring_stack": { 2023-09-20T22:05:08.567 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "secure_monitoring_stack", 2023-09-20T22:05:08.567 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.567 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.567 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.567 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.568 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.568 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.568 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.568 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-09-20T22:05:08.568 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.569 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.569 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.569 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.569 INFO:teuthology.orchestra.run.smithi022.stdout: "service_discovery_port": { 2023-09-20T22:05:08.569 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "service_discovery_port", 2023-09-20T22:05:08.569 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.570 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.570 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.570 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8765", 2023-09-20T22:05:08.570 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.570 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.571 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.571 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "cephadm service discovery port", 2023-09-20T22:05:08.571 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.571 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.571 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.572 INFO:teuthology.orchestra.run.smithi022.stdout: "ssh_config_file": { 2023-09-20T22:05:08.572 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssh_config_file", 2023-09-20T22:05:08.572 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.572 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.573 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.573 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.573 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.573 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.573 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.573 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-09-20T22:05:08.574 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.574 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.574 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.574 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.574 INFO:teuthology.orchestra.run.smithi022.stdout: "use_agent": { 2023-09-20T22:05:08.575 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_agent", 2023-09-20T22:05:08.575 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.575 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.575 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.575 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.576 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.576 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.576 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.576 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-09-20T22:05:08.576 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.576 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.577 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.577 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.577 INFO:teuthology.orchestra.run.smithi022.stdout: "use_repo_digest": { 2023-09-20T22:05:08.577 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "use_repo_digest", 2023-09-20T22:05:08.577 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.578 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.578 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.578 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.578 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.578 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.579 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.579 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-09-20T22:05:08.579 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.579 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.579 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.580 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_failed_host_check": { 2023-09-20T22:05:08.580 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_failed_host_check", 2023-09-20T22:05:08.580 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.580 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.580 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.581 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.581 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.581 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.581 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.581 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if the host check fails", 2023-09-20T22:05:08.581 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.582 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.582 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.582 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_daemons": { 2023-09-20T22:05:08.582 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_daemons", 2023-09-20T22:05:08.583 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.583 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.583 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.583 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.583 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.584 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.584 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.584 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-09-20T22:05:08.584 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.584 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.584 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.585 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_on_stray_hosts": { 2023-09-20T22:05:08.585 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_on_stray_hosts", 2023-09-20T22:05:08.585 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.585 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.586 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.586 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.586 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.586 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.586 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.587 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-09-20T22:05:08.587 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.587 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.587 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.587 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.587 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.588 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crash", 2023-09-20T22:05:08.588 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.588 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.589 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.589 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.589 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.589 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.589 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.589 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.590 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.590 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.590 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.590 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.590 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.591 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.591 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.591 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.591 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.591 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.591 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.592 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.592 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.592 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.592 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.593 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.593 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.593 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.593 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.593 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.594 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.594 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.594 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.594 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.594 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.594 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.595 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.595 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.595 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.595 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.595 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.596 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.596 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.596 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.596 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.596 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.596 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.597 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.597 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.597 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.597 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.597 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.598 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.598 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.598 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.598 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.598 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.599 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.599 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.599 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.599 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.599 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.599 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.600 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.600 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.600 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.600 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.600 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.600 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.601 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.601 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.601 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.601 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.601 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.602 INFO:teuthology.orchestra.run.smithi022.stdout: "retain_interval": { 2023-09-20T22:05:08.602 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retain_interval", 2023-09-20T22:05:08.602 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.602 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.602 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.603 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "31536000", 2023-09-20T22:05:08.603 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.603 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.603 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.603 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain crashes before pruning them", 2023-09-20T22:05:08.603 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.604 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.604 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.604 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.604 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_recent_interval": { 2023-09-20T22:05:08.604 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_recent_interval", 2023-09-20T22:05:08.604 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.605 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.605 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.605 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1209600", 2023-09-20T22:05:08.605 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.605 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.606 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.606 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "time interval in which to warn about recent crashes", 2023-09-20T22:05:08.606 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.606 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.606 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.607 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.607 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.607 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.607 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.607 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dashboard", 2023-09-20T22:05:08.607 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.608 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.608 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.608 INFO:teuthology.orchestra.run.smithi022.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-09-20T22:05:08.608 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-09-20T22:05:08.608 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.609 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.609 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.609 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:08.609 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.609 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.609 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.610 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.610 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.610 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.610 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.610 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.611 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_HOST": { 2023-09-20T22:05:08.611 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_HOST", 2023-09-20T22:05:08.611 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.611 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.611 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.611 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.612 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.612 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.612 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.612 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.612 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.613 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.613 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.613 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.613 INFO:teuthology.orchestra.run.smithi022.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-09-20T22:05:08.613 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-09-20T22:05:08.613 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.614 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.614 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.614 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.614 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.614 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.615 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.615 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.615 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.615 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.615 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.615 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.616 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_ENABLED": { 2023-09-20T22:05:08.616 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_ENABLED", 2023-09-20T22:05:08.616 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.616 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.616 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.617 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.617 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.617 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.617 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.617 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.617 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.618 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.618 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.618 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.618 INFO:teuthology.orchestra.run.smithi022.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-09-20T22:05:08.618 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-09-20T22:05:08.619 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.619 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.619 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.619 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.619 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.620 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.620 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.620 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.620 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.620 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.620 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.621 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.621 INFO:teuthology.orchestra.run.smithi022.stdout: "ENABLE_BROWSABLE_API": { 2023-09-20T22:05:08.621 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ENABLE_BROWSABLE_API", 2023-09-20T22:05:08.621 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.621 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.622 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.622 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.622 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.622 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.622 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.622 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.623 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.623 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.623 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.623 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-09-20T22:05:08.624 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-09-20T22:05:08.624 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.624 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.624 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.624 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.624 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.625 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.625 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.625 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.625 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.625 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.625 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.626 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-09-20T22:05:08.626 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-09-20T22:05:08.626 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.626 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.627 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.627 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.627 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.627 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.627 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.627 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.628 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.628 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.628 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.628 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-09-20T22:05:08.629 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-09-20T22:05:08.629 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.629 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.629 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.629 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.629 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.630 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.630 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.630 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.630 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.630 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.631 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.631 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-09-20T22:05:08.631 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-09-20T22:05:08.631 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.631 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.632 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.632 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.632 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.632 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.632 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.633 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.633 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.633 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.633 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.634 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_NFS": { 2023-09-20T22:05:08.634 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-09-20T22:05:08.634 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.634 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.634 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.634 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.635 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.635 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.635 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.635 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.635 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.636 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.636 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.636 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RBD": { 2023-09-20T22:05:08.636 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-09-20T22:05:08.637 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.637 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.637 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.637 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.637 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.637 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.638 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.638 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.638 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.638 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.638 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.639 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.639 INFO:teuthology.orchestra.run.smithi022.stdout: "FEATURE_TOGGLE_RGW": { 2023-09-20T22:05:08.639 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-09-20T22:05:08.639 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.639 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.639 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.640 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.640 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.640 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.640 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.640 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.641 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.641 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.641 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.641 INFO:teuthology.orchestra.run.smithi022.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-09-20T22:05:08.641 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-09-20T22:05:08.642 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.642 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.642 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.642 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.642 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.643 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.643 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.643 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.643 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.643 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.644 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.644 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_PASSWORD": { 2023-09-20T22:05:08.644 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_PASSWORD", 2023-09-20T22:05:08.644 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.644 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.645 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.645 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2023-09-20T22:05:08.645 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.645 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.645 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.645 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.646 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.646 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.646 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.646 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-09-20T22:05:08.647 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-09-20T22:05:08.647 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.647 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.647 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.647 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.648 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.648 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.648 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.648 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.648 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.648 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.649 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.649 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_URL": { 2023-09-20T22:05:08.649 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_URL", 2023-09-20T22:05:08.649 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.650 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.650 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.650 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.650 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.650 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.651 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.651 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.651 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.651 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.651 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.652 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_API_USERNAME": { 2023-09-20T22:05:08.652 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_API_USERNAME", 2023-09-20T22:05:08.652 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.652 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.652 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.653 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2023-09-20T22:05:08.653 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.653 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.653 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.653 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.653 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.654 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.654 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.654 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-09-20T22:05:08.654 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-09-20T22:05:08.655 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.655 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.655 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.655 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.655 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.655 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.656 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.656 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.656 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.656 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.656 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.657 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.657 INFO:teuthology.orchestra.run.smithi022.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-09-20T22:05:08.657 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-09-20T22:05:08.657 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.657 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.657 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.658 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.658 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.658 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.658 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.658 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.659 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.659 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.659 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.659 INFO:teuthology.orchestra.run.smithi022.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-09-20T22:05:08.660 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-09-20T22:05:08.660 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.660 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.660 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.660 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.660 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.661 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.661 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.661 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.661 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.661 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.662 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.662 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.662 INFO:teuthology.orchestra.run.smithi022.stdout: "ISSUE_TRACKER_API_KEY": { 2023-09-20T22:05:08.662 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-09-20T22:05:08.662 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.662 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.663 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.663 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.663 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.663 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.663 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.664 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.664 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.664 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.664 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.664 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.664 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_HOST": { 2023-09-20T22:05:08.665 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_HOST", 2023-09-20T22:05:08.665 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.665 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.665 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.665 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.666 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.666 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.666 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.666 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.666 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.666 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.667 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.667 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.667 INFO:teuthology.orchestra.run.smithi022.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-09-20T22:05:08.667 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-09-20T22:05:08.667 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.668 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.668 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.668 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.668 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.668 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.668 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.669 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.669 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.669 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.669 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.669 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-09-20T22:05:08.670 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-09-20T22:05:08.670 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.670 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.670 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.670 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.671 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-09-20T22:05:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-09-20T22:05:08.672 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.673 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.673 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.673 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.673 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.673 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.674 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.674 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.674 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.674 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.674 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.675 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-09-20T22:05:08.675 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-09-20T22:05:08.675 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.675 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.675 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.676 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.676 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.676 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.676 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.676 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.676 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.677 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.677 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.677 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-09-20T22:05:08.677 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-09-20T22:05:08.678 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.678 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.678 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.678 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.678 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.679 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.679 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.679 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.679 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.679 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.679 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.680 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-09-20T22:05:08.680 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-09-20T22:05:08.680 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.680 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.681 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.681 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.681 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.681 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.681 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.681 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.682 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.682 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.682 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.682 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-09-20T22:05:08.683 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-09-20T22:05:08.683 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.683 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.683 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.683 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.684 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.684 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.684 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.684 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.684 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.684 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.685 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.685 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-09-20T22:05:08.685 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-09-20T22:05:08.685 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.686 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.686 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.686 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.686 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.686 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.687 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.687 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.687 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.687 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.687 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.688 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_ENABLED": { 2023-09-20T22:05:08.688 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_ENABLED", 2023-09-20T22:05:08.688 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.688 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.688 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.689 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.689 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.689 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.689 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.689 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.690 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.690 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.690 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.690 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-09-20T22:05:08.691 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-09-20T22:05:08.691 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.691 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.691 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.691 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-09-20T22:05:08.691 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.692 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.692 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.692 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.692 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.692 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.693 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.693 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-09-20T22:05:08.693 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-09-20T22:05:08.693 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.693 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.694 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.694 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:08.694 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.694 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.694 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.695 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.695 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.695 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.695 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.696 INFO:teuthology.orchestra.run.smithi022.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-09-20T22:05:08.696 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-09-20T22:05:08.696 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.696 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.696 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.696 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8", 2023-09-20T22:05:08.697 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.697 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.697 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.697 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.697 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.698 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.698 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.698 INFO:teuthology.orchestra.run.smithi022.stdout: "REST_REQUESTS_TIMEOUT": { 2023-09-20T22:05:08.698 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-09-20T22:05:08.698 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.699 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.699 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.699 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "45", 2023-09-20T22:05:08.699 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.699 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.700 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.700 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.700 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.700 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.700 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.701 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ACCESS_KEY": { 2023-09-20T22:05:08.701 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ACCESS_KEY", 2023-09-20T22:05:08.701 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.701 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.701 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.702 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.702 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.702 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.702 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.702 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.702 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.703 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.703 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.703 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-09-20T22:05:08.703 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-09-20T22:05:08.704 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.704 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.704 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.704 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "admin", 2023-09-20T22:05:08.704 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.704 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.705 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.705 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.705 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.705 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.705 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.706 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.706 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SECRET_KEY": { 2023-09-20T22:05:08.706 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SECRET_KEY", 2023-09-20T22:05:08.706 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.706 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.707 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.707 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.707 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.707 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.707 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.707 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.708 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.708 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.708 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.708 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.708 INFO:teuthology.orchestra.run.smithi022.stdout: "RGW_API_SSL_VERIFY": { 2023-09-20T22:05:08.709 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "RGW_API_SSL_VERIFY", 2023-09-20T22:05:08.709 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.709 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.709 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.709 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.709 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.710 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.710 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.710 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.710 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.710 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.711 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.711 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-09-20T22:05:08.711 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-09-20T22:05:08.711 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.712 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.712 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.712 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:08.712 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.712 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.712 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.713 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.713 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.713 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.713 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.713 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.714 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-09-20T22:05:08.714 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-09-20T22:05:08.714 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.714 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.714 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.715 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:08.715 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.715 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.715 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.715 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.715 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.716 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.716 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.716 INFO:teuthology.orchestra.run.smithi022.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-09-20T22:05:08.716 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-09-20T22:05:08.717 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.717 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.717 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.717 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:08.717 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.717 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.718 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.718 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.718 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.718 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.718 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.719 INFO:teuthology.orchestra.run.smithi022.stdout: "cross_origin_url": { 2023-09-20T22:05:08.719 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cross_origin_url", 2023-09-20T22:05:08.719 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.719 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.719 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.720 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.720 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.720 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.720 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.720 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.721 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.721 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.721 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.721 INFO:teuthology.orchestra.run.smithi022.stdout: "crt_file": { 2023-09-20T22:05:08.721 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "crt_file", 2023-09-20T22:05:08.722 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.722 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.722 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.722 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.722 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.723 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.723 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.723 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.723 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.723 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.724 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.724 INFO:teuthology.orchestra.run.smithi022.stdout: "debug": { 2023-09-20T22:05:08.724 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "debug", 2023-09-20T22:05:08.724 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.724 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.725 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.725 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.725 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.725 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.725 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.725 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Enable/disable debug options", 2023-09-20T22:05:08.726 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.726 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.726 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.726 INFO:teuthology.orchestra.run.smithi022.stdout: "jwt_token_ttl": { 2023-09-20T22:05:08.727 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "jwt_token_ttl", 2023-09-20T22:05:08.727 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.727 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.727 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.727 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "28800", 2023-09-20T22:05:08.728 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.728 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.728 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.728 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.728 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.728 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.729 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.729 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.729 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2023-09-20T22:05:08.729 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2023-09-20T22:05:08.729 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.730 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.730 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.730 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.730 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.730 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.731 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.731 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.731 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.731 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.731 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.731 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.732 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.732 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.732 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.732 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.732 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.733 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.733 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.733 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.733 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.733 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.733 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.734 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.734 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.734 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.734 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.734 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.735 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.735 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.735 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.735 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.735 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.735 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.736 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.736 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.736 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.736 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.736 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.737 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.737 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.737 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.737 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.737 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.737 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.738 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.738 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.738 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.738 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.738 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.739 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.739 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.739 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.739 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.739 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.739 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.740 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.740 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.740 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.740 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.740 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.741 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.741 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.741 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.741 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.741 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.741 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.742 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.742 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.742 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.742 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.742 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.743 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.743 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.743 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.743 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.743 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.744 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.744 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.744 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.744 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.744 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.745 INFO:teuthology.orchestra.run.smithi022.stdout: "motd": { 2023-09-20T22:05:08.745 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "motd", 2023-09-20T22:05:08.745 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.745 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.745 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.745 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.746 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.746 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.746 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.746 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "The message of the day", 2023-09-20T22:05:08.746 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.747 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.747 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.747 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.747 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect_resolve_ip_addr": { 2023-09-20T22:05:08.747 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "redirect_resolve_ip_addr", 2023-09-20T22:05:08.747 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.748 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.748 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.748 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.748 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.748 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.749 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.749 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.749 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.749 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.749 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.749 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2023-09-20T22:05:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2023-09-20T22:05:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.750 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2023-09-20T22:05:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.751 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2023-09-20T22:05:08.752 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2023-09-20T22:05:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8080", 2023-09-20T22:05:08.753 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.754 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2023-09-20T22:05:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2023-09-20T22:05:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.755 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.756 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.757 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl_server_port": { 2023-09-20T22:05:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl_server_port", 2023-09-20T22:05:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.758 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8443", 2023-09-20T22:05:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.759 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2023-09-20T22:05:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2023-09-20T22:05:08.760 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "redirect", 2023-09-20T22:05:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.761 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: "redirect" 2023-09-20T22:05:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.762 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2023-09-20T22:05:08.763 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2023-09-20T22:05:08.764 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.764 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.764 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.764 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2023-09-20T22:05:08.764 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2023-09-20T22:05:08.764 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2023-09-20T22:05:08.765 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.765 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.765 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.765 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.765 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.766 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.766 INFO:teuthology.orchestra.run.smithi022.stdout: "url_prefix": { 2023-09-20T22:05:08.766 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url_prefix", 2023-09-20T22:05:08.766 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.766 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.767 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.767 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.767 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.767 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.767 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.767 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.768 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.768 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.768 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.768 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.768 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.769 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.769 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.769 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2023-09-20T22:05:08.769 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.769 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.769 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.770 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_monitoring": { 2023-09-20T22:05:08.770 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_monitoring", 2023-09-20T22:05:08.770 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.770 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.770 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.771 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.771 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.771 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.771 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.771 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "monitor device health metrics", 2023-09-20T22:05:08.771 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.772 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.772 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.772 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.772 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.772 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.773 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.773 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.773 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.773 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.773 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.774 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.774 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.774 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.774 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.774 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.775 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.775 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.775 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.775 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.775 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.775 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.776 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.776 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.776 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.776 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.776 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.777 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.777 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.777 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.777 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.777 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.777 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.778 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.778 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.778 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.778 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.778 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.779 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.779 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.779 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.779 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.779 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.779 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.780 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.780 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.780 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.780 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.780 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.781 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.781 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.781 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.781 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.781 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.781 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.782 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.782 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.782 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.782 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.782 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.783 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.783 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.783 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.783 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.783 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.784 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.784 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.784 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.784 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.784 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.784 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.785 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.785 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.785 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.785 INFO:teuthology.orchestra.run.smithi022.stdout: "mark_out_threshold": { 2023-09-20T22:05:08.785 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mark_out_threshold", 2023-09-20T22:05:08.786 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.786 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.786 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.786 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "2419200", 2023-09-20T22:05:08.786 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.787 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.787 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.787 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-09-20T22:05:08.787 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.787 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.787 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.788 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.788 INFO:teuthology.orchestra.run.smithi022.stdout: "pool_name": { 2023-09-20T22:05:08.788 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pool_name", 2023-09-20T22:05:08.788 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.788 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.789 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.789 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "device_health_metrics", 2023-09-20T22:05:08.789 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.789 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.789 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.789 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name of pool in which to store device health metrics", 2023-09-20T22:05:08.790 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.790 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.790 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.790 INFO:teuthology.orchestra.run.smithi022.stdout: "retention_period": { 2023-09-20T22:05:08.791 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "retention_period", 2023-09-20T22:05:08.791 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.791 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.791 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.791 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15552000", 2023-09-20T22:05:08.792 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.792 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.792 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.792 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long to retain device health metrics", 2023-09-20T22:05:08.792 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.792 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.793 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.793 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.793 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_frequency": { 2023-09-20T22:05:08.793 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_frequency", 2023-09-20T22:05:08.793 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.794 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.794 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.794 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2023-09-20T22:05:08.794 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.794 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.795 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.795 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to scrape device health metrics", 2023-09-20T22:05:08.795 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.795 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.795 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.796 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.796 INFO:teuthology.orchestra.run.smithi022.stdout: "self_heal": { 2023-09-20T22:05:08.796 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "self_heal", 2023-09-20T22:05:08.796 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.796 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.796 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.797 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:08.797 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.797 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.797 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.797 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-09-20T22:05:08.798 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.798 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.798 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.798 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.798 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:08.799 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:08.799 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.799 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.799 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.799 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:08.799 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.800 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.800 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.800 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how frequently to wake up and check device health", 2023-09-20T22:05:08.800 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.800 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.801 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.801 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.801 INFO:teuthology.orchestra.run.smithi022.stdout: "warn_threshold": { 2023-09-20T22:05:08.801 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "warn_threshold", 2023-09-20T22:05:08.801 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.802 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.802 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.802 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7257600", 2023-09-20T22:05:08.802 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.802 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.803 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.803 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-09-20T22:05:08.803 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.803 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.803 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.803 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.804 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.804 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.804 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.804 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "diskprediction_local", 2023-09-20T22:05:08.804 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.804 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.805 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.805 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.805 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.805 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.805 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.806 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.806 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.806 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.806 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.806 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.807 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.807 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.807 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.807 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.807 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.807 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.808 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.808 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.808 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.808 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.808 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.809 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.809 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.809 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.809 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.809 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.809 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.810 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.810 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.810 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.810 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.810 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.811 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.811 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.811 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.811 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.811 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.811 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.812 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.812 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.812 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.812 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.812 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.813 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.813 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.813 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.813 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.813 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.813 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.814 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.814 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.814 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.814 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.814 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.815 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.815 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.815 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.815 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.815 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.816 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.816 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.816 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.816 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.816 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.817 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.817 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.817 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.817 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.817 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.817 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.818 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.818 INFO:teuthology.orchestra.run.smithi022.stdout: "predict_interval": { 2023-09-20T22:05:08.818 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predict_interval", 2023-09-20T22:05:08.818 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.818 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.819 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.819 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "86400", 2023-09-20T22:05:08.819 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.819 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.819 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.819 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.820 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.820 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.820 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.820 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.820 INFO:teuthology.orchestra.run.smithi022.stdout: "predictor_model": { 2023-09-20T22:05:08.821 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "predictor_model", 2023-09-20T22:05:08.821 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.821 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.821 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.821 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "prophetstor", 2023-09-20T22:05:08.821 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.822 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.822 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.822 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.822 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.822 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.823 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.823 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.823 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:08.823 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:08.823 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.824 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.824 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.824 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "600", 2023-09-20T22:05:08.824 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.824 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.824 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.825 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.825 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.825 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.825 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.825 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.826 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.826 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.826 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.826 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "influx", 2023-09-20T22:05:08.826 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": false, 2023-09-20T22:05:08.827 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "influxdb python module not found", 2023-09-20T22:05:08.827 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.827 INFO:teuthology.orchestra.run.smithi022.stdout: "batch_size": { 2023-09-20T22:05:08.827 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "batch_size", 2023-09-20T22:05:08.827 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.827 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.828 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.828 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5000", 2023-09-20T22:05:08.828 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.828 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.828 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.828 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-09-20T22:05:08.829 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.829 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.829 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.829 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.829 INFO:teuthology.orchestra.run.smithi022.stdout: "database": { 2023-09-20T22:05:08.830 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "database", 2023-09-20T22:05:08.830 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.830 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.830 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.830 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "ceph", 2023-09-20T22:05:08.831 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.831 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.831 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.831 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-09-20T22:05:08.831 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.831 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.832 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.832 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.832 INFO:teuthology.orchestra.run.smithi022.stdout: "hostname": { 2023-09-20T22:05:08.832 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "hostname", 2023-09-20T22:05:08.832 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.832 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.833 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.833 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.833 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.833 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.833 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.834 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server hostname", 2023-09-20T22:05:08.834 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.834 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.834 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.834 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.835 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:08.835 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:08.835 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:08.835 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.835 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.835 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "30", 2023-09-20T22:05:08.836 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "5", 2023-09-20T22:05:08.836 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.836 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.836 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-09-20T22:05:08.836 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.837 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.837 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.837 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.837 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.837 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.837 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.838 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.838 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.838 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.838 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.838 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.839 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.839 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.839 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.839 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.839 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.840 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.840 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.840 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.840 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.840 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.840 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.841 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.841 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.841 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.841 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.841 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.842 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.842 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.842 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.842 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.842 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.843 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.843 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.843 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.843 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.843 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.843 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.844 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.844 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.844 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.844 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.844 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.845 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.845 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.845 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.845 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.845 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.845 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.846 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.846 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.846 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.846 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.846 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.847 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.847 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.847 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.847 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.847 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.847 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.848 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.848 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.848 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.848 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.848 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.849 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.849 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.849 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.849 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.849 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.850 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.850 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.850 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.850 INFO:teuthology.orchestra.run.smithi022.stdout: "password": { 2023-09-20T22:05:08.850 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "password", 2023-09-20T22:05:08.850 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.851 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.851 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.851 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.851 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.851 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.852 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.852 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "password of InfluxDB server user", 2023-09-20T22:05:08.852 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.852 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.852 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.853 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.853 INFO:teuthology.orchestra.run.smithi022.stdout: "port": { 2023-09-20T22:05:08.853 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "port", 2023-09-20T22:05:08.853 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.853 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.854 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.854 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "8086", 2023-09-20T22:05:08.854 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.854 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.854 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.855 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "InfluxDB server port", 2023-09-20T22:05:08.855 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.855 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.855 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.855 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.855 INFO:teuthology.orchestra.run.smithi022.stdout: "ssl": { 2023-09-20T22:05:08.856 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ssl", 2023-09-20T22:05:08.856 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.856 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.856 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.856 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "false", 2023-09-20T22:05:08.857 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.857 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.857 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.857 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-09-20T22:05:08.857 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.858 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.858 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.858 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.858 INFO:teuthology.orchestra.run.smithi022.stdout: "threads": { 2023-09-20T22:05:08.859 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threads", 2023-09-20T22:05:08.859 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.859 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.859 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.859 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:08.859 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:08.860 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "32", 2023-09-20T22:05:08.860 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.860 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-09-20T22:05:08.860 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.860 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.861 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.861 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.861 INFO:teuthology.orchestra.run.smithi022.stdout: "username": { 2023-09-20T22:05:08.861 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "username", 2023-09-20T22:05:08.861 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.861 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.862 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.862 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.862 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.862 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.862 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.863 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "username of InfluxDB server user", 2023-09-20T22:05:08.863 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.863 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.863 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.863 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.864 INFO:teuthology.orchestra.run.smithi022.stdout: "verify_ssl": { 2023-09-20T22:05:08.864 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "verify_ssl", 2023-09-20T22:05:08.864 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.864 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.864 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.864 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "true", 2023-09-20T22:05:08.865 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.865 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.865 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.865 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-09-20T22:05:08.865 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.866 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.866 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.866 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.866 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.866 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.867 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.867 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "insights", 2023-09-20T22:05:08.867 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.867 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.867 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.867 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.868 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.868 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.868 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.868 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.868 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.868 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.869 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.869 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.869 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.869 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.869 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.870 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.870 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.870 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.870 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.870 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.870 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.871 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.871 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.871 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.871 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.871 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.872 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.872 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.872 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.872 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.872 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.872 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.873 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.873 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.873 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.873 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.873 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.874 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.874 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.874 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.874 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.874 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.874 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.875 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.875 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.875 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.875 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.875 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.876 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.876 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.876 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.876 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.876 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.876 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.877 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.877 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.877 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.877 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.877 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.878 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.878 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.878 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.878 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.878 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.879 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.879 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.879 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.879 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.879 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.879 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.880 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.880 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.880 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.880 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.880 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.881 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.881 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "iostat", 2023-09-20T22:05:08.881 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.881 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.881 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.881 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.882 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.882 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.882 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.882 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.882 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.883 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.883 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.883 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.883 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.883 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.884 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.884 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.884 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.884 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.884 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.884 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.885 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.885 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.885 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.885 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.885 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.885 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.886 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.886 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.886 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.886 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.886 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.887 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.887 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.887 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.887 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.887 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.887 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.888 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.888 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.888 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.888 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.888 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.889 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.889 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.889 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.889 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.889 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.889 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.890 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.890 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.890 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.890 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.890 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.891 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.891 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.891 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.891 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.891 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.891 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.892 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.892 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.892 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.892 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.892 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.893 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.893 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.893 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.893 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.893 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.894 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.894 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.894 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.894 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.894 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.894 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.895 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.895 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "k8sevents", 2023-09-20T22:05:08.895 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.895 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.895 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.896 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_event_retention_days": { 2023-09-20T22:05:08.896 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "ceph_event_retention_days", 2023-09-20T22:05:08.896 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.896 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.896 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.897 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "7", 2023-09-20T22:05:08.897 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.897 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.897 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.897 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Days to hold ceph event information within local cache", 2023-09-20T22:05:08.898 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.898 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.898 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.898 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.898 INFO:teuthology.orchestra.run.smithi022.stdout: "config_check_secs": { 2023-09-20T22:05:08.898 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "config_check_secs", 2023-09-20T22:05:08.899 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.899 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.899 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.899 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:08.899 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "10", 2023-09-20T22:05:08.900 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.900 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.900 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-09-20T22:05:08.900 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.900 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.900 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.901 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.901 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.901 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.901 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.901 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.902 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.902 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.902 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.902 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.902 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.903 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.903 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.903 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.903 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.903 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.903 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.904 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.904 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.904 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.904 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.904 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.904 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.905 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.905 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.905 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.905 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.905 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.906 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.906 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.906 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.906 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.906 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.907 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.907 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.907 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.907 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.907 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.908 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.908 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.908 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.908 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.908 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.908 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.909 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.909 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.909 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.909 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.909 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.910 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.910 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.910 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.910 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.910 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.911 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.911 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.911 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.911 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.911 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.911 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.912 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.912 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.912 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.912 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.912 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.913 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.913 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.913 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.913 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.913 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.913 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.914 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.914 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.914 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.914 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.914 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "localpool", 2023-09-20T22:05:08.915 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.915 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.915 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.915 INFO:teuthology.orchestra.run.smithi022.stdout: "failure_domain": { 2023-09-20T22:05:08.915 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "failure_domain", 2023-09-20T22:05:08.915 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.916 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.916 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.916 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "host", 2023-09-20T22:05:08.916 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.916 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.917 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.917 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "failure domain for any created local pool", 2023-09-20T22:05:08.917 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-09-20T22:05:08.917 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.917 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.917 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.918 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.918 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.918 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.918 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.918 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.919 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.919 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.919 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.919 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.919 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.920 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.920 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.920 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.920 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.920 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.920 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.921 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.921 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.921 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.921 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.922 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.922 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.922 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.922 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.922 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.923 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.923 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.923 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.923 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.923 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.923 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.924 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.924 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.924 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.924 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.924 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.925 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.925 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.925 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.925 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.925 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.926 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.926 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.926 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.926 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.926 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.927 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.927 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.927 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.927 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.927 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.927 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.928 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.928 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.928 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.928 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.929 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.929 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.929 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.929 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.929 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.929 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.930 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.930 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.930 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.930 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.931 INFO:teuthology.orchestra.run.smithi022.stdout: "min_size": { 2023-09-20T22:05:08.931 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "min_size", 2023-09-20T22:05:08.931 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.931 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.931 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.931 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.932 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.932 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.932 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.932 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default min_size for any created local pool", 2023-09-20T22:05:08.932 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-09-20T22:05:08.932 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.933 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.933 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.933 INFO:teuthology.orchestra.run.smithi022.stdout: "num_rep": { 2023-09-20T22:05:08.933 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "num_rep", 2023-09-20T22:05:08.933 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.934 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.934 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.934 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3", 2023-09-20T22:05:08.934 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.934 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.934 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.935 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default replica count for any created local pool", 2023-09-20T22:05:08.935 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.935 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.935 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.935 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.936 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_num": { 2023-09-20T22:05:08.936 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_num", 2023-09-20T22:05:08.936 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:08.936 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.936 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.936 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "128", 2023-09-20T22:05:08.937 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.937 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.937 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.937 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "default pg_num for any created local pool", 2023-09-20T22:05:08.937 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.938 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.938 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.938 INFO:teuthology.orchestra.run.smithi022.stdout: "prefix": { 2023-09-20T22:05:08.938 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prefix", 2023-09-20T22:05:08.939 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.939 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.939 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.939 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.939 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.939 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.940 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.940 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "name prefix for any created local pool", 2023-09-20T22:05:08.940 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.940 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.940 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.941 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.941 INFO:teuthology.orchestra.run.smithi022.stdout: "subtree": { 2023-09-20T22:05:08.941 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "subtree", 2023-09-20T22:05:08.941 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.941 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.941 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.942 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "rack", 2023-09-20T22:05:08.942 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.942 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.942 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.942 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "CRUSH level for which to create a local pool", 2023-09-20T22:05:08.943 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-09-20T22:05:08.943 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.943 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.943 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.943 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.943 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.944 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.944 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mds_autoscaler", 2023-09-20T22:05:08.944 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.944 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.944 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.945 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.945 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.945 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.945 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.945 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.946 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.946 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.946 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.946 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.946 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.946 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.947 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.947 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.947 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.948 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.949 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.949 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.949 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.949 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.949 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.950 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.950 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.950 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.950 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.950 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.951 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.951 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.951 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.951 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.951 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.951 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.952 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.952 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.952 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.952 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.953 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.953 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.953 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.953 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.953 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.954 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.954 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.954 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.954 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.954 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.955 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.955 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.955 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.955 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.955 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.956 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.956 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.956 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.956 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.956 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.957 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.957 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.957 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.957 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.957 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.957 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.958 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.958 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.958 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.958 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.958 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.959 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.959 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.959 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.959 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.960 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.960 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirroring", 2023-09-20T22:05:08.960 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.960 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.960 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.960 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.961 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.961 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.961 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.961 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.961 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.962 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.962 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.962 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.962 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.962 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.963 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.963 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.963 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.963 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.963 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.963 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.964 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.964 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.964 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.964 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.965 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.965 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.965 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.965 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.965 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.966 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.966 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.966 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.966 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.966 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.967 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.967 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.967 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.967 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.967 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.967 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.968 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.968 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.968 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.968 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.968 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.968 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.969 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.970 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.970 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.970 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.970 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.970 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.971 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.971 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.971 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.971 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.971 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.972 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.972 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.972 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.972 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.972 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.972 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.973 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.973 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.973 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.973 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.973 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.973 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "nfs", 2023-09-20T22:05:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.974 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.975 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.975 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.975 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.975 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.975 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.976 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.976 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.976 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.976 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.976 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.976 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.977 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.977 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.977 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.977 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.977 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.978 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.978 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.978 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.978 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.978 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.979 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.979 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.979 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.979 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.979 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.979 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.980 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.980 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.980 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.980 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.980 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.981 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.981 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.981 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.981 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.981 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.981 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.982 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.982 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.982 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.982 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.982 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.983 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.983 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.983 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.983 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.983 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.984 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.984 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.984 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.984 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.984 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.984 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.985 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.985 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.985 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:08.985 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:08.985 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.986 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.986 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.986 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.986 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.986 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.986 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.987 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.987 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.987 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.987 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.987 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.988 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:08.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.988 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:08.988 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2023-09-20T22:05:08.988 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:08.988 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:08.989 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:08.989 INFO:teuthology.orchestra.run.smithi022.stdout: "fail_fs": { 2023-09-20T22:05:08.989 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "fail_fs", 2023-09-20T22:05:08.989 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.989 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.990 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:08.990 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.990 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.990 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.990 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.991 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-09-20T22:05:08.991 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.991 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.991 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.991 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:08.992 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:08.992 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.992 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.992 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.992 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:08.993 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.993 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.993 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:08.993 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:08.993 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:08.993 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:08.994 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:08.994 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:08.994 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:08.994 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:08.994 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.995 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.995 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.995 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.995 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:08.995 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:08.996 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:08.996 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.996 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.996 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:08.996 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.997 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.997 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:08.997 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:08.997 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:08.997 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:08.998 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:08.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:08.998 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:08.998 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:08.998 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:08.998 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:08.999 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:08.999 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:08.999 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:08.999 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:08.999 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.000 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.000 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.000 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.000 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.000 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.000 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.001 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.001 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.001 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.001 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.001 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.002 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.002 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.002 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.002 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.003 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.003 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.003 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.003 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.003 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.003 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.004 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.004 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.004 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.004 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator": { 2023-09-20T22:05:09.004 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "orchestrator", 2023-09-20T22:05:09.005 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.005 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.005 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.005 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.005 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.006 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.006 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.006 INFO:teuthology.orchestra.run.smithi022.stdout: "cephadm", 2023-09-20T22:05:09.006 INFO:teuthology.orchestra.run.smithi022.stdout: "rook", 2023-09-20T22:05:09.006 INFO:teuthology.orchestra.run.smithi022.stdout: "test_orchestrator" 2023-09-20T22:05:09.006 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.007 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Orchestrator backend", 2023-09-20T22:05:09.007 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.007 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.007 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.007 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.008 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.008 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_perf_query", 2023-09-20T22:05:09.008 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.009 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.009 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.009 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.009 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.009 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.009 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.010 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.010 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.010 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.010 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.010 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.011 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.011 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.011 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.011 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.011 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.011 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.012 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.012 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.012 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.012 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.012 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.013 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.013 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.013 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.013 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.014 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.014 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.014 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.014 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.014 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.015 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.015 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.015 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.015 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.016 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.016 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.016 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.016 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.016 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.016 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.017 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.017 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.017 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.017 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.017 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.018 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.018 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.018 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.018 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.018 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.019 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.019 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.019 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.019 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.020 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.020 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.020 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.020 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.020 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.020 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.021 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.021 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.021 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.021 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.021 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.022 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.022 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.022 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.022 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.023 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "osd_support", 2023-09-20T22:05:09.023 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.023 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.023 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.023 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.024 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.024 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.024 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.024 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.024 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.024 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.025 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.025 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.025 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.025 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.025 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.026 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.026 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.026 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.026 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.026 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.026 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.027 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.027 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.027 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.027 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.028 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.028 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.028 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.028 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.028 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.028 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.029 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.029 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.029 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.029 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.029 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.030 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.030 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.030 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.030 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.031 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.031 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.031 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.031 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.031 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.031 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.032 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.032 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.032 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.032 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.032 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.033 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.033 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.033 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.033 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.033 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.033 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.034 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.034 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.034 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.034 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.034 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.035 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.035 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.035 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.035 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.035 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.035 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.036 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.036 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.036 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.036 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.036 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.037 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.037 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "pg_autoscaler", 2023-09-20T22:05:09.037 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.037 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.037 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.037 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.038 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.038 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.038 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.038 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.038 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.039 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.039 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.039 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.039 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.039 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.039 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.040 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.040 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.040 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.040 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.040 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.041 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.041 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.041 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.041 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.041 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.042 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.042 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.042 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.042 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.042 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.042 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.043 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.043 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.043 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.043 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.043 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.044 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.044 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.044 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.044 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.044 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.044 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.045 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.045 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.045 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.045 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.045 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.046 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.046 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.046 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.046 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.046 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.047 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.047 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.047 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.047 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.047 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.047 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.048 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.048 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.048 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.048 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.048 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.048 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.049 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.049 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.049 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.049 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.049 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.050 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.050 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.050 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.050 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.050 INFO:teuthology.orchestra.run.smithi022.stdout: "noautoscale": { 2023-09-20T22:05:09.051 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "noautoscale", 2023-09-20T22:05:09.051 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.051 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.051 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.051 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.051 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.052 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.052 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.052 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "global autoscale flag", 2023-09-20T22:05:09.052 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-09-20T22:05:09.052 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.053 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.053 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.053 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:09.053 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:09.053 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:09.053 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.054 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.054 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:09.054 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.054 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.054 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.055 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.055 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.055 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.055 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.055 INFO:teuthology.orchestra.run.smithi022.stdout: "threshold": { 2023-09-20T22:05:09.056 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "threshold", 2023-09-20T22:05:09.056 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:09.056 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.056 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.056 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "3.0", 2023-09-20T22:05:09.057 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1.0", 2023-09-20T22:05:09.057 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.057 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.057 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "scaling threshold", 2023-09-20T22:05:09.057 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-09-20T22:05:09.058 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.058 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.058 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.058 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.058 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.059 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.059 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "progress", 2023-09-20T22:05:09.059 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.059 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.059 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.059 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_pg_recovery_event": { 2023-09-20T22:05:09.060 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_pg_recovery_event", 2023-09-20T22:05:09.060 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.060 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.060 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.060 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.061 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.061 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.061 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.061 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow the module to show pg recovery progress", 2023-09-20T22:05:09.061 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.061 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.062 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.062 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2023-09-20T22:05:09.062 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2023-09-20T22:05:09.062 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.063 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.063 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.063 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:09.063 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.063 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.063 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.064 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.064 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.064 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.064 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.064 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.065 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.065 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.065 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.065 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.065 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.065 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.066 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.066 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.066 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.066 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.066 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.067 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.067 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.067 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.067 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.067 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.067 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.068 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.068 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.068 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.068 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.068 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.069 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.069 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.069 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.069 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.069 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.069 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.070 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.070 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.070 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.070 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.070 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.071 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.071 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.071 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.071 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.071 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.071 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.072 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.072 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.072 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.072 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.072 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.072 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.073 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.073 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.073 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.073 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.073 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.074 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.074 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.074 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.074 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.074 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.075 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.075 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.075 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.075 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.075 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.076 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.076 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.076 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.076 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.076 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.076 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.077 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.077 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.077 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.077 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.077 INFO:teuthology.orchestra.run.smithi022.stdout: "max_completed_events": { 2023-09-20T22:05:09.078 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_completed_events", 2023-09-20T22:05:09.078 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.078 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.078 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.078 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "50", 2023-09-20T22:05:09.078 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.079 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.079 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.079 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "number of past completed events to remember", 2023-09-20T22:05:09.079 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.079 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.080 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.080 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.080 INFO:teuthology.orchestra.run.smithi022.stdout: "sleep_interval": { 2023-09-20T22:05:09.080 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "sleep_interval", 2023-09-20T22:05:09.080 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:09.081 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.081 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.081 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "5", 2023-09-20T22:05:09.081 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.081 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.081 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.082 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "how long the module is going to sleep", 2023-09-20T22:05:09.082 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.082 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.082 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.082 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.083 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.083 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.083 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.083 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "prometheus", 2023-09-20T22:05:09.083 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.083 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.084 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.084 INFO:teuthology.orchestra.run.smithi022.stdout: "cache": { 2023-09-20T22:05:09.084 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "cache", 2023-09-20T22:05:09.084 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.084 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.085 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.085 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:09.085 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.085 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.085 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.085 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.086 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.086 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.086 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.086 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.086 INFO:teuthology.orchestra.run.smithi022.stdout: "exclude_perf_counters": { 2023-09-20T22:05:09.087 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "exclude_perf_counters", 2023-09-20T22:05:09.087 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.087 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.087 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.087 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:09.088 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.088 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.088 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.088 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-09-20T22:05:09.088 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-09-20T22:05:09.089 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.089 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.089 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.089 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.089 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.089 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.090 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.090 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.090 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.090 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.090 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.091 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.091 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.091 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.091 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.091 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.091 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.092 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.092 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.092 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.092 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.092 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.093 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.094 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.094 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.094 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.094 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.094 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.095 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.095 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.095 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.095 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.095 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.095 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.096 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.096 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.096 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.096 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.096 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.097 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.097 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.097 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.097 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.097 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.097 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.098 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.098 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.098 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.098 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.098 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.099 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.099 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.099 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.099 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.099 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.100 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.100 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.100 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.100 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.100 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.101 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.101 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.101 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.101 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.101 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.101 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.102 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.102 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.102 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools": { 2023-09-20T22:05:09.102 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools", 2023-09-20T22:05:09.102 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.102 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.103 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.103 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.103 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.103 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.103 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.104 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.104 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.104 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.104 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.104 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.104 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_stats_pools_refresh_interval": { 2023-09-20T22:05:09.105 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-09-20T22:05:09.105 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.105 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.105 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.105 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300", 2023-09-20T22:05:09.105 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.106 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.106 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.106 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.106 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.106 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.107 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.107 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.107 INFO:teuthology.orchestra.run.smithi022.stdout: "scrape_interval": { 2023-09-20T22:05:09.107 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "scrape_interval", 2023-09-20T22:05:09.107 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:09.108 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.108 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.108 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15.0", 2023-09-20T22:05:09.108 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.108 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.108 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.109 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.109 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.109 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.109 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.109 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.110 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2023-09-20T22:05:09.110 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2023-09-20T22:05:09.110 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.110 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.110 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.111 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "::", 2023-09-20T22:05:09.111 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.111 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.111 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.111 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-09-20T22:05:09.111 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.112 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.112 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.112 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.112 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2023-09-20T22:05:09.112 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2023-09-20T22:05:09.112 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.113 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.113 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.113 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "9283", 2023-09-20T22:05:09.113 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.113 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.114 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.114 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-09-20T22:05:09.114 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.114 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.114 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.114 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.115 INFO:teuthology.orchestra.run.smithi022.stdout: "stale_cache_strategy": { 2023-09-20T22:05:09.115 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stale_cache_strategy", 2023-09-20T22:05:09.115 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.115 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.115 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.116 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "log", 2023-09-20T22:05:09.116 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.116 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.116 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.116 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.116 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.117 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.117 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.117 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.117 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_behaviour": { 2023-09-20T22:05:09.117 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_behaviour", 2023-09-20T22:05:09.118 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.118 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.118 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.118 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "default", 2023-09-20T22:05:09.118 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.118 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.119 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.119 INFO:teuthology.orchestra.run.smithi022.stdout: "default", 2023-09-20T22:05:09.119 INFO:teuthology.orchestra.run.smithi022.stdout: "error" 2023-09-20T22:05:09.119 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.119 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.120 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.120 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.120 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.120 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.120 INFO:teuthology.orchestra.run.smithi022.stdout: "standby_error_status_code": { 2023-09-20T22:05:09.121 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "standby_error_status_code", 2023-09-20T22:05:09.121 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.121 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.121 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.121 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "500", 2023-09-20T22:05:09.121 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "400", 2023-09-20T22:05:09.122 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "599", 2023-09-20T22:05:09.122 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.122 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.122 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.122 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.123 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.123 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.123 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.123 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.123 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.123 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2023-09-20T22:05:09.124 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.124 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.124 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.124 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.124 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.125 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.125 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.125 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.125 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.125 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.125 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.126 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.126 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.126 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.126 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.126 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.127 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.127 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.127 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.127 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.127 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.128 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.128 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.128 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.128 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.128 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.128 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.129 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.129 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.129 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.129 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.129 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.129 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.130 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.130 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.130 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.130 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.130 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.131 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.131 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.131 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.131 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.131 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.132 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.132 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.132 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.132 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.132 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.132 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.133 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.133 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.133 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.133 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.133 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.133 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.134 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.134 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.134 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.134 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.135 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.135 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.135 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.135 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.135 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_snap_create": { 2023-09-20T22:05:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_snap_create", 2023-09-20T22:05:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10", 2023-09-20T22:05:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: "mirror_snapshot_schedule": { 2023-09-20T22:05:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "mirror_snapshot_schedule", 2023-09-20T22:05:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: "trash_purge_schedule": { 2023-09-20T22:05:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "trash_purge_schedule", 2023-09-20T22:05:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.145 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.145 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.145 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.145 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "restful", 2023-09-20T22:05:09.145 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.146 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.146 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.146 INFO:teuthology.orchestra.run.smithi022.stdout: "enable_auth": { 2023-09-20T22:05:09.146 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enable_auth", 2023-09-20T22:05:09.147 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.147 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.147 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.147 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:09.147 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.147 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.148 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.148 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.148 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.148 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.148 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.149 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.149 INFO:teuthology.orchestra.run.smithi022.stdout: "key_file": { 2023-09-20T22:05:09.149 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "key_file", 2023-09-20T22:05:09.149 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.149 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.149 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.150 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.151 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.151 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.151 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.151 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.151 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.151 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.152 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.152 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.152 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.152 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.152 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.153 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.153 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.153 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.153 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.153 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.153 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.154 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.154 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.154 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.154 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.154 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.157 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.157 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.157 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.157 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.157 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.158 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.158 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.158 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.158 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.158 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.159 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.160 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.160 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.160 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.160 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.160 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.160 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.161 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.161 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.161 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.161 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.161 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.162 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.162 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.162 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.162 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.162 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.163 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.163 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.163 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.163 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.163 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.163 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.164 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.164 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.164 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.164 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.164 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.165 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.165 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.165 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.165 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.165 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.166 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.166 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.166 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.166 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.166 INFO:teuthology.orchestra.run.smithi022.stdout: "server_addr": { 2023-09-20T22:05:09.167 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_addr", 2023-09-20T22:05:09.167 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.167 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.167 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.167 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.167 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.168 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.168 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.168 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.168 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.168 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.169 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.169 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.169 INFO:teuthology.orchestra.run.smithi022.stdout: "server_port": { 2023-09-20T22:05:09.169 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "server_port", 2023-09-20T22:05:09.169 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.169 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.170 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.170 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.170 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.170 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.170 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.171 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.171 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.171 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.171 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.171 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.171 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.172 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.172 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.172 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rgw", 2023-09-20T22:05:09.172 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.172 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.173 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.173 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.173 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.173 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.173 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.173 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.174 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.174 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.174 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.174 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.174 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.175 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.175 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.175 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.175 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.175 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.175 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.176 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.176 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.176 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.176 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.176 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.177 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.177 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.177 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.177 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.177 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.178 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.178 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.178 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.178 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.178 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.179 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.179 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.179 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.179 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.179 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.179 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.180 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.180 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.180 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.180 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.180 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.180 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.181 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.181 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.181 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.181 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.181 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.182 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.182 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.182 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.182 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.182 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.183 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.183 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.183 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.183 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.183 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.184 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.184 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.184 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.184 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.184 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.185 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.185 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.185 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.185 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.185 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.186 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.186 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.186 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.186 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rook", 2023-09-20T22:05:09.187 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.187 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.187 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.187 INFO:teuthology.orchestra.run.smithi022.stdout: "drive_group_interval": { 2023-09-20T22:05:09.187 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "drive_group_interval", 2023-09-20T22:05:09.187 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:09.188 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.188 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.188 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "300.0", 2023-09-20T22:05:09.188 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.188 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.188 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.189 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-09-20T22:05:09.189 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.189 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.189 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.189 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.189 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.190 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.190 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.190 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.190 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.190 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.191 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.191 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.191 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.191 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.191 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.192 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.192 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.192 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.192 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.192 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.192 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.193 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.193 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.193 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.193 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.193 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.194 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.194 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.194 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.194 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.194 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.195 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.195 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.195 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.195 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.195 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.195 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.196 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.196 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.196 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.196 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.197 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.197 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.197 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.197 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.197 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.197 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.198 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.198 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.198 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.198 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.198 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.199 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.199 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.199 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.199 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.199 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.200 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.200 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.200 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.200 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.200 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.201 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.201 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.201 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.201 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.201 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.202 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.202 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.202 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.202 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.202 INFO:teuthology.orchestra.run.smithi022.stdout: "storage_class": { 2023-09-20T22:05:09.203 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "storage_class", 2023-09-20T22:05:09.203 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.203 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.203 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.203 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "local", 2023-09-20T22:05:09.203 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.204 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.204 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.204 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-09-20T22:05:09.204 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.204 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.204 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.205 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.205 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.205 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.205 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.205 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "selftest", 2023-09-20T22:05:09.206 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.206 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.206 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.206 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.206 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.207 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.207 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.207 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.207 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.207 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.207 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.208 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.208 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.208 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.208 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.208 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.208 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.209 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.209 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.209 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.209 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.209 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.210 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.210 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.210 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.210 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.211 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.211 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.211 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.211 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.211 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.211 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.212 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.212 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.212 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.212 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.213 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.213 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.213 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.213 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.213 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.213 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.214 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.214 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.214 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.214 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.214 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.215 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.215 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.215 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.215 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.215 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.215 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.216 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.216 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.216 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.216 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.216 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.217 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.217 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.217 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.217 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.217 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.217 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.218 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.218 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.218 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.218 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.218 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.219 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.219 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.219 INFO:teuthology.orchestra.run.smithi022.stdout: "roption1": { 2023-09-20T22:05:09.219 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption1", 2023-09-20T22:05:09.219 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.220 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.220 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.220 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.220 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.220 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.220 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.221 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.221 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.221 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.221 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.221 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.222 INFO:teuthology.orchestra.run.smithi022.stdout: "roption2": { 2023-09-20T22:05:09.222 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "roption2", 2023-09-20T22:05:09.222 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.222 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.222 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.223 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "xyz", 2023-09-20T22:05:09.223 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.223 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.223 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.223 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.223 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.224 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.224 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.224 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.224 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption1": { 2023-09-20T22:05:09.224 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption1", 2023-09-20T22:05:09.225 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.225 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.225 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.225 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.225 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.225 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.226 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.226 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.226 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.226 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.226 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.227 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.227 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption2": { 2023-09-20T22:05:09.227 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption2", 2023-09-20T22:05:09.227 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.227 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.227 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.228 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.228 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.228 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.228 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.228 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.229 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.229 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.229 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.229 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption3": { 2023-09-20T22:05:09.229 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption3", 2023-09-20T22:05:09.230 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "float", 2023-09-20T22:05:09.230 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.230 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.230 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.230 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.231 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.231 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.231 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.231 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.231 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.232 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.232 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.232 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption4": { 2023-09-20T22:05:09.232 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption4", 2023-09-20T22:05:09.232 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.232 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.233 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.233 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.233 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.233 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.233 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.234 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.234 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.234 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.234 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.234 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.235 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption5": { 2023-09-20T22:05:09.235 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption5", 2023-09-20T22:05:09.235 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.235 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.235 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.236 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.236 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.236 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.236 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.236 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.237 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.237 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.237 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.237 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.237 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption6": { 2023-09-20T22:05:09.238 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption6", 2023-09-20T22:05:09.238 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.238 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.238 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.238 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:09.239 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.239 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.239 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.239 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.239 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.240 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.240 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.240 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.240 INFO:teuthology.orchestra.run.smithi022.stdout: "rwoption7": { 2023-09-20T22:05:09.240 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rwoption7", 2023-09-20T22:05:09.240 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "1", 2023-09-20T22:05:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "42", 2023-09-20T22:05:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: "testkey": { 2023-09-20T22:05:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testkey", 2023-09-20T22:05:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: "testlkey": { 2023-09-20T22:05:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testlkey", 2023-09-20T22:05:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: "testnewline": { 2023-09-20T22:05:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "testnewline", 2023-09-20T22:05:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.251 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.251 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.251 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snap_schedule", 2023-09-20T22:05:09.251 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.251 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.252 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.252 INFO:teuthology.orchestra.run.smithi022.stdout: "allow_m_granularity": { 2023-09-20T22:05:09.252 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "allow_m_granularity", 2023-09-20T22:05:09.252 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.252 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.252 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.253 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.253 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.253 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.253 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.253 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "allow minute scheduled snapshots", 2023-09-20T22:05:09.253 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.254 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.254 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.254 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.254 INFO:teuthology.orchestra.run.smithi022.stdout: "dump_on_update": { 2023-09-20T22:05:09.254 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "dump_on_update", 2023-09-20T22:05:09.255 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.255 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.255 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.255 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.255 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.256 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.256 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.256 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "dump database to debug log on update", 2023-09-20T22:05:09.256 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.256 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.257 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.257 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.257 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.257 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.257 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.258 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.258 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.258 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.258 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.258 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.258 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.259 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.259 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.259 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.259 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.259 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.260 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.260 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.260 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.260 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.260 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.260 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.261 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.261 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.261 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.261 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.261 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.262 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.262 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.262 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.262 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.262 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.263 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.263 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.263 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.263 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.263 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.263 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.264 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.264 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.264 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.264 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.264 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.265 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.265 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.265 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.265 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.265 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.265 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.266 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.266 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.266 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.266 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.266 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.267 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.267 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.267 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.267 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.267 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.267 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.268 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.268 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.268 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.268 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.268 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.269 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.269 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.269 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.269 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.269 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.270 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.270 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.270 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.270 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.270 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.270 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "stats", 2023-09-20T22:05:09.271 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.271 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.271 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.271 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.271 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.272 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.272 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.272 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.272 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.272 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.273 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.273 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.273 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.273 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.273 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.274 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.274 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.274 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.274 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.274 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.274 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.275 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.275 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.275 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.275 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.275 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.276 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.276 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.276 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.276 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.276 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.276 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.277 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.277 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.277 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.277 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.277 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.278 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.278 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.278 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.278 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.278 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.278 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.279 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.279 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.279 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.279 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.279 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.280 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.280 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.280 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.280 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.280 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.281 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.281 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.281 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.281 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.281 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.281 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.282 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.282 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.282 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.282 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.282 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.283 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.283 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.283 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.283 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.283 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.284 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.284 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.284 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.284 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.284 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.284 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.285 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.285 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "status", 2023-09-20T22:05:09.285 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.285 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.285 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.286 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.286 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.286 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.286 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.286 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.287 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.288 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.288 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.288 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.288 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.288 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.289 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.289 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.289 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.289 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.289 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.290 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.290 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.290 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.290 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.290 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.291 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.291 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.291 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.291 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.291 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.292 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.292 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.292 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.292 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.292 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.293 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.293 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.293 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.293 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.293 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.293 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.294 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.294 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.294 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.294 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.294 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.295 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.295 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.295 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.295 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.295 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.296 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.296 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.296 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.296 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.296 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.296 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.297 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.297 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.297 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.297 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.297 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.298 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.298 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.298 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.298 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.298 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.299 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.299 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.299 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telegraf", 2023-09-20T22:05:09.299 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.300 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.300 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.300 INFO:teuthology.orchestra.run.smithi022.stdout: "address": { 2023-09-20T22:05:09.300 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "address", 2023-09-20T22:05:09.300 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.301 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.301 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.301 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-09-20T22:05:09.301 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.301 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.301 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.302 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.302 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.302 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.302 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.302 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.303 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:09.303 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:09.303 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:09.303 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.303 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.304 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "15", 2023-09-20T22:05:09.304 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.304 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.304 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.304 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.304 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.305 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.305 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.305 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.305 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.305 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.306 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.306 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.306 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.306 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.306 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.307 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.307 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.307 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.307 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.307 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.307 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.308 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.308 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.308 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.308 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.308 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.309 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.309 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.309 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.309 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.309 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.309 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.310 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.310 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.310 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.310 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.310 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.311 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.311 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.311 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.311 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.311 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.312 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.312 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.312 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.312 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.312 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.312 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.313 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.313 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.313 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.313 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.313 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.314 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.314 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.314 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.314 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.314 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.315 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.315 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.315 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.315 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.315 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.315 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.316 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.316 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.316 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.316 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.316 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.317 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.317 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.317 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.317 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.317 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.317 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.318 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.318 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.318 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.318 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.318 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.319 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "telemetry", 2023-09-20T22:05:09.319 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.319 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.319 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.319 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_basic": { 2023-09-20T22:05:09.319 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_basic", 2023-09-20T22:05:09.320 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.320 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.320 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.320 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:09.320 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.321 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.321 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.321 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share basic cluster information (size, version)", 2023-09-20T22:05:09.321 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.321 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.321 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.322 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.322 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_crash": { 2023-09-20T22:05:09.322 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_crash", 2023-09-20T22:05:09.322 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.322 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.322 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.323 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:09.323 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.323 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.323 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.323 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-09-20T22:05:09.323 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.324 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.324 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.324 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.324 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_device": { 2023-09-20T22:05:09.324 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_device", 2023-09-20T22:05:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.325 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.326 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.326 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-09-20T22:05:09.326 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.326 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.326 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.327 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.327 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_ident": { 2023-09-20T22:05:09.327 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_ident", 2023-09-20T22:05:09.327 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.327 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.327 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.328 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.328 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.328 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.328 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.328 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-09-20T22:05:09.329 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.329 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.329 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.329 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.329 INFO:teuthology.orchestra.run.smithi022.stdout: "channel_perf": { 2023-09-20T22:05:09.329 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "channel_perf", 2023-09-20T22:05:09.330 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.330 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.330 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.330 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.330 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.331 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.331 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.331 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Share various performance metrics of a cluster", 2023-09-20T22:05:09.331 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.331 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.331 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.332 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.332 INFO:teuthology.orchestra.run.smithi022.stdout: "contact": { 2023-09-20T22:05:09.332 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "contact", 2023-09-20T22:05:09.332 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.332 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.332 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.333 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.333 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.333 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.333 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.333 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.333 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.334 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.334 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.334 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.334 INFO:teuthology.orchestra.run.smithi022.stdout: "description": { 2023-09-20T22:05:09.334 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "description", 2023-09-20T22:05:09.335 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.335 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.335 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.335 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.335 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.336 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.336 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.336 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.336 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.336 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.336 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.337 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.337 INFO:teuthology.orchestra.run.smithi022.stdout: "device_url": { 2023-09-20T22:05:09.337 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "device_url", 2023-09-20T22:05:09.337 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.337 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.338 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.338 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-09-20T22:05:09.338 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.338 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.338 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.339 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.339 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.339 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.339 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.339 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.339 INFO:teuthology.orchestra.run.smithi022.stdout: "enabled": { 2023-09-20T22:05:09.340 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "enabled", 2023-09-20T22:05:09.340 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.340 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.340 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.340 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.341 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.341 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.341 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.341 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.341 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.341 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.342 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.342 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.342 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:09.342 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:09.342 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.343 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.343 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.343 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "24", 2023-09-20T22:05:09.343 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "8", 2023-09-20T22:05:09.343 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.344 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.344 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.344 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.344 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.344 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.344 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.345 INFO:teuthology.orchestra.run.smithi022.stdout: "last_opt_revision": { 2023-09-20T22:05:09.345 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "last_opt_revision", 2023-09-20T22:05:09.345 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.345 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.345 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.345 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "1", 2023-09-20T22:05:09.346 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.346 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.346 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.346 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.346 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.347 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.347 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.347 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.347 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard": { 2023-09-20T22:05:09.347 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard", 2023-09-20T22:05:09.348 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.348 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.348 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.348 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.348 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.348 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.349 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.349 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.349 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.349 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.349 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.349 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.350 INFO:teuthology.orchestra.run.smithi022.stdout: "leaderboard_description": { 2023-09-20T22:05:09.350 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "leaderboard_description", 2023-09-20T22:05:09.350 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.350 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.350 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.351 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.352 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.354 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.354 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.354 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.354 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.354 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.355 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.355 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.355 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.355 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.355 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.356 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.356 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.356 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.356 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.356 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.356 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.357 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.357 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.357 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.357 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.357 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.358 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.358 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.358 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.358 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.361 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.361 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.361 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.361 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.362 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.362 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.362 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.362 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.362 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.363 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.363 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.363 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.363 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.363 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.363 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.364 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.364 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.364 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.364 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.364 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.365 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.365 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.365 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.365 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.365 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.365 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.366 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.366 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.366 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.366 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.366 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.367 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.367 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.367 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.367 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.367 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.368 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.368 INFO:teuthology.orchestra.run.smithi022.stdout: "organization": { 2023-09-20T22:05:09.368 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "organization", 2023-09-20T22:05:09.368 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.368 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.368 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.369 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.369 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.369 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.369 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.369 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.369 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.370 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.370 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.370 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.370 INFO:teuthology.orchestra.run.smithi022.stdout: "proxy": { 2023-09-20T22:05:09.370 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "proxy", 2023-09-20T22:05:09.371 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.371 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.371 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.371 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.371 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.371 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.372 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.372 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.372 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.372 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.372 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.373 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.373 INFO:teuthology.orchestra.run.smithi022.stdout: "url": { 2023-09-20T22:05:09.373 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "url", 2023-09-20T22:05:09.373 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.373 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.373 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.374 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-09-20T22:05:09.374 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.374 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.374 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.374 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.375 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.375 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.375 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.375 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.375 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.376 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.376 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.376 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "test_orchestrator", 2023-09-20T22:05:09.376 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.376 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.377 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.377 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.377 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.377 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.377 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.377 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.378 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.378 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.378 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.378 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.378 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.379 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.379 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.379 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.379 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.379 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.380 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.380 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.380 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.380 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.380 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.380 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.381 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.381 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.381 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.381 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.381 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.382 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.382 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.382 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.382 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.382 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.382 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.383 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.383 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.383 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.383 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.383 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.384 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.384 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.384 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.384 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.384 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.384 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.385 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.385 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.385 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.385 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.385 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.386 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.386 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.386 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.386 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.386 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.387 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.387 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.387 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.387 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.387 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.387 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.388 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.388 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.388 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.388 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.388 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.388 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.389 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.389 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.389 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.389 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.389 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.390 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.390 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.390 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.390 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2023-09-20T22:05:09.390 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.390 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.391 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.391 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.391 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.391 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.391 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.391 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.392 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.392 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.392 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.392 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.392 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.393 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.393 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.393 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.393 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.393 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.393 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.394 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.394 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.394 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.394 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.394 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.395 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.395 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.395 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.395 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.395 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.396 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.396 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.396 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.396 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.396 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.396 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.397 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.397 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.397 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.397 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.397 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.398 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.398 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.398 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.398 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.398 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.398 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.399 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.399 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.399 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.399 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.399 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.400 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.400 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.400 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.400 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.400 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.400 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.401 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.401 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.401 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.401 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.401 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.402 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.402 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.402 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.402 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.402 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.403 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.403 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.403 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.403 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.403 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.403 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.404 INFO:teuthology.orchestra.run.smithi022.stdout: "max_concurrent_clones": { 2023-09-20T22:05:09.404 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "max_concurrent_clones", 2023-09-20T22:05:09.404 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.404 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.404 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.405 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "4", 2023-09-20T22:05:09.405 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.405 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.405 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.405 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Number of asynchronous cloner threads", 2023-09-20T22:05:09.405 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.406 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.406 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.406 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.406 INFO:teuthology.orchestra.run.smithi022.stdout: "periodic_async_work": { 2023-09-20T22:05:09.406 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "periodic_async_work", 2023-09-20T22:05:09.407 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.407 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.407 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.407 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.407 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.407 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.408 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.408 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Periodically check for async work", 2023-09-20T22:05:09.408 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.408 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.408 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.409 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.409 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_delay": { 2023-09-20T22:05:09.409 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_delay", 2023-09-20T22:05:09.409 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.409 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.409 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.410 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "0", 2023-09-20T22:05:09.410 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.410 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.410 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.410 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-09-20T22:05:09.411 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.411 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.411 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.411 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.411 INFO:teuthology.orchestra.run.smithi022.stdout: "snapshot_clone_no_wait": { 2023-09-20T22:05:09.411 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "snapshot_clone_no_wait", 2023-09-20T22:05:09.412 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.412 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.412 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.412 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "True", 2023-09-20T22:05:09.412 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.413 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.413 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.413 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2023-09-20T22:05:09.413 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.413 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.414 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.414 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.414 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.414 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.414 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.414 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix", 2023-09-20T22:05:09.415 INFO:teuthology.orchestra.run.smithi022.stdout: "can_run": true, 2023-09-20T22:05:09.415 INFO:teuthology.orchestra.run.smithi022.stdout: "error_string": "", 2023-09-20T22:05:09.415 INFO:teuthology.orchestra.run.smithi022.stdout: "module_options": { 2023-09-20T22:05:09.415 INFO:teuthology.orchestra.run.smithi022.stdout: "discovery_interval": { 2023-09-20T22:05:09.415 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "discovery_interval", 2023-09-20T22:05:09.415 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "uint", 2023-09-20T22:05:09.416 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.416 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.416 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "100", 2023-09-20T22:05:09.416 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.416 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.417 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.417 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.417 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.417 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.417 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.418 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.418 INFO:teuthology.orchestra.run.smithi022.stdout: "identifier": { 2023-09-20T22:05:09.418 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "identifier", 2023-09-20T22:05:09.418 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.418 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.419 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.419 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.419 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.419 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.419 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.419 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.420 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.420 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.420 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.420 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.420 INFO:teuthology.orchestra.run.smithi022.stdout: "interval": { 2023-09-20T22:05:09.420 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "interval", 2023-09-20T22:05:09.421 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "secs", 2023-09-20T22:05:09.421 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.421 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.421 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "60", 2023-09-20T22:05:09.421 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.423 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.423 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.423 INFO:teuthology.orchestra.run.smithi022.stdout: "log_level": { 2023-09-20T22:05:09.423 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_level", 2023-09-20T22:05:09.423 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.423 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.424 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.424 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.424 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.424 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.424 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.425 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.425 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.425 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.425 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.425 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.425 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.426 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.426 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.426 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.426 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.426 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.427 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.427 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster": { 2023-09-20T22:05:09.427 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster", 2023-09-20T22:05:09.427 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.427 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.427 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.428 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.428 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.428 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.428 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.428 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.429 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.429 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.429 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.429 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_cluster_level": { 2023-09-20T22:05:09.429 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_cluster_level", 2023-09-20T22:05:09.430 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.430 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.430 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.430 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "info", 2023-09-20T22:05:09.430 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.431 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.431 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [ 2023-09-20T22:05:09.431 INFO:teuthology.orchestra.run.smithi022.stdout: "", 2023-09-20T22:05:09.431 INFO:teuthology.orchestra.run.smithi022.stdout: "critical", 2023-09-20T22:05:09.431 INFO:teuthology.orchestra.run.smithi022.stdout: "debug", 2023-09-20T22:05:09.432 INFO:teuthology.orchestra.run.smithi022.stdout: "error", 2023-09-20T22:05:09.432 INFO:teuthology.orchestra.run.smithi022.stdout: "info", 2023-09-20T22:05:09.432 INFO:teuthology.orchestra.run.smithi022.stdout: "warning" 2023-09-20T22:05:09.432 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.432 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.432 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.433 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.433 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.433 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.433 INFO:teuthology.orchestra.run.smithi022.stdout: "log_to_file": { 2023-09-20T22:05:09.433 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "log_to_file", 2023-09-20T22:05:09.434 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "bool", 2023-09-20T22:05:09.434 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.434 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 1, 2023-09-20T22:05:09.434 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "False", 2023-09-20T22:05:09.434 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.435 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.435 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.435 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.435 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.435 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.435 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.436 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.436 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_host": { 2023-09-20T22:05:09.436 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_host", 2023-09-20T22:05:09.436 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.436 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.437 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.437 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "", 2023-09-20T22:05:09.437 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.437 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.437 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.438 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.438 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.438 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.438 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.438 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.439 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_port": { 2023-09-20T22:05:09.439 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_port", 2023-09-20T22:05:09.439 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "int", 2023-09-20T22:05:09.439 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.439 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.439 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "10051", 2023-09-20T22:05:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.440 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: "zabbix_sender": { 2023-09-20T22:05:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "zabbix_sender", 2023-09-20T22:05:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "str", 2023-09-20T22:05:09.442 INFO:teuthology.orchestra.run.smithi022.stdout: "level": "advanced", 2023-09-20T22:05:09.442 INFO:teuthology.orchestra.run.smithi022.stdout: "flags": 0, 2023-09-20T22:05:09.442 INFO:teuthology.orchestra.run.smithi022.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-09-20T22:05:09.442 INFO:teuthology.orchestra.run.smithi022.stdout: "min": "", 2023-09-20T22:05:09.442 INFO:teuthology.orchestra.run.smithi022.stdout: "max": "", 2023-09-20T22:05:09.443 INFO:teuthology.orchestra.run.smithi022.stdout: "enum_allowed": [], 2023-09-20T22:05:09.443 INFO:teuthology.orchestra.run.smithi022.stdout: "desc": "", 2023-09-20T22:05:09.443 INFO:teuthology.orchestra.run.smithi022.stdout: "long_desc": "", 2023-09-20T22:05:09.443 INFO:teuthology.orchestra.run.smithi022.stdout: "tags": [], 2023-09-20T22:05:09.443 INFO:teuthology.orchestra.run.smithi022.stdout: "see_also": [] 2023-09-20T22:05:09.443 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.444 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.444 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.444 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.444 INFO:teuthology.orchestra.run.smithi022.stdout: "services": { 2023-09-20T22:05:09.444 INFO:teuthology.orchestra.run.smithi022.stdout: "dashboard": "https://172.21.15.22:8443/" 2023-09-20T22:05:09.445 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.445 INFO:teuthology.orchestra.run.smithi022.stdout: "always_on_modules": { 2023-09-20T22:05:09.445 INFO:teuthology.orchestra.run.smithi022.stdout: "octopus": [ 2023-09-20T22:05:09.445 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2023-09-20T22:05:09.445 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2023-09-20T22:05:09.445 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2023-09-20T22:05:09.446 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2023-09-20T22:05:09.446 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2023-09-20T22:05:09.446 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2023-09-20T22:05:09.446 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2023-09-20T22:05:09.446 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2023-09-20T22:05:09.447 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2023-09-20T22:05:09.447 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2023-09-20T22:05:09.447 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.447 INFO:teuthology.orchestra.run.smithi022.stdout: "pacific": [ 2023-09-20T22:05:09.447 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2023-09-20T22:05:09.447 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2023-09-20T22:05:09.448 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2023-09-20T22:05:09.448 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2023-09-20T22:05:09.448 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2023-09-20T22:05:09.448 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2023-09-20T22:05:09.448 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2023-09-20T22:05:09.449 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2023-09-20T22:05:09.449 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2023-09-20T22:05:09.449 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2023-09-20T22:05:09.449 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.449 INFO:teuthology.orchestra.run.smithi022.stdout: "quincy": [ 2023-09-20T22:05:09.449 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2023-09-20T22:05:09.450 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2023-09-20T22:05:09.450 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2023-09-20T22:05:09.450 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2023-09-20T22:05:09.450 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2023-09-20T22:05:09.450 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2023-09-20T22:05:09.451 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2023-09-20T22:05:09.451 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2023-09-20T22:05:09.451 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2023-09-20T22:05:09.451 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2023-09-20T22:05:09.451 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2023-09-20T22:05:09.452 INFO:teuthology.orchestra.run.smithi022.stdout: "reef": [ 2023-09-20T22:05:09.452 INFO:teuthology.orchestra.run.smithi022.stdout: "balancer", 2023-09-20T22:05:09.452 INFO:teuthology.orchestra.run.smithi022.stdout: "crash", 2023-09-20T22:05:09.452 INFO:teuthology.orchestra.run.smithi022.stdout: "devicehealth", 2023-09-20T22:05:09.452 INFO:teuthology.orchestra.run.smithi022.stdout: "orchestrator", 2023-09-20T22:05:09.453 INFO:teuthology.orchestra.run.smithi022.stdout: "pg_autoscaler", 2023-09-20T22:05:09.453 INFO:teuthology.orchestra.run.smithi022.stdout: "progress", 2023-09-20T22:05:09.453 INFO:teuthology.orchestra.run.smithi022.stdout: "rbd_support", 2023-09-20T22:05:09.453 INFO:teuthology.orchestra.run.smithi022.stdout: "status", 2023-09-20T22:05:09.453 INFO:teuthology.orchestra.run.smithi022.stdout: "telemetry", 2023-09-20T22:05:09.453 INFO:teuthology.orchestra.run.smithi022.stdout: "volumes" 2023-09-20T22:05:09.454 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-09-20T22:05:09.454 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.454 INFO:teuthology.orchestra.run.smithi022.stdout: "last_failure_osd_epoch": 3, 2023-09-20T22:05:09.454 INFO:teuthology.orchestra.run.smithi022.stdout: "active_clients": [ 2023-09-20T22:05:09.454 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.455 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "devicehealth", 2023-09-20T22:05:09.455 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2023-09-20T22:05:09.455 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.455 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2023-09-20T22:05:09.455 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2023-09-20T22:05:09.455 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 3109490890 2023-09-20T22:05:09.456 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.456 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-09-20T22:05:09.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.456 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.456 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "libcephsqlite", 2023-09-20T22:05:09.457 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2023-09-20T22:05:09.457 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.457 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2023-09-20T22:05:09.457 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2023-09-20T22:05:09.457 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 2546572633 2023-09-20T22:05:09.457 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.458 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-09-20T22:05:09.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.458 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.458 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "rbd_support", 2023-09-20T22:05:09.458 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2023-09-20T22:05:09.459 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.459 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2023-09-20T22:05:09.459 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2023-09-20T22:05:09.459 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 1576470946 2023-09-20T22:05:09.459 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.460 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-09-20T22:05:09.460 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:09.460 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.460 INFO:teuthology.orchestra.run.smithi022.stdout: "name": "volumes", 2023-09-20T22:05:09.460 INFO:teuthology.orchestra.run.smithi022.stdout: "addrvec": [ 2023-09-20T22:05:09.460 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-09-20T22:05:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "type": "v2", 2023-09-20T22:05:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "addr": "172.21.15.22:0", 2023-09-20T22:05:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: "nonce": 627665965 2023-09-20T22:05:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.461 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-09-20T22:05:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:09.462 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-09-20T22:05:09.462 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:05:09.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:05 smithi089 ceph-mon[134090]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-20T22:05:09.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:05 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1393655927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T22:05:09.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:05 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/171257051' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-20T22:05:09.464 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:07 smithi089 ceph-mon[134090]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2023-09-20T22:05:09.465 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:05 smithi123 ceph-mon[134433]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-20T22:05:09.465 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:05 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1393655927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T22:05:09.465 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:05 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/171257051' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-20T22:05:09.465 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:07 smithi123 ceph-mon[134433]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2023-09-20T22:05:09.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.955+0000 7fdc7b59e700 1 -- 172.21.15.22:0/171257051 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdc640a1470 msgr2=0x7fdc640a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:09.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.955+0000 7fdc7b59e700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdc640a1470 0x7fdc640a3930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000b2e0 tx=0x7fdc70005fb0 comp rx=0 tx=0).stop 2023-09-20T22:05:09.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.955+0000 7fdc7b59e700 1 -- 172.21.15.22:0/171257051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc7c102e60 msgr2=0x7fdc7c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:09.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.955+0000 7fdc7b59e700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc7c102e60 0x7fdc7c1989d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00e9a0 tx=0x7fdc6c00ecb0 comp rx=0 tx=0).stop 2023-09-20T22:05:09.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.956+0000 7fdc7b59e700 1 -- 172.21.15.22:0/171257051 shutdown_connections 2023-09-20T22:05:09.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.956+0000 7fdc7b59e700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fdc640a1470 0x7fdc640a3930 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:09.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.956+0000 7fdc7b59e700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdc7c104970 0x7fdc7c19ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:09.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.956+0000 7fdc7b59e700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc7c102e60 0x7fdc7c1989d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:09.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.956+0000 7fdc7b59e700 1 --2- 172.21.15.22:0/171257051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc7c103fb0 0x7fdc7c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:09.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.956+0000 7fdc7b59e700 1 -- 172.21.15.22:0/171257051 >> 172.21.15.22:0/171257051 conn(0x7fdc7c0fe3b0 msgr2=0x7fdc7c100530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:09.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.956+0000 7fdc7b59e700 1 -- 172.21.15.22:0/171257051 shutdown_connections 2023-09-20T22:05:09.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:05.956+0000 7fdc7b59e700 1 -- 172.21.15.22:0/171257051 wait complete. 2023-09-20T22:05:09.506 DEBUG:tasks.check_counter:Getting stats from a 2023-09-20T22:05:09.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.a perf dump 2023-09-20T22:05:09.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:05 smithi022 ceph-mon[97651]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 KiB/s wr, 13 op/s 2023-09-20T22:05:09.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:05 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1393655927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T22:05:09.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:05 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/171257051' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-09-20T22:05:09.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:07 smithi022 ceph-mon[97651]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2023-09-20T22:05:10.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:09 smithi022 ceph-mon[97651]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2023-09-20T22:05:10.326 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T22:05:10.342 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:09 smithi123 ceph-mon[134433]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2023-09-20T22:05:10.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:09 smithi089 ceph-mon[134090]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2023-09-20T22:05:12.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:11 smithi022 ceph-mon[97651]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2023-09-20T22:05:12.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:11 smithi123 ceph-mon[134433]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2023-09-20T22:05:12.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:11 smithi089 ceph-mon[134090]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2023-09-20T22:05:13.413 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:05:13.413 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:13.413 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 5300, 2023-09-20T22:05:13.414 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 6127, 2023-09-20T22:05:13.414 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1348746, 2023-09-20T22:05:13.414 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 4607743, 2023-09-20T22:05:13.414 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 13, 2023-09-20T22:05:13.414 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 6, 2023-09-20T22:05:13.415 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.582192957, 2023-09-20T22:05:13.415 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.235023536, 2023-09-20T22:05:13.415 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.349846420, 2023-09-20T22:05:13.415 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.050864831, 2023-09-20T22:05:13.415 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:13.415 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 6125, 2023-09-20T22:05:13.416 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 2.560025408, 2023-09-20T22:05:13.416 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000417963 2023-09-20T22:05:13.416 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.416 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:13.416 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 8227, 2023-09-20T22:05:13.417 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.005700173, 2023-09-20T22:05:13.417 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000692 2023-09-20T22:05:13.417 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.417 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-20T22:05:13.417 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2023-09-20T22:05:13.418 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.418 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:13.418 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 4242, 2023-09-20T22:05:13.418 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 5097, 2023-09-20T22:05:13.418 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1416890, 2023-09-20T22:05:13.419 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 2557480, 2023-09-20T22:05:13.419 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 15, 2023-09-20T22:05:13.419 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 5, 2023-09-20T22:05:13.419 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.555478877, 2023-09-20T22:05:13.419 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.338158204, 2023-09-20T22:05:13.420 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.270214995, 2023-09-20T22:05:13.420 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.011122247, 2023-09-20T22:05:13.420 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:13.420 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 5097, 2023-09-20T22:05:13.420 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 2.276086577, 2023-09-20T22:05:13.420 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000446554 2023-09-20T22:05:13.421 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.421 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:13.421 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 4422, 2023-09-20T22:05:13.421 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.004364197, 2023-09-20T22:05:13.421 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000986 2023-09-20T22:05:13.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 312944, 2023-09-20T22:05:13.422 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 187536 2023-09-20T22:05:13.422 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.422 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:13.423 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 503, 2023-09-20T22:05:13.423 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 720, 2023-09-20T22:05:13.423 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 150875, 2023-09-20T22:05:13.423 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 805801, 2023-09-20T22:05:13.423 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 12, 2023-09-20T22:05:13.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 8, 2023-09-20T22:05:13.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.097235290, 2023-09-20T22:05:13.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.050699847, 2023-09-20T22:05:13.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.040410204, 2023-09-20T22:05:13.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.007789836, 2023-09-20T22:05:13.424 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:13.425 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 718, 2023-09-20T22:05:13.425 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.334826608, 2023-09-20T22:05:13.425 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000466332 2023-09-20T22:05:13.425 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.425 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:13.426 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 620, 2023-09-20T22:05:13.426 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000139548, 2023-09-20T22:05:13.426 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000225 2023-09-20T22:05:13.426 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.426 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-20T22:05:13.427 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 560848 2023-09-20T22:05:13.427 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.427 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2023-09-20T22:05:13.427 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 1, 2023-09-20T22:05:13.427 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2023-09-20T22:05:13.428 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.428 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-MDSRank": { 2023-09-20T22:05:13.428 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2023-09-20T22:05:13.428 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2023-09-20T22:05:13.428 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2342, 2023-09-20T22:05:13.429 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 1.383669086, 2023-09-20T22:05:13.429 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000590806 2023-09-20T22:05:13.429 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:13.429 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.429 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-PurgeQueue": { 2023-09-20T22:05:13.430 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2023-09-20T22:05:13.430 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2023-09-20T22:05:13.430 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2, 2023-09-20T22:05:13.430 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000018046, 2023-09-20T22:05:13.430 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000009023 2023-09-20T22:05:13.431 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:13.431 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.431 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2023-09-20T22:05:13.431 INFO:teuthology.orchestra.run.smithi022.stdout: "request": 2453, 2023-09-20T22:05:13.431 INFO:teuthology.orchestra.run.smithi022.stdout: "reply": 2418, 2023-09-20T22:05:13.432 INFO:teuthology.orchestra.run.smithi022.stdout: "reply_latency": { 2023-09-20T22:05:13.432 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2418, 2023-09-20T22:05:13.432 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 23.524908626, 2023-09-20T22:05:13.432 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.009729077 2023-09-20T22:05:13.432 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.433 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_reply": 0, 2023-09-20T22:05:13.433 INFO:teuthology.orchestra.run.smithi022.stdout: "forward": 35, 2023-09-20T22:05:13.433 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_complete": 0, 2023-09-20T22:05:13.433 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_keys": 0, 2023-09-20T22:05:13.433 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_commit": 11, 2023-09-20T22:05:13.434 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_split": 0, 2023-09-20T22:05:13.434 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_merge": 0, 2023-09-20T22:05:13.434 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes": 453, 2023-09-20T22:05:13.434 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_top": 3, 2023-09-20T22:05:13.434 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_bottom": 450, 2023-09-20T22:05:13.435 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pin_tail": 0, 2023-09-20T22:05:13.435 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pinned": 440, 2023-09-20T22:05:13.435 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_expired": 0, 2023-09-20T22:05:13.435 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_with_caps": 63, 2023-09-20T22:05:13.435 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": 63, 2023-09-20T22:05:13.436 INFO:teuthology.orchestra.run.smithi022.stdout: "subtrees": 4, 2023-09-20T22:05:13.436 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse": 12428, 2023-09-20T22:05:13.436 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_hit": 9124, 2023-09-20T22:05:13.436 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_forward": 35, 2023-09-20T22:05:13.436 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_discover": 21, 2023-09-20T22:05:13.437 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_dir_fetch": 0, 2023-09-20T22:05:13.437 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_remote_ino": 0, 2023-09-20T22:05:13.437 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_lock": 2606, 2023-09-20T22:05:13.437 INFO:teuthology.orchestra.run.smithi022.stdout: "load_cent": 361, 2023-09-20T22:05:13.437 INFO:teuthology.orchestra.run.smithi022.stdout: "q": 0, 2023-09-20T22:05:13.438 INFO:teuthology.orchestra.run.smithi022.stdout: "exported": 0, 2023-09-20T22:05:13.438 INFO:teuthology.orchestra.run.smithi022.stdout: "exported_inodes": 0, 2023-09-20T22:05:13.438 INFO:teuthology.orchestra.run.smithi022.stdout: "imported": 1, 2023-09-20T22:05:13.438 INFO:teuthology.orchestra.run.smithi022.stdout: "imported_inodes": 35, 2023-09-20T22:05:13.439 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_dir_fetch": 0, 2023-09-20T22:05:13.439 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_backtrace_fetch": 92, 2023-09-20T22:05:13.439 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_peer_discover": 0, 2023-09-20T22:05:13.439 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rfiles": 1033, 2023-09-20T22:05:13.440 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rbytes": 1632663607, 2023-09-20T22:05:13.440 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rsnaps": 0, 2023-09-20T22:05:13.440 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_fetch": 92, 2023-09-20T22:05:13.440 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_set_tag": 0, 2023-09-20T22:05:13.440 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_repaired": 0, 2023-09-20T22:05:13.441 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_inotable_repaired": 0, 2023-09-20T22:05:13.441 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_inodes": 2, 2023-09-20T22:05:13.441 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_base_inodes": 0, 2023-09-20T22:05:13.441 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dirfrag_rstats": 2, 2023-09-20T22:05:13.441 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_file_inodes": 90, 2023-09-20T22:05:13.442 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_inode_file_caps": 24, 2023-09-20T22:05:13.442 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_revoke": 367, 2023-09-20T22:05:13.442 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_grant": 854, 2023-09-20T22:05:13.442 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_trunc": 0, 2023-09-20T22:05:13.442 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-20T22:05:13.443 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-20T22:05:13.443 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps": 1098, 2023-09-20T22:05:13.443 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps_dirty": 434, 2023-09-20T22:05:13.443 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_cap_release": 52, 2023-09-20T22:05:13.444 INFO:teuthology.orchestra.run.smithi022.stdout: "process_request_cap_release": 970 2023-09-20T22:05:13.444 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.444 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache": { 2023-09-20T22:05:13.444 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update": 288, 2023-09-20T22:05:13.444 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update_receipt": 295, 2023-09-20T22:05:13.445 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_try_discover": 158, 2023-09-20T22:05:13.445 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_send_discover": 256, 2023-09-20T22:05:13.445 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_handle_discover": 311, 2023-09-20T22:05:13.445 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays": 0, 2023-09-20T22:05:13.445 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_delayed": 0, 2023-09-20T22:05:13.445 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_enqueuing": 0, 2023-09-20T22:05:13.446 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_created": 235, 2023-09-20T22:05:13.446 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_enqueued": 0, 2023-09-20T22:05:13.446 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_reintegrated": 968, 2023-09-20T22:05:13.446 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_migrated": 0, 2023-09-20T22:05:13.446 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_processing": 0, 2023-09-20T22:05:13.447 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_enqueued": 0, 2023-09-20T22:05:13.447 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_prioritized": 0, 2023-09-20T22:05:13.447 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_started": 0, 2023-09-20T22:05:13.447 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_completed": 0, 2023-09-20T22:05:13.447 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_enqueue_scrub": 0, 2023-09-20T22:05:13.447 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_exportdir": 0, 2023-09-20T22:05:13.448 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_flush": 0, 2023-09-20T22:05:13.448 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragmentdir": 0, 2023-09-20T22:05:13.448 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragstats": 0, 2023-09-20T22:05:13.448 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_inodestats": 0 2023-09-20T22:05:13.448 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.449 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log": { 2023-09-20T22:05:13.449 INFO:teuthology.orchestra.run.smithi022.stdout: "evlrg": 0, 2023-09-20T22:05:13.449 INFO:teuthology.orchestra.run.smithi022.stdout: "evadd": 1643, 2023-09-20T22:05:13.449 INFO:teuthology.orchestra.run.smithi022.stdout: "evex": 0, 2023-09-20T22:05:13.449 INFO:teuthology.orchestra.run.smithi022.stdout: "evtrm": 0, 2023-09-20T22:05:13.450 INFO:teuthology.orchestra.run.smithi022.stdout: "ev": 1643, 2023-09-20T22:05:13.450 INFO:teuthology.orchestra.run.smithi022.stdout: "evexg": 0, 2023-09-20T22:05:13.450 INFO:teuthology.orchestra.run.smithi022.stdout: "evexd": 0, 2023-09-20T22:05:13.450 INFO:teuthology.orchestra.run.smithi022.stdout: "segadd": 3, 2023-09-20T22:05:13.450 INFO:teuthology.orchestra.run.smithi022.stdout: "segex": 0, 2023-09-20T22:05:13.451 INFO:teuthology.orchestra.run.smithi022.stdout: "segtrm": 0, 2023-09-20T22:05:13.451 INFO:teuthology.orchestra.run.smithi022.stdout: "seg": 3, 2023-09-20T22:05:13.451 INFO:teuthology.orchestra.run.smithi022.stdout: "segmjr": 2, 2023-09-20T22:05:13.451 INFO:teuthology.orchestra.run.smithi022.stdout: "segexg": 0, 2023-09-20T22:05:13.451 INFO:teuthology.orchestra.run.smithi022.stdout: "segexd": 0, 2023-09-20T22:05:13.451 INFO:teuthology.orchestra.run.smithi022.stdout: "expos": 4194304, 2023-09-20T22:05:13.452 INFO:teuthology.orchestra.run.smithi022.stdout: "wrpos": 7015294, 2023-09-20T22:05:13.452 INFO:teuthology.orchestra.run.smithi022.stdout: "rdpos": 0, 2023-09-20T22:05:13.452 INFO:teuthology.orchestra.run.smithi022.stdout: "jlat": { 2023-09-20T22:05:13.452 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1114, 2023-09-20T22:05:13.452 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 6.387815175, 2023-09-20T22:05:13.453 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.005734124 2023-09-20T22:05:13.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.453 INFO:teuthology.orchestra.run.smithi022.stdout: "replayed": 0 2023-09-20T22:05:13.453 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.453 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_mem": { 2023-09-20T22:05:13.454 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 95, 2023-09-20T22:05:13.454 INFO:teuthology.orchestra.run.smithi022.stdout: "ino+": 95, 2023-09-20T22:05:13.454 INFO:teuthology.orchestra.run.smithi022.stdout: "ino-": 0, 2023-09-20T22:05:13.454 INFO:teuthology.orchestra.run.smithi022.stdout: "dir": 26, 2023-09-20T22:05:13.454 INFO:teuthology.orchestra.run.smithi022.stdout: "dir+": 152, 2023-09-20T22:05:13.455 INFO:teuthology.orchestra.run.smithi022.stdout: "dir-": 126, 2023-09-20T22:05:13.455 INFO:teuthology.orchestra.run.smithi022.stdout: "dn": 453, 2023-09-20T22:05:13.455 INFO:teuthology.orchestra.run.smithi022.stdout: "dn+": 453, 2023-09-20T22:05:13.455 INFO:teuthology.orchestra.run.smithi022.stdout: "dn-": 0, 2023-09-20T22:05:13.455 INFO:teuthology.orchestra.run.smithi022.stdout: "cap": 63, 2023-09-20T22:05:13.455 INFO:teuthology.orchestra.run.smithi022.stdout: "cap+": 235, 2023-09-20T22:05:13.456 INFO:teuthology.orchestra.run.smithi022.stdout: "cap-": 172, 2023-09-20T22:05:13.456 INFO:teuthology.orchestra.run.smithi022.stdout: "rss": 55268, 2023-09-20T22:05:13.456 INFO:teuthology.orchestra.run.smithi022.stdout: "heap": 198960 2023-09-20T22:05:13.456 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.456 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_server": { 2023-09-20T22:05:13.457 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_client_request": 7777, 2023-09-20T22:05:13.457 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_server_request": 5, 2023-09-20T22:05:13.457 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_request": 2453, 2023-09-20T22:05:13.457 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_session": 401, 2023-09-20T22:05:13.457 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_peer_request": 63, 2023-09-20T22:05:13.458 INFO:teuthology.orchestra.run.smithi022.stdout: "req_create_latency": { 2023-09-20T22:05:13.458 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 35, 2023-09-20T22:05:13.458 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.039180080, 2023-09-20T22:05:13.458 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.001119430 2023-09-20T22:05:13.458 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.459 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getattr_latency": { 2023-09-20T22:05:13.461 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 5, 2023-09-20T22:05:13.461 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.019815931, 2023-09-20T22:05:13.461 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.003963186 2023-09-20T22:05:13.462 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.462 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getfilelock_latency": { 2023-09-20T22:05:13.462 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.462 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.462 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.463 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.463 INFO:teuthology.orchestra.run.smithi022.stdout: "req_link_latency": { 2023-09-20T22:05:13.463 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 235, 2023-09-20T22:05:13.463 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.928450757, 2023-09-20T22:05:13.463 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.003950854 2023-09-20T22:05:13.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.464 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookup_latency": { 2023-09-20T22:05:13.464 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 844, 2023-09-20T22:05:13.464 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 2.372067423, 2023-09-20T22:05:13.464 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.002810506 2023-09-20T22:05:13.464 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.465 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookuphash_latency": { 2023-09-20T22:05:13.465 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.465 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.465 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.465 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.466 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupino_latency": { 2023-09-20T22:05:13.466 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.466 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.466 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.466 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.467 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupname_latency": { 2023-09-20T22:05:13.467 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.467 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.467 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.467 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.467 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupparent_latency": { 2023-09-20T22:05:13.468 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.468 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.468 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.468 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.468 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupsnap_latency": { 2023-09-20T22:05:13.469 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.469 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.469 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.469 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.469 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lssnap_latency": { 2023-09-20T22:05:13.470 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.470 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.470 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.470 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.470 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mkdir_latency": { 2023-09-20T22:05:13.470 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.471 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.471 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.471 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.471 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mknod_latency": { 2023-09-20T22:05:13.471 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.472 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.472 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.472 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.472 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mksnap_latency": { 2023-09-20T22:05:13.472 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.472 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.473 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.473 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.473 INFO:teuthology.orchestra.run.smithi022.stdout: "req_open_latency": { 2023-09-20T22:05:13.473 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 86, 2023-09-20T22:05:13.473 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.095317736, 2023-09-20T22:05:13.474 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.001108345 2023-09-20T22:05:13.474 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.474 INFO:teuthology.orchestra.run.smithi022.stdout: "req_readdir_latency": { 2023-09-20T22:05:13.474 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 10, 2023-09-20T22:05:13.474 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.020788712, 2023-09-20T22:05:13.475 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.002078871 2023-09-20T22:05:13.475 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.475 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rename_latency": { 2023-09-20T22:05:13.475 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 968, 2023-09-20T22:05:13.475 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 19.121913971, 2023-09-20T22:05:13.476 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.019754043 2023-09-20T22:05:13.476 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.476 INFO:teuthology.orchestra.run.smithi022.stdout: "req_renamesnap_latency": { 2023-09-20T22:05:13.476 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.476 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.476 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.477 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.477 INFO:teuthology.orchestra.run.smithi022.stdout: "req_snapdiff_latency": { 2023-09-20T22:05:13.477 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.477 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.477 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.478 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmdir_latency": { 2023-09-20T22:05:13.478 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.478 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.478 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.479 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.479 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmsnap_latency": { 2023-09-20T22:05:13.479 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.479 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.479 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.479 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.480 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmxattr_latency": { 2023-09-20T22:05:13.480 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.480 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.480 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.480 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.481 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setattr_latency": { 2023-09-20T22:05:13.481 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.481 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.481 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.481 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.482 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setdirlayout_latency": { 2023-09-20T22:05:13.482 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.482 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.482 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.482 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.483 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setfilelock_latency": { 2023-09-20T22:05:13.483 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.483 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.483 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.483 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.484 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setlayout_latency": { 2023-09-20T22:05:13.484 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.484 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.484 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.484 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.484 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setxattr_latency": { 2023-09-20T22:05:13.485 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.485 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.485 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.485 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.486 INFO:teuthology.orchestra.run.smithi022.stdout: "req_symlink_latency": { 2023-09-20T22:05:13.486 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.486 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.486 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.486 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.487 INFO:teuthology.orchestra.run.smithi022.stdout: "req_unlink_latency": { 2023-09-20T22:05:13.487 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 235, 2023-09-20T22:05:13.487 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.927374016, 2023-09-20T22:05:13.487 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.003946272 2023-09-20T22:05:13.487 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.488 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_revoke_eviction": 0, 2023-09-20T22:05:13.488 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_acquisition_throttle": 0, 2023-09-20T22:05:13.488 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getvxattr_latency": { 2023-09-20T22:05:13.488 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.488 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.489 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.489 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:13.489 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.489 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_sessions": { 2023-09-20T22:05:13.489 INFO:teuthology.orchestra.run.smithi022.stdout: "session_count": 1, 2023-09-20T22:05:13.490 INFO:teuthology.orchestra.run.smithi022.stdout: "session_add": 1, 2023-09-20T22:05:13.490 INFO:teuthology.orchestra.run.smithi022.stdout: "session_remove": 0, 2023-09-20T22:05:13.490 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_open": 1, 2023-09-20T22:05:13.490 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_stale": 0, 2023-09-20T22:05:13.490 INFO:teuthology.orchestra.run.smithi022.stdout: "total_load": 281, 2023-09-20T22:05:13.491 INFO:teuthology.orchestra.run.smithi022.stdout: "average_load": 281, 2023-09-20T22:05:13.491 INFO:teuthology.orchestra.run.smithi022.stdout: "avg_session_uptime": 969, 2023-09-20T22:05:13.491 INFO:teuthology.orchestra.run.smithi022.stdout: "mdthresh_evicted": 0 2023-09-20T22:05:13.491 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.491 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2023-09-20T22:05:13.492 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:13.492 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:13.492 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:13.492 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:13.492 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:13.493 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:13.493 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:13.493 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:13.493 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:13.493 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:13.494 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:13.494 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:13.494 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:13.494 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:13.494 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:13.495 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:13.495 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:13.495 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:13.495 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:13.495 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:13.495 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:13.496 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:13.496 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:13.496 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:13.496 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:13.496 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:13.497 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:13.497 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:13.497 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:13.497 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:13.497 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:13.498 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2023-09-20T22:05:13.498 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:13.498 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:13.498 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:13.498 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:13.499 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 798127, 2023-09-20T22:05:13.499 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 318, 2023-09-20T22:05:13.499 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2023-09-20T22:05:13.499 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2023-09-20T22:05:13.499 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2023-09-20T22:05:13.500 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2023-09-20T22:05:13.500 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:13.500 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:13.500 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:13.500 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:13.501 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 9088, 2023-09-20T22:05:13.501 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 175, 2023-09-20T22:05:13.501 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:13.501 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:13.501 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:13.502 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2023-09-20T22:05:13.502 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 625970, 2023-09-20T22:05:13.502 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 16941, 2023-09-20T22:05:13.502 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:13.502 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2023-09-20T22:05:13.502 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:13.503 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2023-09-20T22:05:13.503 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.503 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter": { 2023-09-20T22:05:13.503 INFO:teuthology.orchestra.run.smithi022.stdout: "op_active": 0, 2023-09-20T22:05:13.503 INFO:teuthology.orchestra.run.smithi022.stdout: "op_laggy": 0, 2023-09-20T22:05:13.504 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send": 1417, 2023-09-20T22:05:13.504 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send_bytes": 2869305, 2023-09-20T22:05:13.504 INFO:teuthology.orchestra.run.smithi022.stdout: "op_resend": 27, 2023-09-20T22:05:13.504 INFO:teuthology.orchestra.run.smithi022.stdout: "op_reply": 1417, 2023-09-20T22:05:13.504 INFO:teuthology.orchestra.run.smithi022.stdout: "op_latency": { 2023-09-20T22:05:13.505 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1417, 2023-09-20T22:05:13.505 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 12.919847071, 2023-09-20T22:05:13.505 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.009117746 2023-09-20T22:05:13.505 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.505 INFO:teuthology.orchestra.run.smithi022.stdout: "op_inflight": 0, 2023-09-20T22:05:13.506 INFO:teuthology.orchestra.run.smithi022.stdout: "oplen_avg": { 2023-09-20T22:05:13.506 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1417, 2023-09-20T22:05:13.506 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 1584 2023-09-20T22:05:13.506 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.506 INFO:teuthology.orchestra.run.smithi022.stdout: "op": 1417, 2023-09-20T22:05:13.506 INFO:teuthology.orchestra.run.smithi022.stdout: "op_r": 92, 2023-09-20T22:05:13.507 INFO:teuthology.orchestra.run.smithi022.stdout: "op_w": 1325, 2023-09-20T22:05:13.507 INFO:teuthology.orchestra.run.smithi022.stdout: "op_rmw": 0, 2023-09-20T22:05:13.507 INFO:teuthology.orchestra.run.smithi022.stdout: "op_pg": 0, 2023-09-20T22:05:13.507 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_stat": 0, 2023-09-20T22:05:13.507 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_create": 10, 2023-09-20T22:05:13.508 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_read": 0, 2023-09-20T22:05:13.508 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_write": 1114, 2023-09-20T22:05:13.508 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writefull": 61, 2023-09-20T22:05:13.508 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writesame": 0, 2023-09-20T22:05:13.508 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_append": 0, 2023-09-20T22:05:13.509 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_zero": 0, 2023-09-20T22:05:13.509 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_truncate": 0, 2023-09-20T22:05:13.509 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_delete": 6, 2023-09-20T22:05:13.509 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_mapext": 0, 2023-09-20T22:05:13.509 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_sparse_read": 0, 2023-09-20T22:05:13.509 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_clonerange": 0, 2023-09-20T22:05:13.510 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_getxattr": 92, 2023-09-20T22:05:13.510 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_setxattr": 20, 2023-09-20T22:05:13.510 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_cmpxattr": 0, 2023-09-20T22:05:13.510 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_rmxattr": 0, 2023-09-20T22:05:13.510 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_resetxattrs": 0, 2023-09-20T22:05:13.511 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_call": 0, 2023-09-20T22:05:13.511 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_watch": 0, 2023-09-20T22:05:13.511 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_notify": 0, 2023-09-20T22:05:13.511 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_src_cmpxattr": 0, 2023-09-20T22:05:13.511 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls": 0, 2023-09-20T22:05:13.511 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls_filter": 0, 2023-09-20T22:05:13.512 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_other": 0, 2023-09-20T22:05:13.512 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_active": 0, 2023-09-20T22:05:13.512 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_send": 0, 2023-09-20T22:05:13.512 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_resend": 0, 2023-09-20T22:05:13.512 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_ping": 0, 2023-09-20T22:05:13.513 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_active": 0, 2023-09-20T22:05:13.513 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_send": 0, 2023-09-20T22:05:13.513 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_resend": 0, 2023-09-20T22:05:13.513 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_active": 0, 2023-09-20T22:05:13.513 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_send": 0, 2023-09-20T22:05:13.513 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_resend": 0, 2023-09-20T22:05:13.514 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_active": 0, 2023-09-20T22:05:13.514 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_send": 0, 2023-09-20T22:05:13.514 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_resend": 0, 2023-09-20T22:05:13.514 INFO:teuthology.orchestra.run.smithi022.stdout: "command_active": 0, 2023-09-20T22:05:13.514 INFO:teuthology.orchestra.run.smithi022.stdout: "command_send": 0, 2023-09-20T22:05:13.515 INFO:teuthology.orchestra.run.smithi022.stdout: "command_resend": 0, 2023-09-20T22:05:13.515 INFO:teuthology.orchestra.run.smithi022.stdout: "map_epoch": 93, 2023-09-20T22:05:13.515 INFO:teuthology.orchestra.run.smithi022.stdout: "map_full": 0, 2023-09-20T22:05:13.515 INFO:teuthology.orchestra.run.smithi022.stdout: "map_inc": 4, 2023-09-20T22:05:13.515 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_sessions": 12, 2023-09-20T22:05:13.515 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_open": 13, 2023-09-20T22:05:13.516 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_close": 1, 2023-09-20T22:05:13.516 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_laggy": 0, 2023-09-20T22:05:13.516 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_wr": 204, 2023-09-20T22:05:13.516 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_rd": 0, 2023-09-20T22:05:13.516 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_del": 77 2023-09-20T22:05:13.517 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.517 INFO:teuthology.orchestra.run.smithi022.stdout: "oft": { 2023-09-20T22:05:13.517 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_objs": 1, 2023-09-20T22:05:13.517 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_kv_pairs": 0, 2023-09-20T22:05:13.517 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_updates": 0, 2023-09-20T22:05:13.518 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_removes": 0 2023-09-20T22:05:13.518 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.518 INFO:teuthology.orchestra.run.smithi022.stdout: "osd-slow-ops": { 2023-09-20T22:05:13.518 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_ops_count": 0 2023-09-20T22:05:13.518 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.518 INFO:teuthology.orchestra.run.smithi022.stdout: "purge_queue": { 2023-09-20T22:05:13.519 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops": 0, 2023-09-20T22:05:13.519 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops_high_water": 0, 2023-09-20T22:05:13.519 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing": 0, 2023-09-20T22:05:13.519 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_high_water": 0, 2023-09-20T22:05:13.519 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executed": 0, 2023-09-20T22:05:13.520 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_item_in_journal": 0 2023-09-20T22:05:13.520 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.520 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:13.520 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:13.520 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2023-09-20T22:05:13.520 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:13.521 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 10041, 2023-09-20T22:05:13.521 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 2046640, 2023-09-20T22:05:13.521 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:13.521 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 10041, 2023-09-20T22:05:13.521 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-09-20T22:05:13.522 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-09-20T22:05:13.522 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 10041, 2023-09-20T22:05:13.522 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 2046640, 2023-09-20T22:05:13.522 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:13.522 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.522 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_bytes": { 2023-09-20T22:05:13.523 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2023-09-20T22:05:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-09-20T22:05:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-09-20T22:05:13.524 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 1417, 2023-09-20T22:05:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 2869305, 2023-09-20T22:05:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 1417, 2023-09-20T22:05:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 2869305, 2023-09-20T22:05:13.525 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:13.526 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_ops": { 2023-09-20T22:05:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 1024, 2023-09-20T22:05:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-09-20T22:05:13.527 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-09-20T22:05:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 1417, 2023-09-20T22:05:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 1417, 2023-09-20T22:05:13.528 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 1417, 2023-09-20T22:05:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1417, 2023-09-20T22:05:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.529 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle": { 2023-09-20T22:05:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:13.530 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2023-09-20T22:05:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-09-20T22:05:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-09-20T22:05:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:13.531 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-09-20T22:05:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-09-20T22:05:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 0, 2023-09-20T22:05:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 0, 2023-09-20T22:05:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:13.532 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle-0x55e71d8386e0": { 2023-09-20T22:05:13.533 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2023-09-20T22:05:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 1643, 2023-09-20T22:05:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 2820990, 2023-09-20T22:05:13.534 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 1643, 2023-09-20T22:05:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-09-20T22:05:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-09-20T22:05:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 1114, 2023-09-20T22:05:13.535 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 2820990, 2023-09-20T22:05:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:13.536 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:13.537 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:05:13.985 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-09-20T22:05:13.986 INFO:tasks.check_counter:Daemon mds.a mds.imported=1 2023-09-20T22:05:13.986 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=288 2023-09-20T22:05:13.986 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=295 2023-09-20T22:05:13.987 DEBUG:tasks.check_counter:Getting stats from d 2023-09-20T22:05:13.987 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.d perf dump 2023-09-20T22:05:14.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:13 smithi022 ceph-mon[97651]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-09-20T22:05:14.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:13 smithi123 ceph-mon[134433]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-09-20T22:05:14.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:13 smithi089 ceph-mon[134090]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-09-20T22:05:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T22:05:16.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:15 smithi022 ceph-mon[97651]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2023-09-20T22:05:16.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:15 smithi123 ceph-mon[134433]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2023-09-20T22:05:16.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:15 smithi089 ceph-mon[134090]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2023-09-20T22:05:16.941 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:05:16.941 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:16.942 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 2300, 2023-09-20T22:05:16.942 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2302, 2023-09-20T22:05:16.942 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 1772837, 2023-09-20T22:05:16.942 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 715301, 2023-09-20T22:05:16.943 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 6, 2023-09-20T22:05:16.943 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 2, 2023-09-20T22:05:16.943 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.260272478, 2023-09-20T22:05:16.943 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.076369924, 2023-09-20T22:05:16.943 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.070558205, 2023-09-20T22:05:16.943 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.087656445, 2023-09-20T22:05:16.944 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:16.944 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2300, 2023-09-20T22:05:16.944 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 1.057763474, 2023-09-20T22:05:16.944 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000459897 2023-09-20T22:05:16.944 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.945 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:16.945 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.945 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.945 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.945 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.946 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-20T22:05:16.946 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2023-09-20T22:05:16.946 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.949 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:16.949 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 965, 2023-09-20T22:05:16.949 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 1194, 2023-09-20T22:05:16.949 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 239964, 2023-09-20T22:05:16.950 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 770644, 2023-09-20T22:05:16.950 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 6, 2023-09-20T22:05:16.950 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 4, 2023-09-20T22:05:16.950 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.137966748, 2023-09-20T22:05:16.950 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.048551114, 2023-09-20T22:05:16.951 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.029865825, 2023-09-20T22:05:16.951 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.047468558, 2023-09-20T22:05:16.951 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:16.951 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1193, 2023-09-20T22:05:16.951 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.416579329, 2023-09-20T22:05:16.951 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000349186 2023-09-20T22:05:16.952 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.952 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:16.952 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.952 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.952 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.953 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.954 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-09-20T22:05:16.954 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 472016 2023-09-20T22:05:16.954 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.954 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:16.954 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 319, 2023-09-20T22:05:16.955 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 313, 2023-09-20T22:05:16.955 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 275884, 2023-09-20T22:05:16.955 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 184983, 2023-09-20T22:05:16.955 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2023-09-20T22:05:16.955 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 2, 2023-09-20T22:05:16.956 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.052047560, 2023-09-20T22:05:16.956 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.017250243, 2023-09-20T22:05:16.956 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.021619527, 2023-09-20T22:05:16.956 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.010124034, 2023-09-20T22:05:16.956 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:16.957 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 312, 2023-09-20T22:05:16.957 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.060873766, 2023-09-20T22:05:16.957 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000195108 2023-09-20T22:05:16.957 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.957 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:16.958 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.958 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.958 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.958 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.958 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 275616, 2023-09-20T22:05:16.959 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 184672 2023-09-20T22:05:16.959 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.959 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2023-09-20T22:05:16.959 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 1, 2023-09-20T22:05:16.959 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2023-09-20T22:05:16.960 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.960 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-MDSRank": { 2023-09-20T22:05:16.960 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2023-09-20T22:05:16.960 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2023-09-20T22:05:16.960 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 5455, 2023-09-20T22:05:16.961 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.191765772, 2023-09-20T22:05:16.961 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000035154 2023-09-20T22:05:16.961 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:16.961 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.961 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-PurgeQueue": { 2023-09-20T22:05:16.962 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2023-09-20T22:05:16.962 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2023-09-20T22:05:16.962 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.962 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.962 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.963 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:16.963 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.963 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2023-09-20T22:05:16.963 INFO:teuthology.orchestra.run.smithi022.stdout: "request": 0, 2023-09-20T22:05:16.963 INFO:teuthology.orchestra.run.smithi022.stdout: "reply": 0, 2023-09-20T22:05:16.963 INFO:teuthology.orchestra.run.smithi022.stdout: "reply_latency": { 2023-09-20T22:05:16.964 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.964 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.964 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.964 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.965 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_reply": 0, 2023-09-20T22:05:16.965 INFO:teuthology.orchestra.run.smithi022.stdout: "forward": 0, 2023-09-20T22:05:16.965 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_complete": 0, 2023-09-20T22:05:16.965 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_keys": 0, 2023-09-20T22:05:16.965 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_commit": 0, 2023-09-20T22:05:16.965 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_split": 0, 2023-09-20T22:05:16.966 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_merge": 0, 2023-09-20T22:05:16.966 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes": 288, 2023-09-20T22:05:16.966 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_top": 0, 2023-09-20T22:05:16.966 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_bottom": 0, 2023-09-20T22:05:16.967 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pin_tail": 288, 2023-09-20T22:05:16.967 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pinned": 288, 2023-09-20T22:05:16.967 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_expired": 0, 2023-09-20T22:05:16.967 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_with_caps": 0, 2023-09-20T22:05:16.967 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": 0, 2023-09-20T22:05:16.967 INFO:teuthology.orchestra.run.smithi022.stdout: "subtrees": 9, 2023-09-20T22:05:16.968 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse": 0, 2023-09-20T22:05:16.968 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_hit": 0, 2023-09-20T22:05:16.968 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_forward": 0, 2023-09-20T22:05:16.968 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_discover": 0, 2023-09-20T22:05:16.968 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_dir_fetch": 0, 2023-09-20T22:05:16.969 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_remote_ino": 0, 2023-09-20T22:05:16.969 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_lock": 0, 2023-09-20T22:05:16.969 INFO:teuthology.orchestra.run.smithi022.stdout: "load_cent": 0, 2023-09-20T22:05:16.969 INFO:teuthology.orchestra.run.smithi022.stdout: "q": 0, 2023-09-20T22:05:16.969 INFO:teuthology.orchestra.run.smithi022.stdout: "exported": 0, 2023-09-20T22:05:16.969 INFO:teuthology.orchestra.run.smithi022.stdout: "exported_inodes": 0, 2023-09-20T22:05:16.970 INFO:teuthology.orchestra.run.smithi022.stdout: "imported": 0, 2023-09-20T22:05:16.970 INFO:teuthology.orchestra.run.smithi022.stdout: "imported_inodes": 0, 2023-09-20T22:05:16.970 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_dir_fetch": 0, 2023-09-20T22:05:16.970 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_backtrace_fetch": 0, 2023-09-20T22:05:16.970 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_peer_discover": 0, 2023-09-20T22:05:16.971 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rfiles": 0, 2023-09-20T22:05:16.971 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rbytes": 0, 2023-09-20T22:05:16.971 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rsnaps": 0, 2023-09-20T22:05:16.971 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_fetch": 0, 2023-09-20T22:05:16.971 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_set_tag": 0, 2023-09-20T22:05:16.972 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_repaired": 0, 2023-09-20T22:05:16.972 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_inotable_repaired": 0, 2023-09-20T22:05:16.972 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_inodes": 0, 2023-09-20T22:05:16.972 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_base_inodes": 0, 2023-09-20T22:05:16.972 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dirfrag_rstats": 0, 2023-09-20T22:05:16.973 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_file_inodes": 0, 2023-09-20T22:05:16.973 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_inode_file_caps": 0, 2023-09-20T22:05:16.973 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_revoke": 0, 2023-09-20T22:05:16.973 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_grant": 0, 2023-09-20T22:05:16.973 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_trunc": 0, 2023-09-20T22:05:16.974 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-20T22:05:16.974 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-20T22:05:16.974 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps": 0, 2023-09-20T22:05:16.974 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps_dirty": 0, 2023-09-20T22:05:16.974 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_cap_release": 0, 2023-09-20T22:05:16.975 INFO:teuthology.orchestra.run.smithi022.stdout: "process_request_cap_release": 0 2023-09-20T22:05:16.975 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.975 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache": { 2023-09-20T22:05:16.975 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update": 0, 2023-09-20T22:05:16.975 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update_receipt": 0, 2023-09-20T22:05:16.976 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_try_discover": 0, 2023-09-20T22:05:16.976 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_send_discover": 0, 2023-09-20T22:05:16.976 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_handle_discover": 0, 2023-09-20T22:05:16.976 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays": 0, 2023-09-20T22:05:16.976 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_delayed": 0, 2023-09-20T22:05:16.977 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_enqueuing": 0, 2023-09-20T22:05:16.977 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_created": 0, 2023-09-20T22:05:16.977 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_enqueued": 0, 2023-09-20T22:05:16.977 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_reintegrated": 0, 2023-09-20T22:05:16.977 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_migrated": 0, 2023-09-20T22:05:16.978 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_processing": 0, 2023-09-20T22:05:16.978 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_enqueued": 0, 2023-09-20T22:05:16.978 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_prioritized": 0, 2023-09-20T22:05:16.978 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_started": 0, 2023-09-20T22:05:16.978 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_completed": 0, 2023-09-20T22:05:16.979 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_enqueue_scrub": 0, 2023-09-20T22:05:16.979 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_exportdir": 0, 2023-09-20T22:05:16.979 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_flush": 0, 2023-09-20T22:05:16.979 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragmentdir": 0, 2023-09-20T22:05:16.979 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragstats": 0, 2023-09-20T22:05:16.979 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_inodestats": 0 2023-09-20T22:05:16.980 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.980 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log": { 2023-09-20T22:05:16.980 INFO:teuthology.orchestra.run.smithi022.stdout: "evlrg": 0, 2023-09-20T22:05:16.980 INFO:teuthology.orchestra.run.smithi022.stdout: "evadd": 0, 2023-09-20T22:05:16.980 INFO:teuthology.orchestra.run.smithi022.stdout: "evex": 0, 2023-09-20T22:05:16.981 INFO:teuthology.orchestra.run.smithi022.stdout: "evtrm": 0, 2023-09-20T22:05:16.981 INFO:teuthology.orchestra.run.smithi022.stdout: "ev": 758, 2023-09-20T22:05:16.981 INFO:teuthology.orchestra.run.smithi022.stdout: "evexg": 0, 2023-09-20T22:05:16.981 INFO:teuthology.orchestra.run.smithi022.stdout: "evexd": 0, 2023-09-20T22:05:16.981 INFO:teuthology.orchestra.run.smithi022.stdout: "segadd": 0, 2023-09-20T22:05:16.981 INFO:teuthology.orchestra.run.smithi022.stdout: "segex": 0, 2023-09-20T22:05:16.982 INFO:teuthology.orchestra.run.smithi022.stdout: "segtrm": 0, 2023-09-20T22:05:16.982 INFO:teuthology.orchestra.run.smithi022.stdout: "seg": 2, 2023-09-20T22:05:16.982 INFO:teuthology.orchestra.run.smithi022.stdout: "segmjr": 2, 2023-09-20T22:05:16.982 INFO:teuthology.orchestra.run.smithi022.stdout: "segexg": 0, 2023-09-20T22:05:16.982 INFO:teuthology.orchestra.run.smithi022.stdout: "segexd": 0, 2023-09-20T22:05:16.983 INFO:teuthology.orchestra.run.smithi022.stdout: "expos": 4194304, 2023-09-20T22:05:16.983 INFO:teuthology.orchestra.run.smithi022.stdout: "wrpos": 5287780, 2023-09-20T22:05:16.983 INFO:teuthology.orchestra.run.smithi022.stdout: "rdpos": 5287206, 2023-09-20T22:05:16.983 INFO:teuthology.orchestra.run.smithi022.stdout: "jlat": { 2023-09-20T22:05:16.983 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.984 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.984 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.984 INFO:teuthology.orchestra.run.smithi022.stdout: "replayed": 758 2023-09-20T22:05:16.984 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.985 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_mem": { 2023-09-20T22:05:16.985 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 287, 2023-09-20T22:05:16.985 INFO:teuthology.orchestra.run.smithi022.stdout: "ino+": 288, 2023-09-20T22:05:16.985 INFO:teuthology.orchestra.run.smithi022.stdout: "ino-": 1, 2023-09-20T22:05:16.985 INFO:teuthology.orchestra.run.smithi022.stdout: "dir": 13, 2023-09-20T22:05:16.985 INFO:teuthology.orchestra.run.smithi022.stdout: "dir+": 13, 2023-09-20T22:05:16.986 INFO:teuthology.orchestra.run.smithi022.stdout: "dir-": 0, 2023-09-20T22:05:16.986 INFO:teuthology.orchestra.run.smithi022.stdout: "dn": 288, 2023-09-20T22:05:16.986 INFO:teuthology.orchestra.run.smithi022.stdout: "dn+": 288, 2023-09-20T22:05:16.986 INFO:teuthology.orchestra.run.smithi022.stdout: "dn-": 0, 2023-09-20T22:05:16.987 INFO:teuthology.orchestra.run.smithi022.stdout: "cap": 0, 2023-09-20T22:05:16.987 INFO:teuthology.orchestra.run.smithi022.stdout: "cap+": 0, 2023-09-20T22:05:16.987 INFO:teuthology.orchestra.run.smithi022.stdout: "cap-": 0, 2023-09-20T22:05:16.987 INFO:teuthology.orchestra.run.smithi022.stdout: "rss": 49752, 2023-09-20T22:05:16.987 INFO:teuthology.orchestra.run.smithi022.stdout: "heap": 207152 2023-09-20T22:05:16.988 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.988 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_server": { 2023-09-20T22:05:16.988 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_client_request": 0, 2023-09-20T22:05:16.988 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_server_request": 0, 2023-09-20T22:05:16.988 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_request": 0, 2023-09-20T22:05:16.989 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_session": 0, 2023-09-20T22:05:16.989 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_peer_request": 0, 2023-09-20T22:05:16.989 INFO:teuthology.orchestra.run.smithi022.stdout: "req_create_latency": { 2023-09-20T22:05:16.989 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.989 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.989 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.990 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.990 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getattr_latency": { 2023-09-20T22:05:16.990 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.990 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.990 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.991 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.991 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getfilelock_latency": { 2023-09-20T22:05:16.991 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.991 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.991 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.992 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.992 INFO:teuthology.orchestra.run.smithi022.stdout: "req_link_latency": { 2023-09-20T22:05:16.992 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.992 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.992 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.993 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookup_latency": { 2023-09-20T22:05:16.993 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.993 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.993 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.993 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.994 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookuphash_latency": { 2023-09-20T22:05:16.994 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.994 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.994 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.995 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupino_latency": { 2023-09-20T22:05:16.995 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.995 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.995 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.995 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.996 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupname_latency": { 2023-09-20T22:05:16.996 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.996 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.996 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.997 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupparent_latency": { 2023-09-20T22:05:16.997 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.997 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.997 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.998 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupsnap_latency": { 2023-09-20T22:05:16.998 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.998 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.998 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:16.999 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lssnap_latency": { 2023-09-20T22:05:16.999 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:16.999 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:16.999 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:16.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.000 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mkdir_latency": { 2023-09-20T22:05:17.000 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.000 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.000 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.000 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.001 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mknod_latency": { 2023-09-20T22:05:17.001 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.001 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.001 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.001 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.001 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mksnap_latency": { 2023-09-20T22:05:17.002 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.002 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.002 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.002 INFO:teuthology.orchestra.run.smithi022.stdout: "req_open_latency": { 2023-09-20T22:05:17.003 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.003 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.003 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.003 INFO:teuthology.orchestra.run.smithi022.stdout: "req_readdir_latency": { 2023-09-20T22:05:17.004 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.004 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.004 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.004 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rename_latency": { 2023-09-20T22:05:17.005 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.005 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.005 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.005 INFO:teuthology.orchestra.run.smithi022.stdout: "req_renamesnap_latency": { 2023-09-20T22:05:17.006 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.006 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.006 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.006 INFO:teuthology.orchestra.run.smithi022.stdout: "req_snapdiff_latency": { 2023-09-20T22:05:17.007 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.007 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.007 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.007 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmdir_latency": { 2023-09-20T22:05:17.008 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.008 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.008 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.008 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmsnap_latency": { 2023-09-20T22:05:17.009 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.009 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.009 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.009 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.009 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmxattr_latency": { 2023-09-20T22:05:17.009 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.010 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.010 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.010 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.010 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setattr_latency": { 2023-09-20T22:05:17.011 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.011 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.011 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.011 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.011 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setdirlayout_latency": { 2023-09-20T22:05:17.011 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.012 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.012 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.012 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.012 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setfilelock_latency": { 2023-09-20T22:05:17.012 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.013 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.013 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.013 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.013 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setlayout_latency": { 2023-09-20T22:05:17.013 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.014 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.014 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.014 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.014 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setxattr_latency": { 2023-09-20T22:05:17.014 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.014 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.015 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.015 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.015 INFO:teuthology.orchestra.run.smithi022.stdout: "req_symlink_latency": { 2023-09-20T22:05:17.015 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.015 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.016 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.016 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.016 INFO:teuthology.orchestra.run.smithi022.stdout: "req_unlink_latency": { 2023-09-20T22:05:17.016 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.016 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.017 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.017 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.017 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_revoke_eviction": 0, 2023-09-20T22:05:17.017 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_acquisition_throttle": 0, 2023-09-20T22:05:17.017 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getvxattr_latency": { 2023-09-20T22:05:17.017 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.018 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.018 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.018 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:17.018 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.019 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_sessions": { 2023-09-20T22:05:17.019 INFO:teuthology.orchestra.run.smithi022.stdout: "session_count": 1, 2023-09-20T22:05:17.019 INFO:teuthology.orchestra.run.smithi022.stdout: "session_add": 1, 2023-09-20T22:05:17.019 INFO:teuthology.orchestra.run.smithi022.stdout: "session_remove": 0, 2023-09-20T22:05:17.019 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_open": 1, 2023-09-20T22:05:17.019 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_stale": 0, 2023-09-20T22:05:17.020 INFO:teuthology.orchestra.run.smithi022.stdout: "total_load": 0, 2023-09-20T22:05:17.020 INFO:teuthology.orchestra.run.smithi022.stdout: "average_load": 0, 2023-09-20T22:05:17.020 INFO:teuthology.orchestra.run.smithi022.stdout: "avg_session_uptime": 2592, 2023-09-20T22:05:17.020 INFO:teuthology.orchestra.run.smithi022.stdout: "mdthresh_evicted": 0 2023-09-20T22:05:17.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.021 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2023-09-20T22:05:17.021 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:17.021 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:17.021 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:17.021 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:17.021 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:17.022 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:17.022 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:17.022 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:17.022 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:17.022 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:17.023 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:17.023 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:17.023 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:17.023 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:17.023 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:17.024 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:17.024 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:17.024 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:17.024 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:17.024 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:17.024 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:17.025 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:17.025 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:17.025 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:17.025 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:17.025 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:17.025 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:17.026 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:17.026 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:17.026 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:17.026 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:17.026 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2023-09-20T22:05:17.027 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:17.027 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:17.027 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:17.027 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:17.027 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 202700, 2023-09-20T22:05:17.028 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 337, 2023-09-20T22:05:17.028 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2023-09-20T22:05:17.028 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2023-09-20T22:05:17.028 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2023-09-20T22:05:17.028 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2023-09-20T22:05:17.028 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:17.029 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:17.029 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:17.029 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:17.029 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 9088, 2023-09-20T22:05:17.029 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 175, 2023-09-20T22:05:17.030 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:17.030 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:17.030 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:17.030 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2023-09-20T22:05:17.030 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 867035, 2023-09-20T22:05:17.030 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 8049, 2023-09-20T22:05:17.031 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:17.031 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2023-09-20T22:05:17.031 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:17.031 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2023-09-20T22:05:17.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.032 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter": { 2023-09-20T22:05:17.032 INFO:teuthology.orchestra.run.smithi022.stdout: "op_active": 0, 2023-09-20T22:05:17.032 INFO:teuthology.orchestra.run.smithi022.stdout: "op_laggy": 0, 2023-09-20T22:05:17.032 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send": 3261, 2023-09-20T22:05:17.032 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send_bytes": 21, 2023-09-20T22:05:17.033 INFO:teuthology.orchestra.run.smithi022.stdout: "op_resend": 3, 2023-09-20T22:05:17.033 INFO:teuthology.orchestra.run.smithi022.stdout: "op_reply": 3261, 2023-09-20T22:05:17.033 INFO:teuthology.orchestra.run.smithi022.stdout: "op_latency": { 2023-09-20T22:05:17.033 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 3261, 2023-09-20T22:05:17.033 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 15.452816804, 2023-09-20T22:05:17.033 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.004738674 2023-09-20T22:05:17.034 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.034 INFO:teuthology.orchestra.run.smithi022.stdout: "op_inflight": 0, 2023-09-20T22:05:17.034 INFO:teuthology.orchestra.run.smithi022.stdout: "oplen_avg": { 2023-09-20T22:05:17.034 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 3261, 2023-09-20T22:05:17.034 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 3262 2023-09-20T22:05:17.035 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.035 INFO:teuthology.orchestra.run.smithi022.stdout: "op": 3261, 2023-09-20T22:05:17.035 INFO:teuthology.orchestra.run.smithi022.stdout: "op_r": 3261, 2023-09-20T22:05:17.035 INFO:teuthology.orchestra.run.smithi022.stdout: "op_w": 0, 2023-09-20T22:05:17.035 INFO:teuthology.orchestra.run.smithi022.stdout: "op_rmw": 0, 2023-09-20T22:05:17.036 INFO:teuthology.orchestra.run.smithi022.stdout: "op_pg": 0, 2023-09-20T22:05:17.036 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_stat": 2028, 2023-09-20T22:05:17.036 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_create": 0, 2023-09-20T22:05:17.036 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_read": 1231, 2023-09-20T22:05:17.036 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_write": 0, 2023-09-20T22:05:17.037 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writefull": 0, 2023-09-20T22:05:17.037 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writesame": 0, 2023-09-20T22:05:17.037 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_append": 0, 2023-09-20T22:05:17.037 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_zero": 0, 2023-09-20T22:05:17.037 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_truncate": 0, 2023-09-20T22:05:17.038 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_delete": 0, 2023-09-20T22:05:17.038 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_mapext": 0, 2023-09-20T22:05:17.038 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_sparse_read": 0, 2023-09-20T22:05:17.038 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_clonerange": 0, 2023-09-20T22:05:17.038 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_getxattr": 1, 2023-09-20T22:05:17.038 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_setxattr": 0, 2023-09-20T22:05:17.039 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_cmpxattr": 0, 2023-09-20T22:05:17.039 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_rmxattr": 0, 2023-09-20T22:05:17.039 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_resetxattrs": 0, 2023-09-20T22:05:17.039 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_call": 0, 2023-09-20T22:05:17.039 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_watch": 0, 2023-09-20T22:05:17.040 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_notify": 0, 2023-09-20T22:05:17.040 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_src_cmpxattr": 0, 2023-09-20T22:05:17.040 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls": 0, 2023-09-20T22:05:17.040 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls_filter": 0, 2023-09-20T22:05:17.040 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_other": 0, 2023-09-20T22:05:17.041 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_active": 0, 2023-09-20T22:05:17.041 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_send": 0, 2023-09-20T22:05:17.041 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_resend": 0, 2023-09-20T22:05:17.041 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_ping": 0, 2023-09-20T22:05:17.041 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_active": 0, 2023-09-20T22:05:17.042 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_send": 0, 2023-09-20T22:05:17.042 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_resend": 0, 2023-09-20T22:05:17.042 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_active": 0, 2023-09-20T22:05:17.042 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_send": 0, 2023-09-20T22:05:17.042 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_resend": 0, 2023-09-20T22:05:17.043 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_active": 0, 2023-09-20T22:05:17.043 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_send": 0, 2023-09-20T22:05:17.043 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_resend": 0, 2023-09-20T22:05:17.043 INFO:teuthology.orchestra.run.smithi022.stdout: "command_active": 0, 2023-09-20T22:05:17.043 INFO:teuthology.orchestra.run.smithi022.stdout: "command_send": 0, 2023-09-20T22:05:17.043 INFO:teuthology.orchestra.run.smithi022.stdout: "command_resend": 0, 2023-09-20T22:05:17.044 INFO:teuthology.orchestra.run.smithi022.stdout: "map_epoch": 93, 2023-09-20T22:05:17.044 INFO:teuthology.orchestra.run.smithi022.stdout: "map_full": 0, 2023-09-20T22:05:17.044 INFO:teuthology.orchestra.run.smithi022.stdout: "map_inc": 2, 2023-09-20T22:05:17.044 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_sessions": 5, 2023-09-20T22:05:17.044 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_open": 7, 2023-09-20T22:05:17.045 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_close": 2, 2023-09-20T22:05:17.045 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_laggy": 0, 2023-09-20T22:05:17.045 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_wr": 0, 2023-09-20T22:05:17.045 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_rd": 2, 2023-09-20T22:05:17.045 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_del": 0 2023-09-20T22:05:17.045 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.046 INFO:teuthology.orchestra.run.smithi022.stdout: "oft": { 2023-09-20T22:05:17.046 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_objs": 0, 2023-09-20T22:05:17.046 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_kv_pairs": 0, 2023-09-20T22:05:17.046 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_updates": 0, 2023-09-20T22:05:17.046 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_removes": 0 2023-09-20T22:05:17.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.047 INFO:teuthology.orchestra.run.smithi022.stdout: "osd-slow-ops": { 2023-09-20T22:05:17.047 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_ops_count": 0 2023-09-20T22:05:17.047 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.047 INFO:teuthology.orchestra.run.smithi022.stdout: "purge_queue": { 2023-09-20T22:05:17.048 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops": 0, 2023-09-20T22:05:17.048 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops_high_water": 0, 2023-09-20T22:05:17.048 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing": 0, 2023-09-20T22:05:17.048 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_high_water": 0, 2023-09-20T22:05:17.048 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executed": 0, 2023-09-20T22:05:17.049 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_item_in_journal": 0 2023-09-20T22:05:17.049 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.049 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:17.049 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:17.049 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2023-09-20T22:05:17.049 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:17.050 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 3580, 2023-09-20T22:05:17.050 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1947143, 2023-09-20T22:05:17.050 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:17.050 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 3580, 2023-09-20T22:05:17.050 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-09-20T22:05:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-09-20T22:05:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 3580, 2023-09-20T22:05:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1947143, 2023-09-20T22:05:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.051 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.052 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.052 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:17.052 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.052 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_bytes": { 2023-09-20T22:05:17.052 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:17.053 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2023-09-20T22:05:17.053 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:17.053 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-09-20T22:05:17.053 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-09-20T22:05:17.053 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:17.054 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:17.054 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 3261, 2023-09-20T22:05:17.054 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 1093481, 2023-09-20T22:05:17.054 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 3261, 2023-09-20T22:05:17.054 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1093481, 2023-09-20T22:05:17.055 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:17.055 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.055 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.055 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.055 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:17.055 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.056 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_ops": { 2023-09-20T22:05:17.056 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:17.056 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 1024, 2023-09-20T22:05:17.056 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:17.056 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-09-20T22:05:17.057 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-09-20T22:05:17.057 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:17.057 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:17.057 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 3261, 2023-09-20T22:05:17.057 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 3261, 2023-09-20T22:05:17.057 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 3261, 2023-09-20T22:05:17.058 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 3261, 2023-09-20T22:05:17.058 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:17.058 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.058 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.058 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.059 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:17.059 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.059 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle": { 2023-09-20T22:05:17.059 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:17.059 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2023-09-20T22:05:17.060 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:17.060 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-09-20T22:05:17.060 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-09-20T22:05:17.060 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:17.060 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:17.061 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-09-20T22:05:17.061 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-09-20T22:05:17.061 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 0, 2023-09-20T22:05:17.061 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 0, 2023-09-20T22:05:17.061 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:17.062 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.062 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.062 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.062 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:17.062 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:17.062 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle-0x560435182a00": { 2023-09-20T22:05:17.063 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:17.063 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2023-09-20T22:05:17.063 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:17.063 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-09-20T22:05:17.063 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-09-20T22:05:17.064 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:17.064 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:17.064 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-09-20T22:05:17.064 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-09-20T22:05:17.064 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 0, 2023-09-20T22:05:17.065 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 0, 2023-09-20T22:05:17.065 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:17.065 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:17.065 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:17.065 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:17.066 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:17.066 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:17.066 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:05:17.558 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2023-09-20T22:05:17.558 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2023-09-20T22:05:17.558 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2023-09-20T22:05:17.559 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2023-09-20T22:05:17.559 DEBUG:tasks.check_counter:Getting stats from g 2023-09-20T22:05:17.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.g perf dump 2023-09-20T22:05:18.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:17 smithi022 ceph-mon[97651]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-20T22:05:18.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:17 smithi123 ceph-mon[134433]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-20T22:05:18.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:17 smithi089 ceph-mon[134090]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-09-20T22:05:18.372 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T22:05:20.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:19 smithi022 ceph-mon[97651]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-09-20T22:05:20.284 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:05:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 0, 2023-09-20T22:05:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2, 2023-09-20T22:05:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 0, 2023-09-20T22:05:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 0, 2023-09-20T22:05:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2023-09-20T22:05:20.286 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 0, 2023-09-20T22:05:20.286 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.003626836, 2023-09-20T22:05:20.286 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.000000000, 2023-09-20T22:05:20.286 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.000488047, 2023-09-20T22:05:20.286 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-20T22:05:20.287 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:20.287 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:20.287 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:20.287 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:20.287 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.288 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:20.288 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:20.288 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:20.288 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:20.288 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.289 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-20T22:05:20.289 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2023-09-20T22:05:20.289 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.289 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:20.290 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 2, 2023-09-20T22:05:20.290 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 231, 2023-09-20T22:05:20.290 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 320, 2023-09-20T22:05:20.290 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 193872, 2023-09-20T22:05:20.290 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 4, 2023-09-20T22:05:20.291 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 2, 2023-09-20T22:05:20.291 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.024170146, 2023-09-20T22:05:20.291 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.019349129, 2023-09-20T22:05:20.291 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.002338302, 2023-09-20T22:05:20.291 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-20T22:05:20.291 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:20.292 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 229, 2023-09-20T22:05:20.292 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.037734396, 2023-09-20T22:05:20.292 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000164779 2023-09-20T22:05:20.292 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.292 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:20.293 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:20.293 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:20.293 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:20.293 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.293 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-20T22:05:20.294 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 193872 2023-09-20T22:05:20.294 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.294 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:20.294 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 307, 2023-09-20T22:05:20.294 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 304, 2023-09-20T22:05:20.295 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 228288, 2023-09-20T22:05:20.295 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 182064, 2023-09-20T22:05:20.295 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 2, 2023-09-20T22:05:20.295 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 1, 2023-09-20T22:05:20.295 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.052194511, 2023-09-20T22:05:20.295 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.023473195, 2023-09-20T22:05:20.296 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.015675009, 2023-09-20T22:05:20.296 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.008967465, 2023-09-20T22:05:20.296 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:20.296 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 304, 2023-09-20T22:05:20.296 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.091711530, 2023-09-20T22:05:20.297 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000301682 2023-09-20T22:05:20.297 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.297 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:20.297 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:20.297 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:20.298 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:20.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.298 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 228288, 2023-09-20T22:05:20.298 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 182064 2023-09-20T22:05:20.298 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.299 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2023-09-20T22:05:20.299 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 0, 2023-09-20T22:05:20.299 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2023-09-20T22:05:20.299 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.299 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2023-09-20T22:05:20.300 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:20.300 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:20.300 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:20.300 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:20.301 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:20.301 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:20.301 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:20.301 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:20.301 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:20.302 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:20.302 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:20.302 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:20.302 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:20.302 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:20.303 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:20.303 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:20.303 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:20.303 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:20.304 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:20.304 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:20.304 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:20.304 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:20.304 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:20.304 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:20.305 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:20.305 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:20.305 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:20.305 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:20.305 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:20.306 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:20.306 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:20.306 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2023-09-20T22:05:20.306 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:20.307 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:20.307 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:20.307 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:20.307 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 148721, 2023-09-20T22:05:20.307 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 30, 2023-09-20T22:05:20.308 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2023-09-20T22:05:20.308 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2023-09-20T22:05:20.308 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2023-09-20T22:05:20.308 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2023-09-20T22:05:20.308 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:20.309 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:20.309 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:20.309 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:20.309 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 0, 2023-09-20T22:05:20.309 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 0, 2023-09-20T22:05:20.310 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:20.310 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:20.310 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:20.310 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2023-09-20T22:05:20.310 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 928, 2023-09-20T22:05:20.311 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 2, 2023-09-20T22:05:20.311 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:20.311 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2023-09-20T22:05:20.311 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:20.311 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2023-09-20T22:05:20.311 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:20.312 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:20.312 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:20.312 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2023-09-20T22:05:20.312 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:20.312 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 305, 2023-09-20T22:05:20.313 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 182460, 2023-09-20T22:05:20.313 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:20.313 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 305, 2023-09-20T22:05:20.313 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-09-20T22:05:20.313 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-09-20T22:05:20.313 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 305, 2023-09-20T22:05:20.314 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 182460, 2023-09-20T22:05:20.314 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:20.314 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:20.314 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:20.314 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:20.315 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:20.315 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:20.315 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:05:20.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:19 smithi123 ceph-mon[134433]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-09-20T22:05:20.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:19 smithi089 ceph-mon[134090]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-09-20T22:05:20.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:20 smithi022 ceph-mon[97651]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-20T22:05:21.241 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-09-20T22:05:21.241 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-09-20T22:05:21.241 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-09-20T22:05:21.242 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-09-20T22:05:21.242 DEBUG:tasks.check_counter:Getting stats from j 2023-09-20T22:05:21.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.j perf dump 2023-09-20T22:05:21.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:20 smithi123 ceph-mon[134433]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-20T22:05:21.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:20 smithi089 ceph-mon[134090]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-09-20T22:05:22.077 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.a/config 2023-09-20T22:05:23.497 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:23 smithi022 ceph-mon[97651]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-20T22:05:23.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:23 smithi123 ceph-mon[134433]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-20T22:05:23.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:23 smithi089 ceph-mon[134090]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 9 op/s 2023-09-20T22:05:24.131 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-09-20T22:05:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:24.131 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 1196548, 2023-09-20T22:05:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 1195889, 2023-09-20T22:05:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 194524871, 2023-09-20T22:05:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 190209569, 2023-09-20T22:05:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 11, 2023-09-20T22:05:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 4, 2023-09-20T22:05:24.132 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 88.608942416, 2023-09-20T22:05:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 38.167608210, 2023-09-20T22:05:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 63.185914704, 2023-09-20T22:05:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.238423832, 2023-09-20T22:05:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:24.133 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1195887, 2023-09-20T22:05:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 137.061310515, 2023-09-20T22:05:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000114610 2023-09-20T22:05:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:24.134 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2390752, 2023-09-20T22:05:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.433881668, 2023-09-20T22:05:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000181 2023-09-20T22:05:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-20T22:05:24.135 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2023-09-20T22:05:24.136 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:24.137 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 1162, 2023-09-20T22:05:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 1471, 2023-09-20T22:05:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 568790, 2023-09-20T22:05:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 796404, 2023-09-20T22:05:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 16, 2023-09-20T22:05:24.138 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 5, 2023-09-20T22:05:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.195353731, 2023-09-20T22:05:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.103972561, 2023-09-20T22:05:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.087404290, 2023-09-20T22:05:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.014493730, 2023-09-20T22:05:24.139 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1471, 2023-09-20T22:05:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.925984847, 2023-09-20T22:05:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000629493 2023-09-20T22:05:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:24.140 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 864, 2023-09-20T22:05:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000328317, 2023-09-20T22:05:24.141 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000379 2023-09-20T22:05:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.142 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 295184, 2023-09-20T22:05:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 192832 2023-09-20T22:05:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 436, 2023-09-20T22:05:24.143 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 654, 2023-09-20T22:05:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 157474, 2023-09-20T22:05:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 758469, 2023-09-20T22:05:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 12, 2023-09-20T22:05:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 5, 2023-09-20T22:05:24.144 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.069961528, 2023-09-20T22:05:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.035134051, 2023-09-20T22:05:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.028338895, 2023-09-20T22:05:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.005848584, 2023-09-20T22:05:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:24.145 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 652, 2023-09-20T22:05:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.393493955, 2023-09-20T22:05:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000603518 2023-09-20T22:05:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:24.146 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 451, 2023-09-20T22:05:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000117504, 2023-09-20T22:05:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000260 2023-09-20T22:05:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-20T22:05:24.147 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 565344 2023-09-20T22:05:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2023-09-20T22:05:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 1, 2023-09-20T22:05:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2023-09-20T22:05:24.148 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-MDSRank": { 2023-09-20T22:05:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2023-09-20T22:05:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2023-09-20T22:05:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 729, 2023-09-20T22:05:24.149 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.220152975, 2023-09-20T22:05:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000301993 2023-09-20T22:05:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-PurgeQueue": { 2023-09-20T22:05:24.150 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2023-09-20T22:05:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2023-09-20T22:05:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 14, 2023-09-20T22:05:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000503110, 2023-09-20T22:05:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000035936 2023-09-20T22:05:24.151 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2023-09-20T22:05:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "request": 91, 2023-09-20T22:05:24.152 INFO:teuthology.orchestra.run.smithi022.stdout: "reply": 47, 2023-09-20T22:05:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "reply_latency": { 2023-09-20T22:05:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 47, 2023-09-20T22:05:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.208700811, 2023-09-20T22:05:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.004440442 2023-09-20T22:05:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.153 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_reply": 0, 2023-09-20T22:05:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "forward": 44, 2023-09-20T22:05:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_complete": 0, 2023-09-20T22:05:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_keys": 0, 2023-09-20T22:05:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_commit": 11, 2023-09-20T22:05:24.154 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_split": 0, 2023-09-20T22:05:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_merge": 0, 2023-09-20T22:05:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes": 340, 2023-09-20T22:05:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_top": 4, 2023-09-20T22:05:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_bottom": 336, 2023-09-20T22:05:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pin_tail": 0, 2023-09-20T22:05:24.155 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pinned": 329, 2023-09-20T22:05:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_expired": 0, 2023-09-20T22:05:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_with_caps": 308, 2023-09-20T22:05:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": 308, 2023-09-20T22:05:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "subtrees": 22, 2023-09-20T22:05:24.156 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse": 343, 2023-09-20T22:05:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_hit": 274, 2023-09-20T22:05:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_forward": 44, 2023-09-20T22:05:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_discover": 22, 2023-09-20T22:05:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_dir_fetch": 0, 2023-09-20T22:05:24.157 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_remote_ino": 0, 2023-09-20T22:05:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_lock": 0, 2023-09-20T22:05:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "load_cent": 246, 2023-09-20T22:05:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "q": 0, 2023-09-20T22:05:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "exported": 0, 2023-09-20T22:05:24.158 INFO:teuthology.orchestra.run.smithi022.stdout: "exported_inodes": 0, 2023-09-20T22:05:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "imported": 8, 2023-09-20T22:05:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "imported_inodes": 312, 2023-09-20T22:05:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_dir_fetch": 0, 2023-09-20T22:05:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_backtrace_fetch": 430, 2023-09-20T22:05:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_peer_discover": 0, 2023-09-20T22:05:24.159 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rfiles": 1033, 2023-09-20T22:05:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rbytes": 1632663607, 2023-09-20T22:05:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rsnaps": 0, 2023-09-20T22:05:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_fetch": 430, 2023-09-20T22:05:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_set_tag": 0, 2023-09-20T22:05:24.160 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_repaired": 0, 2023-09-20T22:05:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_inotable_repaired": 0, 2023-09-20T22:05:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_inodes": 0, 2023-09-20T22:05:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_base_inodes": 0, 2023-09-20T22:05:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dirfrag_rstats": 0, 2023-09-20T22:05:24.161 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_file_inodes": 430, 2023-09-20T22:05:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_inode_file_caps": 0, 2023-09-20T22:05:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_revoke": 0, 2023-09-20T22:05:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_grant": 33, 2023-09-20T22:05:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_trunc": 0, 2023-09-20T22:05:24.162 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-20T22:05:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-20T22:05:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps": 198, 2023-09-20T22:05:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps_dirty": 172, 2023-09-20T22:05:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_cap_release": 5, 2023-09-20T22:05:24.163 INFO:teuthology.orchestra.run.smithi022.stdout: "process_request_cap_release": 8 2023-09-20T22:05:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache": { 2023-09-20T22:05:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update": 29, 2023-09-20T22:05:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update_receipt": 428, 2023-09-20T22:05:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_try_discover": 224, 2023-09-20T22:05:24.164 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_send_discover": 394, 2023-09-20T22:05:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_handle_discover": 29, 2023-09-20T22:05:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays": 0, 2023-09-20T22:05:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_delayed": 0, 2023-09-20T22:05:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_enqueuing": 0, 2023-09-20T22:05:24.165 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_created": 1, 2023-09-20T22:05:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_enqueued": 1, 2023-09-20T22:05:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_reintegrated": 0, 2023-09-20T22:05:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_migrated": 0, 2023-09-20T22:05:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_processing": 0, 2023-09-20T22:05:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_enqueued": 0, 2023-09-20T22:05:24.166 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_prioritized": 0, 2023-09-20T22:05:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_started": 0, 2023-09-20T22:05:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_completed": 0, 2023-09-20T22:05:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_enqueue_scrub": 0, 2023-09-20T22:05:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_exportdir": 0, 2023-09-20T22:05:24.167 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_flush": 0, 2023-09-20T22:05:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragmentdir": 0, 2023-09-20T22:05:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragstats": 0, 2023-09-20T22:05:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_inodestats": 0 2023-09-20T22:05:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.168 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log": { 2023-09-20T22:05:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "evlrg": 0, 2023-09-20T22:05:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "evadd": 730, 2023-09-20T22:05:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "evex": 0, 2023-09-20T22:05:24.169 INFO:teuthology.orchestra.run.smithi022.stdout: "evtrm": 0, 2023-09-20T22:05:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "ev": 730, 2023-09-20T22:05:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "evexg": 0, 2023-09-20T22:05:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "evexd": 0, 2023-09-20T22:05:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "segadd": 2, 2023-09-20T22:05:24.170 INFO:teuthology.orchestra.run.smithi022.stdout: "segex": 0, 2023-09-20T22:05:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "segtrm": 0, 2023-09-20T22:05:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "seg": 2, 2023-09-20T22:05:24.171 INFO:teuthology.orchestra.run.smithi022.stdout: "segmjr": 2, 2023-09-20T22:05:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "segexg": 0, 2023-09-20T22:05:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "segexd": 0, 2023-09-20T22:05:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "expos": 4194304, 2023-09-20T22:05:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "wrpos": 5290079, 2023-09-20T22:05:24.172 INFO:teuthology.orchestra.run.smithi022.stdout: "rdpos": 0, 2023-09-20T22:05:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "jlat": { 2023-09-20T22:05:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 205, 2023-09-20T22:05:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 8.674428321, 2023-09-20T22:05:24.173 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.042314284 2023-09-20T22:05:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: "replayed": 0 2023-09-20T22:05:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.174 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_mem": { 2023-09-20T22:05:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 336, 2023-09-20T22:05:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "ino+": 337, 2023-09-20T22:05:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "ino-": 1, 2023-09-20T22:05:24.175 INFO:teuthology.orchestra.run.smithi022.stdout: "dir": 43, 2023-09-20T22:05:24.176 INFO:teuthology.orchestra.run.smithi022.stdout: "dir+": 220, 2023-09-20T22:05:24.176 INFO:teuthology.orchestra.run.smithi022.stdout: "dir-": 177, 2023-09-20T22:05:24.176 INFO:teuthology.orchestra.run.smithi022.stdout: "dn": 340, 2023-09-20T22:05:24.176 INFO:teuthology.orchestra.run.smithi022.stdout: "dn+": 343, 2023-09-20T22:05:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "dn-": 3, 2023-09-20T22:05:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "cap": 308, 2023-09-20T22:05:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "cap+": 601, 2023-09-20T22:05:24.177 INFO:teuthology.orchestra.run.smithi022.stdout: "cap-": 293, 2023-09-20T22:05:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: "rss": 56248, 2023-09-20T22:05:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: "heap": 198960 2023-09-20T22:05:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.178 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_server": { 2023-09-20T22:05:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_client_request": 93, 2023-09-20T22:05:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_server_request": 0, 2023-09-20T22:05:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_request": 91, 2023-09-20T22:05:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_session": 13, 2023-09-20T22:05:24.179 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_peer_request": 0, 2023-09-20T22:05:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: "req_create_latency": { 2023-09-20T22:05:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getattr_latency": { 2023-09-20T22:05:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 15, 2023-09-20T22:05:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.165260146, 2023-09-20T22:05:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.011017343 2023-09-20T22:05:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.181 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getfilelock_latency": { 2023-09-20T22:05:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.182 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "req_link_latency": { 2023-09-20T22:05:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.183 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookup_latency": { 2023-09-20T22:05:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 11, 2023-09-20T22:05:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.022661794, 2023-09-20T22:05:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.002060163 2023-09-20T22:05:24.184 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookuphash_latency": { 2023-09-20T22:05:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.185 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupino_latency": { 2023-09-20T22:05:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.186 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupname_latency": { 2023-09-20T22:05:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.187 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupparent_latency": { 2023-09-20T22:05:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.188 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupsnap_latency": { 2023-09-20T22:05:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.189 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lssnap_latency": { 2023-09-20T22:05:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.190 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mkdir_latency": { 2023-09-20T22:05:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.191 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mknod_latency": { 2023-09-20T22:05:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.192 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mksnap_latency": { 2023-09-20T22:05:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.193 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "req_open_latency": { 2023-09-20T22:05:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 12, 2023-09-20T22:05:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.011713753, 2023-09-20T22:05:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000976146 2023-09-20T22:05:24.194 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: "req_readdir_latency": { 2023-09-20T22:05:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 8, 2023-09-20T22:05:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.007448401, 2023-09-20T22:05:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000931050 2023-09-20T22:05:24.195 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rename_latency": { 2023-09-20T22:05:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.196 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "req_renamesnap_latency": { 2023-09-20T22:05:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.197 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "req_snapdiff_latency": { 2023-09-20T22:05:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.198 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmdir_latency": { 2023-09-20T22:05:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.199 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmsnap_latency": { 2023-09-20T22:05:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.200 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmxattr_latency": { 2023-09-20T22:05:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.201 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setattr_latency": { 2023-09-20T22:05:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.202 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setdirlayout_latency": { 2023-09-20T22:05:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.203 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setfilelock_latency": { 2023-09-20T22:05:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.204 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setlayout_latency": { 2023-09-20T22:05:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.205 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setxattr_latency": { 2023-09-20T22:05:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.206 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "req_symlink_latency": { 2023-09-20T22:05:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.207 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "req_unlink_latency": { 2023-09-20T22:05:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1, 2023-09-20T22:05:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.001616717, 2023-09-20T22:05:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.001616717 2023-09-20T22:05:24.208 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_revoke_eviction": 0, 2023-09-20T22:05:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_acquisition_throttle": 0, 2023-09-20T22:05:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getvxattr_latency": { 2023-09-20T22:05:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.209 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_sessions": { 2023-09-20T22:05:24.210 INFO:teuthology.orchestra.run.smithi022.stdout: "session_count": 1, 2023-09-20T22:05:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "session_add": 1, 2023-09-20T22:05:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "session_remove": 0, 2023-09-20T22:05:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_open": 1, 2023-09-20T22:05:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_stale": 0, 2023-09-20T22:05:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "total_load": 7, 2023-09-20T22:05:24.211 INFO:teuthology.orchestra.run.smithi022.stdout: "average_load": 7, 2023-09-20T22:05:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "avg_session_uptime": 2599, 2023-09-20T22:05:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "mdthresh_evicted": 0 2023-09-20T22:05:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2023-09-20T22:05:24.212 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:24.213 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:24.214 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:24.215 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:24.216 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:24.217 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:24.218 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2023-09-20T22:05:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:24.219 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 323754, 2023-09-20T22:05:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 447, 2023-09-20T22:05:24.220 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 0, 2023-09-20T22:05:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 0, 2023-09-20T22:05:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2023-09-20T22:05:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2023-09-20T22:05:24.221 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:24.222 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 9088, 2023-09-20T22:05:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 175, 2023-09-20T22:05:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:24.223 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2023-09-20T22:05:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 1187540, 2023-09-20T22:05:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 12445, 2023-09-20T22:05:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:24.224 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2023-09-20T22:05:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2023-09-20T22:05:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter": { 2023-09-20T22:05:24.225 INFO:teuthology.orchestra.run.smithi022.stdout: "op_active": 0, 2023-09-20T22:05:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "op_laggy": 0, 2023-09-20T22:05:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send": 749, 2023-09-20T22:05:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send_bytes": 1138361, 2023-09-20T22:05:24.226 INFO:teuthology.orchestra.run.smithi022.stdout: "op_resend": 27, 2023-09-20T22:05:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "op_reply": 749, 2023-09-20T22:05:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "op_latency": { 2023-09-20T22:05:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 749, 2023-09-20T22:05:24.227 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 16.103810280, 2023-09-20T22:05:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.021500414 2023-09-20T22:05:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "op_inflight": 0, 2023-09-20T22:05:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "oplen_avg": { 2023-09-20T22:05:24.228 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 749, 2023-09-20T22:05:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 789 2023-09-20T22:05:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "op": 749, 2023-09-20T22:05:24.229 INFO:teuthology.orchestra.run.smithi022.stdout: "op_r": 431, 2023-09-20T22:05:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: "op_w": 318, 2023-09-20T22:05:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: "op_rmw": 0, 2023-09-20T22:05:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: "op_pg": 0, 2023-09-20T22:05:24.230 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_stat": 0, 2023-09-20T22:05:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_create": 10, 2023-09-20T22:05:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_read": 1, 2023-09-20T22:05:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_write": 206, 2023-09-20T22:05:24.231 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writefull": 59, 2023-09-20T22:05:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writesame": 0, 2023-09-20T22:05:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_append": 0, 2023-09-20T22:05:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_zero": 0, 2023-09-20T22:05:24.232 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_truncate": 0, 2023-09-20T22:05:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_delete": 14, 2023-09-20T22:05:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_mapext": 0, 2023-09-20T22:05:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_sparse_read": 0, 2023-09-20T22:05:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_clonerange": 0, 2023-09-20T22:05:24.233 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_getxattr": 430, 2023-09-20T22:05:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_setxattr": 20, 2023-09-20T22:05:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_cmpxattr": 0, 2023-09-20T22:05:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_rmxattr": 0, 2023-09-20T22:05:24.234 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_resetxattrs": 0, 2023-09-20T22:05:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_call": 0, 2023-09-20T22:05:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_watch": 0, 2023-09-20T22:05:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_notify": 0, 2023-09-20T22:05:24.235 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_src_cmpxattr": 0, 2023-09-20T22:05:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls": 0, 2023-09-20T22:05:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls_filter": 0, 2023-09-20T22:05:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_other": 0, 2023-09-20T22:05:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_active": 0, 2023-09-20T22:05:24.236 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_send": 0, 2023-09-20T22:05:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_resend": 0, 2023-09-20T22:05:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_ping": 0, 2023-09-20T22:05:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_active": 0, 2023-09-20T22:05:24.237 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_send": 0, 2023-09-20T22:05:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_resend": 0, 2023-09-20T22:05:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_active": 0, 2023-09-20T22:05:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_send": 0, 2023-09-20T22:05:24.238 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_resend": 0, 2023-09-20T22:05:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_active": 0, 2023-09-20T22:05:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_send": 0, 2023-09-20T22:05:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_resend": 0, 2023-09-20T22:05:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "command_active": 0, 2023-09-20T22:05:24.239 INFO:teuthology.orchestra.run.smithi022.stdout: "command_send": 0, 2023-09-20T22:05:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "command_resend": 0, 2023-09-20T22:05:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "map_epoch": 93, 2023-09-20T22:05:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "map_full": 0, 2023-09-20T22:05:24.240 INFO:teuthology.orchestra.run.smithi022.stdout: "map_inc": 3, 2023-09-20T22:05:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_sessions": 12, 2023-09-20T22:05:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_open": 12, 2023-09-20T22:05:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_close": 0, 2023-09-20T22:05:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_laggy": 0, 2023-09-20T22:05:24.241 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_wr": 40, 2023-09-20T22:05:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_rd": 0, 2023-09-20T22:05:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_del": 9 2023-09-20T22:05:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.242 INFO:teuthology.orchestra.run.smithi022.stdout: "oft": { 2023-09-20T22:05:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_objs": 1, 2023-09-20T22:05:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_kv_pairs": 0, 2023-09-20T22:05:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_updates": 0, 2023-09-20T22:05:24.243 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_removes": 0 2023-09-20T22:05:24.244 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.244 INFO:teuthology.orchestra.run.smithi022.stdout: "osd-slow-ops": { 2023-09-20T22:05:24.244 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_ops_count": 0 2023-09-20T22:05:24.244 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.245 INFO:teuthology.orchestra.run.smithi022.stdout: "purge_queue": { 2023-09-20T22:05:24.245 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops": 0, 2023-09-20T22:05:24.245 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops_high_water": 2, 2023-09-20T22:05:24.246 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing": 0, 2023-09-20T22:05:24.246 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_high_water": 1, 2023-09-20T22:05:24.246 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executed": 1, 2023-09-20T22:05:24.246 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_item_in_journal": 0 2023-09-20T22:05:24.246 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.247 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:24.247 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:24.247 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2023-09-20T22:05:24.247 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:24.247 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 1198144, 2023-09-20T22:05:24.248 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 87402443, 2023-09-20T22:05:24.248 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:24.248 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 1198144, 2023-09-20T22:05:24.248 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-09-20T22:05:24.248 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-09-20T22:05:24.249 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 1198144, 2023-09-20T22:05:24.249 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 87402443, 2023-09-20T22:05:24.249 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:24.249 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.249 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.250 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.250 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:24.250 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.250 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_bytes": { 2023-09-20T22:05:24.250 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:24.251 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2023-09-20T22:05:24.251 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:24.251 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-09-20T22:05:24.251 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-09-20T22:05:24.251 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:24.251 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:24.251 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 749, 2023-09-20T22:05:24.252 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 1138470, 2023-09-20T22:05:24.252 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 749, 2023-09-20T22:05:24.252 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1138470, 2023-09-20T22:05:24.252 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:24.252 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.252 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.253 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.253 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:24.253 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.253 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_ops": { 2023-09-20T22:05:24.253 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:24.253 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 1024, 2023-09-20T22:05:24.254 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:24.254 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-09-20T22:05:24.254 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-09-20T22:05:24.254 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:24.254 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:24.255 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 749, 2023-09-20T22:05:24.255 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 749, 2023-09-20T22:05:24.255 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 749, 2023-09-20T22:05:24.255 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 749, 2023-09-20T22:05:24.255 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:24.255 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.256 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.256 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.256 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:24.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.256 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle": { 2023-09-20T22:05:24.256 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:24.257 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2023-09-20T22:05:24.257 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:24.257 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 1, 2023-09-20T22:05:24.257 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 109, 2023-09-20T22:05:24.258 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:24.258 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 1, 2023-09-20T22:05:24.258 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-09-20T22:05:24.258 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-09-20T22:05:24.258 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 1, 2023-09-20T22:05:24.259 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 109, 2023-09-20T22:05:24.259 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:24.259 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.260 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.260 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.260 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:24.261 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-09-20T22:05:24.261 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle-0x55fed16686e0": { 2023-09-20T22:05:24.261 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-09-20T22:05:24.261 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2023-09-20T22:05:24.262 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-09-20T22:05:24.262 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 730, 2023-09-20T22:05:24.262 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 1095775, 2023-09-20T22:05:24.262 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:24.262 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 730, 2023-09-20T22:05:24.262 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-09-20T22:05:24.263 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-09-20T22:05:24.263 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 205, 2023-09-20T22:05:24.263 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 1095775, 2023-09-20T22:05:24.263 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-09-20T22:05:24.263 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-09-20T22:05:24.264 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-09-20T22:05:24.264 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-09-20T22:05:24.264 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:24.264 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-09-20T22:05:24.264 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-09-20T22:05:24.746 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2023-09-20T22:05:24.747 INFO:tasks.check_counter:Daemon mds.j mds.imported=8 2023-09-20T22:05:24.747 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update=29 2023-09-20T22:05:24.748 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update_receipt=428 2023-09-20T22:05:24.748 DEBUG:tasks.check_counter:Getting stats from b 2023-09-20T22:05:24.749 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.b perf dump 2023-09-20T22:05:25.675 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T22:05:25.979 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:25 smithi089 ceph-mon[134090]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-20T22:05:25.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:25 smithi022 ceph-mon[97651]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-20T22:05:26.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:25 smithi123 ceph-mon[134433]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2023-09-20T22:05:27.437 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-09-20T22:05:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1222715, 2023-09-20T22:05:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 1222519, 2023-09-20T22:05:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 194320982, 2023-09-20T22:05:27.438 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 201629667, 2023-09-20T22:05:27.439 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 9, 2023-09-20T22:05:27.439 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2023-09-20T22:05:27.439 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 37.641530292, 2023-09-20T22:05:27.439 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 13.916938804, 2023-09-20T22:05:27.439 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 27.352751842, 2023-09-20T22:05:27.440 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.112697489, 2023-09-20T22:05:27.440 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:27.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1222522, 2023-09-20T22:05:27.440 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 68.753415429, 2023-09-20T22:05:27.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000056239 2023-09-20T22:05:27.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.441 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:27.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2440256, 2023-09-20T22:05:27.441 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.236295028, 2023-09-20T22:05:27.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000096 2023-09-20T22:05:27.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.442 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 322736, 2023-09-20T22:05:27.442 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 192768 2023-09-20T22:05:27.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.442 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:27.443 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 664290, 2023-09-20T22:05:27.443 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 739047, 2023-09-20T22:05:27.443 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 172820057, 2023-09-20T22:05:27.443 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 340518238, 2023-09-20T22:05:27.443 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 21, 2023-09-20T22:05:27.444 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 8, 2023-09-20T22:05:27.444 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 19.675672377, 2023-09-20T22:05:27.444 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 8.830166063, 2023-09-20T22:05:27.444 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 21.070313639, 2023-09-20T22:05:27.444 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.006254638, 2023-09-20T22:05:27.445 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:27.445 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 739041, 2023-09-20T22:05:27.445 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 164.833744038, 2023-09-20T22:05:27.445 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000223037 2023-09-20T22:05:27.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.445 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:27.446 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 251867, 2023-09-20T22:05:27.446 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.083918192, 2023-09-20T22:05:27.446 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000333 2023-09-20T22:05:27.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.446 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-20T22:05:27.447 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 580880 2023-09-20T22:05:27.447 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.447 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:27.447 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 361962, 2023-09-20T22:05:27.447 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 363122, 2023-09-20T22:05:27.448 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 98754955, 2023-09-20T22:05:27.448 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 82488038, 2023-09-20T22:05:27.448 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 86, 2023-09-20T22:05:27.448 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 7, 2023-09-20T22:05:27.448 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 15.237447982, 2023-09-20T22:05:27.449 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 5.835018913, 2023-09-20T22:05:27.449 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 11.172733883, 2023-09-20T22:05:27.449 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.161527895, 2023-09-20T22:05:27.449 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:27.449 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 363121, 2023-09-20T22:05:27.449 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 56.726878415, 2023-09-20T22:05:27.450 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000156220 2023-09-20T22:05:27.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.450 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:27.450 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 718476, 2023-09-20T22:05:27.450 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.091133936, 2023-09-20T22:05:27.451 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000126 2023-09-20T22:05:27.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.451 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-09-20T22:05:27.451 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 448 2023-09-20T22:05:27.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.452 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2023-09-20T22:05:27.452 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2023-09-20T22:05:27.452 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2023-09-20T22:05:27.452 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.452 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2023-09-20T22:05:27.453 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2023-09-20T22:05:27.453 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2023-09-20T22:05:27.453 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2133, 2023-09-20T22:05:27.453 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 1.754959295, 2023-09-20T22:05:27.453 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000822765 2023-09-20T22:05:27.454 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:27.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.454 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2023-09-20T22:05:27.454 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2023-09-20T22:05:27.454 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2023-09-20T22:05:27.454 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 130, 2023-09-20T22:05:27.455 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.021140870, 2023-09-20T22:05:27.455 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000162622 2023-09-20T22:05:27.455 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:27.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.455 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2023-09-20T22:05:27.456 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 591098, 2023-09-20T22:05:27.456 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 590827, 2023-09-20T22:05:27.456 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2023-09-20T22:05:27.456 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 590827, 2023-09-20T22:05:27.456 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 1814.859147993, 2023-09-20T22:05:27.456 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.003071726 2023-09-20T22:05:27.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.457 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2023-09-20T22:05:27.457 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 271, 2023-09-20T22:05:27.457 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 0, 2023-09-20T22:05:27.457 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2023-09-20T22:05:27.458 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 39, 2023-09-20T22:05:27.458 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 3, 2023-09-20T22:05:27.458 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 0, 2023-09-20T22:05:27.458 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 253, 2023-09-20T22:05:27.458 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 9, 2023-09-20T22:05:27.459 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 201, 2023-09-20T22:05:27.459 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 43, 2023-09-20T22:05:27.459 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 240, 2023-09-20T22:05:27.459 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 904, 2023-09-20T22:05:27.459 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 189, 2023-09-20T22:05:27.459 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 189, 2023-09-20T22:05:27.460 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 30, 2023-09-20T22:05:27.460 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 837364, 2023-09-20T22:05:27.460 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 836281, 2023-09-20T22:05:27.460 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 246, 2023-09-20T22:05:27.460 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 19, 2023-09-20T22:05:27.461 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 0, 2023-09-20T22:05:27.461 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2023-09-20T22:05:27.461 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 187, 2023-09-20T22:05:27.461 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 37, 2023-09-20T22:05:27.461 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2023-09-20T22:05:27.462 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 28, 2023-09-20T22:05:27.462 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 867, 2023-09-20T22:05:27.462 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 1, 2023-09-20T22:05:27.462 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 6, 2023-09-20T22:05:27.462 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2023-09-20T22:05:27.463 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 695, 2023-09-20T22:05:27.463 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2023-09-20T22:05:27.463 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 1033, 2023-09-20T22:05:27.463 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 1632664011, 2023-09-20T22:05:27.463 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 0, 2023-09-20T22:05:27.464 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 695, 2023-09-20T22:05:27.464 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2023-09-20T22:05:27.464 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2023-09-20T22:05:27.464 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2023-09-20T22:05:27.464 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 50, 2023-09-20T22:05:27.465 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 3, 2023-09-20T22:05:27.465 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 47, 2023-09-20T22:05:27.465 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 645, 2023-09-20T22:05:27.465 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 5, 2023-09-20T22:05:27.465 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 71559, 2023-09-20T22:05:27.465 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 72792, 2023-09-20T22:05:27.466 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 0, 2023-09-20T22:05:27.466 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-20T22:05:27.466 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-20T22:05:27.466 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 72668, 2023-09-20T22:05:27.466 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 1064, 2023-09-20T22:05:27.467 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 39, 2023-09-20T22:05:27.467 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 215 2023-09-20T22:05:27.467 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.467 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2023-09-20T22:05:27.467 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 144, 2023-09-20T22:05:27.468 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 0, 2023-09-20T22:05:27.468 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 0, 2023-09-20T22:05:27.468 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 19, 2023-09-20T22:05:27.468 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 765, 2023-09-20T22:05:27.468 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 0, 2023-09-20T22:05:27.469 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2023-09-20T22:05:27.469 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2023-09-20T22:05:27.469 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 59, 2023-09-20T22:05:27.469 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 31, 2023-09-20T22:05:27.469 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 112, 2023-09-20T22:05:27.470 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2023-09-20T22:05:27.470 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2023-09-20T22:05:27.470 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2023-09-20T22:05:27.470 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2023-09-20T22:05:27.470 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2023-09-20T22:05:27.471 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2023-09-20T22:05:27.471 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 3, 2023-09-20T22:05:27.471 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 28, 2023-09-20T22:05:27.471 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2023-09-20T22:05:27.471 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 3, 2023-09-20T22:05:27.472 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2023-09-20T22:05:27.472 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0 2023-09-20T22:05:27.472 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.472 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2023-09-20T22:05:27.472 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 0, 2023-09-20T22:05:27.473 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 2871, 2023-09-20T22:05:27.473 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 0, 2023-09-20T22:05:27.473 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 0, 2023-09-20T22:05:27.473 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 2871, 2023-09-20T22:05:27.473 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2023-09-20T22:05:27.474 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 0, 2023-09-20T22:05:27.474 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 4, 2023-09-20T22:05:27.474 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 0, 2023-09-20T22:05:27.474 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 0, 2023-09-20T22:05:27.474 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 4, 2023-09-20T22:05:27.475 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 2, 2023-09-20T22:05:27.475 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2023-09-20T22:05:27.475 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 0, 2023-09-20T22:05:27.475 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 4194304, 2023-09-20T22:05:27.475 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 10812997, 2023-09-20T22:05:27.476 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 0, 2023-09-20T22:05:27.476 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2023-09-20T22:05:27.476 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1476, 2023-09-20T22:05:27.476 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 10.486517983, 2023-09-20T22:05:27.476 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.007104686 2023-09-20T22:05:27.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.477 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 0 2023-09-20T22:05:27.477 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.477 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2023-09-20T22:05:27.477 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 215, 2023-09-20T22:05:27.478 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 1082, 2023-09-20T22:05:27.478 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 867, 2023-09-20T22:05:27.478 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 60, 2023-09-20T22:05:27.478 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 68, 2023-09-20T22:05:27.478 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 8, 2023-09-20T22:05:27.479 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 253, 2023-09-20T22:05:27.479 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 1497, 2023-09-20T22:05:27.479 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 1244, 2023-09-20T22:05:27.479 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 189, 2023-09-20T22:05:27.479 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 1232, 2023-09-20T22:05:27.479 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 1043, 2023-09-20T22:05:27.480 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 63972, 2023-09-20T22:05:27.480 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 207152 2023-09-20T22:05:27.480 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.480 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2023-09-20T22:05:27.480 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 805753, 2023-09-20T22:05:27.481 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 95, 2023-09-20T22:05:27.481 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 591098, 2023-09-20T22:05:27.481 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 274, 2023-09-20T22:05:27.481 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 71538, 2023-09-20T22:05:27.481 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2023-09-20T22:05:27.482 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1034, 2023-09-20T22:05:27.482 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 5.934403803, 2023-09-20T22:05:27.482 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.005739268 2023-09-20T22:05:27.482 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2023-09-20T22:05:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 588877, 2023-09-20T22:05:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 1801.866313946, 2023-09-20T22:05:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.003059834 2023-09-20T22:05:27.483 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.484 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2023-09-20T22:05:27.484 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.484 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.484 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.484 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.485 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2023-09-20T22:05:27.485 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 28, 2023-09-20T22:05:27.485 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.046753873, 2023-09-20T22:05:27.485 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.001669781 2023-09-20T22:05:27.485 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.485 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2023-09-20T22:05:27.486 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 96, 2023-09-20T22:05:27.486 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 5.074297052, 2023-09-20T22:05:27.486 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.052857260 2023-09-20T22:05:27.486 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.486 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2023-09-20T22:05:27.487 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.487 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.487 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.487 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.487 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2023-09-20T22:05:27.488 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.488 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.488 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.488 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.488 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2023-09-20T22:05:27.489 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.489 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.489 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.489 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.489 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2023-09-20T22:05:27.490 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.490 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.490 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.490 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.490 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2023-09-20T22:05:27.490 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.491 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.491 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.491 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.491 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2023-09-20T22:05:27.491 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.492 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.492 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.492 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.492 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2023-09-20T22:05:27.492 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 29, 2023-09-20T22:05:27.493 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.650335530, 2023-09-20T22:05:27.493 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.022425363 2023-09-20T22:05:27.493 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.493 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2023-09-20T22:05:27.493 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.494 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.494 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.494 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.494 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2023-09-20T22:05:27.494 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.495 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.495 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.495 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.495 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2023-09-20T22:05:27.495 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 528, 2023-09-20T22:05:27.496 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.142412099, 2023-09-20T22:05:27.496 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000269719 2023-09-20T22:05:27.496 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.496 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2023-09-20T22:05:27.496 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 50, 2023-09-20T22:05:27.497 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.106241908, 2023-09-20T22:05:27.497 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002124838 2023-09-20T22:05:27.497 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.497 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2023-09-20T22:05:27.498 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 136, 2023-09-20T22:05:27.498 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.966058375, 2023-09-20T22:05:27.498 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.007103370 2023-09-20T22:05:27.498 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.498 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2023-09-20T22:05:27.499 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.499 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.499 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.499 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.499 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2023-09-20T22:05:27.500 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.500 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.500 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.500 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.500 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2023-09-20T22:05:27.501 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.501 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.501 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.501 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.501 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2023-09-20T22:05:27.502 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.502 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.502 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.502 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.502 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2023-09-20T22:05:27.502 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.503 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.503 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.503 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.503 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2023-09-20T22:05:27.503 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1, 2023-09-20T22:05:27.504 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000671411, 2023-09-20T22:05:27.504 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000671411 2023-09-20T22:05:27.504 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.504 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2023-09-20T22:05:27.504 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.505 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.505 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.505 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.505 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2023-09-20T22:05:27.505 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.505 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.506 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.506 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.506 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2023-09-20T22:05:27.506 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.506 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.507 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.507 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.507 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2023-09-20T22:05:27.507 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1, 2023-09-20T22:05:27.507 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.002552564, 2023-09-20T22:05:27.508 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002552564 2023-09-20T22:05:27.508 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.508 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2023-09-20T22:05:27.508 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.508 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.508 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.509 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.509 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2023-09-20T22:05:27.509 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 47, 2023-09-20T22:05:27.509 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.069107432, 2023-09-20T22:05:27.509 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.001470370 2023-09-20T22:05:27.510 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.510 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2023-09-20T22:05:27.510 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2023-09-20T22:05:27.510 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2023-09-20T22:05:27.510 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.511 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.511 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.511 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:27.511 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.511 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2023-09-20T22:05:27.511 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 1, 2023-09-20T22:05:27.512 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 2, 2023-09-20T22:05:27.512 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 1, 2023-09-20T22:05:27.512 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 1, 2023-09-20T22:05:27.512 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2023-09-20T22:05:27.512 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 34483, 2023-09-20T22:05:27.513 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 34483, 2023-09-20T22:05:27.513 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 987, 2023-09-20T22:05:27.513 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2023-09-20T22:05:27.513 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.513 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2023-09-20T22:05:27.513 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:27.514 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:27.514 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:27.514 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:27.514 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:27.515 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:27.515 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:27.515 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:27.515 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:27.515 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:27.515 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:27.516 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:27.516 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:27.516 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:27.516 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:27.516 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:27.517 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:27.517 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:27.517 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:27.517 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:27.517 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:27.518 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:27.518 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:27.518 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:27.518 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:27.518 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:27.519 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:27.519 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:27.519 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:27.519 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:27.519 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:27.519 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2023-09-20T22:05:27.520 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:27.520 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:27.520 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:27.520 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:27.520 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 337826, 2023-09-20T22:05:27.521 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 325, 2023-09-20T22:05:27.521 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 264, 2023-09-20T22:05:27.521 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 3, 2023-09-20T22:05:27.521 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2023-09-20T22:05:27.521 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2023-09-20T22:05:27.522 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:27.522 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:27.522 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:27.522 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:27.522 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9088, 2023-09-20T22:05:27.523 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 175, 2023-09-20T22:05:27.523 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:27.523 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:27.523 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:27.523 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2023-09-20T22:05:27.524 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 842220, 2023-09-20T22:05:27.524 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 9940, 2023-09-20T22:05:27.524 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:27.524 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2023-09-20T22:05:27.524 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:27.525 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2023-09-20T22:05:27.525 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.525 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2023-09-20T22:05:27.525 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2023-09-20T22:05:27.525 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2023-09-20T22:05:27.526 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 2452, 2023-09-20T22:05:27.526 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 7034876, 2023-09-20T22:05:27.526 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 31, 2023-09-20T22:05:27.526 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 2452, 2023-09-20T22:05:27.526 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2023-09-20T22:05:27.527 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2452, 2023-09-20T22:05:27.527 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 9.945777569, 2023-09-20T22:05:27.527 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.004056189 2023-09-20T22:05:27.527 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.527 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2023-09-20T22:05:27.528 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2023-09-20T22:05:27.528 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2452, 2023-09-20T22:05:27.528 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 2536 2023-09-20T22:05:27.528 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.528 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 2452, 2023-09-20T22:05:27.529 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 714, 2023-09-20T22:05:27.529 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 1738, 2023-09-20T22:05:27.529 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2023-09-20T22:05:27.529 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2023-09-20T22:05:27.529 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 0, 2023-09-20T22:05:27.529 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 11, 2023-09-20T22:05:27.530 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 16, 2023-09-20T22:05:27.530 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 1489, 2023-09-20T22:05:27.530 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 74, 2023-09-20T22:05:27.530 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2023-09-20T22:05:27.530 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2023-09-20T22:05:27.531 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 0, 2023-09-20T22:05:27.531 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 3, 2023-09-20T22:05:27.531 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 90, 2023-09-20T22:05:27.531 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2023-09-20T22:05:27.531 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2023-09-20T22:05:27.532 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2023-09-20T22:05:27.532 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 698, 2023-09-20T22:05:27.532 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 22, 2023-09-20T22:05:27.532 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2023-09-20T22:05:27.533 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2023-09-20T22:05:27.533 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2023-09-20T22:05:27.533 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2023-09-20T22:05:27.533 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2023-09-20T22:05:27.533 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2023-09-20T22:05:27.533 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2023-09-20T22:05:27.534 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2023-09-20T22:05:27.534 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2023-09-20T22:05:27.534 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 0, 2023-09-20T22:05:27.534 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2023-09-20T22:05:27.534 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2023-09-20T22:05:27.535 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2023-09-20T22:05:27.535 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2023-09-20T22:05:27.535 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2023-09-20T22:05:27.535 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2023-09-20T22:05:27.535 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2023-09-20T22:05:27.536 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2023-09-20T22:05:27.536 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2023-09-20T22:05:27.536 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2023-09-20T22:05:27.536 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2023-09-20T22:05:27.536 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2023-09-20T22:05:27.536 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2023-09-20T22:05:27.537 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2023-09-20T22:05:27.537 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2023-09-20T22:05:27.537 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2023-09-20T22:05:27.537 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 93, 2023-09-20T22:05:27.537 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2023-09-20T22:05:27.538 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 9, 2023-09-20T22:05:27.538 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 12, 2023-09-20T22:05:27.538 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 12, 2023-09-20T22:05:27.538 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 0, 2023-09-20T22:05:27.538 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2023-09-20T22:05:27.539 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 112, 2023-09-20T22:05:27.539 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 0, 2023-09-20T22:05:27.539 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 21 2023-09-20T22:05:27.539 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.539 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2023-09-20T22:05:27.539 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 1, 2023-09-20T22:05:27.540 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2023-09-20T22:05:27.540 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2023-09-20T22:05:27.540 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2023-09-20T22:05:27.540 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.540 INFO:teuthology.orchestra.run.smithi089.stdout: "osd-slow-ops": { 2023-09-20T22:05:27.541 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 0 2023-09-20T22:05:27.541 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.541 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2023-09-20T22:05:27.541 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2023-09-20T22:05:27.541 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 23, 2023-09-20T22:05:27.542 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2023-09-20T22:05:27.542 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 5, 2023-09-20T22:05:27.542 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 31, 2023-09-20T22:05:27.542 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2023-09-20T22:05:27.542 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.543 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:27.543 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:27.543 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-09-20T22:05:27.543 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:27.543 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 2248965, 2023-09-20T22:05:27.543 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 274091314, 2023-09-20T22:05:27.544 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:27.544 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 2248965, 2023-09-20T22:05:27.544 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:27.544 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:27.544 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 2248965, 2023-09-20T22:05:27.545 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 274091314, 2023-09-20T22:05:27.545 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:27.545 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.545 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.545 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.546 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:27.546 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.546 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2023-09-20T22:05:27.546 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:27.546 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-09-20T22:05:27.547 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:27.547 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:27.547 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:27.547 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:27.547 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:27.548 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 2452, 2023-09-20T22:05:27.548 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 7038255, 2023-09-20T22:05:27.548 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 2452, 2023-09-20T22:05:27.548 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 7038255, 2023-09-20T22:05:27.548 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:27.549 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.549 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.549 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.549 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:27.549 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.549 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2023-09-20T22:05:27.550 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:27.550 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2023-09-20T22:05:27.550 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:27.550 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:27.550 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:27.551 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:27.551 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:27.551 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 2452, 2023-09-20T22:05:27.551 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 2452, 2023-09-20T22:05:27.551 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 2452, 2023-09-20T22:05:27.552 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 2452, 2023-09-20T22:05:27.552 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:27.552 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.552 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.552 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.553 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:27.553 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.553 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2023-09-20T22:05:27.553 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:27.553 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2023-09-20T22:05:27.553 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:27.554 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 31, 2023-09-20T22:05:27.554 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 3379, 2023-09-20T22:05:27.554 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:27.554 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 31, 2023-09-20T22:05:27.554 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:27.555 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:27.555 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 13, 2023-09-20T22:05:27.555 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 3379, 2023-09-20T22:05:27.555 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:27.555 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x5628b2b266e0": { 2023-09-20T22:05:27.556 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:27.557 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2023-09-20T22:05:27.557 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:27.557 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 2871, 2023-09-20T22:05:27.557 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 6618693, 2023-09-20T22:05:27.557 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:27.558 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 2871, 2023-09-20T22:05:27.558 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:27.558 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:27.558 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 1476, 2023-09-20T22:05:27.558 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 6618693, 2023-09-20T22:05:27.559 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:27.559 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:27.559 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:27.559 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:27.559 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:27.560 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:27.560 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-09-20T22:05:27.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:27 smithi022 ceph-mon[97651]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T22:05:28.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:27 smithi123 ceph-mon[134433]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T22:05:28.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:27 smithi089 ceph-mon[134090]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T22:05:29.936 INFO:tasks.check_counter:Daemon mds.b mds.exported=28 2023-09-20T22:05:29.937 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2023-09-20T22:05:29.937 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=144 2023-09-20T22:05:29.937 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2023-09-20T22:05:29.937 DEBUG:tasks.check_counter:Getting stats from e 2023-09-20T22:05:29.938 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.e perf dump 2023-09-20T22:05:29.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:29 smithi022 ceph-mon[97651]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T22:05:30.053 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:29 smithi089 ceph-mon[134090]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T22:05:30.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:29 smithi123 ceph-mon[134433]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T22:05:31.247 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T22:05:31.741 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:31 smithi089 ceph-mon[134090]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T22:05:31.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:31 smithi022 ceph-mon[97651]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T22:05:32.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:31 smithi123 ceph-mon[134433]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T22:05:32.849 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-09-20T22:05:32.849 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:32.849 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1303, 2023-09-20T22:05:32.849 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 1305, 2023-09-20T22:05:32.850 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 3056928, 2023-09-20T22:05:32.850 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 405297, 2023-09-20T22:05:32.850 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 7, 2023-09-20T22:05:32.850 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2023-09-20T22:05:32.850 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.111842242, 2023-09-20T22:05:32.851 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.034192261, 2023-09-20T22:05:32.851 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.029232577, 2023-09-20T22:05:32.851 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.038873332, 2023-09-20T22:05:32.851 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:32.851 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1303, 2023-09-20T22:05:32.852 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.419970435, 2023-09-20T22:05:32.852 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000322310 2023-09-20T22:05:32.852 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.852 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:32.852 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.853 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.853 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.853 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.853 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-20T22:05:32.853 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2023-09-20T22:05:32.854 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.854 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:32.854 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1315, 2023-09-20T22:05:32.854 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 1302, 2023-09-20T22:05:32.855 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 546448, 2023-09-20T22:05:32.856 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 493046, 2023-09-20T22:05:32.856 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 5, 2023-09-20T22:05:32.856 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2023-09-20T22:05:32.856 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.125651005, 2023-09-20T22:05:32.856 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.032961009, 2023-09-20T22:05:32.857 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.033823352, 2023-09-20T22:05:32.857 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.049438711, 2023-09-20T22:05:32.857 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:32.857 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1302, 2023-09-20T22:05:32.857 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.488157589, 2023-09-20T22:05:32.857 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000374929 2023-09-20T22:05:32.858 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.858 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:32.858 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.858 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.858 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.859 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 304608, 2023-09-20T22:05:32.859 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 187328 2023-09-20T22:05:32.859 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.859 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:32.860 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1083, 2023-09-20T22:05:32.860 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 1316, 2023-09-20T22:05:32.860 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 363712, 2023-09-20T22:05:32.860 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 813174, 2023-09-20T22:05:32.860 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2023-09-20T22:05:32.861 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2023-09-20T22:05:32.861 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.114621678, 2023-09-20T22:05:32.861 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.034622954, 2023-09-20T22:05:32.861 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.027380929, 2023-09-20T22:05:32.861 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.041563951, 2023-09-20T22:05:32.862 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:32.862 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1314, 2023-09-20T22:05:32.862 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.573873344, 2023-09-20T22:05:32.862 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000436737 2023-09-20T22:05:32.862 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.862 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:32.863 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.863 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.863 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.863 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.863 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2023-09-20T22:05:32.864 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 476672 2023-09-20T22:05:32.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.864 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2023-09-20T22:05:32.864 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2023-09-20T22:05:32.864 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2023-09-20T22:05:32.864 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.865 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2023-09-20T22:05:32.865 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2023-09-20T22:05:32.865 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2023-09-20T22:05:32.865 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 5716, 2023-09-20T22:05:32.865 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.152286897, 2023-09-20T22:05:32.866 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000026642 2023-09-20T22:05:32.866 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:32.866 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.866 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2023-09-20T22:05:32.866 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2023-09-20T22:05:32.866 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2023-09-20T22:05:32.867 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.867 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.867 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.867 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:32.867 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.868 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2023-09-20T22:05:32.868 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 0, 2023-09-20T22:05:32.868 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 0, 2023-09-20T22:05:32.868 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2023-09-20T22:05:32.868 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.868 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.869 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.869 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.869 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2023-09-20T22:05:32.869 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 0, 2023-09-20T22:05:32.869 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 0, 2023-09-20T22:05:32.870 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2023-09-20T22:05:32.870 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 0, 2023-09-20T22:05:32.870 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 0, 2023-09-20T22:05:32.870 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 0, 2023-09-20T22:05:32.870 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 476, 2023-09-20T22:05:32.870 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 0, 2023-09-20T22:05:32.871 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 0, 2023-09-20T22:05:32.871 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 476, 2023-09-20T22:05:32.871 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 476, 2023-09-20T22:05:32.871 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 0, 2023-09-20T22:05:32.871 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 0, 2023-09-20T22:05:32.872 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 0, 2023-09-20T22:05:32.872 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 13, 2023-09-20T22:05:32.872 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 0, 2023-09-20T22:05:32.872 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 0, 2023-09-20T22:05:32.872 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 0, 2023-09-20T22:05:32.872 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 0, 2023-09-20T22:05:32.873 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 0, 2023-09-20T22:05:32.873 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2023-09-20T22:05:32.873 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 0, 2023-09-20T22:05:32.873 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 0, 2023-09-20T22:05:32.873 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2023-09-20T22:05:32.874 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 0, 2023-09-20T22:05:32.874 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 0, 2023-09-20T22:05:32.874 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 0, 2023-09-20T22:05:32.874 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 0, 2023-09-20T22:05:32.874 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2023-09-20T22:05:32.874 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 0, 2023-09-20T22:05:32.875 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2023-09-20T22:05:32.875 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 0, 2023-09-20T22:05:32.875 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 0, 2023-09-20T22:05:32.875 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 0, 2023-09-20T22:05:32.875 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 0, 2023-09-20T22:05:32.876 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2023-09-20T22:05:32.876 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2023-09-20T22:05:32.876 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2023-09-20T22:05:32.876 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 0, 2023-09-20T22:05:32.876 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 0, 2023-09-20T22:05:32.876 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 0, 2023-09-20T22:05:32.877 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 0, 2023-09-20T22:05:32.877 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 0, 2023-09-20T22:05:32.877 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 0, 2023-09-20T22:05:32.877 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 0, 2023-09-20T22:05:32.877 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 0, 2023-09-20T22:05:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-20T22:05:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-20T22:05:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 0, 2023-09-20T22:05:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 0, 2023-09-20T22:05:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 0, 2023-09-20T22:05:32.878 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 0 2023-09-20T22:05:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2023-09-20T22:05:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 0, 2023-09-20T22:05:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 0, 2023-09-20T22:05:32.879 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 0, 2023-09-20T22:05:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 0, 2023-09-20T22:05:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 0, 2023-09-20T22:05:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 0, 2023-09-20T22:05:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2023-09-20T22:05:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2023-09-20T22:05:32.880 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 0, 2023-09-20T22:05:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 0, 2023-09-20T22:05:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 0, 2023-09-20T22:05:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2023-09-20T22:05:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2023-09-20T22:05:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2023-09-20T22:05:32.881 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2023-09-20T22:05:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2023-09-20T22:05:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2023-09-20T22:05:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 0, 2023-09-20T22:05:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 0, 2023-09-20T22:05:32.882 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2023-09-20T22:05:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 0, 2023-09-20T22:05:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2023-09-20T22:05:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0 2023-09-20T22:05:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2023-09-20T22:05:32.883 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 0, 2023-09-20T22:05:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 0, 2023-09-20T22:05:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 0, 2023-09-20T22:05:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 0, 2023-09-20T22:05:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 1768, 2023-09-20T22:05:32.884 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2023-09-20T22:05:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 0, 2023-09-20T22:05:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 0, 2023-09-20T22:05:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 0, 2023-09-20T22:05:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 0, 2023-09-20T22:05:32.885 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 3, 2023-09-20T22:05:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 2, 2023-09-20T22:05:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2023-09-20T22:05:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 0, 2023-09-20T22:05:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 4194304, 2023-09-20T22:05:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 6912695, 2023-09-20T22:05:32.886 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 6912121, 2023-09-20T22:05:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2023-09-20T22:05:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.887 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 1768 2023-09-20T22:05:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2023-09-20T22:05:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 248, 2023-09-20T22:05:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 468, 2023-09-20T22:05:32.888 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 220, 2023-09-20T22:05:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 22, 2023-09-20T22:05:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 23, 2023-09-20T22:05:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 1, 2023-09-20T22:05:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 476, 2023-09-20T22:05:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 483, 2023-09-20T22:05:32.889 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 7, 2023-09-20T22:05:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 0, 2023-09-20T22:05:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 0, 2023-09-20T22:05:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 0, 2023-09-20T22:05:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 49288, 2023-09-20T22:05:32.890 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 207152 2023-09-20T22:05:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2023-09-20T22:05:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 0, 2023-09-20T22:05:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 0, 2023-09-20T22:05:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 0, 2023-09-20T22:05:32.891 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 0, 2023-09-20T22:05:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 0, 2023-09-20T22:05:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2023-09-20T22:05:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.892 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2023-09-20T22:05:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.893 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2023-09-20T22:05:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.894 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2023-09-20T22:05:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.895 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2023-09-20T22:05:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.896 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2023-09-20T22:05:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.897 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2023-09-20T22:05:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2023-09-20T22:05:32.898 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2023-09-20T22:05:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.899 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2023-09-20T22:05:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.900 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2023-09-20T22:05:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.901 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.902 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.902 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2023-09-20T22:05:32.902 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.902 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.902 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.903 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2023-09-20T22:05:32.903 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.903 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.903 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.904 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2023-09-20T22:05:32.904 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.904 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.904 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.904 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.905 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2023-09-20T22:05:32.905 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.905 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.905 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.905 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.906 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2023-09-20T22:05:32.906 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.906 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.906 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.906 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.906 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2023-09-20T22:05:32.907 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.907 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.907 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.907 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.907 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2023-09-20T22:05:32.908 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.908 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.908 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.908 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2023-09-20T22:05:32.908 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.909 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.909 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.909 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.909 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2023-09-20T22:05:32.909 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.910 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.910 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.910 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.910 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2023-09-20T22:05:32.910 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.910 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.911 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.911 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.911 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2023-09-20T22:05:32.911 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.911 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.912 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.912 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.912 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2023-09-20T22:05:32.912 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.912 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.912 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.913 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2023-09-20T22:05:32.913 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.913 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.913 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.914 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2023-09-20T22:05:32.914 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.914 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.914 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.914 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.915 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2023-09-20T22:05:32.915 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.915 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.915 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.915 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.916 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2023-09-20T22:05:32.916 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.916 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.916 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.916 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.916 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2023-09-20T22:05:32.917 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.917 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.917 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.917 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2023-09-20T22:05:32.918 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.918 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.918 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.918 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.918 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2023-09-20T22:05:32.919 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2023-09-20T22:05:32.919 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2023-09-20T22:05:32.919 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.919 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.919 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.919 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:32.920 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.920 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2023-09-20T22:05:32.920 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 1, 2023-09-20T22:05:32.920 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 1, 2023-09-20T22:05:32.920 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 0, 2023-09-20T22:05:32.921 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 1, 2023-09-20T22:05:32.921 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2023-09-20T22:05:32.921 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 0, 2023-09-20T22:05:32.921 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 0, 2023-09-20T22:05:32.921 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 2858, 2023-09-20T22:05:32.921 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2023-09-20T22:05:32.922 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.922 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2023-09-20T22:05:32.922 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:32.922 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:32.922 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:32.923 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:32.923 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:32.923 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:32.923 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:32.923 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:32.923 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:32.924 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:32.924 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:32.924 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:32.924 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:32.924 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:32.925 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:32.925 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:32.925 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:32.925 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:32.926 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:32.926 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:32.926 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:32.926 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:32.926 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:32.926 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:32.927 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:32.927 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:32.927 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:32.927 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:32.927 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:32.928 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:32.928 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:32.928 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2023-09-20T22:05:32.928 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:32.928 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:32.928 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:32.929 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:32.929 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 199215, 2023-09-20T22:05:32.929 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 297, 2023-09-20T22:05:32.929 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2023-09-20T22:05:32.929 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2023-09-20T22:05:32.930 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2023-09-20T22:05:32.930 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2023-09-20T22:05:32.930 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:32.930 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:32.930 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:32.930 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:32.931 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9088, 2023-09-20T22:05:32.931 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 175, 2023-09-20T22:05:32.931 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:32.931 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:32.931 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:32.932 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2023-09-20T22:05:32.932 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 922680, 2023-09-20T22:05:32.932 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 8017, 2023-09-20T22:05:32.932 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:32.932 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2023-09-20T22:05:32.932 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:32.933 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2023-09-20T22:05:32.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.933 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2023-09-20T22:05:32.933 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2023-09-20T22:05:32.933 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2023-09-20T22:05:32.933 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 3368, 2023-09-20T22:05:32.934 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 21, 2023-09-20T22:05:32.934 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 3, 2023-09-20T22:05:32.934 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 3368, 2023-09-20T22:05:32.934 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2023-09-20T22:05:32.934 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3368, 2023-09-20T22:05:32.935 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 25.830430351, 2023-09-20T22:05:32.935 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.007669367 2023-09-20T22:05:32.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.935 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2023-09-20T22:05:32.936 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2023-09-20T22:05:32.936 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3368, 2023-09-20T22:05:32.936 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 3369 2023-09-20T22:05:32.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.936 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 3368, 2023-09-20T22:05:32.936 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 3368, 2023-09-20T22:05:32.937 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 0, 2023-09-20T22:05:32.937 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2023-09-20T22:05:32.937 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2023-09-20T22:05:32.937 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 2063, 2023-09-20T22:05:32.937 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 0, 2023-09-20T22:05:32.938 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 1303, 2023-09-20T22:05:32.938 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 0, 2023-09-20T22:05:32.938 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 0, 2023-09-20T22:05:32.938 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2023-09-20T22:05:32.938 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2023-09-20T22:05:32.938 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 0, 2023-09-20T22:05:32.939 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 0, 2023-09-20T22:05:32.939 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 0, 2023-09-20T22:05:32.939 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2023-09-20T22:05:32.939 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2023-09-20T22:05:32.939 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2023-09-20T22:05:32.940 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 1, 2023-09-20T22:05:32.940 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 0, 2023-09-20T22:05:32.940 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2023-09-20T22:05:32.940 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2023-09-20T22:05:32.940 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2023-09-20T22:05:32.940 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2023-09-20T22:05:32.941 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2023-09-20T22:05:32.941 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2023-09-20T22:05:32.941 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2023-09-20T22:05:32.941 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2023-09-20T22:05:32.941 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2023-09-20T22:05:32.942 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 0, 2023-09-20T22:05:32.942 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2023-09-20T22:05:32.942 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2023-09-20T22:05:32.942 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2023-09-20T22:05:32.942 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2023-09-20T22:05:32.942 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2023-09-20T22:05:32.943 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2023-09-20T22:05:32.943 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2023-09-20T22:05:32.943 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2023-09-20T22:05:32.943 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2023-09-20T22:05:32.943 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2023-09-20T22:05:32.944 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2023-09-20T22:05:32.944 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2023-09-20T22:05:32.944 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2023-09-20T22:05:32.944 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2023-09-20T22:05:32.944 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2023-09-20T22:05:32.944 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2023-09-20T22:05:32.945 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 93, 2023-09-20T22:05:32.945 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2023-09-20T22:05:32.945 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 4, 2023-09-20T22:05:32.945 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 5, 2023-09-20T22:05:32.945 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 7, 2023-09-20T22:05:32.945 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 2, 2023-09-20T22:05:32.946 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2023-09-20T22:05:32.946 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 0, 2023-09-20T22:05:32.946 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 2, 2023-09-20T22:05:32.946 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 0 2023-09-20T22:05:32.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.947 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2023-09-20T22:05:32.947 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 0, 2023-09-20T22:05:32.947 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2023-09-20T22:05:32.947 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2023-09-20T22:05:32.947 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2023-09-20T22:05:32.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.949 INFO:teuthology.orchestra.run.smithi089.stdout: "osd-slow-ops": { 2023-09-20T22:05:32.949 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 0 2023-09-20T22:05:32.949 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.949 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2023-09-20T22:05:32.950 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2023-09-20T22:05:32.950 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 0, 2023-09-20T22:05:32.950 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2023-09-20T22:05:32.950 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 0, 2023-09-20T22:05:32.950 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 0, 2023-09-20T22:05:32.950 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2023-09-20T22:05:32.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.951 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:32.951 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:32.951 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-09-20T22:05:32.951 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:32.952 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 3697, 2023-09-20T22:05:32.952 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 3614512, 2023-09-20T22:05:32.952 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:32.952 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 3697, 2023-09-20T22:05:32.952 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:32.952 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:32.953 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 3697, 2023-09-20T22:05:32.953 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 3614512, 2023-09-20T22:05:32.953 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:32.953 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.953 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.953 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.954 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:32.954 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.954 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2023-09-20T22:05:32.954 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:32.954 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-09-20T22:05:32.955 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:32.955 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:32.955 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:32.955 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:32.955 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:32.956 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 3368, 2023-09-20T22:05:32.956 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 2718396, 2023-09-20T22:05:32.956 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 3368, 2023-09-20T22:05:32.956 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 2718396, 2023-09-20T22:05:32.956 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:32.956 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.957 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.957 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.957 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:32.957 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.957 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2023-09-20T22:05:32.958 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:32.958 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2023-09-20T22:05:32.958 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:32.958 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:32.958 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:32.959 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:32.959 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:32.959 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 3368, 2023-09-20T22:05:32.959 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 3368, 2023-09-20T22:05:32.959 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 3368, 2023-09-20T22:05:32.959 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 3368, 2023-09-20T22:05:32.960 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:32.960 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.960 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.960 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.960 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:32.961 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.961 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2023-09-20T22:05:32.961 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:32.961 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2023-09-20T22:05:32.961 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:32.961 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:32.962 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:32.962 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:32.962 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:32.962 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:32.962 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:32.963 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2023-09-20T22:05:32.963 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2023-09-20T22:05:32.963 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:32.963 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.963 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.963 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.964 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:32.964 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:32.964 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x55d1eb418960": { 2023-09-20T22:05:32.964 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:32.964 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2023-09-20T22:05:32.965 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:32.965 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:32.965 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:32.965 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:32.965 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:32.965 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:32.966 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:32.966 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2023-09-20T22:05:32.966 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2023-09-20T22:05:32.966 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:32.966 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:32.966 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:32.967 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:32.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:32.967 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:32.967 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-09-20T22:05:33.317 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-09-20T22:05:33.318 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-09-20T22:05:33.318 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-09-20T22:05:33.318 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-09-20T22:05:33.318 DEBUG:tasks.check_counter:Getting stats from h 2023-09-20T22:05:33.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.h perf dump 2023-09-20T22:05:33.920 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T22:05:33.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:33 smithi022 ceph-mon[97651]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2023-09-20T22:05:34.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:33 smithi123 ceph-mon[134433]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2023-09-20T22:05:34.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:33 smithi089 ceph-mon[134090]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2023-09-20T22:05:35.855 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:35 smithi089 ceph-mon[134090]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2023-09-20T22:05:35.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:35 smithi022 ceph-mon[97651]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2023-09-20T22:05:36.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:35 smithi123 ceph-mon[134433]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2023-09-20T22:05:36.194 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-09-20T22:05:36.195 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:36.195 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 3494, 2023-09-20T22:05:36.195 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 3017, 2023-09-20T22:05:36.195 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1241295, 2023-09-20T22:05:36.196 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 3683602, 2023-09-20T22:05:36.196 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 10, 2023-09-20T22:05:36.196 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2023-09-20T22:05:36.196 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.251116811, 2023-09-20T22:05:36.196 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.097812922, 2023-09-20T22:05:36.197 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.144661216, 2023-09-20T22:05:36.197 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.031673253, 2023-09-20T22:05:36.197 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:36.197 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3019, 2023-09-20T22:05:36.198 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.631903029, 2023-09-20T22:05:36.198 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000209308 2023-09-20T22:05:36.198 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.198 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:36.198 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 4657, 2023-09-20T22:05:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000876621, 2023-09-20T22:05:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000188 2023-09-20T22:05:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-20T22:05:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2023-09-20T22:05:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 938, 2023-09-20T22:05:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 1192, 2023-09-20T22:05:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 279304, 2023-09-20T22:05:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 868358, 2023-09-20T22:05:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 12, 2023-09-20T22:05:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 9, 2023-09-20T22:05:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.076039720, 2023-09-20T22:05:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.027282626, 2023-09-20T22:05:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.032866692, 2023-09-20T22:05:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.014685547, 2023-09-20T22:05:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1191, 2023-09-20T22:05:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.429330031, 2023-09-20T22:05:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000360478 2023-09-20T22:05:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 820, 2023-09-20T22:05:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000187034, 2023-09-20T22:05:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000228 2023-09-20T22:05:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-09-20T22:05:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 484656 2023-09-20T22:05:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 3521, 2023-09-20T22:05:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 3552, 2023-09-20T22:05:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1417606, 2023-09-20T22:05:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 1854362, 2023-09-20T22:05:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 16, 2023-09-20T22:05:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 10, 2023-09-20T22:05:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.278317047, 2023-09-20T22:05:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.082191667, 2023-09-20T22:05:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.156370861, 2023-09-20T22:05:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.068128573, 2023-09-20T22:05:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3547, 2023-09-20T22:05:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.930547228, 2023-09-20T22:05:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000262347 2023-09-20T22:05:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.209 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:36.209 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2087, 2023-09-20T22:05:36.209 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000974167, 2023-09-20T22:05:36.209 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000466 2023-09-20T22:05:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 308544, 2023-09-20T22:05:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 194048 2023-09-20T22:05:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2023-09-20T22:05:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2023-09-20T22:05:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2023-09-20T22:05:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2023-09-20T22:05:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2023-09-20T22:05:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2023-09-20T22:05:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1214, 2023-09-20T22:05:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.568234113, 2023-09-20T22:05:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000468067 2023-09-20T22:05:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2023-09-20T22:05:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2023-09-20T22:05:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2023-09-20T22:05:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 738, 2023-09-20T22:05:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.038166405, 2023-09-20T22:05:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000051715 2023-09-20T22:05:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2023-09-20T22:05:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 981, 2023-09-20T22:05:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 851, 2023-09-20T22:05:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2023-09-20T22:05:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 851, 2023-09-20T22:05:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 2.269458174, 2023-09-20T22:05:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002666813 2023-09-20T22:05:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2023-09-20T22:05:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 130, 2023-09-20T22:05:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 0, 2023-09-20T22:05:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2023-09-20T22:05:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 11, 2023-09-20T22:05:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 0, 2023-09-20T22:05:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 0, 2023-09-20T22:05:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 266, 2023-09-20T22:05:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 0, 2023-09-20T22:05:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 0, 2023-09-20T22:05:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 266, 2023-09-20T22:05:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 266, 2023-09-20T22:05:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 9, 2023-09-20T22:05:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 242, 2023-09-20T22:05:36.220 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 242, 2023-09-20T22:05:36.220 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 23, 2023-09-20T22:05:36.220 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 1793, 2023-09-20T22:05:36.220 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 1422, 2023-09-20T22:05:36.220 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 129, 2023-09-20T22:05:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 20, 2023-09-20T22:05:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 0, 2023-09-20T22:05:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2023-09-20T22:05:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 80, 2023-09-20T22:05:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 8, 2023-09-20T22:05:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2023-09-20T22:05:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 1, 2023-09-20T22:05:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 6, 2023-09-20T22:05:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 12, 2023-09-20T22:05:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 240, 2023-09-20T22:05:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2023-09-20T22:05:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 355, 2023-09-20T22:05:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2023-09-20T22:05:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 1033, 2023-09-20T22:05:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 1632663607, 2023-09-20T22:05:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 0, 2023-09-20T22:05:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 355, 2023-09-20T22:05:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2023-09-20T22:05:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2023-09-20T22:05:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2023-09-20T22:05:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 8, 2023-09-20T22:05:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 0, 2023-09-20T22:05:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 8, 2023-09-20T22:05:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 347, 2023-09-20T22:05:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 0, 2023-09-20T22:05:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 8, 2023-09-20T22:05:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 642, 2023-09-20T22:05:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 1, 2023-09-20T22:05:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-20T22:05:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-20T22:05:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 527, 2023-09-20T22:05:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 360, 2023-09-20T22:05:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 109, 2023-09-20T22:05:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 906 2023-09-20T22:05:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2023-09-20T22:05:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 199, 2023-09-20T22:05:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 296, 2023-09-20T22:05:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 155, 2023-09-20T22:05:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 270, 2023-09-20T22:05:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 203, 2023-09-20T22:05:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 0, 2023-09-20T22:05:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2023-09-20T22:05:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2023-09-20T22:05:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 219, 2023-09-20T22:05:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 219, 2023-09-20T22:05:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 0, 2023-09-20T22:05:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2023-09-20T22:05:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2023-09-20T22:05:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2023-09-20T22:05:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2023-09-20T22:05:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2023-09-20T22:05:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2023-09-20T22:05:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 0, 2023-09-20T22:05:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 1, 2023-09-20T22:05:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2023-09-20T22:05:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 0, 2023-09-20T22:05:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2023-09-20T22:05:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0 2023-09-20T22:05:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2023-09-20T22:05:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 0, 2023-09-20T22:05:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 1768, 2023-09-20T22:05:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 0, 2023-09-20T22:05:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 0, 2023-09-20T22:05:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 1768, 2023-09-20T22:05:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2023-09-20T22:05:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 0, 2023-09-20T22:05:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 3, 2023-09-20T22:05:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 0, 2023-09-20T22:05:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 0, 2023-09-20T22:05:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 3, 2023-09-20T22:05:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 2, 2023-09-20T22:05:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2023-09-20T22:05:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 0, 2023-09-20T22:05:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 4194304, 2023-09-20T22:05:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 6912695, 2023-09-20T22:05:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 0, 2023-09-20T22:05:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2023-09-20T22:05:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 379, 2023-09-20T22:05:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 6.111181333, 2023-09-20T22:05:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.016124489 2023-09-20T22:05:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 0 2023-09-20T22:05:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2023-09-20T22:05:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 257, 2023-09-20T22:05:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 479, 2023-09-20T22:05:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 222, 2023-09-20T22:05:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 43, 2023-09-20T22:05:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 162, 2023-09-20T22:05:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 119, 2023-09-20T22:05:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 266, 2023-09-20T22:05:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 494, 2023-09-20T22:05:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 228, 2023-09-20T22:05:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 242, 2023-09-20T22:05:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 684, 2023-09-20T22:05:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 442, 2023-09-20T22:05:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 55680, 2023-09-20T22:05:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 198960 2023-09-20T22:05:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2023-09-20T22:05:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 1193, 2023-09-20T22:05:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 0, 2023-09-20T22:05:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 981, 2023-09-20T22:05:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 26, 2023-09-20T22:05:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 42, 2023-09-20T22:05:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2023-09-20T22:05:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 219, 2023-09-20T22:05:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.466487250, 2023-09-20T22:05:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002130078 2023-09-20T22:05:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2023-09-20T22:05:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 8, 2023-09-20T22:05:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.002897882, 2023-09-20T22:05:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000362235 2023-09-20T22:05:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2023-09-20T22:05:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2023-09-20T22:05:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.246 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.246 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.246 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2023-09-20T22:05:36.246 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 233, 2023-09-20T22:05:36.246 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 1.201253406, 2023-09-20T22:05:36.247 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.005155594 2023-09-20T22:05:36.247 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.247 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2023-09-20T22:05:36.247 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.247 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.248 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.248 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.248 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2023-09-20T22:05:36.248 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.248 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.248 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.249 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.249 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2023-09-20T22:05:36.249 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.249 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.249 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.250 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.250 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2023-09-20T22:05:36.250 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.250 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.250 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.251 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2023-09-20T22:05:36.251 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.251 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.251 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.251 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.252 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2023-09-20T22:05:36.252 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.252 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.252 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.252 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.253 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2023-09-20T22:05:36.253 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.253 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.253 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.253 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.253 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2023-09-20T22:05:36.254 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.254 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.254 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.254 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.254 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2023-09-20T22:05:36.255 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.255 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.255 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.255 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.255 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2023-09-20T22:05:36.256 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 152, 2023-09-20T22:05:36.256 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.088924314, 2023-09-20T22:05:36.256 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000585028 2023-09-20T22:05:36.256 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.256 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2023-09-20T22:05:36.256 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 19, 2023-09-20T22:05:36.257 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.010574596, 2023-09-20T22:05:36.257 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000556557 2023-09-20T22:05:36.257 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.257 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2023-09-20T22:05:36.257 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 207, 2023-09-20T22:05:36.258 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.437679071, 2023-09-20T22:05:36.258 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002114391 2023-09-20T22:05:36.258 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.258 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2023-09-20T22:05:36.258 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.259 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.259 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.259 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.259 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2023-09-20T22:05:36.259 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.259 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.260 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.260 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.260 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2023-09-20T22:05:36.260 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.260 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.261 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.261 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.261 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2023-09-20T22:05:36.261 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.261 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.261 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.262 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.262 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2023-09-20T22:05:36.262 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.262 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.262 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.263 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2023-09-20T22:05:36.263 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1, 2023-09-20T22:05:36.263 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.003558525, 2023-09-20T22:05:36.263 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.003558525 2023-09-20T22:05:36.263 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.264 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2023-09-20T22:05:36.264 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.264 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.264 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.264 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.265 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2023-09-20T22:05:36.265 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.265 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.265 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.265 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.265 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2023-09-20T22:05:36.266 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.266 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.266 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.266 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.266 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2023-09-20T22:05:36.267 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.267 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.267 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.267 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.267 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2023-09-20T22:05:36.268 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.268 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.268 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.268 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.268 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2023-09-20T22:05:36.268 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 12, 2023-09-20T22:05:36.269 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.058083130, 2023-09-20T22:05:36.269 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.004840260 2023-09-20T22:05:36.269 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.269 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2023-09-20T22:05:36.269 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2023-09-20T22:05:36.270 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2023-09-20T22:05:36.270 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.270 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.270 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.270 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:36.271 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.271 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2023-09-20T22:05:36.271 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 1, 2023-09-20T22:05:36.271 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 1, 2023-09-20T22:05:36.271 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 0, 2023-09-20T22:05:36.271 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 1, 2023-09-20T22:05:36.272 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2023-09-20T22:05:36.272 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 100, 2023-09-20T22:05:36.272 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 100, 2023-09-20T22:05:36.272 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 1006, 2023-09-20T22:05:36.272 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2023-09-20T22:05:36.273 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.273 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2023-09-20T22:05:36.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:36.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:36.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:36.273 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:36.274 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:36.274 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:36.274 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:36.274 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:36.274 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:36.275 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:36.275 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:36.275 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:36.275 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:36.275 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:36.276 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:36.276 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:36.276 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:36.276 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:36.276 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:36.276 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:36.277 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:36.277 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:36.277 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:36.277 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:36.277 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:36.277 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:36.278 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:36.278 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:36.278 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:36.278 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:36.278 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:36.279 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2023-09-20T22:05:36.279 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:36.279 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:36.279 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:36.279 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:36.280 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 286293, 2023-09-20T22:05:36.280 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 371, 2023-09-20T22:05:36.280 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 704, 2023-09-20T22:05:36.280 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 8, 2023-09-20T22:05:36.280 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2023-09-20T22:05:36.281 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2023-09-20T22:05:36.281 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:36.281 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:36.281 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:36.281 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:36.281 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9088, 2023-09-20T22:05:36.282 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 175, 2023-09-20T22:05:36.282 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:36.282 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:36.283 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:36.283 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2023-09-20T22:05:36.283 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 932185, 2023-09-20T22:05:36.283 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 9959, 2023-09-20T22:05:36.283 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:36.283 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2023-09-20T22:05:36.284 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:36.284 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2023-09-20T22:05:36.284 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.284 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2023-09-20T22:05:36.284 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2023-09-20T22:05:36.285 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2023-09-20T22:05:36.285 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 1599, 2023-09-20T22:05:36.285 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 2814462, 2023-09-20T22:05:36.285 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 28, 2023-09-20T22:05:36.285 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 1599, 2023-09-20T22:05:36.285 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2023-09-20T22:05:36.286 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1599, 2023-09-20T22:05:36.286 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 27.923385649, 2023-09-20T22:05:36.286 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.017463030 2023-09-20T22:05:36.286 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.286 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2023-09-20T22:05:36.287 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2023-09-20T22:05:36.287 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1599, 2023-09-20T22:05:36.287 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 1740 2023-09-20T22:05:36.287 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.287 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 1599, 2023-09-20T22:05:36.288 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 436, 2023-09-20T22:05:36.288 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 1163, 2023-09-20T22:05:36.288 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2023-09-20T22:05:36.288 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2023-09-20T22:05:36.288 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 0, 2023-09-20T22:05:36.288 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 10, 2023-09-20T22:05:36.289 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 81, 2023-09-20T22:05:36.289 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 460, 2023-09-20T22:05:36.289 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 128, 2023-09-20T22:05:36.289 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2023-09-20T22:05:36.289 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2023-09-20T22:05:36.290 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 0, 2023-09-20T22:05:36.290 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 0, 2023-09-20T22:05:36.290 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 450, 2023-09-20T22:05:36.290 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2023-09-20T22:05:36.290 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2023-09-20T22:05:36.291 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2023-09-20T22:05:36.291 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 355, 2023-09-20T22:05:36.291 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 20, 2023-09-20T22:05:36.291 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2023-09-20T22:05:36.291 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2023-09-20T22:05:36.291 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2023-09-20T22:05:36.292 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2023-09-20T22:05:36.292 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2023-09-20T22:05:36.292 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2023-09-20T22:05:36.292 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2023-09-20T22:05:36.292 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2023-09-20T22:05:36.293 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2023-09-20T22:05:36.293 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 1, 2023-09-20T22:05:36.293 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2023-09-20T22:05:36.293 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2023-09-20T22:05:36.293 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2023-09-20T22:05:36.293 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2023-09-20T22:05:36.294 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2023-09-20T22:05:36.294 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2023-09-20T22:05:36.294 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2023-09-20T22:05:36.294 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2023-09-20T22:05:36.294 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2023-09-20T22:05:36.295 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2023-09-20T22:05:36.295 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2023-09-20T22:05:36.295 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2023-09-20T22:05:36.295 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2023-09-20T22:05:36.295 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2023-09-20T22:05:36.296 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2023-09-20T22:05:36.296 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2023-09-20T22:05:36.296 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 93, 2023-09-20T22:05:36.296 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2023-09-20T22:05:36.296 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 4, 2023-09-20T22:05:36.296 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 12, 2023-09-20T22:05:36.297 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 12, 2023-09-20T22:05:36.297 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 0, 2023-09-20T22:05:36.297 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2023-09-20T22:05:36.297 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 174, 2023-09-20T22:05:36.297 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 0, 2023-09-20T22:05:36.298 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 61 2023-09-20T22:05:36.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.298 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2023-09-20T22:05:36.298 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 1, 2023-09-20T22:05:36.298 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2023-09-20T22:05:36.298 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2023-09-20T22:05:36.299 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2023-09-20T22:05:36.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.299 INFO:teuthology.orchestra.run.smithi089.stdout: "osd-slow-ops": { 2023-09-20T22:05:36.299 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 0 2023-09-20T22:05:36.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.300 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2023-09-20T22:05:36.300 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2023-09-20T22:05:36.300 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 10, 2023-09-20T22:05:36.300 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2023-09-20T22:05:36.300 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 5, 2023-09-20T22:05:36.300 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 219, 2023-09-20T22:05:36.301 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2023-09-20T22:05:36.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.301 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:36.301 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:36.301 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-09-20T22:05:36.302 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:36.302 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 7949, 2023-09-20T22:05:36.302 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 2244241, 2023-09-20T22:05:36.302 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:36.302 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 7949, 2023-09-20T22:05:36.302 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:36.303 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:36.303 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 7949, 2023-09-20T22:05:36.303 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 2244241, 2023-09-20T22:05:36.303 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:36.303 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.304 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.304 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.304 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:36.304 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.304 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2023-09-20T22:05:36.304 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:36.305 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-09-20T22:05:36.305 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:36.305 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:36.305 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:36.305 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:36.306 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:36.306 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 1599, 2023-09-20T22:05:36.306 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 2838333, 2023-09-20T22:05:36.306 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 1599, 2023-09-20T22:05:36.306 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 2838333, 2023-09-20T22:05:36.307 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:36.307 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.307 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.307 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.307 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:36.307 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.308 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2023-09-20T22:05:36.308 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:36.308 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2023-09-20T22:05:36.308 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:36.308 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:36.309 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:36.309 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:36.309 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:36.309 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 1599, 2023-09-20T22:05:36.309 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 1599, 2023-09-20T22:05:36.310 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 1599, 2023-09-20T22:05:36.310 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1599, 2023-09-20T22:05:36.310 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:36.310 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.310 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.310 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.311 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:36.311 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.311 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2023-09-20T22:05:36.311 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:36.311 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2023-09-20T22:05:36.312 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:36.312 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 219, 2023-09-20T22:05:36.312 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 23871, 2023-09-20T22:05:36.312 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:36.312 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 219, 2023-09-20T22:05:36.312 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:36.313 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:36.313 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 81, 2023-09-20T22:05:36.313 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 23871, 2023-09-20T22:05:36.313 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:36.313 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.314 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.314 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.314 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:36.314 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:36.314 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x55d3eea446e0": { 2023-09-20T22:05:36.314 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:36.315 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2023-09-20T22:05:36.315 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:36.315 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 1768, 2023-09-20T22:05:36.315 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 2718391, 2023-09-20T22:05:36.315 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:36.316 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 1768, 2023-09-20T22:05:36.316 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:36.316 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:36.316 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 379, 2023-09-20T22:05:36.316 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 2718391, 2023-09-20T22:05:36.317 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:36.317 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:36.317 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:36.317 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:36.317 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:36.317 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:36.318 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-09-20T22:05:36.990 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2023-09-20T22:05:36.990 INFO:tasks.check_counter:Daemon mds.h mds.imported=12 2023-09-20T22:05:36.991 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=199 2023-09-20T22:05:36.991 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=296 2023-09-20T22:05:36.991 DEBUG:tasks.check_counter:Getting stats from k 2023-09-20T22:05:36.991 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.k perf dump 2023-09-20T22:05:37.632 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.b/config 2023-09-20T22:05:37.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:37 smithi022 ceph-mon[97651]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T22:05:38.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:37 smithi123 ceph-mon[134433]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T22:05:38.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:37 smithi089 ceph-mon[134090]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-09-20T22:05:39.871 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2023-09-20T22:05:39.872 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:39.872 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1082, 2023-09-20T22:05:39.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 1082, 2023-09-20T22:05:39.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 365145, 2023-09-20T22:05:39.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 335707, 2023-09-20T22:05:39.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 7, 2023-09-20T22:05:39.873 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2023-09-20T22:05:39.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.102546769, 2023-09-20T22:05:39.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.026030528, 2023-09-20T22:05:39.874 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.026552730, 2023-09-20T22:05:39.875 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.039579067, 2023-09-20T22:05:39.875 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:39.875 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1081, 2023-09-20T22:05:39.876 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.423956633, 2023-09-20T22:05:39.876 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000392189 2023-09-20T22:05:39.877 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.877 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:39.877 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.877 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.877 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.878 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-09-20T22:05:39.878 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 448 2023-09-20T22:05:39.878 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.878 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:39.879 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 2170, 2023-09-20T22:05:39.879 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2407, 2023-09-20T22:05:39.879 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1667039, 2023-09-20T22:05:39.879 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 1239217, 2023-09-20T22:05:39.879 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 6, 2023-09-20T22:05:39.880 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 4, 2023-09-20T22:05:39.880 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.157525899, 2023-09-20T22:05:39.880 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.049312155, 2023-09-20T22:05:39.880 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.036087539, 2023-09-20T22:05:39.880 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.059839584, 2023-09-20T22:05:39.880 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:39.881 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2405, 2023-09-20T22:05:39.881 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.521600982, 2023-09-20T22:05:39.881 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000216881 2023-09-20T22:05:39.881 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.881 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:39.881 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.882 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.882 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.882 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.882 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-20T22:05:39.882 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 564912 2023-09-20T22:05:39.883 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.883 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:39.883 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 328, 2023-09-20T22:05:39.883 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 322, 2023-09-20T22:05:39.884 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 278868, 2023-09-20T22:05:39.884 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 189968, 2023-09-20T22:05:39.884 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 5, 2023-09-20T22:05:39.884 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2023-09-20T22:05:39.884 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.040351214, 2023-09-20T22:05:39.884 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.015181298, 2023-09-20T22:05:39.885 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.013137413, 2023-09-20T22:05:39.885 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.008554049, 2023-09-20T22:05:39.885 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:39.885 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 321, 2023-09-20T22:05:39.885 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.102163782, 2023-09-20T22:05:39.886 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000318267 2023-09-20T22:05:39.886 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.886 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:39.886 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.886 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.887 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.887 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 277520, 2023-09-20T22:05:39.887 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 189024 2023-09-20T22:05:39.887 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.888 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2023-09-20T22:05:39.888 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2023-09-20T22:05:39.888 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2023-09-20T22:05:39.888 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.888 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2023-09-20T22:05:39.889 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2023-09-20T22:05:39.889 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2023-09-20T22:05:39.889 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 5471, 2023-09-20T22:05:39.889 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.137592259, 2023-09-20T22:05:39.889 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000025149 2023-09-20T22:05:39.890 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:39.890 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.890 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2023-09-20T22:05:39.890 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2023-09-20T22:05:39.890 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2023-09-20T22:05:39.891 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.891 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.891 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.891 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:39.891 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.891 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2023-09-20T22:05:39.892 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 0, 2023-09-20T22:05:39.892 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 0, 2023-09-20T22:05:39.892 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2023-09-20T22:05:39.892 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.892 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.893 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.893 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.893 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2023-09-20T22:05:39.893 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 0, 2023-09-20T22:05:39.893 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 0, 2023-09-20T22:05:39.894 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2023-09-20T22:05:39.894 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 0, 2023-09-20T22:05:39.894 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 0, 2023-09-20T22:05:39.894 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 0, 2023-09-20T22:05:39.894 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 320, 2023-09-20T22:05:39.894 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 0, 2023-09-20T22:05:39.895 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 0, 2023-09-20T22:05:39.895 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 320, 2023-09-20T22:05:39.895 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 320, 2023-09-20T22:05:39.895 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 0, 2023-09-20T22:05:39.895 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 0, 2023-09-20T22:05:39.896 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 0, 2023-09-20T22:05:39.896 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 10, 2023-09-20T22:05:39.896 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 0, 2023-09-20T22:05:39.896 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 0, 2023-09-20T22:05:39.896 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 0, 2023-09-20T22:05:39.897 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 0, 2023-09-20T22:05:39.897 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 0, 2023-09-20T22:05:39.897 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2023-09-20T22:05:39.897 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 0, 2023-09-20T22:05:39.897 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 0, 2023-09-20T22:05:39.898 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2023-09-20T22:05:39.898 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 0, 2023-09-20T22:05:39.898 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 0, 2023-09-20T22:05:39.898 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 0, 2023-09-20T22:05:39.898 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 0, 2023-09-20T22:05:39.899 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2023-09-20T22:05:39.899 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 0, 2023-09-20T22:05:39.899 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2023-09-20T22:05:39.899 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 0, 2023-09-20T22:05:39.899 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 0, 2023-09-20T22:05:39.900 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 0, 2023-09-20T22:05:39.900 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 0, 2023-09-20T22:05:39.900 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2023-09-20T22:05:39.900 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2023-09-20T22:05:39.900 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2023-09-20T22:05:39.901 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 0, 2023-09-20T22:05:39.901 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 0, 2023-09-20T22:05:39.901 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 0, 2023-09-20T22:05:39.901 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 0, 2023-09-20T22:05:39.901 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 0, 2023-09-20T22:05:39.902 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 0, 2023-09-20T22:05:39.902 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 0, 2023-09-20T22:05:39.902 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 0, 2023-09-20T22:05:39.902 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-20T22:05:39.902 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-20T22:05:39.903 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 0, 2023-09-20T22:05:39.903 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 0, 2023-09-20T22:05:39.903 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 0, 2023-09-20T22:05:39.903 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 0 2023-09-20T22:05:39.903 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.904 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2023-09-20T22:05:39.904 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 0, 2023-09-20T22:05:39.904 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 0, 2023-09-20T22:05:39.904 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 0, 2023-09-20T22:05:39.904 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 0, 2023-09-20T22:05:39.905 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 0, 2023-09-20T22:05:39.905 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 0, 2023-09-20T22:05:39.905 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2023-09-20T22:05:39.905 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2023-09-20T22:05:39.905 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 0, 2023-09-20T22:05:39.906 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 0, 2023-09-20T22:05:39.906 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 0, 2023-09-20T22:05:39.906 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2023-09-20T22:05:39.906 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2023-09-20T22:05:39.906 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2023-09-20T22:05:39.907 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2023-09-20T22:05:39.907 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2023-09-20T22:05:39.907 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2023-09-20T22:05:39.907 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 0, 2023-09-20T22:05:39.907 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 0, 2023-09-20T22:05:39.908 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2023-09-20T22:05:39.908 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 0, 2023-09-20T22:05:39.908 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2023-09-20T22:05:39.908 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0 2023-09-20T22:05:39.908 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.908 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2023-09-20T22:05:39.909 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 0, 2023-09-20T22:05:39.909 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 0, 2023-09-20T22:05:39.909 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 0, 2023-09-20T22:05:39.909 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 0, 2023-09-20T22:05:39.909 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 748, 2023-09-20T22:05:39.910 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2023-09-20T22:05:39.910 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 0, 2023-09-20T22:05:39.910 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 0, 2023-09-20T22:05:39.910 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 0, 2023-09-20T22:05:39.910 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 0, 2023-09-20T22:05:39.911 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 2, 2023-09-20T22:05:39.911 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 2, 2023-09-20T22:05:39.911 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2023-09-20T22:05:39.911 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 0, 2023-09-20T22:05:39.912 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 4194304, 2023-09-20T22:05:39.912 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 5310399, 2023-09-20T22:05:39.912 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 5309271, 2023-09-20T22:05:39.912 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2023-09-20T22:05:39.912 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.912 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 748 2023-09-20T22:05:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2023-09-20T22:05:39.914 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 315, 2023-09-20T22:05:39.914 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 316, 2023-09-20T22:05:39.914 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 1, 2023-09-20T22:05:39.914 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 14, 2023-09-20T22:05:39.914 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 14, 2023-09-20T22:05:39.914 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 0, 2023-09-20T22:05:39.915 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 320, 2023-09-20T22:05:39.915 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 320, 2023-09-20T22:05:39.915 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 0, 2023-09-20T22:05:39.915 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 0, 2023-09-20T22:05:39.916 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 0, 2023-09-20T22:05:39.916 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 0, 2023-09-20T22:05:39.916 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 51904, 2023-09-20T22:05:39.917 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 207152 2023-09-20T22:05:39.917 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.917 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2023-09-20T22:05:39.917 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 0, 2023-09-20T22:05:39.917 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 0, 2023-09-20T22:05:39.917 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 0, 2023-09-20T22:05:39.918 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 0, 2023-09-20T22:05:39.918 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 0, 2023-09-20T22:05:39.918 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2023-09-20T22:05:39.919 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.921 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.921 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.921 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.922 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2023-09-20T22:05:39.922 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.922 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.923 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.923 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.923 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2023-09-20T22:05:39.923 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.924 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.924 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.924 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.924 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2023-09-20T22:05:39.924 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.925 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.925 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.925 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.925 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2023-09-20T22:05:39.925 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.925 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.926 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.926 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.926 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2023-09-20T22:05:39.926 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.927 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.927 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.927 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.927 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2023-09-20T22:05:39.928 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.928 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.928 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.928 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.928 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2023-09-20T22:05:39.928 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.929 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.929 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.929 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.930 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2023-09-20T22:05:39.930 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.930 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.931 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.931 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.931 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2023-09-20T22:05:39.931 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.932 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.932 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.932 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.932 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2023-09-20T22:05:39.932 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.933 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.933 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.933 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.933 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2023-09-20T22:05:39.933 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.934 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.934 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.934 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.934 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2023-09-20T22:05:39.934 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.935 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.935 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.935 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.935 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2023-09-20T22:05:39.935 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.936 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.936 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.936 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.936 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2023-09-20T22:05:39.936 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.937 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.937 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.937 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.937 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2023-09-20T22:05:39.937 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.938 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.938 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.938 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.938 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2023-09-20T22:05:39.939 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.939 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.939 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.939 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.939 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2023-09-20T22:05:39.939 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.940 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.940 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.940 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.940 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2023-09-20T22:05:39.940 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.941 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.941 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.941 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.941 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2023-09-20T22:05:39.941 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.942 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.942 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.942 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.942 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2023-09-20T22:05:39.942 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.943 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.943 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.943 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.943 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2023-09-20T22:05:39.944 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.944 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.944 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.944 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.944 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2023-09-20T22:05:39.944 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.945 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.945 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.945 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.945 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2023-09-20T22:05:39.945 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.946 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.946 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.946 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.946 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2023-09-20T22:05:39.946 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.947 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.947 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.947 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.947 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2023-09-20T22:05:39.948 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.948 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.948 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.948 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.948 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2023-09-20T22:05:39.949 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.949 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.949 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.950 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.950 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2023-09-20T22:05:39.950 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.950 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.951 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.951 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.951 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2023-09-20T22:05:39.951 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.952 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.952 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.953 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.953 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2023-09-20T22:05:39.953 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2023-09-20T22:05:39.954 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2023-09-20T22:05:39.954 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.954 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.955 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.955 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:39.955 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.955 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2023-09-20T22:05:39.955 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 1, 2023-09-20T22:05:39.956 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 1, 2023-09-20T22:05:39.956 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 0, 2023-09-20T22:05:39.957 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 1, 2023-09-20T22:05:39.957 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2023-09-20T22:05:39.957 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 0, 2023-09-20T22:05:39.957 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 0, 2023-09-20T22:05:39.957 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 2865, 2023-09-20T22:05:39.958 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2023-09-20T22:05:39.958 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.958 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2023-09-20T22:05:39.958 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:39.959 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:39.959 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:39.959 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:39.959 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:39.959 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:39.959 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:39.960 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:39.960 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:39.960 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:39.960 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:39.960 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:39.961 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:39.961 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:39.961 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:39.961 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:39.961 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:39.962 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:39.962 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:39.962 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:39.962 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:39.962 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:39.962 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:39.963 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:39.963 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:39.963 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:39.963 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:39.963 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:39.964 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:39.964 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:39.964 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:39.964 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2023-09-20T22:05:39.964 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:39.965 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:39.965 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:39.965 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:39.965 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 210522, 2023-09-20T22:05:39.965 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 368, 2023-09-20T22:05:39.965 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2023-09-20T22:05:39.966 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2023-09-20T22:05:39.966 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2023-09-20T22:05:39.966 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2023-09-20T22:05:39.966 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:39.966 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:39.967 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:39.967 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:39.967 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9088, 2023-09-20T22:05:39.967 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 175, 2023-09-20T22:05:39.967 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:39.968 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:39.968 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:39.968 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2023-09-20T22:05:39.968 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 953136, 2023-09-20T22:05:39.968 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 8855, 2023-09-20T22:05:39.969 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:39.969 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2023-09-20T22:05:39.969 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:39.969 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2023-09-20T22:05:39.969 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.969 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2023-09-20T22:05:39.970 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2023-09-20T22:05:39.970 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2023-09-20T22:05:39.970 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 3249, 2023-09-20T22:05:39.970 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 21, 2023-09-20T22:05:39.970 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 3, 2023-09-20T22:05:39.971 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 3249, 2023-09-20T22:05:39.971 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2023-09-20T22:05:39.971 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3249, 2023-09-20T22:05:39.971 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 33.738254135, 2023-09-20T22:05:39.971 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.010384196 2023-09-20T22:05:39.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.972 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2023-09-20T22:05:39.972 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2023-09-20T22:05:39.972 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3249, 2023-09-20T22:05:39.972 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 3250 2023-09-20T22:05:39.972 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.973 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 3249, 2023-09-20T22:05:39.973 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 3249, 2023-09-20T22:05:39.973 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 0, 2023-09-20T22:05:39.973 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2023-09-20T22:05:39.973 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2023-09-20T22:05:39.974 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 2012, 2023-09-20T22:05:39.974 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 0, 2023-09-20T22:05:39.974 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 1235, 2023-09-20T22:05:39.974 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 0, 2023-09-20T22:05:39.974 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 0, 2023-09-20T22:05:39.975 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2023-09-20T22:05:39.975 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2023-09-20T22:05:39.975 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 0, 2023-09-20T22:05:39.975 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 0, 2023-09-20T22:05:39.975 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 0, 2023-09-20T22:05:39.975 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2023-09-20T22:05:39.976 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2023-09-20T22:05:39.976 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2023-09-20T22:05:39.976 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 1, 2023-09-20T22:05:39.976 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 0, 2023-09-20T22:05:39.976 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2023-09-20T22:05:39.977 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2023-09-20T22:05:39.977 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2023-09-20T22:05:39.977 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2023-09-20T22:05:39.977 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2023-09-20T22:05:39.977 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2023-09-20T22:05:39.977 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2023-09-20T22:05:39.978 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2023-09-20T22:05:39.978 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2023-09-20T22:05:39.978 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 0, 2023-09-20T22:05:39.978 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2023-09-20T22:05:39.978 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2023-09-20T22:05:39.979 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2023-09-20T22:05:39.979 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2023-09-20T22:05:39.979 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2023-09-20T22:05:39.979 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2023-09-20T22:05:39.979 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2023-09-20T22:05:39.980 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2023-09-20T22:05:39.980 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2023-09-20T22:05:39.980 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2023-09-20T22:05:39.980 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2023-09-20T22:05:39.980 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2023-09-20T22:05:39.980 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2023-09-20T22:05:39.981 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2023-09-20T22:05:39.981 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2023-09-20T22:05:39.981 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2023-09-20T22:05:39.981 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 93, 2023-09-20T22:05:39.981 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2023-09-20T22:05:39.981 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 2, 2023-09-20T22:05:39.982 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 6, 2023-09-20T22:05:39.982 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 8, 2023-09-20T22:05:39.982 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 2, 2023-09-20T22:05:39.982 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2023-09-20T22:05:39.982 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 0, 2023-09-20T22:05:39.983 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 2, 2023-09-20T22:05:39.983 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 0 2023-09-20T22:05:39.983 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.983 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2023-09-20T22:05:39.983 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 0, 2023-09-20T22:05:39.983 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2023-09-20T22:05:39.984 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2023-09-20T22:05:39.984 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2023-09-20T22:05:39.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.984 INFO:teuthology.orchestra.run.smithi089.stdout: "osd-slow-ops": { 2023-09-20T22:05:39.984 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 0 2023-09-20T22:05:39.984 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.985 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2023-09-20T22:05:39.985 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2023-09-20T22:05:39.985 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 0, 2023-09-20T22:05:39.985 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2023-09-20T22:05:39.985 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 0, 2023-09-20T22:05:39.986 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 0, 2023-09-20T22:05:39.986 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2023-09-20T22:05:39.986 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.986 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:39.986 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:39.987 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-09-20T22:05:39.987 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:39.987 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 3576, 2023-09-20T22:05:39.987 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 1969463, 2023-09-20T22:05:39.987 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:39.987 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 3576, 2023-09-20T22:05:39.988 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:39.988 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:39.988 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 3576, 2023-09-20T22:05:39.988 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1969463, 2023-09-20T22:05:39.988 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:39.988 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.989 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.989 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.989 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:39.989 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.989 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2023-09-20T22:05:39.989 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:39.990 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2023-09-20T22:05:39.990 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:39.990 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:39.990 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:39.990 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:39.990 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:39.991 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 3249, 2023-09-20T22:05:39.991 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 1116100, 2023-09-20T22:05:39.991 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 3249, 2023-09-20T22:05:39.991 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1116100, 2023-09-20T22:05:39.991 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:39.991 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.992 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.992 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.992 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:39.992 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.992 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2023-09-20T22:05:39.992 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2023-09-20T22:05:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:39.993 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 3249, 2023-09-20T22:05:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 3249, 2023-09-20T22:05:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 3249, 2023-09-20T22:05:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 3249, 2023-09-20T22:05:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:39.994 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.995 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2023-09-20T22:05:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2023-09-20T22:05:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:39.996 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2023-09-20T22:05:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2023-09-20T22:05:39.997 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:39.998 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2023-09-20T22:05:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x560e4395a960": { 2023-09-20T22:05:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2023-09-20T22:05:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2023-09-20T22:05:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2023-09-20T22:05:39.999 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2023-09-20T22:05:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2023-09-20T22:05:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2023-09-20T22:05:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2023-09-20T22:05:40.000 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2023-09-20T22:05:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2023-09-20T22:05:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2023-09-20T22:05:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2023-09-20T22:05:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2023-09-20T22:05:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2023-09-20T22:05:40.001 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:40.002 INFO:teuthology.orchestra.run.smithi089.stdout: } 2023-09-20T22:05:40.002 INFO:teuthology.orchestra.run.smithi089.stdout:} 2023-09-20T22:05:40.002 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:39 smithi022 ceph-mon[97651]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T22:05:40.022 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:39 smithi089 ceph-mon[134090]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T22:05:40.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:39 smithi123 ceph-mon[134433]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-09-20T22:05:40.297 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-09-20T22:05:40.298 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-09-20T22:05:40.298 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2023-09-20T22:05:40.298 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2023-09-20T22:05:40.299 DEBUG:tasks.check_counter:Getting stats from c 2023-09-20T22:05:40.299 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.c perf dump 2023-09-20T22:05:40.938 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T22:05:41.820 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:41 smithi123 ceph-mon[134433]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-09-20T22:05:41.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:41 smithi022 ceph-mon[97651]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-09-20T22:05:42.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:41 smithi089 ceph-mon[134090]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-09-20T22:05:42.402 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-20T22:05:42.402 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:42.402 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1432, 2023-09-20T22:05:42.403 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1421, 2023-09-20T22:05:42.403 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 672432, 2023-09-20T22:05:42.403 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 530850, 2023-09-20T22:05:42.403 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2023-09-20T22:05:42.403 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2023-09-20T22:05:42.404 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.150820794, 2023-09-20T22:05:42.404 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.044831725, 2023-09-20T22:05:42.404 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.041319386, 2023-09-20T22:05:42.404 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.050890684, 2023-09-20T22:05:42.404 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:42.405 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1420, 2023-09-20T22:05:42.405 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.427950288, 2023-09-20T22:05:42.405 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000301373 2023-09-20T22:05:42.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.405 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:42.405 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.406 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.406 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.406 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 302160, 2023-09-20T22:05:42.407 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 188128 2023-09-20T22:05:42.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.407 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:42.407 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 7, 2023-09-20T22:05:42.407 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 240, 2023-09-20T22:05:42.407 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3708, 2023-09-20T22:05:42.408 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 481324, 2023-09-20T22:05:42.408 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2023-09-20T22:05:42.408 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2023-09-20T22:05:42.408 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.011325368, 2023-09-20T22:05:42.408 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.007871955, 2023-09-20T22:05:42.409 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.001519594, 2023-09-20T22:05:42.409 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000085406, 2023-09-20T22:05:42.409 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:42.409 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 239, 2023-09-20T22:05:42.409 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.042225859, 2023-09-20T22:05:42.409 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000176677 2023-09-20T22:05:42.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.410 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:42.410 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.410 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.410 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.411 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-09-20T22:05:42.411 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 480640 2023-09-20T22:05:42.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.411 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:42.411 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2309, 2023-09-20T22:05:42.412 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2311, 2023-09-20T22:05:42.412 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3461977, 2023-09-20T22:05:42.412 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 718106, 2023-09-20T22:05:42.412 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2023-09-20T22:05:42.412 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2023-09-20T22:05:42.413 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.153000529, 2023-09-20T22:05:42.413 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.041897451, 2023-09-20T22:05:42.413 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.038676207, 2023-09-20T22:05:42.413 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.063104488, 2023-09-20T22:05:42.413 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:42.413 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2309, 2023-09-20T22:05:42.414 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.542301835, 2023-09-20T22:05:42.414 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000234864 2023-09-20T22:05:42.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.414 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:42.414 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.415 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.415 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.415 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-20T22:05:42.415 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2023-09-20T22:05:42.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.416 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2023-09-20T22:05:42.416 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2023-09-20T22:05:42.416 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2023-09-20T22:05:42.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.416 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2023-09-20T22:05:42.416 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-20T22:05:42.417 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-20T22:05:42.417 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 5589, 2023-09-20T22:05:42.417 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.143672638, 2023-09-20T22:05:42.417 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000025706 2023-09-20T22:05:42.417 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:42.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.418 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2023-09-20T22:05:42.418 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-20T22:05:42.418 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-20T22:05:42.418 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.419 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.419 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.419 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:42.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.419 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2023-09-20T22:05:42.419 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2023-09-20T22:05:42.420 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2023-09-20T22:05:42.420 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2023-09-20T22:05:42.420 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.420 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.420 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.421 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2023-09-20T22:05:42.421 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2023-09-20T22:05:42.421 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2023-09-20T22:05:42.421 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2023-09-20T22:05:42.421 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2023-09-20T22:05:42.422 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2023-09-20T22:05:42.422 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2023-09-20T22:05:42.422 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 430, 2023-09-20T22:05:42.422 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 0, 2023-09-20T22:05:42.422 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 0, 2023-09-20T22:05:42.423 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 430, 2023-09-20T22:05:42.423 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 430, 2023-09-20T22:05:42.423 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2023-09-20T22:05:42.423 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2023-09-20T22:05:42.423 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2023-09-20T22:05:42.423 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 3, 2023-09-20T22:05:42.424 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2023-09-20T22:05:42.424 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2023-09-20T22:05:42.424 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2023-09-20T22:05:42.424 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2023-09-20T22:05:42.424 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2023-09-20T22:05:42.425 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2023-09-20T22:05:42.425 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2023-09-20T22:05:42.425 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2023-09-20T22:05:42.425 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2023-09-20T22:05:42.425 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2023-09-20T22:05:42.425 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2023-09-20T22:05:42.426 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2023-09-20T22:05:42.426 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2023-09-20T22:05:42.426 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2023-09-20T22:05:42.426 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2023-09-20T22:05:42.426 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2023-09-20T22:05:42.427 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 0, 2023-09-20T22:05:42.427 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 0, 2023-09-20T22:05:42.427 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2023-09-20T22:05:42.427 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2023-09-20T22:05:42.427 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2023-09-20T22:05:42.428 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2023-09-20T22:05:42.428 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2023-09-20T22:05:42.428 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2023-09-20T22:05:42.428 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2023-09-20T22:05:42.428 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2023-09-20T22:05:42.428 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2023-09-20T22:05:42.429 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2023-09-20T22:05:42.429 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2023-09-20T22:05:42.429 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2023-09-20T22:05:42.429 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2023-09-20T22:05:42.429 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-20T22:05:42.430 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-20T22:05:42.430 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2023-09-20T22:05:42.430 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2023-09-20T22:05:42.430 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2023-09-20T22:05:42.430 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2023-09-20T22:05:42.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.431 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2023-09-20T22:05:42.431 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2023-09-20T22:05:42.431 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2023-09-20T22:05:42.431 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2023-09-20T22:05:42.431 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2023-09-20T22:05:42.432 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2023-09-20T22:05:42.432 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2023-09-20T22:05:42.432 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2023-09-20T22:05:42.432 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2023-09-20T22:05:42.432 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2023-09-20T22:05:42.433 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2023-09-20T22:05:42.433 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2023-09-20T22:05:42.433 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2023-09-20T22:05:42.433 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2023-09-20T22:05:42.433 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2023-09-20T22:05:42.433 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2023-09-20T22:05:42.434 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2023-09-20T22:05:42.434 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2023-09-20T22:05:42.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2023-09-20T22:05:42.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2023-09-20T22:05:42.434 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2023-09-20T22:05:42.435 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2023-09-20T22:05:42.435 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2023-09-20T22:05:42.435 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2023-09-20T22:05:42.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.435 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2023-09-20T22:05:42.435 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2023-09-20T22:05:42.436 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2023-09-20T22:05:42.436 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2023-09-20T22:05:42.436 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2023-09-20T22:05:42.436 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 1686, 2023-09-20T22:05:42.436 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2023-09-20T22:05:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2023-09-20T22:05:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2023-09-20T22:05:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2023-09-20T22:05:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2023-09-20T22:05:42.437 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 3, 2023-09-20T22:05:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2023-09-20T22:05:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2023-09-20T22:05:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2023-09-20T22:05:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2023-09-20T22:05:42.438 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 7069990, 2023-09-20T22:05:42.439 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 7068718, 2023-09-20T22:05:42.439 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2023-09-20T22:05:42.439 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.439 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.439 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.440 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 1686 2023-09-20T22:05:42.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.440 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2023-09-20T22:05:42.440 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 72, 2023-09-20T22:05:42.440 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 72, 2023-09-20T22:05:42.441 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 0, 2023-09-20T22:05:42.441 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 8, 2023-09-20T22:05:42.441 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 8, 2023-09-20T22:05:42.441 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 0, 2023-09-20T22:05:42.441 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 430, 2023-09-20T22:05:42.441 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 430, 2023-09-20T22:05:42.442 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 0, 2023-09-20T22:05:42.442 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2023-09-20T22:05:42.442 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2023-09-20T22:05:42.442 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2023-09-20T22:05:42.442 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 50964, 2023-09-20T22:05:42.443 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 207152 2023-09-20T22:05:42.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.443 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2023-09-20T22:05:42.443 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2023-09-20T22:05:42.443 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2023-09-20T22:05:42.444 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2023-09-20T22:05:42.444 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2023-09-20T22:05:42.444 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2023-09-20T22:05:42.444 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2023-09-20T22:05:42.444 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.444 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.445 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.445 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2023-09-20T22:05:42.445 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.445 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.446 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.446 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2023-09-20T22:05:42.446 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.446 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.447 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.447 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2023-09-20T22:05:42.447 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.447 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.447 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.448 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2023-09-20T22:05:42.448 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.448 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.448 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.449 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2023-09-20T22:05:42.449 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.449 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.449 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.450 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2023-09-20T22:05:42.450 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.450 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.450 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.451 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2023-09-20T22:05:42.451 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.451 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.451 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.451 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2023-09-20T22:05:42.452 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.452 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.452 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.452 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2023-09-20T22:05:42.453 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.453 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.453 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.453 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2023-09-20T22:05:42.453 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.454 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.454 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.454 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2023-09-20T22:05:42.454 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.455 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.455 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.455 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2023-09-20T22:05:42.455 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.456 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.456 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.456 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2023-09-20T22:05:42.456 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.457 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.457 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.457 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2023-09-20T22:05:42.457 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.457 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.458 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.458 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2023-09-20T22:05:42.458 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.458 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.459 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.459 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2023-09-20T22:05:42.459 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.459 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.460 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.460 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2023-09-20T22:05:42.460 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.460 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.460 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.461 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2023-09-20T22:05:42.461 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.461 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.461 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.462 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2023-09-20T22:05:42.462 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.462 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.462 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.463 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2023-09-20T22:05:42.463 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.463 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.463 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.463 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2023-09-20T22:05:42.464 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.464 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.464 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.464 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2023-09-20T22:05:42.465 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.465 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.465 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.465 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2023-09-20T22:05:42.465 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.466 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.466 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.466 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2023-09-20T22:05:42.466 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.467 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.467 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.467 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2023-09-20T22:05:42.467 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.467 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.468 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.468 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2023-09-20T22:05:42.468 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.468 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.469 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.469 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2023-09-20T22:05:42.469 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.469 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.469 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.470 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2023-09-20T22:05:42.470 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.470 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.470 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.471 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2023-09-20T22:05:42.471 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2023-09-20T22:05:42.471 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2023-09-20T22:05:42.471 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.471 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.472 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.472 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:42.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.472 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2023-09-20T22:05:42.472 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2023-09-20T22:05:42.473 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 1, 2023-09-20T22:05:42.473 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 0, 2023-09-20T22:05:42.473 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2023-09-20T22:05:42.473 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2023-09-20T22:05:42.473 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2023-09-20T22:05:42.473 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2023-09-20T22:05:42.474 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 2877, 2023-09-20T22:05:42.474 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2023-09-20T22:05:42.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.474 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2023-09-20T22:05:42.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:42.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:42.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:42.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:42.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:42.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:42.475 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:42.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:42.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:42.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:42.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:42.476 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:42.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:42.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:42.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:42.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:42.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:42.477 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:42.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:42.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:42.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:42.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:42.478 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:42.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:42.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:42.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:42.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:42.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:42.479 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:42.480 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:42.480 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:42.480 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2023-09-20T22:05:42.480 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:42.480 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:42.480 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:42.481 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:42.481 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 184606, 2023-09-20T22:05:42.481 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 116, 2023-09-20T22:05:42.481 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2023-09-20T22:05:42.481 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2023-09-20T22:05:42.481 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2023-09-20T22:05:42.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2023-09-20T22:05:42.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:42.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:42.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:42.482 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:42.483 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 9088, 2023-09-20T22:05:42.483 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 175, 2023-09-20T22:05:42.483 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:42.483 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:42.483 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:42.483 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2023-09-20T22:05:42.484 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 482075, 2023-09-20T22:05:42.484 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 12910, 2023-09-20T22:05:42.484 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:42.484 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2023-09-20T22:05:42.484 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:42.484 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2023-09-20T22:05:42.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.485 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2023-09-20T22:05:42.485 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-09-20T22:05:42.485 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-09-20T22:05:42.485 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 3413, 2023-09-20T22:05:42.486 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 21, 2023-09-20T22:05:42.486 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 3, 2023-09-20T22:05:42.486 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 3413, 2023-09-20T22:05:42.486 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-09-20T22:05:42.486 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3413, 2023-09-20T22:05:42.487 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 15.051656025, 2023-09-20T22:05:42.487 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004410095 2023-09-20T22:05:42.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.487 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-09-20T22:05:42.487 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-09-20T22:05:42.487 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3413, 2023-09-20T22:05:42.488 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3414 2023-09-20T22:05:42.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.488 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 3413, 2023-09-20T22:05:42.488 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 3413, 2023-09-20T22:05:42.488 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2023-09-20T22:05:42.488 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-09-20T22:05:42.489 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-09-20T22:05:42.489 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 2088, 2023-09-20T22:05:42.489 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2023-09-20T22:05:42.489 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 1323, 2023-09-20T22:05:42.489 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2023-09-20T22:05:42.489 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2023-09-20T22:05:42.490 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-09-20T22:05:42.490 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-09-20T22:05:42.490 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-09-20T22:05:42.490 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2023-09-20T22:05:42.490 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2023-09-20T22:05:42.491 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-09-20T22:05:42.491 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-09-20T22:05:42.491 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-09-20T22:05:42.491 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2023-09-20T22:05:42.491 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2023-09-20T22:05:42.492 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2023-09-20T22:05:42.492 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-09-20T22:05:42.492 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-09-20T22:05:42.492 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2023-09-20T22:05:42.492 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-09-20T22:05:42.492 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-09-20T22:05:42.493 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-09-20T22:05:42.493 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-09-20T22:05:42.493 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-09-20T22:05:42.493 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2023-09-20T22:05:42.493 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-09-20T22:05:42.493 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-09-20T22:05:42.494 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-09-20T22:05:42.494 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-09-20T22:05:42.494 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-09-20T22:05:42.494 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-09-20T22:05:42.494 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-09-20T22:05:42.494 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-09-20T22:05:42.495 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-09-20T22:05:42.495 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-09-20T22:05:42.495 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-09-20T22:05:42.495 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-09-20T22:05:42.495 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-09-20T22:05:42.496 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-09-20T22:05:42.496 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2023-09-20T22:05:42.496 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-09-20T22:05:42.496 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 93, 2023-09-20T22:05:42.496 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-09-20T22:05:42.496 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 4, 2023-09-20T22:05:42.497 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 4, 2023-09-20T22:05:42.497 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 6, 2023-09-20T22:05:42.497 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 2, 2023-09-20T22:05:42.497 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-09-20T22:05:42.497 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2023-09-20T22:05:42.497 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2023-09-20T22:05:42.498 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2023-09-20T22:05:42.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.498 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2023-09-20T22:05:42.498 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2023-09-20T22:05:42.498 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2023-09-20T22:05:42.499 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2023-09-20T22:05:42.499 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2023-09-20T22:05:42.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.499 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2023-09-20T22:05:42.499 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2023-09-20T22:05:42.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.500 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2023-09-20T22:05:42.500 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2023-09-20T22:05:42.500 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2023-09-20T22:05:42.500 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2023-09-20T22:05:42.500 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2023-09-20T22:05:42.501 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2023-09-20T22:05:42.501 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2023-09-20T22:05:42.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.501 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:42.501 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:42.501 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-20T22:05:42.502 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:42.502 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3744, 2023-09-20T22:05:42.502 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 3781207, 2023-09-20T22:05:42.502 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:42.502 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3744, 2023-09-20T22:05:42.503 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-20T22:05:42.503 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-20T22:05:42.503 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3744, 2023-09-20T22:05:42.503 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 3781207, 2023-09-20T22:05:42.503 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:42.503 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.504 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.504 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.504 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:42.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.504 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2023-09-20T22:05:42.504 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:42.505 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-20T22:05:42.505 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:42.505 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-20T22:05:42.505 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-20T22:05:42.505 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:42.505 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:42.506 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3413, 2023-09-20T22:05:42.506 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 2875691, 2023-09-20T22:05:42.506 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3413, 2023-09-20T22:05:42.506 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2875691, 2023-09-20T22:05:42.506 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:42.507 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.507 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.507 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.507 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:42.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.507 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2023-09-20T22:05:42.508 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:42.508 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-09-20T22:05:42.508 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:42.508 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-20T22:05:42.508 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-20T22:05:42.508 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:42.509 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:42.509 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3413, 2023-09-20T22:05:42.509 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 3413, 2023-09-20T22:05:42.509 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3413, 2023-09-20T22:05:42.509 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 3413, 2023-09-20T22:05:42.509 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:42.510 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.510 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.510 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.510 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:42.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.511 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2023-09-20T22:05:42.511 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:42.511 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-20T22:05:42.511 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:42.511 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-20T22:05:42.511 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-20T22:05:42.512 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:42.512 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:42.512 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-20T22:05:42.512 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-20T22:05:42.512 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-09-20T22:05:42.512 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-09-20T22:05:42.513 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:42.513 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.513 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.513 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.513 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:42.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:42.514 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x5588ba74a960": { 2023-09-20T22:05:42.514 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:42.514 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-20T22:05:42.514 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:42.514 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-20T22:05:42.515 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-20T22:05:42.515 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:42.515 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:42.515 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-20T22:05:42.515 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-20T22:05:42.515 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-09-20T22:05:42.516 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-09-20T22:05:42.516 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:42.516 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:42.516 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:42.516 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:42.517 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:42.517 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:42.517 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-20T22:05:42.882 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-09-20T22:05:42.883 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-09-20T22:05:42.883 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-09-20T22:05:42.883 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-09-20T22:05:42.883 DEBUG:tasks.check_counter:Getting stats from f 2023-09-20T22:05:42.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.f perf dump 2023-09-20T22:05:43.437 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T22:05:43.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:43 smithi022 ceph-mon[97651]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2023-09-20T22:05:44.021 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:43 smithi123 ceph-mon[134433]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2023-09-20T22:05:44.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:43 smithi089 ceph-mon[134090]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2023-09-20T22:05:44.796 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-20T22:05:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 814, 2023-09-20T22:05:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 823, 2023-09-20T22:05:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 251509, 2023-09-20T22:05:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 1414087, 2023-09-20T22:05:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 14, 2023-09-20T22:05:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2023-09-20T22:05:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.064514517, 2023-09-20T22:05:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.021282769, 2023-09-20T22:05:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.026063627, 2023-09-20T22:05:44.799 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.015109214, 2023-09-20T22:05:44.799 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:44.799 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 821, 2023-09-20T22:05:44.799 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.139994341, 2023-09-20T22:05:44.799 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000170516 2023-09-20T22:05:44.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.800 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:44.800 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 607, 2023-09-20T22:05:44.800 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000120907, 2023-09-20T22:05:44.800 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000199 2023-09-20T22:05:44.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.801 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-20T22:05:44.801 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2023-09-20T22:05:44.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.801 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:44.801 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 550, 2023-09-20T22:05:44.801 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 769, 2023-09-20T22:05:44.802 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 221669, 2023-09-20T22:05:44.802 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 783048, 2023-09-20T22:05:44.802 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 10, 2023-09-20T22:05:44.802 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2023-09-20T22:05:44.802 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.042256035, 2023-09-20T22:05:44.803 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.017927120, 2023-09-20T22:05:44.803 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.021033282, 2023-09-20T22:05:44.803 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.004079387, 2023-09-20T22:05:44.803 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:44.803 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 770, 2023-09-20T22:05:44.804 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.217630562, 2023-09-20T22:05:44.804 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000282637 2023-09-20T22:05:44.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.804 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:44.804 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 655, 2023-09-20T22:05:44.804 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000108830, 2023-09-20T22:05:44.805 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000166 2023-09-20T22:05:44.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.805 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-09-20T22:05:44.805 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 572576 2023-09-20T22:05:44.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.806 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:44.806 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 431603, 2023-09-20T22:05:44.806 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 502613, 2023-09-20T22:05:44.806 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 100946551, 2023-09-20T22:05:44.806 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 147468419, 2023-09-20T22:05:44.807 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 15, 2023-09-20T22:05:44.807 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2023-09-20T22:05:44.807 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 19.180213668, 2023-09-20T22:05:44.807 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 7.661891090, 2023-09-20T22:05:44.807 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 14.292678864, 2023-09-20T22:05:44.808 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.016336809, 2023-09-20T22:05:44.808 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:44.808 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 502610, 2023-09-20T22:05:44.808 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 27.651377594, 2023-09-20T22:05:44.808 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000055015 2023-09-20T22:05:44.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.809 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:44.809 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 842024, 2023-09-20T22:05:44.809 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.117196116, 2023-09-20T22:05:44.809 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000139 2023-09-20T22:05:44.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.810 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 303600, 2023-09-20T22:05:44.810 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 194896 2023-09-20T22:05:44.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.810 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2023-09-20T22:05:44.810 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2023-09-20T22:05:44.811 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2023-09-20T22:05:44.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.811 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2023-09-20T22:05:44.811 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-20T22:05:44.811 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-20T22:05:44.812 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 984, 2023-09-20T22:05:44.812 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.162489601, 2023-09-20T22:05:44.812 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000165131 2023-09-20T22:05:44.812 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:44.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.813 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2023-09-20T22:05:44.813 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-20T22:05:44.813 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-20T22:05:44.813 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 16, 2023-09-20T22:05:44.813 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000290137, 2023-09-20T22:05:44.813 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000018133 2023-09-20T22:05:44.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:44.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.814 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2023-09-20T22:05:44.814 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 146, 2023-09-20T22:05:44.814 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 25, 2023-09-20T22:05:44.815 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2023-09-20T22:05:44.815 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 25, 2023-09-20T22:05:44.815 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.017472297, 2023-09-20T22:05:44.815 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000698891 2023-09-20T22:05:44.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.816 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2023-09-20T22:05:44.816 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 121, 2023-09-20T22:05:44.816 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2023-09-20T22:05:44.816 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2023-09-20T22:05:44.816 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 11, 2023-09-20T22:05:44.817 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2023-09-20T22:05:44.817 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2023-09-20T22:05:44.817 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 308, 2023-09-20T22:05:44.817 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 5, 2023-09-20T22:05:44.817 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 303, 2023-09-20T22:05:44.817 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2023-09-20T22:05:44.818 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 297, 2023-09-20T22:05:44.818 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2023-09-20T22:05:44.818 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 280, 2023-09-20T22:05:44.818 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 280, 2023-09-20T22:05:44.819 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 22, 2023-09-20T22:05:44.819 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 378, 2023-09-20T22:05:44.819 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 232, 2023-09-20T22:05:44.819 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 121, 2023-09-20T22:05:44.819 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 22, 2023-09-20T22:05:44.819 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2023-09-20T22:05:44.820 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2023-09-20T22:05:44.820 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2023-09-20T22:05:44.820 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 3006, 2023-09-20T22:05:44.820 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 3, 2023-09-20T22:05:44.820 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2023-09-20T22:05:44.820 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2023-09-20T22:05:44.821 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 7, 2023-09-20T22:05:44.821 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 280, 2023-09-20T22:05:44.821 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2023-09-20T22:05:44.821 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 444, 2023-09-20T22:05:44.821 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2023-09-20T22:05:44.822 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 1033, 2023-09-20T22:05:44.822 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 1632663607, 2023-09-20T22:05:44.822 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2023-09-20T22:05:44.822 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 444, 2023-09-20T22:05:44.822 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2023-09-20T22:05:44.823 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2023-09-20T22:05:44.823 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2023-09-20T22:05:44.823 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2023-09-20T22:05:44.823 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2023-09-20T22:05:44.823 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2023-09-20T22:05:44.823 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 444, 2023-09-20T22:05:44.824 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 3, 2023-09-20T22:05:44.824 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 71384, 2023-09-20T22:05:44.824 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 71421, 2023-09-20T22:05:44.824 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2023-09-20T22:05:44.824 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-20T22:05:44.824 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-20T22:05:44.825 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 71616, 2023-09-20T22:05:44.825 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 225, 2023-09-20T22:05:44.825 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 6, 2023-09-20T22:05:44.825 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 7 2023-09-20T22:05:44.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.826 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2023-09-20T22:05:44.826 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 25, 2023-09-20T22:05:44.826 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 425, 2023-09-20T22:05:44.826 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 222, 2023-09-20T22:05:44.826 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 394, 2023-09-20T22:05:44.827 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 25, 2023-09-20T22:05:44.827 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2023-09-20T22:05:44.827 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2023-09-20T22:05:44.827 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2023-09-20T22:05:44.827 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 1, 2023-09-20T22:05:44.827 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 1, 2023-09-20T22:05:44.828 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2023-09-20T22:05:44.828 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2023-09-20T22:05:44.828 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2023-09-20T22:05:44.828 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2023-09-20T22:05:44.828 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2023-09-20T22:05:44.828 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2023-09-20T22:05:44.829 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2023-09-20T22:05:44.829 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2023-09-20T22:05:44.829 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2023-09-20T22:05:44.829 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2023-09-20T22:05:44.829 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2023-09-20T22:05:44.830 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2023-09-20T22:05:44.830 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2023-09-20T22:05:44.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.830 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2023-09-20T22:05:44.830 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2023-09-20T22:05:44.830 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 773, 2023-09-20T22:05:44.831 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2023-09-20T22:05:44.831 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2023-09-20T22:05:44.831 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 773, 2023-09-20T22:05:44.831 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2023-09-20T22:05:44.831 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2023-09-20T22:05:44.832 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 2, 2023-09-20T22:05:44.832 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2023-09-20T22:05:44.832 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2023-09-20T22:05:44.832 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 2, 2023-09-20T22:05:44.832 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2023-09-20T22:05:44.832 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2023-09-20T22:05:44.833 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2023-09-20T22:05:44.833 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2023-09-20T22:05:44.833 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 5303045, 2023-09-20T22:05:44.833 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2023-09-20T22:05:44.833 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2023-09-20T22:05:44.834 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 262, 2023-09-20T22:05:44.834 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 7.938839775, 2023-09-20T22:05:44.834 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.030300915 2023-09-20T22:05:44.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.834 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2023-09-20T22:05:44.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.835 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2023-09-20T22:05:44.835 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 308, 2023-09-20T22:05:44.835 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 309, 2023-09-20T22:05:44.835 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 1, 2023-09-20T22:05:44.835 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 43, 2023-09-20T22:05:44.836 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 220, 2023-09-20T22:05:44.836 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 177, 2023-09-20T22:05:44.836 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 308, 2023-09-20T22:05:44.836 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 311, 2023-09-20T22:05:44.836 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 3, 2023-09-20T22:05:44.837 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 280, 2023-09-20T22:05:44.837 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 553, 2023-09-20T22:05:44.837 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 273, 2023-09-20T22:05:44.837 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 57760, 2023-09-20T22:05:44.837 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 198960 2023-09-20T22:05:44.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.838 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2023-09-20T22:05:44.838 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 149, 2023-09-20T22:05:44.838 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 71381, 2023-09-20T22:05:44.838 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 146, 2023-09-20T22:05:44.839 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 14, 2023-09-20T22:05:44.839 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 142762, 2023-09-20T22:05:44.839 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2023-09-20T22:05:44.839 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.839 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.840 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.840 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2023-09-20T22:05:44.840 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.840 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.840 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.841 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2023-09-20T22:05:44.841 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.841 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.841 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.842 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2023-09-20T22:05:44.842 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.842 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.842 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.843 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2023-09-20T22:05:44.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7, 2023-09-20T22:05:44.843 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002320009, 2023-09-20T22:05:44.843 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000331429 2023-09-20T22:05:44.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.844 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2023-09-20T22:05:44.844 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.844 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.844 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.845 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2023-09-20T22:05:44.845 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.845 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.845 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.846 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2023-09-20T22:05:44.846 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.846 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.846 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.847 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2023-09-20T22:05:44.847 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.847 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.847 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.848 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2023-09-20T22:05:44.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.848 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.849 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2023-09-20T22:05:44.849 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.849 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.849 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.850 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2023-09-20T22:05:44.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.850 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.851 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2023-09-20T22:05:44.851 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.851 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.851 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.853 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2023-09-20T22:05:44.853 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.853 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.853 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.854 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2023-09-20T22:05:44.854 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 10, 2023-09-20T22:05:44.854 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.007055172, 2023-09-20T22:05:44.854 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000705517 2023-09-20T22:05:44.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.855 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2023-09-20T22:05:44.855 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7, 2023-09-20T22:05:44.855 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.007238708, 2023-09-20T22:05:44.855 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001034101 2023-09-20T22:05:44.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.856 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2023-09-20T22:05:44.856 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.856 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.856 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.857 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2023-09-20T22:05:44.857 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.857 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.857 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.858 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2023-09-20T22:05:44.858 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.858 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.858 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.859 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2023-09-20T22:05:44.859 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.859 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.859 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.860 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2023-09-20T22:05:44.860 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.860 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.860 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.861 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2023-09-20T22:05:44.861 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.861 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.861 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.861 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2023-09-20T22:05:44.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.862 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.862 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2023-09-20T22:05:44.863 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.863 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.863 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.863 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2023-09-20T22:05:44.864 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.864 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.864 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.864 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2023-09-20T22:05:44.865 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.865 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.865 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.865 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2023-09-20T22:05:44.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.866 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.866 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2023-09-20T22:05:44.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.867 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.867 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2023-09-20T22:05:44.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2023-09-20T22:05:44.868 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000858408, 2023-09-20T22:05:44.868 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000858408 2023-09-20T22:05:44.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.868 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2023-09-20T22:05:44.868 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2023-09-20T22:05:44.869 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2023-09-20T22:05:44.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.869 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.869 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:44.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.870 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2023-09-20T22:05:44.870 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2023-09-20T22:05:44.870 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 1, 2023-09-20T22:05:44.870 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 0, 2023-09-20T22:05:44.870 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2023-09-20T22:05:44.871 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2023-09-20T22:05:44.871 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 6, 2023-09-20T22:05:44.871 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 6, 2023-09-20T22:05:44.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1001, 2023-09-20T22:05:44.871 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2023-09-20T22:05:44.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.872 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2023-09-20T22:05:44.872 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:44.872 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:44.872 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:44.873 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:44.873 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:44.873 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:44.873 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:44.873 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:44.873 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:44.874 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:44.874 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:44.874 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:44.874 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:44.874 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:44.875 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:44.875 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:44.875 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:44.875 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:44.875 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:44.876 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:44.876 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:44.876 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:44.876 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:44.876 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:44.876 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:44.877 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:44.877 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:44.877 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:44.877 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:44.877 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:44.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:44.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2023-09-20T22:05:44.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:44.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:44.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:44.879 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:44.879 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 307160, 2023-09-20T22:05:44.879 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 398, 2023-09-20T22:05:44.879 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2023-09-20T22:05:44.879 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2023-09-20T22:05:44.880 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2023-09-20T22:05:44.880 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2023-09-20T22:05:44.880 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:44.880 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:44.880 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:44.881 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:44.881 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 9088, 2023-09-20T22:05:44.881 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 175, 2023-09-20T22:05:44.881 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:44.881 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:44.881 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:44.882 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2023-09-20T22:05:44.882 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 1088783, 2023-09-20T22:05:44.882 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 11335, 2023-09-20T22:05:44.882 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:44.882 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2023-09-20T22:05:44.883 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:44.883 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2023-09-20T22:05:44.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.883 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2023-09-20T22:05:44.883 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-09-20T22:05:44.884 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-09-20T22:05:44.884 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 812, 2023-09-20T22:05:44.884 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 1149087, 2023-09-20T22:05:44.884 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 33, 2023-09-20T22:05:44.884 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 812, 2023-09-20T22:05:44.885 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-09-20T22:05:44.885 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 812, 2023-09-20T22:05:44.885 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 10.402762511, 2023-09-20T22:05:44.885 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.012811283 2023-09-20T22:05:44.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.885 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-09-20T22:05:44.886 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-09-20T22:05:44.886 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 812, 2023-09-20T22:05:44.886 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 847 2023-09-20T22:05:44.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.886 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 812, 2023-09-20T22:05:44.887 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 445, 2023-09-20T22:05:44.887 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 367, 2023-09-20T22:05:44.887 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-09-20T22:05:44.887 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-09-20T22:05:44.887 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2023-09-20T22:05:44.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2023-09-20T22:05:44.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 1, 2023-09-20T22:05:44.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 263, 2023-09-20T22:05:44.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 56, 2023-09-20T22:05:44.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-09-20T22:05:44.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-09-20T22:05:44.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-09-20T22:05:44.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2023-09-20T22:05:44.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 14, 2023-09-20T22:05:44.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-09-20T22:05:44.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-09-20T22:05:44.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-09-20T22:05:44.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 444, 2023-09-20T22:05:44.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2023-09-20T22:05:44.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2023-09-20T22:05:44.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-09-20T22:05:44.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-09-20T22:05:44.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2023-09-20T22:05:44.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-09-20T22:05:44.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-09-20T22:05:44.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-09-20T22:05:44.891 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-09-20T22:05:44.892 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-09-20T22:05:44.892 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2023-09-20T22:05:44.892 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-09-20T22:05:44.892 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-09-20T22:05:44.892 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-09-20T22:05:44.893 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-09-20T22:05:44.893 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-09-20T22:05:44.893 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-09-20T22:05:44.893 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-09-20T22:05:44.893 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-09-20T22:05:44.893 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-09-20T22:05:44.894 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-09-20T22:05:44.894 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-09-20T22:05:44.894 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-09-20T22:05:44.894 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-09-20T22:05:44.894 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-09-20T22:05:44.895 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2023-09-20T22:05:44.895 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-09-20T22:05:44.895 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 93, 2023-09-20T22:05:44.895 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-09-20T22:05:44.895 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 4, 2023-09-20T22:05:44.895 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2023-09-20T22:05:44.896 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2023-09-20T22:05:44.896 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2023-09-20T22:05:44.896 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-09-20T22:05:44.896 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 32, 2023-09-20T22:05:44.896 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2023-09-20T22:05:44.897 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 7 2023-09-20T22:05:44.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.897 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2023-09-20T22:05:44.897 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2023-09-20T22:05:44.897 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2023-09-20T22:05:44.898 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2023-09-20T22:05:44.898 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2023-09-20T22:05:44.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.898 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2023-09-20T22:05:44.898 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2023-09-20T22:05:44.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.899 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2023-09-20T22:05:44.899 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2023-09-20T22:05:44.899 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 2, 2023-09-20T22:05:44.899 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2023-09-20T22:05:44.899 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 1, 2023-09-20T22:05:44.900 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 1, 2023-09-20T22:05:44.900 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2023-09-20T22:05:44.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.900 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:44.900 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:44.900 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-20T22:05:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 432963, 2023-09-20T22:05:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 63580087, 2023-09-20T22:05:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 432963, 2023-09-20T22:05:44.901 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-20T22:05:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-20T22:05:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 432963, 2023-09-20T22:05:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 63580087, 2023-09-20T22:05:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.902 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.903 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.903 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:44.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.903 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2023-09-20T22:05:44.903 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:44.904 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-20T22:05:44.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:44.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-20T22:05:44.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-20T22:05:44.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:44.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:44.905 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 812, 2023-09-20T22:05:44.905 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 1149196, 2023-09-20T22:05:44.905 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 812, 2023-09-20T22:05:44.905 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1149196, 2023-09-20T22:05:44.905 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:44.905 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.906 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.906 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.906 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:44.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.907 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2023-09-20T22:05:44.907 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:44.907 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-09-20T22:05:44.907 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:44.907 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-20T22:05:44.907 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-20T22:05:44.908 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:44.908 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:44.908 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 812, 2023-09-20T22:05:44.908 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 812, 2023-09-20T22:05:44.908 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 812, 2023-09-20T22:05:44.909 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 812, 2023-09-20T22:05:44.909 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:44.909 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.909 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.909 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.909 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:44.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.910 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2023-09-20T22:05:44.910 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:44.910 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-20T22:05:44.910 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:44.911 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 1, 2023-09-20T22:05:44.911 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 109, 2023-09-20T22:05:44.911 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:44.911 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 1, 2023-09-20T22:05:44.911 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-20T22:05:44.912 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-20T22:05:44.912 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1, 2023-09-20T22:05:44.912 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 109, 2023-09-20T22:05:44.912 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:44.912 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.912 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.913 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:44.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:44.913 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x555e6f7426e0": { 2023-09-20T22:05:44.913 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:44.914 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-20T22:05:44.914 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:44.914 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 773, 2023-09-20T22:05:44.914 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1108741, 2023-09-20T22:05:44.914 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:44.915 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 773, 2023-09-20T22:05:44.915 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-20T22:05:44.915 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-20T22:05:44.915 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 262, 2023-09-20T22:05:44.915 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1108741, 2023-09-20T22:05:44.916 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:44.916 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:44.916 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:44.916 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:44.916 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:44.917 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-20T22:05:45.279 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-09-20T22:05:45.279 INFO:tasks.check_counter:Daemon mds.f mds.imported=7 2023-09-20T22:05:45.279 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=25 2023-09-20T22:05:45.280 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=425 2023-09-20T22:05:45.280 DEBUG:tasks.check_counter:Getting stats from i 2023-09-20T22:05:45.280 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.i perf dump 2023-09-20T22:05:45.838 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T22:05:45.997 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:45 smithi022 ceph-mon[97651]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2023-09-20T22:05:46.075 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:45 smithi123 ceph-mon[134433]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2023-09-20T22:05:46.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:45 smithi089 ceph-mon[134090]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2023-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1265, 2023-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1267, 2023-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 4608033, 2023-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 393425, 2023-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2023-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2023-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.125448955, 2023-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.032202293, 2023-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.034777855, 2023-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.046753871, 2023-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1265, 2023-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.390664403, 2023-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000308825 2023-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2023-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2023-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 335, 2023-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 324, 2023-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 303976, 2023-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 189598, 2023-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2023-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2023-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.042531777, 2023-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.016016243, 2023-09-20T22:05:47.244 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.014805233, 2023-09-20T22:05:47.244 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.008496270, 2023-09-20T22:05:47.244 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:47.244 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 323, 2023-09-20T22:05:47.244 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.081727601, 2023-09-20T22:05:47.245 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000253026 2023-09-20T22:05:47.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.245 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:47.245 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.246 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.246 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.246 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 302320, 2023-09-20T22:05:47.246 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 188288 2023-09-20T22:05:47.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.247 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:47.247 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2353, 2023-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2585, 2023-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3034973, 2023-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 1210693, 2023-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 5, 2023-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2023-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.182651365, 2023-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.057565212, 2023-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.042898330, 2023-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.069023304, 2023-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2584, 2023-09-20T22:05:47.250 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.696363905, 2023-09-20T22:05:47.250 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000269490 2023-09-20T22:05:47.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:47.251 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.251 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.251 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 480464 2023-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2023-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2023-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2023-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2023-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6037, 2023-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.162526583, 2023-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000026921 2023-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2023-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2023-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2023-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2023-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2023-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2023-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2023-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2023-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2023-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2023-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2023-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2023-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2023-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2023-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 261, 2023-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 0, 2023-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 0, 2023-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 261, 2023-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 261, 2023-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2023-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2023-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2023-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 29, 2023-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2023-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2023-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2023-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2023-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2023-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2023-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2023-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2023-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2023-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2023-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2023-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2023-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2023-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2023-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2023-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2023-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 1033, 2023-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 1632664011, 2023-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2023-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2023-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2023-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2023-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2023-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2023-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2023-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2023-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2023-09-20T22:05:47.269 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2023-09-20T22:05:47.269 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2023-09-20T22:05:47.269 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2023-09-20T22:05:47.269 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2023-09-20T22:05:47.269 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-09-20T22:05:47.270 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2023-09-20T22:05:47.270 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2023-09-20T22:05:47.270 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2023-09-20T22:05:47.270 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2023-09-20T22:05:47.270 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2023-09-20T22:05:47.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.271 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2023-09-20T22:05:47.271 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2023-09-20T22:05:47.271 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2023-09-20T22:05:47.271 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2023-09-20T22:05:47.272 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2023-09-20T22:05:47.272 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2023-09-20T22:05:47.272 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2023-09-20T22:05:47.272 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2023-09-20T22:05:47.273 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2023-09-20T22:05:47.273 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2023-09-20T22:05:47.273 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2023-09-20T22:05:47.273 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2023-09-20T22:05:47.273 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2023-09-20T22:05:47.274 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2023-09-20T22:05:47.274 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2023-09-20T22:05:47.274 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2023-09-20T22:05:47.274 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2023-09-20T22:05:47.274 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2023-09-20T22:05:47.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2023-09-20T22:05:47.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2023-09-20T22:05:47.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2023-09-20T22:05:47.275 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2023-09-20T22:05:47.276 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2023-09-20T22:05:47.276 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2023-09-20T22:05:47.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.276 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2023-09-20T22:05:47.276 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2023-09-20T22:05:47.276 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2023-09-20T22:05:47.277 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2023-09-20T22:05:47.277 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2023-09-20T22:05:47.277 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 2876, 2023-09-20T22:05:47.277 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2023-09-20T22:05:47.277 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2023-09-20T22:05:47.278 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2023-09-20T22:05:47.278 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2023-09-20T22:05:47.278 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2023-09-20T22:05:47.278 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 4, 2023-09-20T22:05:47.278 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2023-09-20T22:05:47.279 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2023-09-20T22:05:47.279 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2023-09-20T22:05:47.279 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2023-09-20T22:05:47.279 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 10827086, 2023-09-20T22:05:47.280 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 10825050, 2023-09-20T22:05:47.280 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2023-09-20T22:05:47.280 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.280 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.281 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.281 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 2876 2023-09-20T22:05:47.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.282 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2023-09-20T22:05:47.282 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 191, 2023-09-20T22:05:47.282 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 1069, 2023-09-20T22:05:47.283 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 878, 2023-09-20T22:05:47.283 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 50, 2023-09-20T22:05:47.283 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 58, 2023-09-20T22:05:47.283 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 8, 2023-09-20T22:05:47.284 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 261, 2023-09-20T22:05:47.284 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 1128, 2023-09-20T22:05:47.284 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 867, 2023-09-20T22:05:47.284 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2023-09-20T22:05:47.285 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2023-09-20T22:05:47.285 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2023-09-20T22:05:47.285 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 50504, 2023-09-20T22:05:47.285 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 207152 2023-09-20T22:05:47.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.286 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2023-09-20T22:05:47.286 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2023-09-20T22:05:47.286 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2023-09-20T22:05:47.286 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2023-09-20T22:05:47.287 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2023-09-20T22:05:47.287 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2023-09-20T22:05:47.287 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2023-09-20T22:05:47.287 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.287 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.288 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2023-09-20T22:05:47.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.288 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.289 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2023-09-20T22:05:47.289 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.289 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.289 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.290 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2023-09-20T22:05:47.290 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.290 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.290 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.291 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2023-09-20T22:05:47.291 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.291 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.291 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.292 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2023-09-20T22:05:47.292 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.292 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.292 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.293 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2023-09-20T22:05:47.293 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.293 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.294 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.294 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2023-09-20T22:05:47.294 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.294 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.295 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.295 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2023-09-20T22:05:47.295 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.295 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.296 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.296 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2023-09-20T22:05:47.296 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.296 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.297 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.297 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2023-09-20T22:05:47.297 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.298 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.298 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.299 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2023-09-20T22:05:47.299 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.299 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.299 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.300 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2023-09-20T22:05:47.300 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.300 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.301 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.301 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2023-09-20T22:05:47.302 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.302 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.303 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.303 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2023-09-20T22:05:47.304 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.304 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.304 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.305 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2023-09-20T22:05:47.305 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.306 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.306 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.307 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2023-09-20T22:05:47.307 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.308 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.308 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.309 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2023-09-20T22:05:47.309 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.309 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.310 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.311 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2023-09-20T22:05:47.311 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.311 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.312 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.312 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2023-09-20T22:05:47.313 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.313 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.313 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.314 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2023-09-20T22:05:47.314 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.314 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.314 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.315 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2023-09-20T22:05:47.315 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.315 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.315 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.316 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2023-09-20T22:05:47.316 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.316 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.316 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.317 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2023-09-20T22:05:47.317 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.317 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.317 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.318 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2023-09-20T22:05:47.318 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.318 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.318 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.319 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2023-09-20T22:05:47.319 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.319 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.319 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.320 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2023-09-20T22:05:47.320 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.320 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.320 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.321 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2023-09-20T22:05:47.321 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.321 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.321 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.321 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2023-09-20T22:05:47.322 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.322 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.322 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.323 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2023-09-20T22:05:47.323 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2023-09-20T22:05:47.323 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2023-09-20T22:05:47.323 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.325 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.325 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.325 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:47.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.326 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2023-09-20T22:05:47.326 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2023-09-20T22:05:47.327 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 2, 2023-09-20T22:05:47.327 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 1, 2023-09-20T22:05:47.328 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2023-09-20T22:05:47.328 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2023-09-20T22:05:47.328 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2023-09-20T22:05:47.329 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2023-09-20T22:05:47.329 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 2882, 2023-09-20T22:05:47.329 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2023-09-20T22:05:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2023-09-20T22:05:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:47.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:47.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:47.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:47.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:47.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:47.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:47.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:47.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:47.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:47.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:47.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:47.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:47.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:47.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:47.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:47.334 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:47.334 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:47.334 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:47.335 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:47.335 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:47.335 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:47.335 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:47.336 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:47.336 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:47.336 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:47.337 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:47.337 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:47.337 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:47.337 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2023-09-20T22:05:47.337 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:47.338 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:47.338 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:47.338 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:47.338 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 205876, 2023-09-20T22:05:47.338 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 243, 2023-09-20T22:05:47.339 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2023-09-20T22:05:47.339 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2023-09-20T22:05:47.339 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2023-09-20T22:05:47.339 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2023-09-20T22:05:47.340 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:47.340 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:47.340 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:47.340 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:47.340 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 9088, 2023-09-20T22:05:47.340 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 175, 2023-09-20T22:05:47.341 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:47.341 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:47.341 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:47.341 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2023-09-20T22:05:47.341 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 682510, 2023-09-20T22:05:47.342 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 5865, 2023-09-20T22:05:47.342 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:47.342 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2023-09-20T22:05:47.342 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:47.342 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2023-09-20T22:05:47.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.343 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2023-09-20T22:05:47.343 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2023-09-20T22:05:47.343 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2023-09-20T22:05:47.343 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 3617, 2023-09-20T22:05:47.343 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 26, 2023-09-20T22:05:47.344 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 4, 2023-09-20T22:05:47.344 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 3617, 2023-09-20T22:05:47.344 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2023-09-20T22:05:47.344 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3617, 2023-09-20T22:05:47.344 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 13.005702896, 2023-09-20T22:05:47.345 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003595715 2023-09-20T22:05:47.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.345 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2023-09-20T22:05:47.345 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2023-09-20T22:05:47.345 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3617, 2023-09-20T22:05:47.346 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3618 2023-09-20T22:05:47.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.346 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 3617, 2023-09-20T22:05:47.346 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 3617, 2023-09-20T22:05:47.347 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2023-09-20T22:05:47.347 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2023-09-20T22:05:47.347 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2023-09-20T22:05:47.347 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 2214, 2023-09-20T22:05:47.348 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2023-09-20T22:05:47.348 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 1400, 2023-09-20T22:05:47.348 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2023-09-20T22:05:47.348 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2023-09-20T22:05:47.349 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2023-09-20T22:05:47.349 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2023-09-20T22:05:47.349 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2023-09-20T22:05:47.349 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2023-09-20T22:05:47.349 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2023-09-20T22:05:47.350 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2023-09-20T22:05:47.350 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2023-09-20T22:05:47.350 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2023-09-20T22:05:47.350 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 2, 2023-09-20T22:05:47.350 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2023-09-20T22:05:47.351 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2023-09-20T22:05:47.351 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2023-09-20T22:05:47.351 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2023-09-20T22:05:47.351 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2023-09-20T22:05:47.351 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2023-09-20T22:05:47.351 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2023-09-20T22:05:47.352 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2023-09-20T22:05:47.352 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2023-09-20T22:05:47.352 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2023-09-20T22:05:47.352 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2023-09-20T22:05:47.352 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2023-09-20T22:05:47.353 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2023-09-20T22:05:47.353 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2023-09-20T22:05:47.353 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2023-09-20T22:05:47.353 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2023-09-20T22:05:47.353 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2023-09-20T22:05:47.353 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2023-09-20T22:05:47.354 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2023-09-20T22:05:47.354 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2023-09-20T22:05:47.354 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2023-09-20T22:05:47.354 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2023-09-20T22:05:47.354 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2023-09-20T22:05:47.355 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2023-09-20T22:05:47.355 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2023-09-20T22:05:47.355 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2023-09-20T22:05:47.355 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2023-09-20T22:05:47.355 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 93, 2023-09-20T22:05:47.356 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2023-09-20T22:05:47.356 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 4, 2023-09-20T22:05:47.356 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 6, 2023-09-20T22:05:47.356 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 8, 2023-09-20T22:05:47.356 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 2, 2023-09-20T22:05:47.357 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2023-09-20T22:05:47.357 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2023-09-20T22:05:47.357 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2023-09-20T22:05:47.357 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2023-09-20T22:05:47.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.358 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2023-09-20T22:05:47.358 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2023-09-20T22:05:47.358 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2023-09-20T22:05:47.359 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2023-09-20T22:05:47.359 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2023-09-20T22:05:47.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.359 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2023-09-20T22:05:47.359 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2023-09-20T22:05:47.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.360 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2023-09-20T22:05:47.360 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2023-09-20T22:05:47.360 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2023-09-20T22:05:47.360 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2023-09-20T22:05:47.361 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2023-09-20T22:05:47.361 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2023-09-20T22:05:47.361 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2023-09-20T22:05:47.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.361 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:47.362 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:47.362 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-20T22:05:47.362 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:47.362 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3949, 2023-09-20T22:05:47.362 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 7571576, 2023-09-20T22:05:47.363 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:47.363 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3949, 2023-09-20T22:05:47.363 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-20T22:05:47.363 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-20T22:05:47.363 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3949, 2023-09-20T22:05:47.364 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 7571576, 2023-09-20T22:05:47.364 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:47.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.364 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.364 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:47.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.365 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2023-09-20T22:05:47.365 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:47.365 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-20T22:05:47.365 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:47.366 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-20T22:05:47.366 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-20T22:05:47.366 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:47.366 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:47.366 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3617, 2023-09-20T22:05:47.367 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 6632792, 2023-09-20T22:05:47.367 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3617, 2023-09-20T22:05:47.367 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 6632792, 2023-09-20T22:05:47.367 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:47.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.368 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.368 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.368 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:47.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.369 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2023-09-20T22:05:47.369 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:47.369 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2023-09-20T22:05:47.369 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:47.369 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-20T22:05:47.370 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-20T22:05:47.370 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:47.370 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:47.370 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 3617, 2023-09-20T22:05:47.371 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 3617, 2023-09-20T22:05:47.371 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 3617, 2023-09-20T22:05:47.372 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 3617, 2023-09-20T22:05:47.372 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:47.372 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.372 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.373 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.373 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:47.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.374 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2023-09-20T22:05:47.374 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:47.375 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-20T22:05:47.375 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:47.376 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-20T22:05:47.376 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-20T22:05:47.376 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:47.377 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:47.377 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-20T22:05:47.377 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-20T22:05:47.378 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-09-20T22:05:47.378 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-09-20T22:05:47.379 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:47.379 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.379 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.380 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.380 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:47.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:47.381 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x5639705e2960": { 2023-09-20T22:05:47.381 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:47.381 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2023-09-20T22:05:47.381 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:47.382 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2023-09-20T22:05:47.382 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2023-09-20T22:05:47.382 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:47.382 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2023-09-20T22:05:47.382 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-20T22:05:47.383 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-20T22:05:47.383 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2023-09-20T22:05:47.383 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2023-09-20T22:05:47.383 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:47.383 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:47.384 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:47.384 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:47.384 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:47.384 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:47.384 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-20T22:05:47.683 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2023-09-20T22:05:47.684 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2023-09-20T22:05:47.684 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2023-09-20T22:05:47.684 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2023-09-20T22:05:47.684 DEBUG:tasks.check_counter:Getting stats from l 2023-09-20T22:05:47.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bf8a061ff16ac1d393cf108f740d92dfa49f2e54 shell --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da -- ceph daemon mds.l perf dump 2023-09-20T22:05:48.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:47 smithi123 ceph-mon[134433]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2023-09-20T22:05:48.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:47 smithi089 ceph-mon[134090]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2023-09-20T22:05:48.238 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/mon.c/config 2023-09-20T22:05:48.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:47 smithi022 ceph-mon[97651]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2023-09-20T22:05:49.659 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-09-20T22:05:49.659 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2023-09-20T22:05:49.660 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4, 2023-09-20T22:05:49.660 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 4, 2023-09-20T22:05:49.660 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 2992, 2023-09-20T22:05:49.660 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 448, 2023-09-20T22:05:49.660 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2023-09-20T22:05:49.661 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2023-09-20T22:05:49.661 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.001349348, 2023-09-20T22:05:49.661 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000028539, 2023-09-20T22:05:49.661 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000532501, 2023-09-20T22:05:49.661 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-20T22:05:49.661 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:49.662 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3, 2023-09-20T22:05:49.662 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000658561, 2023-09-20T22:05:49.662 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000219520 2023-09-20T22:05:49.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.662 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:49.663 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:49.663 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:49.663 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:49.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.663 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-09-20T22:05:49.664 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2023-09-20T22:05:49.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.664 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2023-09-20T22:05:49.664 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 316, 2023-09-20T22:05:49.664 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 315, 2023-09-20T22:05:49.664 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 228832, 2023-09-20T22:05:49.665 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 189712, 2023-09-20T22:05:49.665 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2023-09-20T22:05:49.665 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2023-09-20T22:05:49.665 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.039180651, 2023-09-20T22:05:49.665 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.015325044, 2023-09-20T22:05:49.665 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.011369916, 2023-09-20T22:05:49.666 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.008498170, 2023-09-20T22:05:49.666 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:49.666 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 314, 2023-09-20T22:05:49.666 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.084624580, 2023-09-20T22:05:49.666 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000269505 2023-09-20T22:05:49.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.667 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:49.667 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:49.667 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:49.667 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:49.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.668 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 228832, 2023-09-20T22:05:49.668 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 189712 2023-09-20T22:05:49.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.668 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2023-09-20T22:05:49.668 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2, 2023-09-20T22:05:49.668 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 241, 2023-09-20T22:05:49.669 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 320, 2023-09-20T22:05:49.669 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 195632, 2023-09-20T22:05:49.669 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2023-09-20T22:05:49.669 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2023-09-20T22:05:49.669 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.011128289, 2023-09-20T22:05:49.670 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.008130537, 2023-09-20T22:05:49.670 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000956290, 2023-09-20T22:05:49.670 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-09-20T22:05:49.670 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2023-09-20T22:05:49.670 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 239, 2023-09-20T22:05:49.670 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.074399329, 2023-09-20T22:05:49.671 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000311294 2023-09-20T22:05:49.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.671 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2023-09-20T22:05:49.671 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:49.671 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:49.672 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:49.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.672 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2023-09-20T22:05:49.672 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 195632 2023-09-20T22:05:49.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.672 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2023-09-20T22:05:49.673 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2023-09-20T22:05:49.673 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2023-09-20T22:05:49.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.673 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2023-09-20T22:05:49.673 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2023-09-20T22:05:49.673 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2023-09-20T22:05:49.674 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2023-09-20T22:05:49.674 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2023-09-20T22:05:49.674 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2023-09-20T22:05:49.674 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2023-09-20T22:05:49.674 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2023-09-20T22:05:49.675 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2023-09-20T22:05:49.675 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2023-09-20T22:05:49.675 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2023-09-20T22:05:49.675 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2023-09-20T22:05:49.675 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2023-09-20T22:05:49.675 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2023-09-20T22:05:49.676 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2023-09-20T22:05:49.676 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2023-09-20T22:05:49.676 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2023-09-20T22:05:49.676 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2023-09-20T22:05:49.676 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2023-09-20T22:05:49.676 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2023-09-20T22:05:49.677 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2023-09-20T22:05:49.677 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2023-09-20T22:05:49.677 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2023-09-20T22:05:49.677 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2023-09-20T22:05:49.677 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2023-09-20T22:05:49.677 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2023-09-20T22:05:49.678 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2023-09-20T22:05:49.678 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2023-09-20T22:05:49.678 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2023-09-20T22:05:49.678 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2023-09-20T22:05:49.678 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2023-09-20T22:05:49.678 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2023-09-20T22:05:49.679 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2023-09-20T22:05:49.679 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2023-09-20T22:05:49.679 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2023-09-20T22:05:49.679 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2023-09-20T22:05:49.679 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2023-09-20T22:05:49.679 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148744, 2023-09-20T22:05:49.680 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 30, 2023-09-20T22:05:49.680 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2023-09-20T22:05:49.680 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2023-09-20T22:05:49.680 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2023-09-20T22:05:49.680 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2023-09-20T22:05:49.681 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2023-09-20T22:05:49.681 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2023-09-20T22:05:49.681 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2023-09-20T22:05:49.681 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2023-09-20T22:05:49.681 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2023-09-20T22:05:49.681 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2023-09-20T22:05:49.682 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2023-09-20T22:05:49.682 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2023-09-20T22:05:49.682 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2023-09-20T22:05:49.682 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2023-09-20T22:05:49.682 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2023-09-20T22:05:49.683 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2023-09-20T22:05:49.683 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2023-09-20T22:05:49.683 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2023-09-20T22:05:49.683 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2023-09-20T22:05:49.683 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2023-09-20T22:05:49.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-09-20T22:05:49.684 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-09-20T22:05:49.684 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2023-09-20T22:05:49.684 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2023-09-20T22:05:49.684 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2023-09-20T22:05:49.684 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 318, 2023-09-20T22:05:49.684 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 184150, 2023-09-20T22:05:49.685 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2023-09-20T22:05:49.685 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 318, 2023-09-20T22:05:49.685 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2023-09-20T22:05:49.685 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2023-09-20T22:05:49.685 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 318, 2023-09-20T22:05:49.686 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 184150, 2023-09-20T22:05:49.686 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2023-09-20T22:05:49.686 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2023-09-20T22:05:49.686 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2023-09-20T22:05:49.686 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2023-09-20T22:05:49.686 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:49.687 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-09-20T22:05:49.687 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-09-20T22:05:50.089 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:49 smithi123 ceph-mon[134433]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-20T22:05:50.090 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:49 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:05:50.091 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-09-20T22:05:50.091 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-09-20T22:05:50.091 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-09-20T22:05:50.092 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-09-20T22:05:50.092 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-09-20T22:05:50.102 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-20T22:05:50.103 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T22:05:50.103 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-20T22:05:50.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:49 smithi089 ceph-mon[134090]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-20T22:05:50.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:49 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:05:50.170 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T22:05:50.170 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-20T22:05:50.220 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T22:05:50.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:49 smithi022 ceph-mon[97651]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-09-20T22:05:50.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:49 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-09-20T22:05:50.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.484+0000 7f32f9bff700 1 -- 172.21.15.22:0/1088109082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f4101fd0 msgr2=0x7f32f4102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:50.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.484+0000 7f32f9bff700 1 --2- 172.21.15.22:0/1088109082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f4101fd0 0x7f32f4102430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32e80099c0 tx=0x7f32e8009cd0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.485+0000 7f32f9bff700 1 -- 172.21.15.22:0/1088109082 shutdown_connections 2023-09-20T22:05:50.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.485+0000 7f32f9bff700 1 --2- 172.21.15.22:0/1088109082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4102970 0x7f32f4109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.485+0000 7f32f9bff700 1 --2- 172.21.15.22:0/1088109082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f4100de0 0x7f32f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.485+0000 7f32f9bff700 1 --2- 172.21.15.22:0/1088109082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f4101fd0 0x7f32f4102430 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.485+0000 7f32f9bff700 1 -- 172.21.15.22:0/1088109082 >> 172.21.15.22:0/1088109082 conn(0x7f32f40fc370 msgr2=0x7f32f40fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:50.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.485+0000 7f32f9bff700 1 -- 172.21.15.22:0/1088109082 shutdown_connections 2023-09-20T22:05:50.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.485+0000 7f32f9bff700 1 -- 172.21.15.22:0/1088109082 wait complete. 2023-09-20T22:05:50.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.486+0000 7f32f9bff700 1 Processor -- start 2023-09-20T22:05:50.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.486+0000 7f32f9bff700 1 -- start start 2023-09-20T22:05:50.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f9bff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f4100de0 0x7f32f4196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:50.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f9bff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4101fd0 0x7f32f4196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:50.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f9bff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f4102970 0x7f32f419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:50.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f9bff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f419e610 con 0x7f32f4101fd0 2023-09-20T22:05:50.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f9bff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f419e750 con 0x7f32f4102970 2023-09-20T22:05:50.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f9bff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f419e8e0 con 0x7f32f4100de0 2023-09-20T22:05:50.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f4102970 0x7f32f419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:50.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f4100de0 0x7f32f4196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:50.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f3fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f4102970 0x7f32f419e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60488/0 (socket says 172.21.15.22:60488) 2023-09-20T22:05:50.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4101fd0 0x7f32f4196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:50.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f3fff700 1 -- 172.21.15.22:0/426736305 learned_addr learned my addr 172.21.15.22:0/426736305 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:50.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f3fff700 1 -- 172.21.15.22:0/426736305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f4100de0 msgr2=0x7f32f4196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:50.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f3fff700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f4100de0 0x7f32f4196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f3fff700 1 -- 172.21.15.22:0/426736305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4101fd0 msgr2=0x7f32f4196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:50.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f3fff700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4101fd0 0x7f32f4196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.487+0000 7f32f3fff700 1 -- 172.21.15.22:0/426736305 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e80095f0 con 0x7f32f4102970 2023-09-20T22:05:50.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.488+0000 7f32f3fff700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f4102970 0x7f32f419e040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00ef30 tx=0x7f32ec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:50.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.488+0000 7f32f0ff9700 1 -- 172.21.15.22:0/426736305 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec018070 con 0x7f32f4102970 2023-09-20T22:05:50.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.488+0000 7f32f0ff9700 1 -- 172.21.15.22:0/426736305 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32ec00f040 con 0x7f32f4102970 2023-09-20T22:05:50.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.488+0000 7f32f9bff700 1 -- 172.21.15.22:0/426736305 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f419eb60 con 0x7f32f4102970 2023-09-20T22:05:50.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.488+0000 7f32f0ff9700 1 -- 172.21.15.22:0/426736305 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec00cd00 con 0x7f32f4102970 2023-09-20T22:05:50.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.488+0000 7f32f9bff700 1 -- 172.21.15.22:0/426736305 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f419f0c0 con 0x7f32f4102970 2023-09-20T22:05:50.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.489+0000 7f32f9bff700 1 -- 172.21.15.22:0/426736305 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f4107cc0 con 0x7f32f4102970 2023-09-20T22:05:50.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.492+0000 7f32f0ff9700 1 -- 172.21.15.22:0/426736305 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f32ec003a00 con 0x7f32f4102970 2023-09-20T22:05:50.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.493+0000 7f32f0ff9700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f32dc0a13d0 0x7f32dc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:50.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.494+0000 7f32f0ff9700 1 -- 172.21.15.22:0/426736305 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f32ec0cca20 con 0x7f32f4102970 2023-09-20T22:05:50.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.494+0000 7f32f37fe700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f32dc0a13d0 0x7f32dc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:50.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.494+0000 7f32f37fe700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f32dc0a13d0 0x7f32dc0a3890 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f32f4101e30 tx=0x7f32e4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:50.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.498+0000 7f32f0ff9700 1 -- 172.21.15.22:0/426736305 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f32ec095df0 con 0x7f32f4102970 2023-09-20T22:05:50.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.771+0000 7f32f9bff700 1 -- 172.21.15.22:0/426736305 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32f410c640 con 0x7f32f4102970 2023-09-20T22:05:50.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.773+0000 7f32f0ff9700 1 -- 172.21.15.22:0/426736305 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7f32ec099410 con 0x7f32f4102970 2023-09-20T22:05:50.774 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:05:50.774 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T22:05:50.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.775+0000 7f32f9bff700 1 -- 172.21.15.22:0/426736305 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f32dc0a13d0 msgr2=0x7f32dc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.775+0000 7f32f9bff700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f32dc0a13d0 0x7f32dc0a3890 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f32f4101e30 tx=0x7f32e4009260 comp rx=0 tx=0).stop 2023-09-20T22:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.775+0000 7f32f9bff700 1 -- 172.21.15.22:0/426736305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f4102970 msgr2=0x7f32f419e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:50.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.775+0000 7f32f9bff700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f4102970 0x7f32f419e040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00ef30 tx=0x7f32ec00c3a0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.775+0000 7f32f9bff700 1 -- 172.21.15.22:0/426736305 shutdown_connections 2023-09-20T22:05:50.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.775+0000 7f32f9bff700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f32dc0a13d0 0x7f32dc0a3890 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.775+0000 7f32f9bff700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32f4102970 0x7f32f419e040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.775+0000 7f32f9bff700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32f4100de0 0x7f32f4196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.775+0000 7f32f9bff700 1 --2- 172.21.15.22:0/426736305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4101fd0 0x7f32f4196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:50.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.776+0000 7f32f9bff700 1 -- 172.21.15.22:0/426736305 >> 172.21.15.22:0/426736305 conn(0x7f32f40fc370 msgr2=0x7f32f40fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:50.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.776+0000 7f32f9bff700 1 -- 172.21.15.22:0/426736305 shutdown_connections 2023-09-20T22:05:50.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:50.776+0000 7f32f9bff700 1 -- 172.21.15.22:0/426736305 wait complete. 2023-09-20T22:05:50.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-20T21:42:25.633235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-20T21:46:38.981332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-20T21:46:40.478907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-20T21:46:44.069558+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-20T22:05:50.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-20T22:05:51.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:50 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:50 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:50 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:50 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:50 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:05:51.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:50 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:05:51.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:50 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:05:51.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.099+0000 7f35abdff700 1 -- 172.21.15.22:0/3742398971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a4101fc0 msgr2=0x7f35a4102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:51.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.099+0000 7f35abdff700 1 --2- 172.21.15.22:0/3742398971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a4101fc0 0x7f35a4102420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3594009990 tx=0x7f3594009ca0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.100+0000 7f35abdff700 1 -- 172.21.15.22:0/3742398971 shutdown_connections 2023-09-20T22:05:51.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.100+0000 7f35abdff700 1 --2- 172.21.15.22:0/3742398971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a4102960 0x7f35a4109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.100+0000 7f35abdff700 1 --2- 172.21.15.22:0/3742398971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35a4100dd0 0x7f35a41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.100+0000 7f35abdff700 1 --2- 172.21.15.22:0/3742398971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a4101fc0 0x7f35a4102420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.100+0000 7f35abdff700 1 -- 172.21.15.22:0/3742398971 >> 172.21.15.22:0/3742398971 conn(0x7f35a40fc360 msgr2=0x7f35a40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:51.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.100+0000 7f35abdff700 1 -- 172.21.15.22:0/3742398971 shutdown_connections 2023-09-20T22:05:51.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.100+0000 7f35abdff700 1 -- 172.21.15.22:0/3742398971 wait complete. 2023-09-20T22:05:51.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.101+0000 7f35abdff700 1 Processor -- start 2023-09-20T22:05:51.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.101+0000 7f35abdff700 1 -- start start 2023-09-20T22:05:51.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.101+0000 7f35abdff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a4100dd0 0x7f35a4198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:51.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.101+0000 7f35abdff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35a4101fc0 0x7f35a4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:51.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.101+0000 7f35abdff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a4102960 0x7f35a41a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:51.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.101+0000 7f35abdff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a41a0850 con 0x7f35a4101fc0 2023-09-20T22:05:51.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.101+0000 7f35abdff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a41a09c0 con 0x7f35a4100dd0 2023-09-20T22:05:51.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.101+0000 7f35abdff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a41a0b80 con 0x7f35a4102960 2023-09-20T22:05:51.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35a9b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a4100dd0 0x7f35a4198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:51.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35aa39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a4102960 0x7f35a41a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:51.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35aa39c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a4102960 0x7f35a41a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:36900/0 (socket says 172.21.15.22:36900) 2023-09-20T22:05:51.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35a9b9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a4100dd0 0x7f35a4198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60520/0 (socket says 172.21.15.22:60520) 2023-09-20T22:05:51.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35aa39c700 1 -- 172.21.15.22:0/376090232 learned_addr learned my addr 172.21.15.22:0/376090232 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:51.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35a939a700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35a4101fc0 0x7f35a4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:51.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35a9b9b700 1 -- 172.21.15.22:0/376090232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a4102960 msgr2=0x7f35a41a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:51.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35a9b9b700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a4102960 0x7f35a41a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35a9b9b700 1 -- 172.21.15.22:0/376090232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35a4101fc0 msgr2=0x7f35a4199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:51.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35a9b9b700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35a4101fc0 0x7f35a4199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35a9b9b700 1 -- 172.21.15.22:0/376090232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c009c40 con 0x7f35a4100dd0 2023-09-20T22:05:51.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.102+0000 7f35a9b9b700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a4100dd0 0x7f35a4198d10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f35a000e9c0 tx=0x7f35a000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:51.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.103+0000 7f359affd700 1 -- 172.21.15.22:0/376090232 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0004220 con 0x7f35a4100dd0 2023-09-20T22:05:51.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.103+0000 7f359affd700 1 -- 172.21.15.22:0/376090232 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a0004380 con 0x7f35a4100dd0 2023-09-20T22:05:51.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.103+0000 7f35abdff700 1 -- 172.21.15.22:0/376090232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35940095f0 con 0x7f35a4100dd0 2023-09-20T22:05:51.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.103+0000 7f359affd700 1 -- 172.21.15.22:0/376090232 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0003be0 con 0x7f35a4100dd0 2023-09-20T22:05:51.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.103+0000 7f35abdff700 1 -- 172.21.15.22:0/376090232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a41a1190 con 0x7f35a4100dd0 2023-09-20T22:05:51.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.104+0000 7f35abdff700 1 -- 172.21.15.22:0/376090232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a4193420 con 0x7f35a4100dd0 2023-09-20T22:05:51.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.107+0000 7f359affd700 1 -- 172.21.15.22:0/376090232 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f35a0003d40 con 0x7f35a4100dd0 2023-09-20T22:05:51.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.109+0000 7f359affd700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f35900a1300 0x7f35900a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:51.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.109+0000 7f359affd700 1 -- 172.21.15.22:0/376090232 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f35a0014070 con 0x7f35a4100dd0 2023-09-20T22:05:51.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.109+0000 7f35a939a700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f35900a1300 0x7f35900a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:51.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:50 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:50 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:50 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:50 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:50 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:05:51.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:50 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:05:51.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:50 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:05:51.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.110+0000 7f35a939a700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f35900a1300 0x7f35900a37c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f359400b2e0 tx=0x7f3594005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:51.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.113+0000 7f359affd700 1 -- 172.21.15.22:0/376090232 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f35a0096590 con 0x7f35a4100dd0 2023-09-20T22:05:51.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:50 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:50 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:50 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:50 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-09-20T22:05:51.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:50 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-09-20T22:05:51.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:50 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-09-20T22:05:51.249 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:50 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' 2023-09-20T22:05:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.351+0000 7f35abdff700 1 -- 172.21.15.22:0/376090232 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35a410c630 con 0x7f35a4100dd0 2023-09-20T22:05:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.351+0000 7f359affd700 1 -- 172.21.15.22:0/376090232 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35a410c630 con 0x7f35a4100dd0 2023-09-20T22:05:51.352 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-09-20T22:05:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 -- 172.21.15.22:0/376090232 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f35900a1300 msgr2=0x7f35900a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f35900a1300 0x7f35900a37c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f359400b2e0 tx=0x7f3594005fb0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 -- 172.21.15.22:0/376090232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a4100dd0 msgr2=0x7f35a4198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a4100dd0 0x7f35a4198d10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f35a000e9c0 tx=0x7f35a000ecd0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 -- 172.21.15.22:0/376090232 shutdown_connections 2023-09-20T22:05:51.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a4102960 0x7f35a41a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f35900a1300 0x7f35900a37c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a4100dd0 0x7f35a4198d10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 --2- 172.21.15.22:0/376090232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35a4101fc0 0x7f35a4199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 -- 172.21.15.22:0/376090232 >> 172.21.15.22:0/376090232 conn(0x7f35a40fc360 msgr2=0x7f35a4107900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 -- 172.21.15.22:0/376090232 shutdown_connections 2023-09-20T22:05:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.354+0000 7f35abdff700 1 -- 172.21.15.22:0/376090232 wait complete. 2023-09-20T22:05:51.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-20T22:05:51.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.624+0000 7fb8641ff700 1 -- 172.21.15.22:0/169401792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb85c1041f0 msgr2=0x7fb85c104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:51.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.624+0000 7fb8641ff700 1 --2- 172.21.15.22:0/169401792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb85c1041f0 0x7fb85c104650 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0099c0 tx=0x7fb84c009cd0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.624+0000 7fb8641ff700 1 -- 172.21.15.22:0/169401792 shutdown_connections 2023-09-20T22:05:51.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.624+0000 7fb8641ff700 1 --2- 172.21.15.22:0/169401792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c104b90 0x7fb85c10b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.624+0000 7fb8641ff700 1 --2- 172.21.15.22:0/169401792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c103000 0x7fb85c103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.624+0000 7fb8641ff700 1 --2- 172.21.15.22:0/169401792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb85c1041f0 0x7fb85c104650 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.624+0000 7fb8641ff700 1 -- 172.21.15.22:0/169401792 >> 172.21.15.22:0/169401792 conn(0x7fb85c0fe590 msgr2=0x7fb85c1009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:51.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.625+0000 7fb8641ff700 1 -- 172.21.15.22:0/169401792 shutdown_connections 2023-09-20T22:05:51.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.625+0000 7fb8641ff700 1 -- 172.21.15.22:0/169401792 wait complete. 2023-09-20T22:05:51.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb8641ff700 1 Processor -- start 2023-09-20T22:05:51.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb8641ff700 1 -- start start 2023-09-20T22:05:51.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb8641ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c103000 0x7fb85c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb8641ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c1041f0 0x7fb85c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb8641ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb85c104b90 0x7fb85c1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb8641ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1a08c0 con 0x7fb85c1041f0 2023-09-20T22:05:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb8641ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1a0a00 con 0x7fb85c104b90 2023-09-20T22:05:51.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb8641ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1a0b90 con 0x7fb85c103000 2023-09-20T22:05:51.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb86279c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb85c104b90 0x7fb85c1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:51.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb86279c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb85c104b90 0x7fb85c1a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60550/0 (socket says 172.21.15.22:60550) 2023-09-20T22:05:51.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb86279c700 1 -- 172.21.15.22:0/1721652277 learned_addr learned my addr 172.21.15.22:0/1721652277 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:51.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb86179a700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c1041f0 0x7fb85c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:51.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.626+0000 7fb861f9b700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c103000 0x7fb85c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.627+0000 7fb86279c700 1 -- 172.21.15.22:0/1721652277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c103000 msgr2=0x7fb85c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.627+0000 7fb86279c700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c103000 0x7fb85c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.627+0000 7fb86279c700 1 -- 172.21.15.22:0/1721652277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c1041f0 msgr2=0x7fb85c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.627+0000 7fb86279c700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c1041f0 0x7fb85c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.627+0000 7fb86279c700 1 -- 172.21.15.22:0/1721652277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c0095f0 con 0x7fb85c104b90 2023-09-20T22:05:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.627+0000 7fb86279c700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb85c104b90 0x7fb85c1a02f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb854007ee0 tx=0x7fb85400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.627+0000 7fb852ffd700 1 -- 172.21.15.22:0/1721652277 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85400dd20 con 0x7fb85c104b90 2023-09-20T22:05:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.627+0000 7fb852ffd700 1 -- 172.21.15.22:0/1721652277 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb85400f040 con 0x7fb85c104b90 2023-09-20T22:05:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.627+0000 7fb8641ff700 1 -- 172.21.15.22:0/1721652277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c1a0e70 con 0x7fb85c104b90 2023-09-20T22:05:51.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.627+0000 7fb852ffd700 1 -- 172.21.15.22:0/1721652277 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854015400 con 0x7fb85c104b90 2023-09-20T22:05:51.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.628+0000 7fb8641ff700 1 -- 172.21.15.22:0/1721652277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c1a1340 con 0x7fb85c104b90 2023-09-20T22:05:51.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.629+0000 7fb8641ff700 1 -- 172.21.15.22:0/1721652277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c10e870 con 0x7fb85c104b90 2023-09-20T22:05:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.631+0000 7fb852ffd700 1 -- 172.21.15.22:0/1721652277 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fb854003a00 con 0x7fb85c104b90 2023-09-20T22:05:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.633+0000 7fb852ffd700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb8480a1380 0x7fb8480a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.633+0000 7fb852ffd700 1 -- 172.21.15.22:0/1721652277 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fb8540cc900 con 0x7fb85c104b90 2023-09-20T22:05:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.633+0000 7fb861f9b700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb8480a1380 0x7fb8480a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:51.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.634+0000 7fb852ffd700 1 -- 172.21.15.22:0/1721652277 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fb854095be0 con 0x7fb85c104b90 2023-09-20T22:05:51.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.634+0000 7fb861f9b700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb8480a1380 0x7fb8480a3840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb85c104050 tx=0x7fb858009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.906+0000 7fb8641ff700 1 -- 172.21.15.22:0/1721652277 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb85c109f70 con 0x7fb85c104b90 2023-09-20T22:05:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.907+0000 7fb852ffd700 1 -- 172.21.15.22:0/1721652277 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb854099200 con 0x7fb85c104b90 2023-09-20T22:05:51.907 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T22:05:51.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.909+0000 7fb8641ff700 1 -- 172.21.15.22:0/1721652277 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb8480a1380 msgr2=0x7fb8480a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:51.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.909+0000 7fb8641ff700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb8480a1380 0x7fb8480a3840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb85c104050 tx=0x7fb858009260 comp rx=0 tx=0).stop 2023-09-20T22:05:51.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.909+0000 7fb8641ff700 1 -- 172.21.15.22:0/1721652277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb85c104b90 msgr2=0x7fb85c1a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.909+0000 7fb8641ff700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb85c104b90 0x7fb85c1a02f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb854007ee0 tx=0x7fb85400d3b0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.910+0000 7fb8641ff700 1 -- 172.21.15.22:0/1721652277 shutdown_connections 2023-09-20T22:05:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.910+0000 7fb8641ff700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fb8480a1380 0x7fb8480a3840 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.910+0000 7fb8641ff700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb85c104b90 0x7fb85c1a02f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.910+0000 7fb8641ff700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb85c103000 0x7fb85c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.910+0000 7fb8641ff700 1 --2- 172.21.15.22:0/1721652277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c1041f0 0x7fb85c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:51.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.910+0000 7fb8641ff700 1 -- 172.21.15.22:0/1721652277 >> 172.21.15.22:0/1721652277 conn(0x7fb85c0fe590 msgr2=0x7fb85c100820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:51.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.910+0000 7fb8641ff700 1 -- 172.21.15.22:0/1721652277 shutdown_connections 2023-09-20T22:05:51.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:51.910+0000 7fb8641ff700 1 -- 172.21.15.22:0/1721652277 wait complete. 2023-09-20T22:05:51.934 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-20T22:05:52.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:51 smithi123 ceph-mon[134433]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-20T22:05:52.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:51 smithi123 ceph-mon[134433]: Adjusting osd_memory_target on smithi022 to 222.0M 2023-09-20T22:05:52.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:51 smithi123 ceph-mon[134433]: Unable to set osd_memory_target on smithi022 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-20T22:05:52.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:51 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/426736305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T22:05:52.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:51 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/376090232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T22:05:52.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:51 smithi089 ceph-mon[134090]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-20T22:05:52.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:51 smithi089 ceph-mon[134090]: Adjusting osd_memory_target on smithi022 to 222.0M 2023-09-20T22:05:52.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:51 smithi089 ceph-mon[134090]: Unable to set osd_memory_target on smithi022 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-20T22:05:52.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:51 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/426736305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T22:05:52.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:51 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/376090232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T22:05:52.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.171+0000 7f78b8fff700 1 -- 172.21.15.22:0/680559927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b4101880 msgr2=0x7f78b4103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:52.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.171+0000 7f78b8fff700 1 --2- 172.21.15.22:0/680559927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b4101880 0x7f78b4103c70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009990 tx=0x7f78a4009ca0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.171+0000 7f78b8fff700 1 -- 172.21.15.22:0/680559927 shutdown_connections 2023-09-20T22:05:52.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.171+0000 7f78b8fff700 1 --2- 172.21.15.22:0/680559927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78b4106ae0 0x7f78b4108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.171+0000 7f78b8fff700 1 --2- 172.21.15.22:0/680559927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b4101880 0x7f78b4103c70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.171+0000 7f78b8fff700 1 --2- 172.21.15.22:0/680559927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78b41041b0 0x7f78b41065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.171+0000 7f78b8fff700 1 -- 172.21.15.22:0/680559927 >> 172.21.15.22:0/680559927 conn(0x7f78b40fb270 msgr2=0x7f78b40fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.171+0000 7f78b8fff700 1 -- 172.21.15.22:0/680559927 shutdown_connections 2023-09-20T22:05:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.171+0000 7f78b8fff700 1 -- 172.21.15.22:0/680559927 wait complete. 2023-09-20T22:05:52.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.172+0000 7f78b8fff700 1 Processor -- start 2023-09-20T22:05:52.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.172+0000 7f78b8fff700 1 -- start start 2023-09-20T22:05:52.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b8fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78b4101880 0x7f78b4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b8fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b41041b0 0x7f78b4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b8fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78b4106ae0 0x7f78b41a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b8fff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b41a0830 con 0x7f78b4101880 2023-09-20T22:05:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b8fff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b41a09a0 con 0x7f78b41041b0 2023-09-20T22:05:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b8fff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b41a0b60 con 0x7f78b4106ae0 2023-09-20T22:05:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b41041b0 0x7f78b4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b1d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b41041b0 0x7f78b4199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60560/0 (socket says 172.21.15.22:60560) 2023-09-20T22:05:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b1d9b700 1 -- 172.21.15.22:0/291707055 learned_addr learned my addr 172.21.15.22:0/291707055 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b2d9d700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78b4106ae0 0x7f78b41a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.173+0000 7f78b259c700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78b4101880 0x7f78b4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78b1d9b700 1 -- 172.21.15.22:0/291707055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78b4106ae0 msgr2=0x7f78b41a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78b1d9b700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78b4106ae0 0x7f78b41a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78b1d9b700 1 -- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78b4101880 msgr2=0x7f78b4198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78b1d9b700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78b4101880 0x7f78b4198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78b1d9b700 1 -- 172.21.15.22:0/291707055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a40095f0 con 0x7f78b41041b0 2023-09-20T22:05:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78b259c700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78b4101880 0x7f78b4198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-20T22:05:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78b1d9b700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b41041b0 0x7f78b4199230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f78a800e9a0 tx=0x7f78a800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78a37fe700 1 -- 172.21.15.22:0/291707055 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8009790 con 0x7f78b41041b0 2023-09-20T22:05:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78a37fe700 1 -- 172.21.15.22:0/291707055 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a800cbd0 con 0x7f78b41041b0 2023-09-20T22:05:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78a37fe700 1 -- 172.21.15.22:0/291707055 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8018680 con 0x7f78b41041b0 2023-09-20T22:05:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78b8fff700 1 -- 172.21.15.22:0/291707055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b41a0e40 con 0x7f78b41041b0 2023-09-20T22:05:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.174+0000 7f78b8fff700 1 -- 172.21.15.22:0/291707055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b41a1260 con 0x7f78b41041b0 2023-09-20T22:05:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.175+0000 7f78b8fff700 1 -- 172.21.15.22:0/291707055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b41086b0 con 0x7f78b41041b0 2023-09-20T22:05:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.178+0000 7f78a37fe700 1 -- 172.21.15.22:0/291707055 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f78a800cd40 con 0x7f78b41041b0 2023-09-20T22:05:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.179+0000 7f78a37fe700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f789c0a1380 0x7f789c0a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.179+0000 7f78a37fe700 1 -- 172.21.15.22:0/291707055 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f78a8014070 con 0x7f78b41041b0 2023-09-20T22:05:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.179+0000 7f78b259c700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f789c0a1380 0x7f789c0a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.180+0000 7f78b259c700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f789c0a1380 0x7f789c0a3840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f78a400b2e0 tx=0x7f78a4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.181+0000 7f78a37fe700 1 -- 172.21.15.22:0/291707055 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f78a80965e0 con 0x7f78b41041b0 2023-09-20T22:05:52.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:51 smithi022 ceph-mon[97651]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 12 op/s 2023-09-20T22:05:52.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:51 smithi022 ceph-mon[97651]: Adjusting osd_memory_target on smithi022 to 222.0M 2023-09-20T22:05:52.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:51 smithi022 ceph-mon[97651]: Unable to set osd_memory_target on smithi022 to 232784384: error parsing value: Value '232784384' is below minimum 939524096 2023-09-20T22:05:52.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:51 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/426736305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T22:05:52.248 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:51 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/376090232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-09-20T22:05:52.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.462+0000 7f78b8fff700 1 -- 172.21.15.22:0/291707055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78b410c7c0 con 0x7f78b41041b0 2023-09-20T22:05:52.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.463+0000 7f78a37fe700 1 -- 172.21.15.22:0/291707055 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78a8099c00 con 0x7f78b41041b0 2023-09-20T22:05:52.463 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-09-20T22:05:52.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 -- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f789c0a1380 msgr2=0x7f789c0a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:52.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f789c0a1380 0x7f789c0a3840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f78a400b2e0 tx=0x7f78a4005fb0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 -- 172.21.15.22:0/291707055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b41041b0 msgr2=0x7f78b4199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b41041b0 0x7f78b4199230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f78a800e9a0 tx=0x7f78a800ecb0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 -- 172.21.15.22:0/291707055 shutdown_connections 2023-09-20T22:05:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78b4106ae0 0x7f78b41a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78b4101880 0x7f78b4198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f789c0a1380 0x7f789c0a3840 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 --2- 172.21.15.22:0/291707055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78b41041b0 0x7f78b4199230 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 -- 172.21.15.22:0/291707055 >> 172.21.15.22:0/291707055 conn(0x7f78b40fb270 msgr2=0x7f78b40fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 -- 172.21.15.22:0/291707055 shutdown_connections 2023-09-20T22:05:52.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.465+0000 7f78b8fff700 1 -- 172.21.15.22:0/291707055 wait complete. 2023-09-20T22:05:52.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-20T22:05:52.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.788+0000 7f64b239d700 1 -- 172.21.15.22:0/3732620732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac102e60 msgr2=0x7f64ac103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:52.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.788+0000 7f64b239d700 1 --2- 172.21.15.22:0/3732620732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac102e60 0x7f64ac103280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009990 tx=0x7f64a8009ca0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.788+0000 7f64b239d700 1 -- 172.21.15.22:0/3732620732 shutdown_connections 2023-09-20T22:05:52.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.788+0000 7f64b239d700 1 --2- 172.21.15.22:0/3732620732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac104970 0x7f64ac10b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.788+0000 7f64b239d700 1 --2- 172.21.15.22:0/3732620732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac102e60 0x7f64ac103280 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.788+0000 7f64b239d700 1 --2- 172.21.15.22:0/3732620732 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64ac103fb0 0x7f64ac104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.788+0000 7f64b239d700 1 -- 172.21.15.22:0/3732620732 >> 172.21.15.22:0/3732620732 conn(0x7f64ac0fe3b0 msgr2=0x7f64ac100810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:52.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.788+0000 7f64b239d700 1 -- 172.21.15.22:0/3732620732 shutdown_connections 2023-09-20T22:05:52.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.788+0000 7f64b239d700 1 -- 172.21.15.22:0/3732620732 wait complete. 2023-09-20T22:05:52.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.789+0000 7f64b239d700 1 Processor -- start 2023-09-20T22:05:52.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.789+0000 7f64b239d700 1 -- start start 2023-09-20T22:05:52.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b239d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64ac102e60 0x7f64ac198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:52.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b239d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac103fb0 0x7f64ac198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:52.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b239d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac104970 0x7f64ac1a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:52.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b239d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1a05f0 con 0x7f64ac102e60 2023-09-20T22:05:52.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b239d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1a0730 con 0x7f64ac104970 2023-09-20T22:05:52.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b239d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1a08c0 con 0x7f64ac103fb0 2023-09-20T22:05:52.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b139b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64ac102e60 0x7f64ac198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:52.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b139b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64ac102e60 0x7f64ac198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49680/0 (socket says 172.21.15.22:49680) 2023-09-20T22:05:52.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b139b700 1 -- 172.21.15.22:0/1075118230 learned_addr learned my addr 172.21.15.22:0/1075118230 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:52.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b0b9a700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac103fb0 0x7f64ac198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:52.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.790+0000 7f64b1b9c700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac104970 0x7f64ac1a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:52.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.791+0000 7f64b139b700 1 -- 172.21.15.22:0/1075118230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac103fb0 msgr2=0x7f64ac198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:52.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.791+0000 7f64b139b700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac103fb0 0x7f64ac198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.791+0000 7f64b139b700 1 -- 172.21.15.22:0/1075118230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac104970 msgr2=0x7f64ac1a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:52.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.791+0000 7f64b139b700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac104970 0x7f64ac1a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:52.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.791+0000 7f64b139b700 1 -- 172.21.15.22:0/1075118230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a80095f0 con 0x7f64ac102e60 2023-09-20T22:05:52.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.791+0000 7f64b139b700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64ac102e60 0x7f64ac198a70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f64a8004760 tx=0x7f64a8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:52.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.791+0000 7f64a27fc700 1 -- 172.21.15.22:0/1075118230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a801d070 con 0x7f64ac102e60 2023-09-20T22:05:52.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.791+0000 7f64b239d700 1 -- 172.21.15.22:0/1075118230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac1a0b40 con 0x7f64ac102e60 2023-09-20T22:05:52.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.791+0000 7f64b239d700 1 -- 172.21.15.22:0/1075118230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac1a0f60 con 0x7f64ac102e60 2023-09-20T22:05:52.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.792+0000 7f64a27fc700 1 -- 172.21.15.22:0/1075118230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64a8021410 con 0x7f64ac102e60 2023-09-20T22:05:52.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.792+0000 7f64a27fc700 1 -- 172.21.15.22:0/1075118230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8017650 con 0x7f64ac102e60 2023-09-20T22:05:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.793+0000 7f64b239d700 1 -- 172.21.15.22:0/1075118230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6490005320 con 0x7f64ac102e60 2023-09-20T22:05:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.795+0000 7f64a27fc700 1 -- 172.21.15.22:0/1075118230 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f64a800fad0 con 0x7f64ac102e60 2023-09-20T22:05:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.796+0000 7f64a27fc700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f64980a1420 0x7f64980a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:52.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.796+0000 7f64a27fc700 1 -- 172.21.15.22:0/1075118230 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f64a80ce1a0 con 0x7f64ac102e60 2023-09-20T22:05:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.797+0000 7f64b0b9a700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f64980a1420 0x7f64980a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:52.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.797+0000 7f64b0b9a700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f64980a1420 0x7f64980a38e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f649c009c60 tx=0x7f649c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:52.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:52.801+0000 7f64a27fc700 1 -- 172.21.15.22:0/1075118230 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f64a8097510 con 0x7f64ac102e60 2023-09-20T22:05:53.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.078+0000 7f64b239d700 1 -- 172.21.15.22:0/1075118230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6490005b50 con 0x7f64ac102e60 2023-09-20T22:05:53.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.078+0000 7f64a27fc700 1 -- 172.21.15.22:0/1075118230 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f64a809ab30 con 0x7f64ac102e60 2023-09-20T22:05:53.079 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-09-20T22:05:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 -- 172.21.15.22:0/1075118230 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f64980a1420 msgr2=0x7f64980a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:53.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f64980a1420 0x7f64980a38e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f649c009c60 tx=0x7f649c009260 comp rx=0 tx=0).stop 2023-09-20T22:05:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 -- 172.21.15.22:0/1075118230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64ac102e60 msgr2=0x7f64ac198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64ac102e60 0x7f64ac198a70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f64a8004760 tx=0x7f64a8004790 comp rx=0 tx=0).stop 2023-09-20T22:05:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 -- 172.21.15.22:0/1075118230 shutdown_connections 2023-09-20T22:05:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f64980a1420 0x7f64980a38e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f64ac104970 0x7f64ac1a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64ac102e60 0x7f64ac198a70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 --2- 172.21.15.22:0/1075118230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64ac103fb0 0x7f64ac198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 -- 172.21.15.22:0/1075118230 >> 172.21.15.22:0/1075118230 conn(0x7f64ac0fe3b0 msgr2=0x7f64ac100590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 -- 172.21.15.22:0/1075118230 shutdown_connections 2023-09-20T22:05:53.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.081+0000 7f64b239d700 1 -- 172.21.15.22:0/1075118230 wait complete. 2023-09-20T22:05:53.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:52 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1721652277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T22:05:53.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:52 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/291707055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T22:05:53.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:52 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1721652277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T22:05:53.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:52 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/291707055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T22:05:53.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-20T22:05:53.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:52 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1721652277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-09-20T22:05:53.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:52 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/291707055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-09-20T22:05:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.340+0000 7f67727ff700 1 -- 172.21.15.22:0/2819972320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f676c100df0 msgr2=0x7f676c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.340+0000 7f67727ff700 1 --2- 172.21.15.22:0/2819972320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f676c100df0 0x7f676c101210 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f675c0099c0 tx=0x7f675c009cd0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.340+0000 7f67727ff700 1 -- 172.21.15.22:0/2819972320 shutdown_connections 2023-09-20T22:05:53.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.340+0000 7f67727ff700 1 --2- 172.21.15.22:0/2819972320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f676c102980 0x7f676c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.340+0000 7f67727ff700 1 --2- 172.21.15.22:0/2819972320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f676c100df0 0x7f676c101210 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.340+0000 7f67727ff700 1 --2- 172.21.15.22:0/2819972320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f676c101fe0 0x7f676c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.340+0000 7f67727ff700 1 -- 172.21.15.22:0/2819972320 >> 172.21.15.22:0/2819972320 conn(0x7f676c0fc360 msgr2=0x7f676c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:53.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.341+0000 7f67727ff700 1 -- 172.21.15.22:0/2819972320 shutdown_connections 2023-09-20T22:05:53.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.341+0000 7f67727ff700 1 -- 172.21.15.22:0/2819972320 wait complete. 2023-09-20T22:05:53.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f67727ff700 1 Processor -- start 2023-09-20T22:05:53.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f67727ff700 1 -- start start 2023-09-20T22:05:53.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f67727ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f676c100df0 0x7f676c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:53.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f67727ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f676c101fe0 0x7f676c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:53.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f67727ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f676c102980 0x7f676c1a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:53.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f67727ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c1a08d0 con 0x7f676c102980 2023-09-20T22:05:53.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f67727ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c1a0a10 con 0x7f676c100df0 2023-09-20T22:05:53.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f67727ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c1a0ba0 con 0x7f676c101fe0 2023-09-20T22:05:53.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f676bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f676c100df0 0x7f676c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:53.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f676bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f676c100df0 0x7f676c198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60590/0 (socket says 172.21.15.22:60590) 2023-09-20T22:05:53.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.342+0000 7f676bfff700 1 -- 172.21.15.22:0/3091627326 learned_addr learned my addr 172.21.15.22:0/3091627326 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:53.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.343+0000 7f6770d9c700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f676c102980 0x7f676c1a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:53.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.343+0000 7f676b7fe700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f676c101fe0 0x7f676c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:53.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.343+0000 7f676bfff700 1 -- 172.21.15.22:0/3091627326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f676c101fe0 msgr2=0x7f676c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:53.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.343+0000 7f676bfff700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f676c101fe0 0x7f676c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.343+0000 7f676bfff700 1 -- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f676c102980 msgr2=0x7f676c1a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:53.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.343+0000 7f676bfff700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f676c102980 0x7f676c1a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.343+0000 7f676bfff700 1 -- 172.21.15.22:0/3091627326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764009c40 con 0x7f676c100df0 2023-09-20T22:05:53.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.343+0000 7f6770d9c700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f676c102980 0x7f676c1a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:05:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.343+0000 7f676bfff700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f676c100df0 0x7f676c198d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f675c009990 tx=0x7f675c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.344+0000 7f67697fa700 1 -- 172.21.15.22:0/3091627326 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c01d070 con 0x7f676c100df0 2023-09-20T22:05:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.344+0000 7f67697fa700 1 -- 172.21.15.22:0/3091627326 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f675c0054f0 con 0x7f676c100df0 2023-09-20T22:05:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.344+0000 7f67727ff700 1 -- 172.21.15.22:0/3091627326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c0095f0 con 0x7f676c100df0 2023-09-20T22:05:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.344+0000 7f67697fa700 1 -- 172.21.15.22:0/3091627326 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c004a40 con 0x7f676c100df0 2023-09-20T22:05:53.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.344+0000 7f67727ff700 1 -- 172.21.15.22:0/3091627326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c1a1110 con 0x7f676c100df0 2023-09-20T22:05:53.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.345+0000 7f67727ff700 1 -- 172.21.15.22:0/3091627326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c193440 con 0x7f676c100df0 2023-09-20T22:05:53.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.347+0000 7f67697fa700 1 -- 172.21.15.22:0/3091627326 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f675c004be0 con 0x7f676c100df0 2023-09-20T22:05:53.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.349+0000 7f67697fa700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67540a1380 0x7f67540a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:53.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.349+0000 7f67697fa700 1 -- 172.21.15.22:0/3091627326 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f675c0cdd60 con 0x7f676c100df0 2023-09-20T22:05:53.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.349+0000 7f676b7fe700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67540a1380 0x7f67540a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:53.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.350+0000 7f676b7fe700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67540a1380 0x7f67540a3840 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6760009c60 tx=0x7f6760009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:53.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.350+0000 7f67697fa700 1 -- 172.21.15.22:0/3091627326 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f675c0971f0 con 0x7f676c100df0 2023-09-20T22:05:53.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.635+0000 7f67727ff700 1 -- 172.21.15.22:0/3091627326 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f676c02d160 con 0x7f676c100df0 2023-09-20T22:05:53.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.636+0000 7f67697fa700 1 -- 172.21.15.22:0/3091627326 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) v1 ==== 81+0+591 (secure 0 0 0) 0x7f675c026260 con 0x7f676c100df0 2023-09-20T22:05:53.636 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/3260510802 2023-09-21T21:40:07.017182+0000 2023-09-20T22:05:53.637 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/3260510802 2023-09-21T21:40:07.017182+0000 2023-09-20T22:05:53.637 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6801/1128310681 2023-09-21T21:39:39.363251+0000 2023-09-20T22:05:53.637 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/3419565055 2023-09-21T21:40:07.017182+0000 2023-09-20T22:05:53.637 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6800/1128310681 2023-09-21T21:39:39.363251+0000 2023-09-20T22:05:53.637 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/4209005662 2023-09-21T21:39:39.363251+0000 2023-09-20T22:05:53.638 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1007451083 2023-09-21T21:39:39.363251+0000 2023-09-20T22:05:53.638 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/1994934339 2023-09-21T21:40:07.017182+0000 2023-09-20T22:05:53.638 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/875874054 2023-09-21T21:40:07.017182+0000 2023-09-20T22:05:53.638 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:0/2863894527 2023-09-21T21:39:39.363251+0000 2023-09-20T22:05:53.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.638+0000 7f67727ff700 1 -- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67540a1380 msgr2=0x7f67540a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:53.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.638+0000 7f67727ff700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67540a1380 0x7f67540a3840 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6760009c60 tx=0x7f6760009260 comp rx=0 tx=0).stop 2023-09-20T22:05:53.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.638+0000 7f67727ff700 1 -- 172.21.15.22:0/3091627326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f676c100df0 msgr2=0x7f676c198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:53.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.638+0000 7f67727ff700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f676c100df0 0x7f676c198d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f675c009990 tx=0x7f675c00f690 comp rx=0 tx=0).stop 2023-09-20T22:05:53.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.638+0000 7f67727ff700 1 -- 172.21.15.22:0/3091627326 shutdown_connections 2023-09-20T22:05:53.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.638+0000 7f67727ff700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f676c102980 0x7f676c1a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.638+0000 7f67727ff700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f67540a1380 0x7f67540a3840 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.638+0000 7f67727ff700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f676c100df0 0x7f676c198d50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.638+0000 7f67727ff700 1 --2- 172.21.15.22:0/3091627326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f676c101fe0 0x7f676c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:53.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.638+0000 7f67727ff700 1 -- 172.21.15.22:0/3091627326 >> 172.21.15.22:0/3091627326 conn(0x7f676c0fc360 msgr2=0x7f676c107940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:53.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.639+0000 7f67727ff700 1 -- 172.21.15.22:0/3091627326 shutdown_connections 2023-09-20T22:05:53.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:53.639+0000 7f67727ff700 1 -- 172.21.15.22:0/3091627326 wait complete. 2023-09-20T22:05:53.642 INFO:teuthology.orchestra.run.smithi022.stderr:listed 10 entries 2023-09-20T22:05:53.661 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-09-20T22:05:53.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T22:05:53.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /tmp/cephfs 2023-09-20T22:05:54.000 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:53 smithi022 ceph-mon[97651]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2023-09-20T22:05:54.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:53 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/1075118230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T22:05:54.001 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:53 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3091627326' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-20T22:05:54.009 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-09-20T22:05:54.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T22:05:54.010 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-09-20T22:05:54.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:53 smithi123 ceph-mon[134433]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2023-09-20T22:05:54.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:53 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/1075118230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T22:05:54.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:53 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3091627326' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-20T22:05:54.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T22:05:54.101 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-09-20T22:05:54.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:53 smithi089 ceph-mon[134090]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2023-09-20T22:05:54.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:53 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/1075118230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-09-20T22:05:54.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:53 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3091627326' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-09-20T22:05:54.171 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-09-20T22:05:54.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T22:05:54.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-09-20T22:05:54.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T22:05:54.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2023-09-20T22:05:54.370 INFO:tasks.kclient:Unmounting kernel clients... 2023-09-20T22:05:54.371 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-09-20T22:05:54.372 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-20T22:05:54.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348692k,nr_inodes=4087173,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/7e9eec72cec624eabb2e642fcf6b478f60c58a62d58d07a4b2a8c09077f30513/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/WM6QL2IHK46VARHZ32GEYPI6SD:/var/lib/containers/storage/overlay/l/RYFHNLIWBHFMLDDGBGO4DWBFOK,upperdir=/var/lib/containers/storage/overlay/7e9eec72cec624eabb2e642fcf6b478f60c58a62d58d07a4b2a8c09077f30513/diff,workdir=/var/lib/containers/storage/overlay/7e9eec72cec624eabb2e642fcf6b478f60c58a62d58d07a4b2a8c09077f30513/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c2b14720fb8b011592e4386aa420c77ccd753101a90a162a0e50b37531f2dc40/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/WM6QL2IHK46VARHZ32GEYPI6SD:/var/lib/containers/storage/overlay/l/RYFHNLIWBHFMLDDGBGO4DWBFOK,upperdir=/var/lib/containers/storage/overlay/c2b14720fb8b011592e4386aa420c77ccd753101a90a162a0e50b37531f2dc40/diff,workdir=/var/lib/containers/storage/overlay/c2b14720fb8b011592e4386aa420c77ccd753101a90a162a0e50b37531f2dc40/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275628k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/9f0184f2feb6c1eab1021e8b874e9fb9ffb3d677108b59cfaea19490990b7b2e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/WM6QL2IHK46VARHZ32GEYPI6SD:/var/lib/containers/storage/overlay/l/RYFHNLIWBHFMLDDGBGO4DWBFOK,upperdir=/var/lib/containers/storage/overlay/9f0184f2feb6c1eab1021e8b874e9fb9ffb3d677108b59cfaea19490990b7b2e/diff,workdir=/var/lib/containers/storage/overlay/9f0184f2feb6c1eab1021e8b874e9fb9ffb3d677108b59cfaea19490990b7b2e/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/54d948e602e99cba805f93dd23a3d7f1cab7ba5b58c2378a01ef115d8619c92f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/WM6QL2IHK46VARHZ32GEYPI6SD:/var/lib/containers/storage/overlay/l/RYFHNLIWBHFMLDDGBGO4DWBFOK,upperdir=/var/lib/containers/storage/overlay/54d948e602e99cba805f93dd23a3d7f1cab7ba5b58c2378a01ef115d8619c92f/diff,workdir=/var/lib/containers/storage/overlay/54d948e602e99cba805f93dd23a3d7f1cab7ba5b58c2378a01ef115d8619c92f/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6de1b0e2850e8598f15a9b8ceb8c3e044f176c1c1ac58450107e2af69b5faa9d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/WM6QL2IHK46VARHZ32GEYPI6SD:/var/lib/containers/storage/overlay/l/RYFHNLIWBHFMLDDGBGO4DWBFOK,upperdir=/var/lib/containers/storage/overlay/6de1b0e2850e8598f15a9b8ceb8c3e044f176c1c1ac58450107e2af69b5faa9d/diff,workdir=/var/lib/containers/storage/overlay/6de1b0e2850e8598f15a9b8ceb8c3e044f176c1c1ac58450107e2af69b5faa9d/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/cd810c33b651d0c48a9be6950eeda6d98b179c6a26862a0c1add05e218303e20/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/WM6QL2IHK46VARHZ32GEYPI6SD:/var/lib/containers/storage/overlay/l/RYFHNLIWBHFMLDDGBGO4DWBFOK,upperdir=/var/lib/containers/storage/overlay/cd810c33b651d0c48a9be6950eeda6d98b179c6a26862a0c1add05e218303e20/diff,workdir=/var/lib/containers/storage/overlay/cd810c33b651d0c48a9be6950eeda6d98b179c6a26862a0c1add05e218303e20/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6dcf9da55e6fcb321d50699d53674c6e95729a91f6420189787ce90e15a71eb7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/WM6QL2IHK46VARHZ32GEYPI6SD:/var/lib/containers/storage/overlay/l/RYFHNLIWBHFMLDDGBGO4DWBFOK,upperdir=/var/lib/containers/storage/overlay/6dcf9da55e6fcb321d50699d53674c6e95729a91f6420189787ce90e15a71eb7/diff,workdir=/var/lib/containers/storage/overlay/6dcf9da55e6fcb321d50699d53674c6e95729a91f6420189787ce90e15a71eb7/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5c6de95337d52592beec7196eadad5874b553fe4718082e1250a754387247234/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/WM6QL2IHK46VARHZ32GEYPI6SD:/var/lib/containers/storage/overlay/l/RYFHNLIWBHFMLDDGBGO4DWBFOK,upperdir=/var/lib/containers/storage/overlay/5c6de95337d52592beec7196eadad5874b553fe4718082e1250a754387247234/diff,workdir=/var/lib/containers/storage/overlay/5c6de95337d52592beec7196eadad5874b553fe4718082e1250a754387247234/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/abbd8f4f5dc2f1beee23076227896d55ccde4bf4c961879c032ff60b40e96c77/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/WM6QL2IHK46VARHZ32GEYPI6SD:/var/lib/containers/storage/overlay/l/RYFHNLIWBHFMLDDGBGO4DWBFOK,upperdir=/var/lib/containers/storage/overlay/abbd8f4f5dc2f1beee23076227896d55ccde4bf4c961879c032ff60b40e96c77/diff,workdir=/var/lib/containers/storage/overlay/abbd8f4f5dc2f1beee23076227896d55ccde4bf4c961879c032ff60b40e96c77/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/22f946ef49cf3496bedb7babb1d02cb994b52e8cd733617bbb3039229e65a6e0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/WM6QL2IHK46VARHZ32GEYPI6SD:/var/lib/containers/storage/overlay/l/RYFHNLIWBHFMLDDGBGO4DWBFOK,upperdir=/var/lib/containers/storage/overlay/22f946ef49cf3496bedb7babb1d02cb994b52e8cd733617bbb3039229e65a6e0/diff,workdir=/var/lib/containers/storage/overlay/22f946ef49cf3496bedb7babb1d02cb994b52e8cd733617bbb3039229e65a6e0/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-09-20T22:05:54.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T22:05:54.428 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-09-20T22:05:54.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-20T22:05:54.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2023-09-20T22:05:54.651 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2023-09-20T22:05:54.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T22:05:54.656 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-09-20T22:05:54.666 INFO:tasks.cephadm:Teardown begin 2023-09-20T22:05:54.667 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-20T22:05:54.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T22:05:54.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.955+0000 7fced53ff700 1 -- 172.21.15.22:0/990954077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fced0104210 msgr2=0x7fced0104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:54.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.955+0000 7fced53ff700 1 --2- 172.21.15.22:0/990954077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fced0104210 0x7fced0104670 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcec40099c0 tx=0x7fcec4009cd0 comp rx=0 tx=0).stop 2023-09-20T22:05:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.955+0000 7fced53ff700 1 -- 172.21.15.22:0/990954077 shutdown_connections 2023-09-20T22:05:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.955+0000 7fced53ff700 1 --2- 172.21.15.22:0/990954077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced0104bb0 0x7fced010b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.955+0000 7fced53ff700 1 --2- 172.21.15.22:0/990954077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced0103020 0x7fced0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.955+0000 7fced53ff700 1 --2- 172.21.15.22:0/990954077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fced0104210 0x7fced0104670 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:54.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.955+0000 7fced53ff700 1 -- 172.21.15.22:0/990954077 >> 172.21.15.22:0/990954077 conn(0x7fced00fe590 msgr2=0x7fced01009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.955+0000 7fced53ff700 1 -- 172.21.15.22:0/990954077 shutdown_connections 2023-09-20T22:05:54.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.955+0000 7fced53ff700 1 -- 172.21.15.22:0/990954077 wait complete. 2023-09-20T22:05:54.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.956+0000 7fced53ff700 1 Processor -- start 2023-09-20T22:05:54.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fced53ff700 1 -- start start 2023-09-20T22:05:54.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fced53ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced0103020 0x7fced0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:54.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fced53ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fced0104210 0x7fced01991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:54.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fced53ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced0104bb0 0x7fced01a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:54.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fced53ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced01a0810 con 0x7fced0104bb0 2023-09-20T22:05:54.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fced53ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced01a0980 con 0x7fced0104210 2023-09-20T22:05:54.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fced53ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced01a0b40 con 0x7fced0103020 2023-09-20T22:05:54.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fcece7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fced0104210 0x7fced01991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fcece7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fced0104210 0x7fced01991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60598/0 (socket says 172.21.15.22:60598) 2023-09-20T22:05:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fceceffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced0103020 0x7fced0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fcece7fc700 1 -- 172.21.15.22:0/3162918678 learned_addr learned my addr 172.21.15.22:0/3162918678 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.957+0000 7fcecf7fe700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced0104bb0 0x7fced01a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.958+0000 7fcece7fc700 1 -- 172.21.15.22:0/3162918678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced0103020 msgr2=0x7fced0198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.958+0000 7fcece7fc700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced0103020 0x7fced0198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.958+0000 7fcece7fc700 1 -- 172.21.15.22:0/3162918678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced0104bb0 msgr2=0x7fced01a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.958+0000 7fcece7fc700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced0104bb0 0x7fced01a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.958+0000 7fcece7fc700 1 -- 172.21.15.22:0/3162918678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec8009c40 con 0x7fced0104210 2023-09-20T22:05:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.958+0000 7fcece7fc700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fced0104210 0x7fced01991e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcec40058e0 tx=0x7fcec400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:54.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.958+0000 7fceb7fff700 1 -- 172.21.15.22:0/3162918678 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec401d070 con 0x7fced0104210 2023-09-20T22:05:54.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.958+0000 7fceb7fff700 1 -- 172.21.15.22:0/3162918678 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcec4005050 con 0x7fced0104210 2023-09-20T22:05:54.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.958+0000 7fced53ff700 1 -- 172.21.15.22:0/3162918678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec40095f0 con 0x7fced0104210 2023-09-20T22:05:54.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.958+0000 7fceb7fff700 1 -- 172.21.15.22:0/3162918678 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4004b30 con 0x7fced0104210 2023-09-20T22:05:54.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.959+0000 7fced53ff700 1 -- 172.21.15.22:0/3162918678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced01a10b0 con 0x7fced0104210 2023-09-20T22:05:54.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.960+0000 7fced53ff700 1 -- 172.21.15.22:0/3162918678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced010a570 con 0x7fced0104210 2023-09-20T22:05:54.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.963+0000 7fceb7fff700 1 -- 172.21.15.22:0/3162918678 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fcec40051c0 con 0x7fced0104210 2023-09-20T22:05:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.964+0000 7fceb7fff700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fceb80a13d0 0x7fceb80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.964+0000 7fceb7fff700 1 -- 172.21.15.22:0/3162918678 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fcec40cdc50 con 0x7fced0104210 2023-09-20T22:05:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.964+0000 7fceceffd700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fceb80a13d0 0x7fceb80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.965+0000 7fceb7fff700 1 -- 172.21.15.22:0/3162918678 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fcec4097020 con 0x7fced0104210 2023-09-20T22:05:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:54.965+0000 7fceceffd700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fceb80a13d0 0x7fceb80a3890 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fced0199b70 tx=0x7fcec0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:55.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.298+0000 7fced53ff700 1 -- 172.21.15.22:0/3162918678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fced010ead0 con 0x7fced0104210 2023-09-20T22:05:55.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.299+0000 7fceb7fff700 1 -- 172.21.15.22:0/3162918678 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11362 (secure 0 0 0) 0x7fcec409a640 con 0x7fced0104210 2023-09-20T22:05:55.299 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:05:55.299 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T22:04:39.172497+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T22:05:55.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 -- 172.21.15.22:0/3162918678 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fceb80a13d0 msgr2=0x7fceb80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:55.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fceb80a13d0 0x7fceb80a3890 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fced0199b70 tx=0x7fcec0009260 comp rx=0 tx=0).stop 2023-09-20T22:05:55.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 -- 172.21.15.22:0/3162918678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fced0104210 msgr2=0x7fced01991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:55.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fced0104210 0x7fced01991e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcec40058e0 tx=0x7fcec400f690 comp rx=0 tx=0).stop 2023-09-20T22:05:55.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 -- 172.21.15.22:0/3162918678 shutdown_connections 2023-09-20T22:05:55.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fced0104bb0 0x7fced01a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fced0103020 0x7fced0198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fceb80a13d0 0x7fceb80a3890 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 --2- 172.21.15.22:0/3162918678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fced0104210 0x7fced01991e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 -- 172.21.15.22:0/3162918678 >> 172.21.15.22:0/3162918678 conn(0x7fced00fe590 msgr2=0x7fced01076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 -- 172.21.15.22:0/3162918678 shutdown_connections 2023-09-20T22:05:55.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.302+0000 7fced53ff700 1 -- 172.21.15.22:0/3162918678 wait complete. 2023-09-20T22:05:55.305 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2023-09-20T22:05:55.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-20T22:05:55.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.551+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/3740654270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8106ab0 msgr2=0x7f2df8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:55.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.551+0000 7f2dfe1ff700 1 --2- 172.21.15.22:0/3740654270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8106ab0 0x7f2df8108f20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2df000b0a0 tx=0x7f2df000b3b0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.551+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/3740654270 shutdown_connections 2023-09-20T22:05:55.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.551+0000 7f2dfe1ff700 1 --2- 172.21.15.22:0/3740654270 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8106ab0 0x7f2df8108f20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.551+0000 7f2dfe1ff700 1 --2- 172.21.15.22:0/3740654270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2df8101850 0x7f2df8103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.551+0000 7f2dfe1ff700 1 --2- 172.21.15.22:0/3740654270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df8104180 0x7f2df8106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.551+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/3740654270 >> 172.21.15.22:0/3740654270 conn(0x7f2df80fb240 msgr2=0x7f2df80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.551+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/3740654270 shutdown_connections 2023-09-20T22:05:55.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.551+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/3740654270 wait complete. 2023-09-20T22:05:55.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.553+0000 7f2dfe1ff700 1 Processor -- start 2023-09-20T22:05:55.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.553+0000 7f2dfe1ff700 1 -- start start 2023-09-20T22:05:55.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.553+0000 7f2dfe1ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df8101850 0x7f2df8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:55.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.553+0000 7f2dfe1ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2df8104180 0x7f2df81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.553+0000 7f2dfe1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8106ab0 0x7f2df81a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.553+0000 7f2dfe1ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df81a08f0 con 0x7f2df8104180 2023-09-20T22:05:55.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.553+0000 7f2dfe1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df81a0a60 con 0x7f2df8106ab0 2023-09-20T22:05:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.553+0000 7f2dfe1ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df81a0c20 con 0x7f2df8101850 2023-09-20T22:05:55.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2df8104180 0x7f2df81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:55.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8106ab0 0x7f2df81a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:55.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df8101850 0x7f2df8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:55.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8106ab0 0x7f2df81a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:60610/0 (socket says 172.21.15.22:60610) 2023-09-20T22:05:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df8101850 0x7f2df8198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:37020/0 (socket says 172.21.15.22:37020) 2023-09-20T22:05:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df7fff700 1 -- 172.21.15.22:0/2941934244 learned_addr learned my addr 172.21.15.22:0/2941934244 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:55.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df7fff700 1 -- 172.21.15.22:0/2941934244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df8101850 msgr2=0x7f2df8198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df7fff700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df8101850 0x7f2df8198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df7fff700 1 -- 172.21.15.22:0/2941934244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2df8104180 msgr2=0x7f2df81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:55.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df7fff700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2df8104180 0x7f2df81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df7fff700 1 -- 172.21.15.22:0/2941934244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df0009d00 con 0x7f2df8106ab0 2023-09-20T22:05:55.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.554+0000 7f2df7fff700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8106ab0 0x7f2df81a0320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009fd0 tx=0x7f2df000bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:55.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.555+0000 7f2df4ff9700 1 -- 172.21.15.22:0/2941934244 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df000e040 con 0x7f2df8106ab0 2023-09-20T22:05:55.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.555+0000 7f2df4ff9700 1 -- 172.21.15.22:0/2941934244 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2df00094a0 con 0x7f2df8106ab0 2023-09-20T22:05:55.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.555+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/2941934244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df81a0ea0 con 0x7f2df8106ab0 2023-09-20T22:05:55.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.555+0000 7f2df4ff9700 1 -- 172.21.15.22:0/2941934244 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df001b6c0 con 0x7f2df8106ab0 2023-09-20T22:05:55.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.555+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/2941934244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df81a1260 con 0x7f2df8106ab0 2023-09-20T22:05:55.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.556+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/2941934244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df8108630 con 0x7f2df8106ab0 2023-09-20T22:05:55.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.558+0000 7f2df4ff9700 1 -- 172.21.15.22:0/2941934244 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f2df001d470 con 0x7f2df8106ab0 2023-09-20T22:05:55.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.560+0000 7f2df4ff9700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2de00a13d0 0x7f2de00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:55.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.560+0000 7f2df4ff9700 1 -- 172.21.15.22:0/2941934244 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7f2df00cdee0 con 0x7f2df8106ab0 2023-09-20T22:05:55.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.560+0000 7f2df77fe700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2de00a13d0 0x7f2de00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.561+0000 7f2df77fe700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2de00a13d0 0x7f2de00a3890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2df819a280 tx=0x7f2de8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:55.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.564+0000 7f2df4ff9700 1 -- 172.21.15.22:0/2941934244 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f2df00972b0 con 0x7f2df8106ab0 2023-09-20T22:05:55.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.873+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/2941934244 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2df810c600 con 0x7f2df8106ab0 2023-09-20T22:05:55.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.874+0000 7f2df4ff9700 1 -- 172.21.15.22:0/2941934244 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11362 (secure 0 0 0) 0x7f2df009a8d0 con 0x7f2df8106ab0 2023-09-20T22:05:55.874 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:05:55.875 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6839/3781158526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6838","nonce":3781158526},{"type":"v1","addr":"172.21.15.22:6839","nonce":3781158526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2368905204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2368905204},{"type":"v1","addr":"172.21.15.123:6833","nonce":2368905204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-09-20T21:46:41.044280+0000","modified":"2023-09-20T22:04:39.172497+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24530,"mds_3":24487,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/3516975495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3516975495},{"type":"v1","addr":"172.21.15.89:6833","nonce":3516975495}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.22:6837/4039103472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6836","nonce":4039103472},{"type":"v1","addr":"172.21.15.22:6837","nonce":4039103472}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"j","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.22:6835/56798476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":56798476},{"type":"v1","addr":"172.21.15.22:6835","nonce":56798476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6835/835271994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":835271994},{"type":"v1","addr":"172.21.15.123:6835","nonce":835271994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/794452586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":794452586},{"type":"v1","addr":"172.21.15.89:6835","nonce":794452586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24499":{"gid":24499,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6837/1168160038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":1168160038},{"type":"v1","addr":"172.21.15.123:6837","nonce":1168160038}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6839/2368757728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2368757728},{"type":"v1","addr":"172.21.15.123:6839","nonce":2368757728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.89:6837/3887702690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3887702690},{"type":"v1","addr":"172.21.15.89:6837","nonce":3887702690}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6841/1806616417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6840","nonce":1806616417},{"type":"v1","addr":"172.21.15.22:6841","nonce":1806616417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.89:6839/2165264535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":2165264535},{"type":"v1","addr":"172.21.15.89:6839","nonce":2165264535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-20T22:05:55.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.876+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/2941934244 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2de00a13d0 msgr2=0x7f2de00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:55.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.876+0000 7f2dfe1ff700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2de00a13d0 0x7f2de00a3890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2df819a280 tx=0x7f2de8009260 comp rx=0 tx=0).stop 2023-09-20T22:05:55.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.876+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/2941934244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8106ab0 msgr2=0x7f2df81a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:55.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.876+0000 7f2dfe1ff700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8106ab0 0x7f2df81a0320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009fd0 tx=0x7f2df000bce0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.877+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/2941934244 shutdown_connections 2023-09-20T22:05:55.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.877+0000 7f2dfe1ff700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f2de00a13d0 0x7f2de00a3890 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.877+0000 7f2dfe1ff700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2df8106ab0 0x7f2df81a0320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.877+0000 7f2dfe1ff700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df8101850 0x7f2df8198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.877+0000 7f2dfe1ff700 1 --2- 172.21.15.22:0/2941934244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2df8104180 0x7f2df81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.877+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/2941934244 >> 172.21.15.22:0/2941934244 conn(0x7f2df80fb240 msgr2=0x7f2df81024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.877+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/2941934244 shutdown_connections 2023-09-20T22:05:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:55.877+0000 7f2dfe1ff700 1 -- 172.21.15.22:0/2941934244 wait complete. 2023-09-20T22:05:55.879 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 33 2023-09-20T22:05:55.903 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-20T22:05:56.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:55 smithi123 ceph-mon[134433]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-09-20T22:05:56.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:55 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/3162918678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T22:05:56.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:55 smithi089 ceph-mon[134090]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-09-20T22:05:56.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:55 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/3162918678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T22:05:56.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.165+0000 7fa26bfff700 1 -- 172.21.15.22:0/571496893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa26c101780 msgr2=0x7fa26c103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:56.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.165+0000 7fa26bfff700 1 --2- 172.21.15.22:0/571496893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa26c101780 0x7fa26c103b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009990 tx=0x7fa25c009ca0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.165+0000 7fa26bfff700 1 -- 172.21.15.22:0/571496893 shutdown_connections 2023-09-20T22:05:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.165+0000 7fa26bfff700 1 --2- 172.21.15.22:0/571496893 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa26c1069e0 0x7fa26c108dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.165+0000 7fa26bfff700 1 --2- 172.21.15.22:0/571496893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa26c101780 0x7fa26c103b70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.165+0000 7fa26bfff700 1 --2- 172.21.15.22:0/571496893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa26c1040b0 0x7fa26c1064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.165+0000 7fa26bfff700 1 -- 172.21.15.22:0/571496893 >> 172.21.15.22:0/571496893 conn(0x7fa26c0fb0b0 msgr2=0x7fa26c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:56.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.165+0000 7fa26bfff700 1 -- 172.21.15.22:0/571496893 shutdown_connections 2023-09-20T22:05:56.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.165+0000 7fa26bfff700 1 -- 172.21.15.22:0/571496893 wait complete. 2023-09-20T22:05:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.166+0000 7fa26bfff700 1 Processor -- start 2023-09-20T22:05:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.166+0000 7fa26bfff700 1 -- start start 2023-09-20T22:05:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa26c101780 0x7fa26c1010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:56.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa26c1040b0 0x7fa26c0ff730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:56.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa26c1069e0 0x7fa26c0ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:56.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26bfff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c100410 con 0x7fa26c1069e0 2023-09-20T22:05:56.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26bfff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c100580 con 0x7fa26c1040b0 2023-09-20T22:05:56.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26bfff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c1a0800 con 0x7fa26c101780 2023-09-20T22:05:56.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa26c1040b0 0x7fa26c0ff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:56.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26a7fc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa26c1040b0 0x7fa26c0ff730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:41946/0 (socket says 172.21.15.22:41946) 2023-09-20T22:05:56.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26a7fc700 1 -- 172.21.15.22:0/2985541124 learned_addr learned my addr 172.21.15.22:0/2985541124 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:56.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26affd700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa26c101780 0x7fa26c1010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:56.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26b7fe700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa26c1069e0 0x7fa26c0ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:56.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26a7fc700 1 -- 172.21.15.22:0/2985541124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa26c101780 msgr2=0x7fa26c1010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:56.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26a7fc700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa26c101780 0x7fa26c1010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26a7fc700 1 -- 172.21.15.22:0/2985541124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa26c1069e0 msgr2=0x7fa26c0ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:56.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26a7fc700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa26c1069e0 0x7fa26c0ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.167+0000 7fa26a7fc700 1 -- 172.21.15.22:0/2985541124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c0095f0 con 0x7fa26c1040b0 2023-09-20T22:05:56.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.168+0000 7fa26a7fc700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa26c1040b0 0x7fa26c0ff730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa26000d980 tx=0x7fa26000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:56.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.168+0000 7fa253fff700 1 -- 172.21.15.22:0/2985541124 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260004220 con 0x7fa26c1040b0 2023-09-20T22:05:56.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.168+0000 7fa253fff700 1 -- 172.21.15.22:0/2985541124 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa260007640 con 0x7fa26c1040b0 2023-09-20T22:05:56.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.168+0000 7fa26bfff700 1 -- 172.21.15.22:0/2985541124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c1a0ae0 con 0x7fa26c1040b0 2023-09-20T22:05:56.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.168+0000 7fa253fff700 1 -- 172.21.15.22:0/2985541124 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260005690 con 0x7fa26c1040b0 2023-09-20T22:05:56.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.168+0000 7fa26bfff700 1 -- 172.21.15.22:0/2985541124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c1a1040 con 0x7fa26c1040b0 2023-09-20T22:05:56.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.169+0000 7fa26bfff700 1 -- 172.21.15.22:0/2985541124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c0fcc30 con 0x7fa26c1040b0 2023-09-20T22:05:56.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.171+0000 7fa253fff700 1 -- 172.21.15.22:0/2985541124 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fa260004750 con 0x7fa26c1040b0 2023-09-20T22:05:56.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.173+0000 7fa253fff700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa2540a1470 0x7fa2540a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:56.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.173+0000 7fa253fff700 1 -- 172.21.15.22:0/2985541124 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fa2600ccdb0 con 0x7fa26c1040b0 2023-09-20T22:05:56.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.173+0000 7fa26affd700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa2540a1470 0x7fa2540a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:56.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.174+0000 7fa26affd700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa2540a1470 0x7fa2540a3930 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00b2e0 tx=0x7fa25c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:56.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.176+0000 7fa253fff700 1 -- 172.21.15.22:0/2985541124 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fa2600d1070 con 0x7fa26c1040b0 2023-09-20T22:05:56.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:55 smithi022 ceph-mon[97651]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-09-20T22:05:56.247 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:55 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/3162918678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T22:05:56.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.408+0000 7fa26bfff700 1 -- 172.21.15.22:0/2985541124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa26c02d140 con 0x7fa26c1040b0 2023-09-20T22:05:56.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.410+0000 7fa253fff700 1 -- 172.21.15.22:0/2985541124 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24645 (secure 0 0 0) 0x7fa260016090 con 0x7fa26c1040b0 2023-09-20T22:05:56.411 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-09-20T22:05:56.411 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":93,"fsid":"f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da","created":"2023-09-20T21:39:07.129064+0000","modified":"2023-09-20T21:48:09.582452+0000","last_up_change":"2023-09-20T21:45:52.802908+0000","last_in_change":"2023-09-20T21:45:35.247472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-20T21:42:25.633235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-20T21:46:38.981332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-20T21:46:40.478907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-20T21:46:44.069558+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6bdd85c1-5ec1-40a9-be24-e1c0701d30a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6803","nonce":3853681188}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6805","nonce":3853681188}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6809","nonce":3853681188}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3853681188},{"type":"v1","addr":"172.21.15.22:6807","nonce":3853681188}]},"public_addr":"172.21.15.22:6803/3853681188","cluster_addr":"172.21.15.22:6805/3853681188","heartbeat_back_addr":"172.21.15.22:6809/3853681188","heartbeat_front_addr":"172.21.15.22:6807/3853681188","state":["exists","up"]},{"osd":1,"uuid":"acd88d1d-b602-4722-a8b3-4ef144e4da4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6801","nonce":2519597559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6803","nonce":2519597559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6807","nonce":2519597559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2519597559},{"type":"v1","addr":"172.21.15.89:6805","nonce":2519597559}]},"public_addr":"172.21.15.89:6801/2519597559","cluster_addr":"172.21.15.89:6803/2519597559","heartbeat_back_addr":"172.21.15.89:6807/2519597559","heartbeat_front_addr":"172.21.15.89:6805/2519597559","state":["exists","up"]},{"osd":2,"uuid":"28865364-a5f4-426f-98a6-19cb758151a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6801","nonce":877674627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6803","nonce":877674627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6807","nonce":877674627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":877674627},{"type":"v1","addr":"172.21.15.123:6805","nonce":877674627}]},"public_addr":"172.21.15.123:6801/877674627","cluster_addr":"172.21.15.123:6803/877674627","heartbeat_back_addr":"172.21.15.123:6807/877674627","heartbeat_front_addr":"172.21.15.123:6805/877674627","state":["exists","up"]},{"osd":3,"uuid":"d24fd98b-40c7-4b54-9ec1-b0cd2a34e1a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6811","nonce":143368577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6813","nonce":143368577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6817","nonce":143368577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":143368577},{"type":"v1","addr":"172.21.15.22:6815","nonce":143368577}]},"public_addr":"172.21.15.22:6811/143368577","cluster_addr":"172.21.15.22:6813/143368577","heartbeat_back_addr":"172.21.15.22:6817/143368577","heartbeat_front_addr":"172.21.15.22:6815/143368577","state":["exists","up"]},{"osd":4,"uuid":"e4f26e77-aca2-47d8-8158-4d080560576e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6809","nonce":1802541493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6811","nonce":1802541493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6815","nonce":1802541493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":1802541493},{"type":"v1","addr":"172.21.15.89:6813","nonce":1802541493}]},"public_addr":"172.21.15.89:6809/1802541493","cluster_addr":"172.21.15.89:6811/1802541493","heartbeat_back_addr":"172.21.15.89:6815/1802541493","heartbeat_front_addr":"172.21.15.89:6813/1802541493","state":["exists","up"]},{"osd":5,"uuid":"9d8866df-2568-4a6d-83e7-1be50e327005","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6809","nonce":402706185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6811","nonce":402706185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6815","nonce":402706185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":402706185},{"type":"v1","addr":"172.21.15.123:6813","nonce":402706185}]},"public_addr":"172.21.15.123:6809/402706185","cluster_addr":"172.21.15.123:6811/402706185","heartbeat_back_addr":"172.21.15.123:6815/402706185","heartbeat_front_addr":"172.21.15.123:6813/402706185","state":["exists","up"]},{"osd":6,"uuid":"8a625e71-db36-4eab-a821-68672c3284e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6819","nonce":1850518910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6821","nonce":1850518910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6825","nonce":1850518910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":1850518910},{"type":"v1","addr":"172.21.15.22:6823","nonce":1850518910}]},"public_addr":"172.21.15.22:6819/1850518910","cluster_addr":"172.21.15.22:6821/1850518910","heartbeat_back_addr":"172.21.15.22:6825/1850518910","heartbeat_front_addr":"172.21.15.22:6823/1850518910","state":["exists","up"]},{"osd":7,"uuid":"7674a06e-2830-441e-9178-687e42dfce80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6817","nonce":4165516997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6819","nonce":4165516997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6823","nonce":4165516997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":4165516997},{"type":"v1","addr":"172.21.15.89:6821","nonce":4165516997}]},"public_addr":"172.21.15.89:6817/4165516997","cluster_addr":"172.21.15.89:6819/4165516997","heartbeat_back_addr":"172.21.15.89:6823/4165516997","heartbeat_front_addr":"172.21.15.89:6821/4165516997","state":["exists","up"]},{"osd":8,"uuid":"bf42a69e-43d0-47b2-bd13-9378e765e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6817","nonce":1023938376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6819","nonce":1023938376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6823","nonce":1023938376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1023938376},{"type":"v1","addr":"172.21.15.123:6821","nonce":1023938376}]},"public_addr":"172.21.15.123:6817/1023938376","cluster_addr":"172.21.15.123:6819/1023938376","heartbeat_back_addr":"172.21.15.123:6823/1023938376","heartbeat_front_addr":"172.21.15.123:6821/1023938376","state":["exists","up"]},{"osd":9,"uuid":"80ebf3d2-d3f8-46c9-b9cd-fbd1649b6d06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6827","nonce":4238100695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6829","nonce":4238100695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6833","nonce":4238100695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":4238100695},{"type":"v1","addr":"172.21.15.22:6831","nonce":4238100695}]},"public_addr":"172.21.15.22:6827/4238100695","cluster_addr":"172.21.15.22:6829/4238100695","heartbeat_back_addr":"172.21.15.22:6833/4238100695","heartbeat_front_addr":"172.21.15.22:6831/4238100695","state":["exists","up"]},{"osd":10,"uuid":"3d118a3d-5e31-4f50-aabd-72d9125546ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6825","nonce":2445411927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6827","nonce":2445411927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6831","nonce":2445411927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":2445411927},{"type":"v1","addr":"172.21.15.89:6829","nonce":2445411927}]},"public_addr":"172.21.15.89:6825/2445411927","cluster_addr":"172.21.15.89:6827/2445411927","heartbeat_back_addr":"172.21.15.89:6831/2445411927","heartbeat_front_addr":"172.21.15.89:6829/2445411927","state":["exists","up"]},{"osd":11,"uuid":"41242a75-3273-4217-ad0d-c5da81c6b369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6825","nonce":2107937412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6827","nonce":2107937412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6831","nonce":2107937412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2107937412},{"type":"v1","addr":"172.21.15.123:6829","nonce":2107937412}]},"public_addr":"172.21.15.123:6825/2107937412","cluster_addr":"172.21.15.123:6827/2107937412","heartbeat_back_addr":"172.21.15.123:6831/2107937412","heartbeat_front_addr":"172.21.15.123:6829/2107937412","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:41:39.094399+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:01.884543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:23.153924+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:42:49.852777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:11.570913+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:32.164266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:43:58.511576+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:20.450386+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:44:41.366196+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:07.922021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:29.863980+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-20T21:45:51.035535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6801/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/3260510802":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6801/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/3419565055":"2023-09-21T21:40:07.017182+0000","172.21.15.22:6800/1128310681":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/4209005662":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1007451083":"2023-09-21T21:39:39.363251+0000","172.21.15.22:0/1994934339":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/875874054":"2023-09-21T21:40:07.017182+0000","172.21.15.22:0/2863894527":"2023-09-21T21:39:39.363251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-20T22:05:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.412+0000 7fa26bfff700 1 -- 172.21.15.22:0/2985541124 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa2540a1470 msgr2=0x7fa2540a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.412+0000 7fa26bfff700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa2540a1470 0x7fa2540a3930 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00b2e0 tx=0x7fa25c005fb0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.412+0000 7fa26bfff700 1 -- 172.21.15.22:0/2985541124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa26c1040b0 msgr2=0x7fa26c0ff730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.412+0000 7fa26bfff700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa26c1040b0 0x7fa26c0ff730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa26000d980 tx=0x7fa26000dc90 comp rx=0 tx=0).stop 2023-09-20T22:05:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.412+0000 7fa26bfff700 1 -- 172.21.15.22:0/2985541124 shutdown_connections 2023-09-20T22:05:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.412+0000 7fa26bfff700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa26c1069e0 0x7fa26c0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.412+0000 7fa26bfff700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa26c101780 0x7fa26c1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.412+0000 7fa26bfff700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fa2540a1470 0x7fa2540a3930 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.412+0000 7fa26bfff700 1 --2- 172.21.15.22:0/2985541124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa26c1040b0 0x7fa26c0ff730 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.412+0000 7fa26bfff700 1 -- 172.21.15.22:0/2985541124 >> 172.21.15.22:0/2985541124 conn(0x7fa26c0fb0b0 msgr2=0x7fa26c1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:56.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.413+0000 7fa26bfff700 1 -- 172.21.15.22:0/2985541124 shutdown_connections 2023-09-20T22:05:56.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.413+0000 7fa26bfff700 1 -- 172.21.15.22:0/2985541124 wait complete. 2023-09-20T22:05:56.433 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-20T22:05:56.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.671+0000 7ff04c9ff700 1 -- 172.21.15.22:0/749401012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff048101fe0 msgr2=0x7ff048102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:56.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.671+0000 7ff04c9ff700 1 --2- 172.21.15.22:0/749401012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff048101fe0 0x7ff048102440 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0099c0 tx=0x7ff03c009cd0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.671+0000 7ff04c9ff700 1 -- 172.21.15.22:0/749401012 shutdown_connections 2023-09-20T22:05:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.671+0000 7ff04c9ff700 1 --2- 172.21.15.22:0/749401012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff048102980 0x7ff048109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.671+0000 7ff04c9ff700 1 --2- 172.21.15.22:0/749401012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff048100df0 0x7ff048101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.671+0000 7ff04c9ff700 1 --2- 172.21.15.22:0/749401012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff048101fe0 0x7ff048102440 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.671+0000 7ff04c9ff700 1 -- 172.21.15.22:0/749401012 >> 172.21.15.22:0/749401012 conn(0x7ff0480fc360 msgr2=0x7ff0480fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:56.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.671+0000 7ff04c9ff700 1 -- 172.21.15.22:0/749401012 shutdown_connections 2023-09-20T22:05:56.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.671+0000 7ff04c9ff700 1 -- 172.21.15.22:0/749401012 wait complete. 2023-09-20T22:05:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.672+0000 7ff04c9ff700 1 Processor -- start 2023-09-20T22:05:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.672+0000 7ff04c9ff700 1 -- start start 2023-09-20T22:05:56.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff04c9ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff048100df0 0x7ff048074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff04c9ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff048101fe0 0x7ff0480731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff04c9ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff048102980 0x7ff048073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:56.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff04c9ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff048073eb0 con 0x7ff048100df0 2023-09-20T22:05:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff04c9ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff048074020 con 0x7ff048101fe0 2023-09-20T22:05:56.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff04c9ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0481a0b30 con 0x7ff048102980 2023-09-20T22:05:56.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff045d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff048101fe0 0x7ff0480731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:56.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff045d9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff048101fe0 0x7ff0480731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.22:41956/0 (socket says 172.21.15.22:41956) 2023-09-20T22:05:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff045d9b700 1 -- 172.21.15.22:0/649989131 learned_addr learned my addr 172.21.15.22:0/649989131 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff04659c700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff048100df0 0x7ff048074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:56.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff046d9d700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff048102980 0x7ff048073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:56.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff045d9b700 1 -- 172.21.15.22:0/649989131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff048102980 msgr2=0x7ff048073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:56.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff045d9b700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff048102980 0x7ff048073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff045d9b700 1 -- 172.21.15.22:0/649989131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff048100df0 msgr2=0x7ff048074b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:56.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff045d9b700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff048100df0 0x7ff048074b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:56.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.673+0000 7ff045d9b700 1 -- 172.21.15.22:0/649989131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c0095f0 con 0x7ff048101fe0 2023-09-20T22:05:56.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.674+0000 7ff045d9b700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff048101fe0 0x7ff0480731d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00bf40 tx=0x7ff03c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:56.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.674+0000 7ff0377fe700 1 -- 172.21.15.22:0/649989131 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c01d070 con 0x7ff048101fe0 2023-09-20T22:05:56.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.674+0000 7ff0377fe700 1 -- 172.21.15.22:0/649989131 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff03c005510 con 0x7ff048101fe0 2023-09-20T22:05:56.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.674+0000 7ff04c9ff700 1 -- 172.21.15.22:0/649989131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0481a0db0 con 0x7ff048101fe0 2023-09-20T22:05:56.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.674+0000 7ff0377fe700 1 -- 172.21.15.22:0/649989131 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c004c00 con 0x7ff048101fe0 2023-09-20T22:05:56.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.674+0000 7ff04c9ff700 1 -- 172.21.15.22:0/649989131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0481a11d0 con 0x7ff048101fe0 2023-09-20T22:05:56.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.676+0000 7ff04c9ff700 1 -- 172.21.15.22:0/649989131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0481083a0 con 0x7ff048101fe0 2023-09-20T22:05:56.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.678+0000 7ff0377fe700 1 -- 172.21.15.22:0/649989131 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7ff03c00fb80 con 0x7ff048101fe0 2023-09-20T22:05:56.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.679+0000 7ff0377fe700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0300a1380 0x7ff0300a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:56.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.679+0000 7ff0377fe700 1 -- 172.21.15.22:0/649989131 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9144+0+0 (secure 0 0 0) 0x7ff03c0cdec0 con 0x7ff048101fe0 2023-09-20T22:05:56.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.679+0000 7ff04659c700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0300a1380 0x7ff0300a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:56.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.680+0000 7ff04659c700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0300a1380 0x7ff0300a3840 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff048101e40 tx=0x7ff038009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:56.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.683+0000 7ff0377fe700 1 -- 172.21.15.22:0/649989131 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7ff03c097230 con 0x7ff048101fe0 2023-09-20T22:05:56.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:56.992+0000 7ff04c9ff700 1 -- 172.21.15.22:0/649989131 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff04810c650 con 0x7ff048101fe0 2023-09-20T22:05:57.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.000+0000 7ff0377fe700 1 -- 172.21.15.22:0/649989131 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff03c09a850 con 0x7ff048101fe0 2023-09-20T22:05:57.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 -- 172.21.15.22:0/649989131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0300a1380 msgr2=0x7ff0300a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:57.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0300a1380 0x7ff0300a3840 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff048101e40 tx=0x7ff038009260 comp rx=0 tx=0).stop 2023-09-20T22:05:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 -- 172.21.15.22:0/649989131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff048101fe0 msgr2=0x7ff0480731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff048101fe0 0x7ff0480731d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00bf40 tx=0x7ff03c00f690 comp rx=0 tx=0).stop 2023-09-20T22:05:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 -- 172.21.15.22:0/649989131 shutdown_connections 2023-09-20T22:05:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff048102980 0x7ff048073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:57.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff048100df0 0x7ff048074b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7ff0300a1380 0x7ff0300a3840 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 --2- 172.21.15.22:0/649989131 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff048101fe0 0x7ff0480731d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 -- 172.21.15.22:0/649989131 >> 172.21.15.22:0/649989131 conn(0x7ff0480fc360 msgr2=0x7ff0481054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:57.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.003+0000 7ff04c9ff700 1 -- 172.21.15.22:0/649989131 shutdown_connections 2023-09-20T22:05:57.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.004+0000 7ff04c9ff700 1 -- 172.21.15.22:0/649989131 wait complete. 2023-09-20T22:05:57.007 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-20T22:05:57.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:56 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2941934244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T22:05:57.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:56 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2985541124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T22:05:57.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:56 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-a[97627]: 2023-09-20T22:05:56.993+0000 7f30e3295700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-20T22:05:57.035 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-20T22:05:57.038 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:56 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2941934244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T22:05:57.038 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:56 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2985541124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T22:05:57.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:56 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2941934244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-09-20T22:05:57.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:56 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2985541124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-09-20T22:05:57.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.257+0000 7fbb055ff700 1 -- 172.21.15.22:0/3854422526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb00073840 msgr2=0x7fbb00073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:57.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.257+0000 7fbb055ff700 1 --2- 172.21.15.22:0/3854422526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb00073840 0x7fbb00073cc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009990 tx=0x7fbaf4009ca0 comp rx=0 tx=0).stop 2023-09-20T22:05:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.258+0000 7fbb055ff700 1 -- 172.21.15.22:0/3854422526 shutdown_connections 2023-09-20T22:05:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.258+0000 7fbb055ff700 1 --2- 172.21.15.22:0/3854422526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb00108c90 0x7fbb0010b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.258+0000 7fbb055ff700 1 --2- 172.21.15.22:0/3854422526 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb00074e20 0x7fbb00073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.258+0000 7fbb055ff700 1 --2- 172.21.15.22:0/3854422526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb00073840 0x7fbb00073cc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.258+0000 7fbb055ff700 1 -- 172.21.15.22:0/3854422526 >> 172.21.15.22:0/3854422526 conn(0x7fbb000fc320 msgr2=0x7fbb000fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.258+0000 7fbb055ff700 1 -- 172.21.15.22:0/3854422526 shutdown_connections 2023-09-20T22:05:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.258+0000 7fbb055ff700 1 -- 172.21.15.22:0/3854422526 wait complete. 2023-09-20T22:05:57.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbb055ff700 1 Processor -- start 2023-09-20T22:05:57.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbb055ff700 1 -- start start 2023-09-20T22:05:57.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbb055ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb00073840 0x7fbb00198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbb055ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb00074e20 0x7fbb00199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbb055ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb00108c90 0x7fbb001a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbb055ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb001a08c0 con 0x7fbb00074e20 2023-09-20T22:05:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbb055ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb001a0a00 con 0x7fbb00108c90 2023-09-20T22:05:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbb055ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb001a0b90 con 0x7fbb00073840 2023-09-20T22:05:57.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbafe7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb00074e20 0x7fbb00199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:57.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbafe7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb00074e20 0x7fbb00199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48568/0 (socket says 172.21.15.22:48568) 2023-09-20T22:05:57.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbafe7fc700 1 -- 172.21.15.22:0/2839757843 learned_addr learned my addr 172.21.15.22:0/2839757843 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:57.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbafeffd700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb00073840 0x7fbb00198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:57.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.259+0000 7fbaff7fe700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb00108c90 0x7fbb001a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbafe7fc700 1 -- 172.21.15.22:0/2839757843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb00073840 msgr2=0x7fbb00198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbafe7fc700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb00073840 0x7fbb00198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbafe7fc700 1 -- 172.21.15.22:0/2839757843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb00108c90 msgr2=0x7fbb001a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbafe7fc700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb00108c90 0x7fbb001a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:57.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbafe7fc700 1 -- 172.21.15.22:0/2839757843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf40095f0 con 0x7fbb00074e20 2023-09-20T22:05:57.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbafeffd700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb00073840 0x7fbb00198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:05:57.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbaff7fe700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb00108c90 0x7fbb001a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:05:57.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbafe7fc700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb00074e20 0x7fbb00199280 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005720 tx=0x7fbaf4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:57.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbae7fff700 1 -- 172.21.15.22:0/2839757843 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf401d070 con 0x7fbb00074e20 2023-09-20T22:05:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbb055ff700 1 -- 172.21.15.22:0/2839757843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb001a0e10 con 0x7fbb00074e20 2023-09-20T22:05:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbae7fff700 1 -- 172.21.15.22:0/2839757843 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaf4021410 con 0x7fbb00074e20 2023-09-20T22:05:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.260+0000 7fbae7fff700 1 -- 172.21.15.22:0/2839757843 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4017650 con 0x7fbb00074e20 2023-09-20T22:05:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.261+0000 7fbb055ff700 1 -- 172.21.15.22:0/2839757843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb001a1180 con 0x7fbb00074e20 2023-09-20T22:05:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.262+0000 7fbb055ff700 1 -- 172.21.15.22:0/2839757843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb00107340 con 0x7fbb00074e20 2023-09-20T22:05:57.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.264+0000 7fbae7fff700 1 -- 172.21.15.22:0/2839757843 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7fbaf4021580 con 0x7fbb00074e20 2023-09-20T22:05:57.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.266+0000 7fbae7fff700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbae80a1300 0x7fbae80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:57.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.266+0000 7fbae7fff700 1 -- 172.21.15.22:0/2839757843 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9574+0+0 (secure 0 0 0) 0x7fbaf40ce340 con 0x7fbb00074e20 2023-09-20T22:05:57.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.266+0000 7fbafeffd700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbae80a1300 0x7fbae80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:57.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.267+0000 7fbafeffd700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbae80a1300 0x7fbae80a37c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0011490 tx=0x7fbaf0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:57.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.270+0000 7fbae7fff700 1 -- 172.21.15.22:0/2839757843 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7fbaf4097620 con 0x7fbb00074e20 2023-09-20T22:05:57.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-c[172252]: ignoring --setuser ceph since I am not root 2023-09-20T22:05:57.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-c[172252]: ignoring --setgroup ceph since I am not root 2023-09-20T22:05:57.343 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mds[172276]: -- 172.21.15.123:0/1207755947 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a2ea4951e0 con 0x55a2ea46b000 2023-09-20T22:05:57.344 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-c[172252]: starting mds.c at 2023-09-20T22:05:57.344 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-i[172592]: ignoring --setuser ceph since I am not root 2023-09-20T22:05:57.344 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-i[172592]: ignoring --setgroup ceph since I am not root 2023-09-20T22:05:57.344 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mds[172616]: -- 172.21.15.123:0/2189731688 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56032ff0d1e0 con 0x56032fee2c00 2023-09-20T22:05:57.345 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-i[172592]: starting mds.i at 2023-09-20T22:05:57.345 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-f[171914]: ignoring --setuser ceph since I am not root 2023-09-20T22:05:57.345 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-f[171914]: ignoring --setgroup ceph since I am not root 2023-09-20T22:05:57.345 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mds[171938]: -- 172.21.15.123:0/3210879910 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f31f92f1e0 con 0x55f31f904800 2023-09-20T22:05:57.346 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-f[171914]: starting mds.f at 2023-09-20T22:05:57.497 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-d[147971]: ignoring --setuser ceph since I am not root 2023-09-20T22:05:57.498 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-d[147971]: ignoring --setgroup ceph since I am not root 2023-09-20T22:05:57.498 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mds[147996]: -- 172.21.15.22:0/694237982 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a6ab08f1e0 con 0x55a6ab064800 2023-09-20T22:05:57.498 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-d[147971]: starting mds.d at 2023-09-20T22:05:57.498 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-a[148978]: ignoring --setuser ceph since I am not root 2023-09-20T22:05:57.499 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-a[148978]: ignoring --setgroup ceph since I am not root 2023-09-20T22:05:57.499 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mds[149002]: -- 172.21.15.22:0/1734309250 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5609657b11e0 con 0x560965786c00 2023-09-20T22:05:57.499 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-a[148978]: starting mds.a at 2023-09-20T22:05:57.499 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-j[147584]: ignoring --setuser ceph since I am not root 2023-09-20T22:05:57.500 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-j[147584]: ignoring --setgroup ceph since I am not root 2023-09-20T22:05:57.500 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mds[147608]: -- 172.21.15.22:0/3625579516 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5650822531e0 con 0x565082228800 2023-09-20T22:05:57.500 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-j[147584]: starting mds.j at 2023-09-20T22:05:57.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:57.537+0000 7fbb055ff700 1 -- 172.21.15.22:0/2839757843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbb0002cfa0 con 0x7fbb00074e20 2023-09-20T22:05:57.610 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-e[170732]: ignoring --setuser ceph since I am not root 2023-09-20T22:05:57.611 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-e[170732]: ignoring --setgroup ceph since I am not root 2023-09-20T22:05:57.611 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-e[170732]: starting mds.e at 2023-09-20T22:05:57.611 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-h[171074]: ignoring --setuser ceph since I am not root 2023-09-20T22:05:57.611 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-h[171074]: ignoring --setgroup ceph since I am not root 2023-09-20T22:05:57.612 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mds[171099]: -- 172.21.15.89:0/4070703055 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564172ae31e0 con 0x564172ab8800 2023-09-20T22:05:57.612 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-h[171074]: starting mds.h at 2023-09-20T22:05:57.612 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-k[170392]: ignoring --setuser ceph since I am not root 2023-09-20T22:05:57.613 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-k[170392]: ignoring --setgroup ceph since I am not root 2023-09-20T22:05:57.613 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mds[170416]: -- 172.21.15.89:0/862442526 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563cba9f11e0 con 0x563cba9c7000 2023-09-20T22:05:57.613 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-k[170392]: starting mds.k at 2023-09-20T22:05:57.613 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b[171416]: ignoring --setuser ceph since I am not root 2023-09-20T22:05:57.613 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b[171416]: ignoring --setgroup ceph since I am not root 2023-09-20T22:05:57.614 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mds[171440]: -- 172.21.15.89:0/1128324858 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c4f8cfd1e0 con 0x55c4f8cd2c00 2023-09-20T22:05:57.614 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b[171416]: starting mds.b at 2023-09-20T22:05:58.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.000+0000 7fbae7fff700 1 -- 172.21.15.22:0/2839757843 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbaf409ac40 con 0x7fbb00074e20 2023-09-20T22:05:58.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.003+0000 7fbb055ff700 1 -- 172.21.15.22:0/2839757843 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbae80a1300 msgr2=0x7fbae80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:58.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.003+0000 7fbb055ff700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbae80a1300 0x7fbae80a37c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0011490 tx=0x7fbaf0009260 comp rx=0 tx=0).stop 2023-09-20T22:05:58.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.004+0000 7fbb055ff700 1 -- 172.21.15.22:0/2839757843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb00074e20 msgr2=0x7fbb00199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:58.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.004+0000 7fbb055ff700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb00074e20 0x7fbb00199280 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005720 tx=0x7fbaf4009580 comp rx=0 tx=0).stop 2023-09-20T22:05:58.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.004+0000 7fbb055ff700 1 -- 172.21.15.22:0/2839757843 shutdown_connections 2023-09-20T22:05:58.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.004+0000 7fbb055ff700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7fbae80a1300 0x7fbae80a37c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:58.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.004+0000 7fbb055ff700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbb00108c90 0x7fbb001a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:58.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.004+0000 7fbb055ff700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb00073840 0x7fbb00198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:58.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.004+0000 7fbb055ff700 1 --2- 172.21.15.22:0/2839757843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb00074e20 0x7fbb00199280 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:58.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.004+0000 7fbb055ff700 1 -- 172.21.15.22:0/2839757843 >> 172.21.15.22:0/2839757843 conn(0x7fbb000fc320 msgr2=0x7fbb000fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:58.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.004+0000 7fbb055ff700 1 -- 172.21.15.22:0/2839757843 shutdown_connections 2023-09-20T22:05:58.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.005+0000 7fbb055ff700 1 -- 172.21.15.22:0/2839757843 wait complete. 2023-09-20T22:05:58.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-20T22:05:58.092 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mon[97651]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2023-09-20T22:05:58.092 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/649989131' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-20T22:05:58.092 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-20T22:05:58.092 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mon[97651]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-20T22:05:58.092 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mon[97651]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-20T22:05:58.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mon[97651]: osdmap e94: 12 total, 12 up, 12 in 2023-09-20T22:05:58.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-20T22:05:58.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mon[97651]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-09-20T22:05:58.093 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:57 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2839757843' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-20T22:05:58.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mon[134433]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2023-09-20T22:05:58.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/649989131' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-20T22:05:58.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-20T22:05:58.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mon[134433]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-20T22:05:58.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mon[134433]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-20T22:05:58.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mon[134433]: osdmap e94: 12 total, 12 up, 12 in 2023-09-20T22:05:58.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-20T22:05:58.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mon[134433]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-09-20T22:05:58.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:57 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2839757843' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-20T22:05:58.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mon[134090]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2023-09-20T22:05:58.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/649989131' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-20T22:05:58.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-09-20T22:05:58.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mon[134090]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-09-20T22:05:58.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mon[134090]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-20T22:05:58.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mon[134090]: osdmap e94: 12 total, 12 up, 12 in 2023-09-20T22:05:58.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-09-20T22:05:58.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mon[134090]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-09-20T22:05:58.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:57 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2839757843' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-09-20T22:05:58.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.240+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/1622589952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c18102ff0 msgr2=0x7f5c18103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:58.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.240+0000 7f5c1e1ff700 1 --2- 172.21.15.22:0/1622589952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c18102ff0 0x7f5c18103410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5c080099c0 tx=0x7f5c08009cd0 comp rx=0 tx=0).stop 2023-09-20T22:05:58.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.240+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/1622589952 shutdown_connections 2023-09-20T22:05:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.240+0000 7f5c1e1ff700 1 --2- 172.21.15.22:0/1622589952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c18104b80 0x7f5c1810b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.240+0000 7f5c1e1ff700 1 --2- 172.21.15.22:0/1622589952 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c18102ff0 0x7f5c18103410 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.240+0000 7f5c1e1ff700 1 --2- 172.21.15.22:0/1622589952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c181041e0 0x7f5c18104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.240+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/1622589952 >> 172.21.15.22:0/1622589952 conn(0x7f5c180fe560 msgr2=0x7f5c181009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.240+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/1622589952 shutdown_connections 2023-09-20T22:05:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.241+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/1622589952 wait complete. 2023-09-20T22:05:58.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.242+0000 7f5c1e1ff700 1 Processor -- start 2023-09-20T22:05:58.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.242+0000 7f5c1e1ff700 1 -- start start 2023-09-20T22:05:58.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c1e1ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c18102ff0 0x7f5c18198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:58.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c1e1ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c181041e0 0x7f5c18199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:58.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c1e1ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c18104b80 0x7f5c181a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c1e1ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c181a0850 con 0x7f5c18102ff0 2023-09-20T22:05:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c1e1ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c181a09c0 con 0x7f5c18104b80 2023-09-20T22:05:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c1e1ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c181a0b80 con 0x7f5c181041e0 2023-09-20T22:05:58.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c177fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c18102ff0 0x7f5c18198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c177fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c18102ff0 0x7f5c18198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48594/0 (socket says 172.21.15.22:48594) 2023-09-20T22:05:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c177fe700 1 -- 172.21.15.22:0/2013806167 learned_addr learned my addr 172.21.15.22:0/2013806167 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c16ffd700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c181041e0 0x7f5c18199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:58.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c17fff700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c18104b80 0x7f5c181a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c177fe700 1 -- 172.21.15.22:0/2013806167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c181041e0 msgr2=0x7f5c18199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c177fe700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c181041e0 0x7f5c18199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c177fe700 1 -- 172.21.15.22:0/2013806167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c18104b80 msgr2=0x7f5c181a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c177fe700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c18104b80 0x7f5c181a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:58.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.243+0000 7f5c177fe700 1 -- 172.21.15.22:0/2013806167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c080095f0 con 0x7f5c18102ff0 2023-09-20T22:05:58.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.244+0000 7f5c177fe700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c18102ff0 0x7f5c18198d10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009fd0 tx=0x7f5c0800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:58.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.244+0000 7f5c14ff9700 1 -- 172.21.15.22:0/2013806167 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0801d070 con 0x7f5c18102ff0 2023-09-20T22:05:58.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.244+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/2013806167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c181a0e00 con 0x7f5c18102ff0 2023-09-20T22:05:58.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.244+0000 7f5c14ff9700 1 -- 172.21.15.22:0/2013806167 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c08005050 con 0x7f5c18102ff0 2023-09-20T22:05:58.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.244+0000 7f5c14ff9700 1 -- 172.21.15.22:0/2013806167 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08004a70 con 0x7f5c18102ff0 2023-09-20T22:05:58.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.244+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/2013806167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c181a11d0 con 0x7f5c18102ff0 2023-09-20T22:05:58.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.245+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/2013806167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c18109f70 con 0x7f5c18102ff0 2023-09-20T22:05:58.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.247+0000 7f5c14ff9700 1 -- 172.21.15.22:0/2013806167 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f5c08005450 con 0x7f5c18102ff0 2023-09-20T22:05:58.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.248+0000 7f5c14ff9700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5c000a1300 0x7f5c000a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:58.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.248+0000 7f5c14ff9700 1 -- 172.21.15.22:0/2013806167 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9574+0+0 (secure 0 0 0) 0x7f5c080cdf40 con 0x7f5c18102ff0 2023-09-20T22:05:58.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.249+0000 7f5c16ffd700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5c000a1300 0x7f5c000a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:58.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.249+0000 7f5c16ffd700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5c000a1300 0x7f5c000a37c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009c60 tx=0x7f5c0c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:58.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.252+0000 7f5c14ff9700 1 -- 172.21.15.22:0/2013806167 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f5c08097100 con 0x7f5c18102ff0 2023-09-20T22:05:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:58.467+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/2013806167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5c1802d040 con 0x7f5c18102ff0 2023-09-20T22:05:59.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.007+0000 7f5c14ff9700 1 -- 172.21.15.22:0/2013806167 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5c0809a720 con 0x7f5c18102ff0 2023-09-20T22:05:59.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/2013806167 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5c000a1300 msgr2=0x7f5c000a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:59.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5c000a1300 0x7f5c000a37c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009c60 tx=0x7f5c0c009260 comp rx=0 tx=0).stop 2023-09-20T22:05:59.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/2013806167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c18102ff0 msgr2=0x7f5c18198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c18102ff0 0x7f5c18198d10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009fd0 tx=0x7f5c0800f690 comp rx=0 tx=0).stop 2023-09-20T22:05:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/2013806167 shutdown_connections 2023-09-20T22:05:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f5c000a1300 0x7f5c000a37c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:59.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5c18104b80 0x7f5c181a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c18102ff0 0x7f5c18198d10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 --2- 172.21.15.22:0/2013806167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c181041e0 0x7f5c18199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/2013806167 >> 172.21.15.22:0/2013806167 conn(0x7f5c180fe560 msgr2=0x7f5c18100800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:59.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/2013806167 shutdown_connections 2023-09-20T22:05:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.012+0000 7f5c1e1ff700 1 -- 172.21.15.22:0/2013806167 wait complete. 2023-09-20T22:05:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2023-09-20T22:05:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-20T22:05:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-20T22:05:59.032 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: Cluster is now healthy 2023-09-20T22:05:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.123:6836/2837740683,v1:172.21.15.123:6838/2837740683] up:boot 2023-09-20T22:05:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.123:6837/2262244027,v1:172.21.15.123:6839/2262244027] up:boot 2023-09-20T22:05:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.123:6834/1525351531,v1:172.21.15.123:6835/1525351531] up:boot 2023-09-20T22:05:59.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.89:6835/2426010508,v1:172.21.15.89:6837/2426010508] up:boot 2023-09-20T22:05:59.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.89:6832/3297837561,v1:172.21.15.89:6839/3297837561] up:boot 2023-09-20T22:05:59.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.89:6833/2596066773,v1:172.21.15.89:6836/2596066773] up:boot 2023-09-20T22:05:59.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.89:6834/786660201,v1:172.21.15.89:6838/786660201] up:boot 2023-09-20T22:05:59.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.22:6834/3974487960,v1:172.21.15.22:6835/3974487960] up:boot 2023-09-20T22:05:59.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.22:6836/2460943271,v1:172.21.15.22:6840/2460943271] up:boot 2023-09-20T22:05:59.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: mds.? [v2:172.21.15.22:6837/2207825619,v1:172.21.15.22:6841/2207825619] up:boot 2023-09-20T22:05:59.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2839757843' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-20T22:05:59.036 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: fsmap 12 up:standby 2023-09-20T22:05:59.036 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-20T22:05:59.036 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-20T22:05:59.036 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-20T22:05:59.036 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-20T22:05:59.037 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-20T22:05:59.037 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-20T22:05:59.037 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-20T22:05:59.037 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-20T22:05:59.037 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-20T22:05:59.038 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-20T22:05:59.038 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:58 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2013806167' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:05:59.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-20T22:05:59.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-20T22:05:59.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-20T22:05:59.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: Cluster is now healthy 2023-09-20T22:05:59.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.123:6836/2837740683,v1:172.21.15.123:6838/2837740683] up:boot 2023-09-20T22:05:59.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.123:6837/2262244027,v1:172.21.15.123:6839/2262244027] up:boot 2023-09-20T22:05:59.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.123:6834/1525351531,v1:172.21.15.123:6835/1525351531] up:boot 2023-09-20T22:05:59.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.89:6835/2426010508,v1:172.21.15.89:6837/2426010508] up:boot 2023-09-20T22:05:59.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.89:6832/3297837561,v1:172.21.15.89:6839/3297837561] up:boot 2023-09-20T22:05:59.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.89:6833/2596066773,v1:172.21.15.89:6836/2596066773] up:boot 2023-09-20T22:05:59.095 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.89:6834/786660201,v1:172.21.15.89:6838/786660201] up:boot 2023-09-20T22:05:59.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.22:6834/3974487960,v1:172.21.15.22:6835/3974487960] up:boot 2023-09-20T22:05:59.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.22:6836/2460943271,v1:172.21.15.22:6840/2460943271] up:boot 2023-09-20T22:05:59.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: mds.? [v2:172.21.15.22:6837/2207825619,v1:172.21.15.22:6841/2207825619] up:boot 2023-09-20T22:05:59.096 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2839757843' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-20T22:05:59.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: fsmap 12 up:standby 2023-09-20T22:05:59.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-20T22:05:59.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-20T22:05:59.097 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-20T22:05:59.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-20T22:05:59.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-20T22:05:59.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-20T22:05:59.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-20T22:05:59.098 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-20T22:05:59.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-20T22:05:59.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-20T22:05:59.099 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:58 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2013806167' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:05:59.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-09-20T22:05:59.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-09-20T22:05:59.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: Cluster is now healthy 2023-09-20T22:05:59.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.123:6836/2837740683,v1:172.21.15.123:6838/2837740683] up:boot 2023-09-20T22:05:59.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.123:6837/2262244027,v1:172.21.15.123:6839/2262244027] up:boot 2023-09-20T22:05:59.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.123:6834/1525351531,v1:172.21.15.123:6835/1525351531] up:boot 2023-09-20T22:05:59.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.89:6835/2426010508,v1:172.21.15.89:6837/2426010508] up:boot 2023-09-20T22:05:59.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.89:6832/3297837561,v1:172.21.15.89:6839/3297837561] up:boot 2023-09-20T22:05:59.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.89:6833/2596066773,v1:172.21.15.89:6836/2596066773] up:boot 2023-09-20T22:05:59.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.89:6834/786660201,v1:172.21.15.89:6838/786660201] up:boot 2023-09-20T22:05:59.113 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.22:6834/3974487960,v1:172.21.15.22:6835/3974487960] up:boot 2023-09-20T22:05:59.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.22:6836/2460943271,v1:172.21.15.22:6840/2460943271] up:boot 2023-09-20T22:05:59.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: mds.? [v2:172.21.15.22:6837/2207825619,v1:172.21.15.22:6841/2207825619] up:boot 2023-09-20T22:05:59.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2839757843' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-09-20T22:05:59.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: fsmap 12 up:standby 2023-09-20T22:05:59.114 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-09-20T22:05:59.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-09-20T22:05:59.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-09-20T22:05:59.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-09-20T22:05:59.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-09-20T22:05:59.115 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-09-20T22:05:59.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-09-20T22:05:59.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-09-20T22:05:59.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-09-20T22:05:59.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='mgr.14154 172.21.15.22:0/2667093595' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-09-20T22:05:59.116 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:58 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2013806167' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:05:59.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.250+0000 7f3ca49ff700 1 -- 172.21.15.22:0/3410529898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ca0101fe0 msgr2=0x7f3ca0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.250+0000 7f3ca49ff700 1 --2- 172.21.15.22:0/3410529898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ca0101fe0 0x7f3ca0102440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3c940099c0 tx=0x7f3c94009cd0 comp rx=0 tx=0).stop 2023-09-20T22:05:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.250+0000 7f3ca49ff700 1 -- 172.21.15.22:0/3410529898 shutdown_connections 2023-09-20T22:05:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.250+0000 7f3ca49ff700 1 --2- 172.21.15.22:0/3410529898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ca0102980 0x7f3ca0109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:59.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.250+0000 7f3ca49ff700 1 --2- 172.21.15.22:0/3410529898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0100df0 0x7f3ca0101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.250+0000 7f3ca49ff700 1 --2- 172.21.15.22:0/3410529898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ca0101fe0 0x7f3ca0102440 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.250+0000 7f3ca49ff700 1 -- 172.21.15.22:0/3410529898 >> 172.21.15.22:0/3410529898 conn(0x7f3ca00fc360 msgr2=0x7f3ca00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:05:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.250+0000 7f3ca49ff700 1 -- 172.21.15.22:0/3410529898 shutdown_connections 2023-09-20T22:05:59.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.250+0000 7f3ca49ff700 1 -- 172.21.15.22:0/3410529898 wait complete. 2023-09-20T22:05:59.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.251+0000 7f3ca49ff700 1 Processor -- start 2023-09-20T22:05:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.251+0000 7f3ca49ff700 1 -- start start 2023-09-20T22:05:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3ca49ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0100df0 0x7f3ca0074b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3ca49ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ca0101fe0 0x7f3ca00731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3ca49ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ca0102980 0x7f3ca0073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:59.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3ca49ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0073eb0 con 0x7f3ca0102980 2023-09-20T22:05:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3ca49ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0074020 con 0x7f3ca0101fe0 2023-09-20T22:05:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3ca49ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca01a0b00 con 0x7f3ca0100df0 2023-09-20T22:05:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9ed9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ca0102980 0x7f3ca0073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0100df0 0x7f3ca0074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:59.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9dd9b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ca0101fe0 0x7f3ca00731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:59.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0100df0 0x7f3ca0074b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.22:58654/0 (socket says 172.21.15.22:58654) 2023-09-20T22:05:59.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9e59c700 1 -- 172.21.15.22:0/4033039981 learned_addr learned my addr 172.21.15.22:0/4033039981 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:05:59.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9ed9d700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ca0102980 0x7f3ca0073710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48598/0 (socket says 172.21.15.22:48598) 2023-09-20T22:05:59.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9e59c700 1 -- 172.21.15.22:0/4033039981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ca0101fe0 msgr2=0x7f3ca00731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:59.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9e59c700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ca0101fe0 0x7f3ca00731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:59.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9e59c700 1 -- 172.21.15.22:0/4033039981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ca0102980 msgr2=0x7f3ca0073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:05:59.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9e59c700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ca0102980 0x7f3ca0073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:05:59.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.252+0000 7f3c9e59c700 1 -- 172.21.15.22:0/4033039981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c940095f0 con 0x7f3ca0100df0 2023-09-20T22:05:59.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.253+0000 7f3c9e59c700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0100df0 0x7f3ca0074b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000e970 tx=0x7f3c9000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:59.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.253+0000 7f3c8f7fe700 1 -- 172.21.15.22:0/4033039981 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90009790 con 0x7f3ca0100df0 2023-09-20T22:05:59.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.253+0000 7f3ca49ff700 1 -- 172.21.15.22:0/4033039981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca01a0de0 con 0x7f3ca0100df0 2023-09-20T22:05:59.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.253+0000 7f3c8f7fe700 1 -- 172.21.15.22:0/4033039981 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c9000c7e0 con 0x7f3ca0100df0 2023-09-20T22:05:59.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.253+0000 7f3c8f7fe700 1 -- 172.21.15.22:0/4033039981 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90003c70 con 0x7f3ca0100df0 2023-09-20T22:05:59.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.253+0000 7f3ca49ff700 1 -- 172.21.15.22:0/4033039981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca01a1210 con 0x7f3ca0100df0 2023-09-20T22:05:59.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.254+0000 7f3ca49ff700 1 -- 172.21.15.22:0/4033039981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca010c670 con 0x7f3ca0100df0 2023-09-20T22:05:59.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.256+0000 7f3c8f7fe700 1 -- 172.21.15.22:0/4033039981 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f3c9000cc50 con 0x7f3ca0100df0 2023-09-20T22:05:59.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.257+0000 7f3c8f7fe700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3c880a1300 0x7f3c880a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:05:59.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.257+0000 7f3c8f7fe700 1 -- 172.21.15.22:0/4033039981 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9041+0+0 (secure 0 0 0) 0x7f3c90014070 con 0x7f3ca0100df0 2023-09-20T22:05:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.258+0000 7f3c9dd9b700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3c880a1300 0x7f3c880a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:05:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.258+0000 7f3c9dd9b700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3c880a1300 0x7f3c880a37c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400b580 tx=0x7f3c94005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:05:59.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.259+0000 7f3c8f7fe700 1 -- 172.21.15.22:0/4033039981 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f3c900965e0 con 0x7f3ca0100df0 2023-09-20T22:05:59.472 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:05:59.472+0000 7f3ca49ff700 1 -- 172.21.15.22:0/4033039981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3ca0107cd0 con 0x7f3ca0100df0 2023-09-20T22:06:00.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.008+0000 7f3c8f7fe700 1 -- 172.21.15.22:0/4033039981 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3c90003dd0 con 0x7f3ca0100df0 2023-09-20T22:06:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.012+0000 7f3ca49ff700 1 -- 172.21.15.22:0/4033039981 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3c880a1300 msgr2=0x7f3c880a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:06:00.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.012+0000 7f3ca49ff700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3c880a1300 0x7f3c880a37c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400b580 tx=0x7f3c94005fb0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.012+0000 7f3ca49ff700 1 -- 172.21.15.22:0/4033039981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0100df0 msgr2=0x7f3ca0074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:06:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.012+0000 7f3ca49ff700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0100df0 0x7f3ca0074b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000e970 tx=0x7f3c9000ec80 comp rx=0 tx=0).stop 2023-09-20T22:06:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.012+0000 7f3ca49ff700 1 -- 172.21.15.22:0/4033039981 shutdown_connections 2023-09-20T22:06:00.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.012+0000 7f3ca49ff700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ca0102980 0x7f3ca0073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.012+0000 7f3ca49ff700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ca0100df0 0x7f3ca0074b90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.012+0000 7f3ca49ff700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f3c880a1300 0x7f3c880a37c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.012+0000 7f3ca49ff700 1 --2- 172.21.15.22:0/4033039981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ca0101fe0 0x7f3ca00731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.012+0000 7f3ca49ff700 1 -- 172.21.15.22:0/4033039981 >> 172.21.15.22:0/4033039981 conn(0x7f3ca00fc360 msgr2=0x7f3ca00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:06:00.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.013+0000 7f3ca49ff700 1 -- 172.21.15.22:0/4033039981 shutdown_connections 2023-09-20T22:06:00.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.013+0000 7f3ca49ff700 1 -- 172.21.15.22:0/4033039981 wait complete. 2023-09-20T22:06:00.016 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2023-09-20T22:06:00.033 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:59 smithi022 ceph-mon[97651]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2023-09-20T22:06:00.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:59 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/2013806167' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-20T22:06:00.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:59 smithi022 ceph-mon[97651]: osdmap e95: 12 total, 12 up, 12 in 2023-09-20T22:06:00.034 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:59 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4033039981' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:06:00.035 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:05:59 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:06:00.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-09-20T22:06:00.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:59 smithi123 ceph-mon[134433]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2023-09-20T22:06:00.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:59 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/2013806167' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-20T22:06:00.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:59 smithi123 ceph-mon[134433]: osdmap e95: 12 total, 12 up, 12 in 2023-09-20T22:06:00.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:59 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4033039981' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:06:00.094 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:05:59 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:06:00.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:59 smithi089 ceph-mon[134090]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2023-09-20T22:06:00.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:59 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/2013806167' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-09-20T22:06:00.110 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:59 smithi089 ceph-mon[134090]: osdmap e95: 12 total, 12 up, 12 in 2023-09-20T22:06:00.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:59 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4033039981' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:06:00.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:05:59 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:06:00.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.247+0000 7f85f99ff700 1 -- 172.21.15.22:0/4081542421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f4101fc0 msgr2=0x7f85f4102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:06:00.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.247+0000 7f85f99ff700 1 --2- 172.21.15.22:0/4081542421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f4101fc0 0x7f85f4102420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f85e80099c0 tx=0x7f85e8009cd0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.247+0000 7f85f99ff700 1 -- 172.21.15.22:0/4081542421 shutdown_connections 2023-09-20T22:06:00.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.247+0000 7f85f99ff700 1 --2- 172.21.15.22:0/4081542421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85f4102960 0x7f85f4109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.247+0000 7f85f99ff700 1 --2- 172.21.15.22:0/4081542421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85f4100dd0 0x7f85f41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.247+0000 7f85f99ff700 1 --2- 172.21.15.22:0/4081542421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f4101fc0 0x7f85f4102420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.247+0000 7f85f99ff700 1 -- 172.21.15.22:0/4081542421 >> 172.21.15.22:0/4081542421 conn(0x7f85f40fc320 msgr2=0x7f85f40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:06:00.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.248+0000 7f85f99ff700 1 -- 172.21.15.22:0/4081542421 shutdown_connections 2023-09-20T22:06:00.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.248+0000 7f85f99ff700 1 -- 172.21.15.22:0/4081542421 wait complete. 2023-09-20T22:06:00.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f99ff700 1 Processor -- start 2023-09-20T22:06:00.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f99ff700 1 -- start start 2023-09-20T22:06:00.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f99ff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f4100dd0 0x7f85f4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:06:00.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f99ff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85f4101fc0 0x7f85f4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:06:00.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f99ff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85f4102960 0x7f85f41a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:06:00.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f99ff700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f41a0830 con 0x7f85f4100dd0 2023-09-20T22:06:00.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f99ff700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f41a09a0 con 0x7f85f4101fc0 2023-09-20T22:06:00.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f99ff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f41a0b60 con 0x7f85f4102960 2023-09-20T22:06:00.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f4100dd0 0x7f85f4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:06:00.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f4100dd0 0x7f85f4198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48606/0 (socket says 172.21.15.22:48606) 2023-09-20T22:06:00.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f2ffd700 1 -- 172.21.15.22:0/4257849767 learned_addr learned my addr 172.21.15.22:0/4257849767 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-09-20T22:06:00.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.249+0000 7f85f27fc700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85f4101fc0 0x7f85f4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:06:00.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f37fe700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85f4102960 0x7f85f41a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:06:00.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f2ffd700 1 -- 172.21.15.22:0/4257849767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85f4102960 msgr2=0x7f85f41a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:06:00.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f2ffd700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85f4102960 0x7f85f41a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f2ffd700 1 -- 172.21.15.22:0/4257849767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85f4101fc0 msgr2=0x7f85f4199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:06:00.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f2ffd700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85f4101fc0 0x7f85f4199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:00.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f2ffd700 1 -- 172.21.15.22:0/4257849767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e80095f0 con 0x7f85f4100dd0 2023-09-20T22:06:00.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f27fc700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85f4101fc0 0x7f85f4199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-20T22:06:00.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f2ffd700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f4100dd0 0x7f85f4198cf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f85e400e970 tx=0x7f85e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:06:00.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f89fd700 1 -- 172.21.15.22:0/4257849767 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4009790 con 0x7f85f4100dd0 2023-09-20T22:06:00.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f99ff700 1 -- 172.21.15.22:0/4257849767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f41a0e40 con 0x7f85f4100dd0 2023-09-20T22:06:00.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f89fd700 1 -- 172.21.15.22:0/4257849767 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e400c7e0 con 0x7f85f4100dd0 2023-09-20T22:06:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.250+0000 7f85f89fd700 1 -- 172.21.15.22:0/4257849767 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4003c70 con 0x7f85f4100dd0 2023-09-20T22:06:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.251+0000 7f85f99ff700 1 -- 172.21.15.22:0/4257849767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f41a1260 con 0x7f85f4100dd0 2023-09-20T22:06:00.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.252+0000 7f85f99ff700 1 -- 172.21.15.22:0/4257849767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f410c5e0 con 0x7f85f4100dd0 2023-09-20T22:06:00.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.254+0000 7f85f89fd700 1 -- 172.21.15.22:0/4257849767 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 16) v1 ==== 136120+0+0 (secure 0 0 0) 0x7f85e400cc50 con 0x7f85f4100dd0 2023-09-20T22:06:00.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.255+0000 7f85f89fd700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f85dc0a1300 0x7f85dc0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-20T22:06:00.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.255+0000 7f85f89fd700 1 -- 172.21.15.22:0/4257849767 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8424+0+0 (secure 0 0 0) 0x7f85e4014070 con 0x7f85f4100dd0 2023-09-20T22:06:00.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.255+0000 7f85f27fc700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f85dc0a1300 0x7f85dc0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-20T22:06:00.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.256+0000 7f85f27fc700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f85dc0a1300 0x7f85dc0a37c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f85e800b580 tx=0x7f85e8005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-20T22:06:00.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.257+0000 7f85f89fd700 1 -- 172.21.15.22:0/4257849767 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179712 (secure 0 0 0) 0x7f85e4096620 con 0x7f85f4100dd0 2023-09-20T22:06:00.468 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:00.468+0000 7f85f99ff700 1 -- 172.21.15.22:0/4257849767 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f85f410c7c0 con 0x7f85f4100dd0 2023-09-20T22:06:01.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.015+0000 7f85f89fd700 1 -- 172.21.15.22:0/4257849767 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) v1 ==== 178+0+0 (secure 0 0 0) 0x7f85e4099c40 con 0x7f85f4100dd0 2023-09-20T22:06:01.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.021+0000 7f85f99ff700 1 -- 172.21.15.22:0/4257849767 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f85dc0a1300 msgr2=0x7f85dc0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:06:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.021+0000 7f85f99ff700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f85dc0a1300 0x7f85dc0a37c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f85e800b580 tx=0x7f85e8005fb0 comp rx=0 tx=0).stop 2023-09-20T22:06:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.021+0000 7f85f99ff700 1 -- 172.21.15.22:0/4257849767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f4100dd0 msgr2=0x7f85f4198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-20T22:06:01.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.021+0000 7f85f99ff700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f4100dd0 0x7f85f4198cf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f85e400e970 tx=0x7f85e400ec80 comp rx=0 tx=0).stop 2023-09-20T22:06:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.022+0000 7f85f99ff700 1 -- 172.21.15.22:0/4257849767 shutdown_connections 2023-09-20T22:06:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.022+0000 7f85f99ff700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85f4102960 0x7f85f41a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.022+0000 7f85f99ff700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f4100dd0 0x7f85f4198cf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.022+0000 7f85f99ff700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.22:6800/718010003,v1:172.21.15.22:6801/718010003] conn(0x7f85dc0a1300 0x7f85dc0a37c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:01.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.022+0000 7f85f99ff700 1 --2- 172.21.15.22:0/4257849767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f85f4101fc0 0x7f85f4199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-20T22:06:01.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.022+0000 7f85f99ff700 1 -- 172.21.15.22:0/4257849767 >> 172.21.15.22:0/4257849767 conn(0x7f85f40fc320 msgr2=0x7f85f40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-20T22:06:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.025+0000 7f85f99ff700 1 -- 172.21.15.22:0/4257849767 shutdown_connections 2023-09-20T22:06:01.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-09-20T22:06:01.025+0000 7f85f99ff700 1 -- 172.21.15.22:0/4257849767 wait complete. 2023-09-20T22:06:01.036 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data_ec' removed 2023-09-20T22:06:01.062 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-20T22:06:01.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-20T22:06:01.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-20T22:06:01.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:06:01 smithi123 ceph-mon[134433]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-20T22:06:01.281 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:06:01 smithi123 ceph-mon[134433]: osdmap e96: 12 total, 12 up, 12 in 2023-09-20T22:06:01.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:06:01 smithi123 ceph-mon[134433]: pgmap v862: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T22:06:01.282 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:06:01 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4257849767' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:06:01.299 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-09-20T22:06:01.299 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-20T22:06:01.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:06:01 smithi089 ceph-mon[134090]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-20T22:06:01.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:06:01 smithi089 ceph-mon[134090]: osdmap e96: 12 total, 12 up, 12 in 2023-09-20T22:06:01.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:06:01 smithi089 ceph-mon[134090]: pgmap v862: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T22:06:01.361 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:06:01 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4257849767' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:06:01.372 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-20T22:06:01.393 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-09-20T22:06:01.411 INFO:tasks.cephadm:Stopping all daemons... 2023-09-20T22:06:01.412 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-09-20T22:06:01.412 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.a 2023-09-20T22:06:01.474 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:06:01 smithi022 ceph-mon[97651]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-09-20T22:06:01.475 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:06:01 smithi022 ceph-mon[97651]: osdmap e96: 12 total, 12 up, 12 in 2023-09-20T22:06:01.475 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:06:01 smithi022 ceph-mon[97651]: pgmap v862: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2023-09-20T22:06:01.475 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:06:01 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4257849767' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-09-20T22:06:01.748 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:06:01 smithi022 systemd[1]: Stopping Ceph mon.a for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:02.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:06:02 smithi123 ceph-mon[134433]: from='client.? 172.21.15.22:0/4257849767' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-09-20T22:06:02.343 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:06:02 smithi123 ceph-mon[134433]: osdmap e97: 12 total, 12 up, 12 in 2023-09-20T22:06:02.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:06:02 smithi089 ceph-mon[134090]: from='client.? 172.21.15.22:0/4257849767' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-09-20T22:06:02.360 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:06:02 smithi089 ceph-mon[134090]: osdmap e97: 12 total, 12 up, 12 in 2023-09-20T22:06:02.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:06:02 smithi022 ceph-mon[97651]: from='client.? 172.21.15.22:0/4257849767' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2023-09-20T22:06:02.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:06:02 smithi022 ceph-mon[97651]: osdmap e97: 12 total, 12 up, 12 in 2023-09-20T22:06:02.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:06:02 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-a[97627]: 2023-09-20T22:06:02.039+0000 7f30e8aa0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:06:02.418 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:06:02 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-a[97627]: 2023-09-20T22:06:02.039+0000 7f30e8aa0700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-09-20T22:06:02.747 INFO:journalctl@ceph.mon.a.smithi022.stdout:Sep 20 22:06:02 smithi022 bash[174828]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-a 2023-09-20T22:06:03.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.a.service' 2023-09-20T22:06:03.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:03.139 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-09-20T22:06:03.140 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-09-20T22:06:03.140 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.b 2023-09-20T22:06:03.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:06:03 smithi089 systemd[1]: Stopping Ceph mon.b for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:04.111 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:06:03 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-b[134066]: 2023-09-20T22:06:03.623+0000 7fd5da5bb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:06:04.112 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:06:03 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-b[134066]: 2023-09-20T22:06:03.623+0000 7fd5da5bb700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-09-20T22:06:04.610 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 20 22:06:04 smithi089 bash[177448]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-b 2023-09-20T22:06:04.691 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.b.service' 2023-09-20T22:06:04.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:04.755 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-09-20T22:06:04.755 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-09-20T22:06:04.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.c 2023-09-20T22:06:05.093 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:06:04 smithi123 systemd[1]: Stopping Ceph mon.c for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:05.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:06:05 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-c[134409]: 2023-09-20T22:06:05.183+0000 7f144ad1e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:06:05.593 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:06:05 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-c[134409]: 2023-09-20T22:06:05.183+0000 7f144ad1e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-09-20T22:06:06.049 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 20 22:06:05 smithi123 bash[178694]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mon-c 2023-09-20T22:06:06.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mon.c.service' 2023-09-20T22:06:06.113 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:06.113 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-09-20T22:06:06.114 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-09-20T22:06:06.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.x 2023-09-20T22:06:06.497 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 22:06:06 smithi022 systemd[1]: Stopping Ceph mgr.x for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:09.247 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 22:06:08 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T22:06:08.764+0000 7f516b9f9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:06:09.248 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 22:06:08 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x[97899]: 2023-09-20T22:06:08.764+0000 7f516b9f9700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-09-20T22:06:09.997 INFO:journalctl@ceph.mgr.x.smithi022.stdout:Sep 20 22:06:09 smithi022 bash[174972]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-x 2023-09-20T22:06:10.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.x.service' 2023-09-20T22:06:10.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:10.328 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-09-20T22:06:10.329 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-09-20T22:06:10.329 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.y 2023-09-20T22:06:10.860 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 22:06:10 smithi089 systemd[1]: Stopping Ceph mgr.y for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:12.110 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 22:06:11 smithi089 bash[177603]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-y 2023-09-20T22:06:12.110 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 20 22:06:11 smithi089 systemd[1]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-09-20T22:06:12.134 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.y.service' 2023-09-20T22:06:12.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:12.173 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-09-20T22:06:12.173 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-09-20T22:06:12.174 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.z 2023-09-20T22:06:12.593 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 22:06:12 smithi123 systemd[1]: Stopping Ceph mgr.z for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:12.884 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 22:06:12 smithi123 bash[178852]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mgr-z 2023-09-20T22:06:13.065 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 20 22:06:12 smithi123 systemd[1]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-09-20T22:06:13.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mgr.z.service' 2023-09-20T22:06:13.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:13.107 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-09-20T22:06:13.107 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-09-20T22:06:13.107 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.0 2023-09-20T22:06:13.497 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 22:06:13 smithi022 systemd[1]: Stopping Ceph osd.0 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:13.997 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 22:06:13 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-0[116162]: 2023-09-20T22:06:13.628+0000 7fb6c428e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:06:13.997 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 22:06:13 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-0[116162]: 2023-09-20T22:06:13.628+0000 7fb6c428e700 -1 osd.0 97 *** Got signal Terminated *** 2023-09-20T22:06:13.998 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 22:06:13 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-0[116162]: 2023-09-20T22:06:13.628+0000 7fb6c428e700 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:06:20.247 INFO:journalctl@ceph.osd.0.smithi022.stdout:Sep 20 22:06:19 smithi022 bash[175117]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-0 2023-09-20T22:06:22.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.0.service' 2023-09-20T22:06:22.380 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:22.380 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-09-20T22:06:22.381 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-09-20T22:06:22.381 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.1 2023-09-20T22:06:22.750 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 22:06:22 smithi089 systemd[1]: Stopping Ceph osd.1 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:23.110 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 22:06:22 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-1[139100]: 2023-09-20T22:06:22.749+0000 7f282a9c9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:06:23.110 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 22:06:22 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-1[139100]: 2023-09-20T22:06:22.749+0000 7f282a9c9700 -1 osd.1 97 *** Got signal Terminated *** 2023-09-20T22:06:23.111 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 22:06:22 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-1[139100]: 2023-09-20T22:06:22.749+0000 7f282a9c9700 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:06:28.860 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 20 22:06:28 smithi089 bash[177751]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-1 2023-09-20T22:06:30.248 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.1.service' 2023-09-20T22:06:30.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:30.288 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-09-20T22:06:30.289 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-09-20T22:06:30.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.2 2023-09-20T22:06:30.593 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 22:06:30 smithi123 systemd[1]: Stopping Ceph osd.2 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:31.093 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 22:06:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-2[140228]: 2023-09-20T22:06:30.645+0000 7f25297e5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:06:31.093 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 22:06:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-2[140228]: 2023-09-20T22:06:30.645+0000 7f25297e5700 -1 osd.2 97 *** Got signal Terminated *** 2023-09-20T22:06:31.093 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 22:06:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-2[140228]: 2023-09-20T22:06:30.645+0000 7f25297e5700 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:06:36.527 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 20 22:06:36 smithi123 bash[179002]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-2 2023-09-20T22:06:37.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.2.service' 2023-09-20T22:06:37.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:37.828 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-09-20T22:06:37.829 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-09-20T22:06:37.829 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.3 2023-09-20T22:06:38.247 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 22:06:37 smithi022 systemd[1]: Stopping Ceph osd.3 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:38.747 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 22:06:38 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3[124679]: 2023-09-20T22:06:38.365+0000 7f4e829cf700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:06:38.747 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 22:06:38 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3[124679]: 2023-09-20T22:06:38.365+0000 7f4e829cf700 -1 osd.3 97 *** Got signal Terminated *** 2023-09-20T22:06:38.748 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 22:06:38 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3[124679]: 2023-09-20T22:06:38.365+0000 7f4e829cf700 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:06:40.497 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 22:06:40 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3[124679]: 2023-09-20T22:06:40.103+0000 7f4e7afbd700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.315040+0000 front 2023-09-20T22:06:17.315223+0000 (oldest deadline 2023-09-20T22:06:39.615240+0000) 2023-09-20T22:06:40.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:40 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:40.434+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:41.497 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 22:06:41 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3[124679]: 2023-09-20T22:06:41.117+0000 7f4e7afbd700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.315040+0000 front 2023-09-20T22:06:17.315223+0000 (oldest deadline 2023-09-20T22:06:39.615240+0000) 2023-09-20T22:06:41.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:41 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:41.163+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:41.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:41 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:41.414+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:42.497 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 22:06:42 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3[124679]: 2023-09-20T22:06:42.161+0000 7f4e7afbd700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.315040+0000 front 2023-09-20T22:06:17.315223+0000 (oldest deadline 2023-09-20T22:06:39.615240+0000) 2023-09-20T22:06:42.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:42 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:42.051+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:42.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:42 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:42.117+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:42.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:42 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:42.389+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:43.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:43.080+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:43.497 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 22:06:43 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3[124679]: 2023-09-20T22:06:43.137+0000 7f4e7afbd700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.315040+0000 front 2023-09-20T22:06:17.315223+0000 (oldest deadline 2023-09-20T22:06:39.615240+0000) 2023-09-20T22:06:43.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:43 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:43.069+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:43.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:43 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:43.385+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:44.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:44 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:44.090+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:44.455 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:44 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:44.054+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:44.716 INFO:journalctl@ceph.osd.3.smithi022.stdout:Sep 20 22:06:44 smithi022 bash[175397]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-3 2023-09-20T22:06:44.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:44 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:44.381+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:44.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:44 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:44.376+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:45.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:45 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:45.048+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:45.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:45 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:45.084+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:45.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:45 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:45.354+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:45.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:45 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:45.379+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:46.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:46 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:46.082+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:46.354 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:46 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:46.065+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:46.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.3.service' 2023-09-20T22:06:46.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:46.751 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-09-20T22:06:46.751 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-09-20T22:06:46.752 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.4 2023-09-20T22:06:46.781 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:46 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:46.363+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:46.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:46 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:46.349+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:47.110 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 20 22:06:46 smithi089 systemd[1]: Stopping Ceph osd.4 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:47.373 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:47 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:47.094+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:47.388 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 20 22:06:47 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-4[147603]: 2023-09-20T22:06:47.111+0000 7f01db5f1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:06:47.388 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 20 22:06:47 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-4[147603]: 2023-09-20T22:06:47.111+0000 7f01db5f1700 -1 osd.4 97 *** Got signal Terminated *** 2023-09-20T22:06:47.388 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 20 22:06:47 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-4[147603]: 2023-09-20T22:06:47.111+0000 7f01db5f1700 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:06:47.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:47 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:47.062+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:47.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:47 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:47.374+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:47.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:47 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:47.388+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:48.409 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:48 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:48.139+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:48.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:48 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:48.097+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:48.780 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:48 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:48.370+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:48.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:48 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:48.409+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:49.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:48 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:48.780+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:49.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:49 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:49.094+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:49.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:49 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:49.080+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:49.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:49 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:49.146+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:49.751 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:49 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:49.413+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:49.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:49 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:49.373+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:50.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:49 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:49.750+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:50.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:50 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:50.044+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:50.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:50 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:50.130+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:50.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:50 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:50.128+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:50.610 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 20 22:06:50 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-4[147603]: 2023-09-20T22:06:50.208+0000 7f01d4be1700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.798641+0000 front 2023-09-20T22:06:26.798670+0000 (oldest deadline 2023-09-20T22:06:50.199307+0000) 2023-09-20T22:06:50.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:50 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:50.384+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:50.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:50 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:50.421+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:51.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:50 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:50.724+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:51.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:51 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:51.083+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:51.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:51 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:51.147+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:51.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:51 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:51.141+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:51.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:51 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:51.062+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:06:51.610 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 20 22:06:51 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-4[147603]: 2023-09-20T22:06:51.224+0000 7f01d4be1700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.798641+0000 front 2023-09-20T22:06:26.798670+0000 (oldest deadline 2023-09-20T22:06:50.199307+0000) 2023-09-20T22:06:51.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:51 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:51.371+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:51.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:51 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:51.460+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:52.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:51 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:51.714+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:52.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:52 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:52.066+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:52.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:52 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:52.160+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:52.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:52 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:52.111+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:06:52.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:52 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:52.117+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:52.709 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:52 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:52.405+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:52.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:52 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:52.505+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:52.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:52 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:52.505+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.565407+0000 front 2023-09-20T22:06:27.565267+0000 (oldest deadline 2023-09-20T22:06:51.665419+0000) 2023-09-20T22:06:53.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:52 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:52.709+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:53.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:53 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:53.196+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:53.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:53 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:53.054+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:53.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:53 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:53.146+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:06:53.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:53 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:53.109+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:53.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:53 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:53.109+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:06:53.610 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 20 22:06:53 smithi089 bash[178024]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-4 2023-09-20T22:06:53.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:53 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:53.373+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:53.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:53 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:53.373+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:06:53.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:53 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:53.476+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:53.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:53 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:53.476+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.565407+0000 front 2023-09-20T22:06:27.565267+0000 (oldest deadline 2023-09-20T22:06:51.665419+0000) 2023-09-20T22:06:54.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:53 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:53.690+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:54.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:54 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:54.028+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:54.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:54 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:54.200+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:54.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:54 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:54.126+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:06:54.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:54 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:54.144+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:54.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:54 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:54.144+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:06:54.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:54 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:54.341+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:54.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:54 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:54.341+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:06:54.657 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.4.service' 2023-09-20T22:06:54.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:06:54.738 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-09-20T22:06:54.738 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-09-20T22:06:54.738 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.5 2023-09-20T22:06:54.769 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:54 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:54.490+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:54.769 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:54 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:54.490+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.565407+0000 front 2023-09-20T22:06:27.565267+0000 (oldest deadline 2023-09-20T22:06:51.665419+0000) 2023-09-20T22:06:55.093 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:54 smithi123 systemd[1]: Stopping Ceph osd.5 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:06:55.093 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:54 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:54.994+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:55.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:54 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:54.661+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:55.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:55 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:55.109+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:06:55.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:55 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:55.189+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:55.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:55 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:55.189+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:06:55.527 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:55 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:55.212+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:55.527 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:55 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:55.099+0000 7f0d497d2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:06:55.528 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:55 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:55.099+0000 7f0d497d2700 -1 osd.5 97 *** Got signal Terminated *** 2023-09-20T22:06:55.528 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:55 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:55.099+0000 7f0d497d2700 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:06:55.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:55 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:55.341+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:55.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:55 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:55.341+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:06:55.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:55 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:55.527+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:55.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:55 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:55.527+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.565407+0000 front 2023-09-20T22:06:27.565267+0000 (oldest deadline 2023-09-20T22:06:51.665419+0000) 2023-09-20T22:06:56.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:55 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:55.658+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:56.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:55 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:55.980+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:56.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:56 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:56.207+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:56.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:56 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:56.085+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:06:56.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:56 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:56.193+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:56.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:56 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:56.193+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:06:56.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:56 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:56.324+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:56.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:56 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:56.324+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:06:56.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:56 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:56.479+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:56.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:56 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:56.479+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.565407+0000 front 2023-09-20T22:06:27.565267+0000 (oldest deadline 2023-09-20T22:06:51.665419+0000) 2023-09-20T22:06:57.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:56 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:56.705+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:57.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:56 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:56.950+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:57.344 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:57.187+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:57.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:57.132+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:06:57.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:57.132+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:06:57.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:57.176+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:57.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:57 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:57.176+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:06:57.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:57.298+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:57.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:57.298+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:06:57.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:57.449+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:57.844 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:57.449+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.565407+0000 front 2023-09-20T22:06:27.565267+0000 (oldest deadline 2023-09-20T22:06:51.665419+0000) 2023-09-20T22:06:58.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:57 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:57.683+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:58.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:57 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:57.962+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:58.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:58 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:58.192+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:58.344 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:58 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:58.192+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:06:58.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:58.180+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:06:58.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:58.180+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:06:58.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:58.140+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:58.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:58.140+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:06:58.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:58 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:58.271+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:58.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:58 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:58.271+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:06:58.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:58 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:58.451+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:58.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:58 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:58.451+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.565407+0000 front 2023-09-20T22:06:27.565267+0000 (oldest deadline 2023-09-20T22:06:51.665419+0000) 2023-09-20T22:06:59.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:58 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:58.677+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:06:59.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:06:58 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:06:58.962+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:06:59.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:59 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:59.168+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:06:59.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:06:59 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:06:59.168+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:06:59.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:59 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:59.182+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:06:59.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:06:59 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:06:59.182+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:06:59.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:59 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:59.092+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:06:59.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:06:59 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:06:59.092+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:06:59.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:59 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:59.289+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:06:59.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:06:59 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:06:59.289+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:06:59.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:59 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:59.487+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.164238+0000 front 2023-09-20T22:06:18.164223+0000 (oldest deadline 2023-09-20T22:06:44.064160+0000) 2023-09-20T22:06:59.843 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:06:59 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5[148132]: 2023-09-20T22:06:59.487+0000 7f0d445c5700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.565407+0000 front 2023-09-20T22:06:27.565267+0000 (oldest deadline 2023-09-20T22:06:51.665419+0000) 2023-09-20T22:07:00.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:06:59 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:06:59.726+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:00.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:00 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:00.008+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:00.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:00 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:00.159+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:00.344 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:00 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:00.159+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:00.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:00 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:00.177+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:07:00.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:00 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:00.177+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:07:00.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:00 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:00.103+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:00.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:00 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:00.103+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:00.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:00 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:00.249+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:00.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:00 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:00.249+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:01.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:00 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:00.768+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:01.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:00 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:00.768+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:01.181 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:00 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:00.964+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:01.181 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:00 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:00.964+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:01.182 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:01 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:01.129+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:01.182 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:01 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:01.129+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:01.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:01 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:01.171+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:07:01.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:01 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:01.171+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:07:01.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:01 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:01.130+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:01.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:01 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:01.130+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:01.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:01 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:01.130+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:01.593 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:07:01 smithi123 bash[179264]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-5 2023-09-20T22:07:01.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:01 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:01.215+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:01.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:01 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:01.215+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:01.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:01 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:01.215+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:02.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:01 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:01.774+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:02.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:01 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:01.774+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:02.144 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:01 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:01.937+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:02.145 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:01 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:01.937+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:02.414 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:02 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:02.144+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:02.415 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:02 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:02.144+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:02.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:02 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:02.136+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:07:02.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:02 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:02.136+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:07:02.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:02 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:02.126+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:02.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:02 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:02.126+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:02.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:02 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:02.126+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:02.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:02 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:02.184+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:02.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:02 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:02.184+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:02.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:02 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:02.184+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:02.638 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.5.service' 2023-09-20T22:07:02.704 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:07:02 smithi123 systemd[1]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.5.service: Succeeded. 2023-09-20T22:07:02.705 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 20 22:07:02 smithi123 systemd[1]: Stopped Ceph osd.5 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da. 2023-09-20T22:07:02.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:07:02.718 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-09-20T22:07:02.719 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-09-20T22:07:02.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.6 2023-09-20T22:07:03.093 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:02 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:02.920+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:03.093 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:02 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:02.920+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:03.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:02 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:02.787+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:03.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:02 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:02.787+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:03.172 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:02 smithi022 systemd[1]: Stopping Ceph osd.6 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:07:03.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:03.173+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:07:03.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:03.174+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:07:03.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:03.253+0000 7f281dcc8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:07:03.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:03.253+0000 7f281dcc8700 -1 osd.6 97 *** Got signal Terminated *** 2023-09-20T22:07:03.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:03.253+0000 7f281dcc8700 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:07:03.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:03.171+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:03.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:03.171+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:03.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:03.171+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:03.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:03 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:03.190+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:03.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:03 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:03.190+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:03.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:03 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:03.182+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:03.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:03 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:03.182+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:03.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:03 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:03.182+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:04.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:03 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:03.770+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:04.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:03 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:03.770+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:04.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:04 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:04.142+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:04.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:04 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:04.142+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:04.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:03 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:03.926+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:04.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:03 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:03.926+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:04.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:04 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:04.206+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:07:04.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:04 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:04.206+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:07:04.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:04 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:04.186+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:04.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:04 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:04.186+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:04.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:04 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:04.186+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:04.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:04 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:04.175+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:04.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:04 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:04.175+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:04.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:04 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:04.175+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:05.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:04 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:04.753+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:05.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:04 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:04.753+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:05.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:05 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:05.138+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:05.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:05 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:05.138+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:05.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:04 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:04.972+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:05.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:04 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:04.972+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:05.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:04 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:04.972+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:05.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:05.245+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:07:05.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:05.245+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:07:05.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:05.178+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:05.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:05.178+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:05.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:05 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:05.178+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:05.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:05 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:05.134+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:05.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:05 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:05.134+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:05.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:05 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:05.134+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:05.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:05 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:05.134+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:06.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:05 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:05.779+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:06.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:05 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:05.779+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:06.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:05 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:05.779+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:06.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:06 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:06.134+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:06.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:06 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:06.134+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:06.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:05 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:05.978+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:06.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:05 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:05.978+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:06.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:05 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:05.978+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:06.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:06 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:06.250+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:07:06.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:06 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:06.250+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:07:06.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:06 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:06.250+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.972980+0000 front 2023-09-20T22:06:42.972911+0000 (oldest deadline 2023-09-20T22:07:05.273115+0000) 2023-09-20T22:07:06.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:06 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:06.191+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:06.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:06 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:06.191+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:06.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:06 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:06.191+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:06.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:06 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:06.170+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:06.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:06 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:06.170+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:06.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:06 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:06.170+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:06.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:06 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:06.170+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:07.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:06 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:06.810+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:07.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:06 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:06.810+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:07.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:06 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:06.810+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:07.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:07 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:07.156+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:07.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:07 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:07.156+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:07.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:07 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:07.026+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:07.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:07 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:07.026+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:07.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:07 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:07.026+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:07.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:07 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:07.249+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:07:07.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:07 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:07.249+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:07:07.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:07 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:07.249+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.972980+0000 front 2023-09-20T22:06:42.972911+0000 (oldest deadline 2023-09-20T22:07:05.273115+0000) 2023-09-20T22:07:07.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:07 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:07.240+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:07.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:07 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:07.240+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:07.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:07 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:07.240+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:07.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:07 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:07.145+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:07.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:07 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:07.145+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:07.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:07 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:07.145+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:07.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:07 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:07.145+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:08.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:07 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:07.795+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:08.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:07 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:07.795+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:08.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:07 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:07.795+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:08.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:08 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:08.120+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:08.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:08 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:08.120+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:08.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:08 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:08.004+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:08.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:08 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:08.004+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:08.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:08 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:08.004+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:08.497 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:08 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:08.217+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.267809+0000 front 2023-09-20T22:06:26.268040+0000 (oldest deadline 2023-09-20T22:06:50.268991+0000) 2023-09-20T22:07:08.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:08 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:08.217+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.471217+0000 front 2023-09-20T22:06:35.471383+0000 (oldest deadline 2023-09-20T22:06:56.571362+0000) 2023-09-20T22:07:08.498 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:08 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6[132554]: 2023-09-20T22:07:08.217+0000 7f28172b8700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.972980+0000 front 2023-09-20T22:06:42.972911+0000 (oldest deadline 2023-09-20T22:07:05.273115+0000) 2023-09-20T22:07:08.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:08 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:08.266+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:08.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:08 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:08.266+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:08.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:08 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:08.266+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:08.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:08 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:08.125+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:08.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:08 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:08.125+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:08.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:08 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:08.125+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:08.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:08 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:08.125+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:09.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:08 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:08.746+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:09.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:08 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:08.746+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:09.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:08 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:08.746+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:09.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:09 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:09.149+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:09.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:09 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:09.149+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:09.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:09 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:09.149+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:09.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:08 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:08.967+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:09.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:08 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:08.967+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:09.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:08 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:08.967+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:09.490 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:09 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:09.307+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:09.490 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:09 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:09.307+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:09.490 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:09 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:09.307+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:09.491 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:09 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:09.307+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:09.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:09 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:09.160+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:09.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:09 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:09.160+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:09.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:09 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:09.160+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:09.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:09 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:09.160+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:09.747 INFO:journalctl@ceph.osd.6.smithi022.stdout:Sep 20 22:07:09 smithi022 bash[175669]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-6 2023-09-20T22:07:10.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:09 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:09.741+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:10.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:09 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:09.741+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:10.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:09 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:09.741+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:10.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:10 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:10.188+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:10.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:10 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:10.188+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:10.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:10 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:10.188+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:10.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:09 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:09.978+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:10.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:09 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:09.978+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:10.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:09 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:09.978+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:10.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:10.148+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:10.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:10.148+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:10.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:10.148+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:10.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:10.148+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:10.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:10 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:10.297+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:10.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:10 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:10.297+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:10.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:10 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:10.297+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:10.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:10 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:10.297+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:11.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:10.766+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:11.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:10.766+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:11.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:10.766+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:11.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:11.184+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:11.343 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:11.184+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:11.344 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:11.184+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:11.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:10 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:10.939+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:11.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:10 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:10.939+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:11.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:10 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:10.939+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:11.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.6.service' 2023-09-20T22:07:11.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:07:11.468 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-09-20T22:07:11.469 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-09-20T22:07:11.469 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.7 2023-09-20T22:07:11.501 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:11.173+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:11.501 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:11.173+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:11.502 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:11.173+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:11.502 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:11.173+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:11.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:11 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:11.313+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:11.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:11 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:11.313+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:11.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:11 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:11.313+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:11.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:11 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:11.313+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:11.813 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:11 smithi089 systemd[1]: Stopping Ceph osd.7 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:07:12.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:11.813+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:12.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:11.813+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:12.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:11.814+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:12.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:11.851+0000 7f82099d1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:07:12.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:11.851+0000 7f82099d1700 -1 osd.7 97 *** Got signal Terminated *** 2023-09-20T22:07:12.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:11 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:11.851+0000 7f82099d1700 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:07:12.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:11.980+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:12.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:11.980+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:12.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:11 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:11.980+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:12.344 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:12 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:12.191+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:12.344 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:12 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:12.191+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:12.345 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:12 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:12.191+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:12.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:12 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:12.131+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:12.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:12 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:12.131+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:12.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:12 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:12.131+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:12.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:12 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:12.131+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:12.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:12 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:12.310+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:12.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:12 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:12.310+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:12.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:12 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:12.310+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:12.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:12 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:12.310+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:13.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:12 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:12.851+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:13.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:12 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:12.851+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:13.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:12 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:12.851+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:13.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:12 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:12.996+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:13.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:12 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:12.996+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:13.344 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:12 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:12.996+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:13.345 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:13 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:13.143+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:13.345 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:13 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:13.143+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:13.345 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:13 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:13.143+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:13.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:13.155+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:13.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:13.155+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:13.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:13.155+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:13.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:13.155+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:13.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:13 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:13.312+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:13.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:13 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:13.312+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:13.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:13 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:13.312+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:13.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:13 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:13.312+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:14.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:13.832+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:14.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:13.832+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:14.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:13.832+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:14.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:13 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:13.832+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:47.750683+0000 front 2023-09-20T22:06:47.750628+0000 (oldest deadline 2023-09-20T22:07:13.650726+0000) 2023-09-20T22:07:14.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:13 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:13.980+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:14.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:13 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:13.980+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:14.343 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:13 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:13.980+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:14.344 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:14 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:14.165+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:14.344 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:14 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:14.165+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:14.344 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:14 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:14.165+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:14.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:14.174+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:14.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:14.174+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:14.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:14.174+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:14.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:14.174+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:14.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:14 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:14.327+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:14.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:14 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:14.327+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:14.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:14 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:14.327+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:14.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:14 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:14.327+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:14.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:14 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:14.327+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:15.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:14.791+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:15.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:14.791+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:15.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:14.791+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:15.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:14.791+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:47.750683+0000 front 2023-09-20T22:06:47.750628+0000 (oldest deadline 2023-09-20T22:07:13.650726+0000) 2023-09-20T22:07:15.210 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:14 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:14.956+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:15.211 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:14 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:14.956+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:15.211 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:14 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:14.956+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:15.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:15 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:15.209+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:15.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:15 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:15.209+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:15.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:15 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:15.209+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:15.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:15.163+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:15.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:15.163+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:15.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:15.163+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:15.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:15.163+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:15.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:15.163+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:15.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:15.357+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:15.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:15.357+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:15.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:15.357+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:15.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:15.357+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:15.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:15 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:15.357+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:16.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:15.774+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:16.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:15.774+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:16.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:15.774+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:16.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:15 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:15.774+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:47.750683+0000 front 2023-09-20T22:06:47.750628+0000 (oldest deadline 2023-09-20T22:07:13.650726+0000) 2023-09-20T22:07:16.259 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:15 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:15.917+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:16.259 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:15 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:15.917+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:16.259 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:15 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:15.917+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:16.260 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:15 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:15.917+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:52.080945+0000 front 2023-09-20T22:06:52.080853+0000 (oldest deadline 2023-09-20T22:07:15.580955+0000) 2023-09-20T22:07:16.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:16.258+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:16.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:16.258+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:16.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:16.258+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:16.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:16.258+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:16.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:16.177+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:16.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:16.177+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:16.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:16.177+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:16.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:16.177+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:16.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:16.177+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:16.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:16 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:16.311+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:16.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:16 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:16.311+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:16.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:16 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:16.311+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:16.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:16 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:16.311+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:16.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:16 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:16.311+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:17.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:16.808+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:26.346414+0000 front 2023-09-20T22:06:26.346378+0000 (oldest deadline 2023-09-20T22:06:48.646533+0000) 2023-09-20T22:07:17.110 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:16.808+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.347669+0000 front 2023-09-20T22:06:33.347646+0000 (oldest deadline 2023-09-20T22:06:59.748154+0000) 2023-09-20T22:07:17.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:16.808+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.550219+0000 front 2023-09-20T22:06:42.550022+0000 (oldest deadline 2023-09-20T22:07:04.849915+0000) 2023-09-20T22:07:17.111 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:16 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7[155556]: 2023-09-20T22:07:16.808+0000 7f82037c2700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:47.750683+0000 front 2023-09-20T22:06:47.750628+0000 (oldest deadline 2023-09-20T22:07:13.650726+0000) 2023-09-20T22:07:17.240 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:16.948+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:17.241 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:16.948+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:17.241 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:16.948+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:17.241 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:16 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:16.948+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:52.080945+0000 front 2023-09-20T22:06:52.080853+0000 (oldest deadline 2023-09-20T22:07:15.580955+0000) 2023-09-20T22:07:17.523 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:17 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:17.209+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:17.523 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:17 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:17.209+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:17.523 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:17 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:17.209+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:17.524 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:17 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:17.209+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:17.524 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:17 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:17.209+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:17.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:17 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:17.240+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:17.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:17 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:17.240+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:17.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:17 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:17.240+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:17.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:17 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:17.240+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:17.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:17 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:17.301+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:17.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:17 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:17.301+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:17.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:17 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:17.301+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:17.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:17 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:17.301+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:17.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:17 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:17.301+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:17.811 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 20 22:07:17 smithi089 bash[178296]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-7 2023-09-20T22:07:18.252 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:17 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:17.930+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:18.252 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:17 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:17.930+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:18.252 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:17 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:17.930+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:18.253 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:17 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:17.930+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:52.080945+0000 front 2023-09-20T22:06:52.080853+0000 (oldest deadline 2023-09-20T22:07:15.580955+0000) 2023-09-20T22:07:18.429 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:18.170+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:18.429 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:18.170+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:18.429 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:18.170+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:18.430 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:18.170+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:18.430 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:18.170+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:18.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:18 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:18.251+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:18.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:18 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:18.251+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:18.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:18 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:18.251+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:18.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:18 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:18.251+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:18.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:18 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:18.299+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:18.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:18 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:18.299+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:18.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:18 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:18.299+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:18.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:18 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:18.299+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:18.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:18 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:18.299+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:19.165 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.7.service' 2023-09-20T22:07:19.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:07:19.246 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-09-20T22:07:19.246 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-09-20T22:07:19.247 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.8 2023-09-20T22:07:19.277 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:18 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:18.920+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:19.277 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:18 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:18.920+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:19.278 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:18 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:18.920+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:19.278 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:18 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:18.920+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:52.080945+0000 front 2023-09-20T22:06:52.080853+0000 (oldest deadline 2023-09-20T22:07:15.580955+0000) 2023-09-20T22:07:19.593 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:19 smithi123 systemd[1]: Stopping Ceph osd.8 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:07:19.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:19.294+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:19.595 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:19.295+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:19.595 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:19.295+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:19.595 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:19.295+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:19.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:19 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:19.151+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:19.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:19 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:19.151+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:19.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:19 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:19.151+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:19.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:19 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:19.151+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:19.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:19 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:19.151+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:19.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:19 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:19.293+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:19.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:19 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:19.293+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:19.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:19 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:19.293+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:19.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:19 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:19.293+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:19.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:19 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:19.293+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:19.947 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:19.642+0000 7f1d7da6f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:07:19.948 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:19.642+0000 7f1d7da6f700 -1 osd.8 97 *** Got signal Terminated *** 2023-09-20T22:07:19.948 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:19.642+0000 7f1d7da6f700 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:07:20.320 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:19.947+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:20.320 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:19.947+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:20.320 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:19.947+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:20.321 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:19 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:19.947+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:52.080945+0000 front 2023-09-20T22:06:52.080853+0000 (oldest deadline 2023-09-20T22:07:15.580955+0000) 2023-09-20T22:07:20.360 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:20 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:20.107+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:20.360 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:20 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:20.107+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:20.360 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:20 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:20.107+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:20.361 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:20 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:20.107+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:20.361 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:20 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:20.107+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:20.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:20 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:20.320+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:20.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:20 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:20.320+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:20.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:20 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:20.320+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:20.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:20 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:20.320+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:20.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:20 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:20.266+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:20.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:20 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:20.266+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:20.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:20 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:20.266+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:20.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:20 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:20.266+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:20.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:20 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:20.266+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:21.270 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:20 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:20.929+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:21.271 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:20 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:20.929+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:21.271 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:20 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:20.929+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:21.271 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:20 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:20.929+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:52.080945+0000 front 2023-09-20T22:06:52.080853+0000 (oldest deadline 2023-09-20T22:07:15.580955+0000) 2023-09-20T22:07:21.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:21 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:21.217+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:21.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:21 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:21.217+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:21.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:21 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:21.217+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:21.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:21 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:21.217+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:21.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:21 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:21.217+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:21.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:21 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:21.270+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:21.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:21 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:21.270+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:21.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:21 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:21.270+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:21.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:21 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:21.270+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:21.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:21 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:21.145+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:21.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:21 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:21.145+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:21.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:21 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:21.145+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:21.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:21 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:21.145+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:21.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:21 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:21.145+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:22.234 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:21 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:21.956+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:22.234 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:21 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:21.956+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:22.235 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:21 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:21.956+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:22.235 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:21 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:21.956+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:52.080945+0000 front 2023-09-20T22:06:52.080853+0000 (oldest deadline 2023-09-20T22:07:15.580955+0000) 2023-09-20T22:07:22.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:22 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:22.222+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:22.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:22 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:22.222+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:22.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:22 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:22.222+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:22.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:22 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:22.222+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:22.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:22 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:22.222+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:22.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:22.234+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:22.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:22.234+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:22.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:22.234+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:22.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:22.234+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:22.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:22 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:22.182+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:22.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:22 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:22.182+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:22.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:22 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:22.182+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:22.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:22 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:22.182+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:22.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:22 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:22.182+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:23.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:22.937+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:23.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:22.937+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:23.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:22.937+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:23.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:22.937+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:52.080945+0000 front 2023-09-20T22:06:52.080853+0000 (oldest deadline 2023-09-20T22:07:15.580955+0000) 2023-09-20T22:07:23.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:23 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:23.228+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:23.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:23 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:23.228+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:23.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:23 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:23.228+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:23.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:23 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:23.228+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:23.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:23 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:23.228+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:23.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:23.220+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:23.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:23.220+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:23.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:23.220+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:23.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:23.220+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:23.595 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:23.220+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:23.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:23 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:23.150+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:23.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:23 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:23.150+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:23.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:23 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:23.150+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:23.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:23 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:23.150+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:23.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:23 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:23.150+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:24.230 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:23.944+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:22.374977+0000 front 2023-09-20T22:06:22.374861+0000 (oldest deadline 2023-09-20T22:06:48.274844+0000) 2023-09-20T22:07:24.230 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:23.944+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.276604+0000 front 2023-09-20T22:06:35.276628+0000 (oldest deadline 2023-09-20T22:07:00.576831+0000) 2023-09-20T22:07:24.231 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:23.944+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.878219+0000 front 2023-09-20T22:06:42.878394+0000 (oldest deadline 2023-09-20T22:07:04.578260+0000) 2023-09-20T22:07:24.231 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:23.944+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:52.080945+0000 front 2023-09-20T22:06:52.080853+0000 (oldest deadline 2023-09-20T22:07:15.580955+0000) 2023-09-20T22:07:24.231 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:23 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8[156801]: 2023-09-20T22:07:23.944+0000 7f1d7605d700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.081961+0000 front 2023-09-20T22:06:59.081980+0000 (oldest deadline 2023-09-20T22:07:23.182051+0000) 2023-09-20T22:07:24.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:24 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:24.230+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:24.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:24 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:24.230+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:24.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:24 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:24.230+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:24.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:24 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:24.230+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:24.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:24 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:24.230+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:24.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:24 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:24.158+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:24.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:24 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:24.158+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:24.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:24 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:24.158+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:24.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:24 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:24.158+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:24.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:24 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:24.158+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:24.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:24 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:24.260+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:24.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:24 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:24.260+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:24.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:24 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:24.260+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:24.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:24 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:24.260+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:24.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:24 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:24.260+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:25.513 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:25 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:25.259+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:25.513 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:25 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:25.259+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:25.513 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:25 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:25.259+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:25.513 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:25 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:25.259+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:25.514 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:25 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:25.259+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:25.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:25 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:25.190+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:25.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:25 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:25.190+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:25.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:25 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:25.190+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:25.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:25 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:25.190+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:25.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:25 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:25.190+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:25.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:25 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:25.190+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:25.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:25 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:25.286+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:25.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:25 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:25.286+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:25.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:25 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:25.286+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:25.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:25 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:25.286+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:25.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:25 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:25.286+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:25.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:25 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:25.286+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.726331+0000 front 2023-09-20T22:06:59.726477+0000 (oldest deadline 2023-09-20T22:07:25.026286+0000) 2023-09-20T22:07:25.843 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 20 22:07:25 smithi123 bash[179539]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-8 2023-09-20T22:07:26.574 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:26 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:26.292+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:26.575 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:26 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:26.292+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:26.575 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:26 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:26.292+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:26.575 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:26 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:26.292+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:26.575 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:26 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:26.292+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:26.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:26 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:26.201+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:26.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:26 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:26.201+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:26.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:26 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:26.201+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:26.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:26 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:26.201+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:26.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:26 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:26.201+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:26.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:26 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:26.201+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:26.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:26 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:26.294+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:26.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:26 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:26.294+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:26.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:26 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:26.294+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:26.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:26 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:26.294+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:26.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:26 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:26.294+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:26.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:26 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:26.294+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.726331+0000 front 2023-09-20T22:06:59.726477+0000 (oldest deadline 2023-09-20T22:07:25.026286+0000) 2023-09-20T22:07:26.936 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.8.service' 2023-09-20T22:07:27.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:07:27.017 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-09-20T22:07:27.017 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-09-20T22:07:27.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.9 2023-09-20T22:07:27.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:27 smithi022 systemd[1]: Stopping Ceph osd.9 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:07:27.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:27 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:27.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:27.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:27 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:27.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:27.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:27 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:27.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:27.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:27 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:27.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:27.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:27 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:27.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:27.499 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:27 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:27.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.726331+0000 front 2023-09-20T22:06:59.726477+0000 (oldest deadline 2023-09-20T22:07:25.026286+0000) 2023-09-20T22:07:27.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:27 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:27.314+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:27.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:27 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:27.314+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:27.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:27 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:27.314+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:27.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:27 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:27.314+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:27.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:27 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:27.314+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:27.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:27 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:27.238+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:27.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:27 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:27.238+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:27.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:27 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:27.238+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:27.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:27 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:27.238+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:27.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:27 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:27.238+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:27.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:27 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:27.238+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:27.997 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:27 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:27.666+0000 7f642236d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:07:27.997 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:27 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:27.666+0000 7f642236d700 -1 osd.9 97 *** Got signal Terminated *** 2023-09-20T22:07:27.997 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:27 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:27.666+0000 7f642236d700 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:07:28.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:28 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:28.336+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:28.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:28 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:28.336+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:28.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:28 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:28.336+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:28.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:28 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:28.336+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:28.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:28 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:28.336+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:28.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:28 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:28.282+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:28.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:28 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:28.282+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:28.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:28 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:28.282+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:28.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:28 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:28.282+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:28.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:28 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:28.282+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:28.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:28 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:28.282+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:28.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:28 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:28.323+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:28.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:28 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:28.323+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:28.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:28 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:28.323+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:28.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:28 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:28.323+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:28.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:28 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:28.323+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:28.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:28 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:28.323+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.726331+0000 front 2023-09-20T22:06:59.726477+0000 (oldest deadline 2023-09-20T22:07:25.026286+0000) 2023-09-20T22:07:29.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:29 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:29.316+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:29.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:29 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:29.316+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:29.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:29 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:29.316+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:29.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:29 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:29.316+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:29.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:29 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:29.316+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:29.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:29 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:29.285+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:29.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:29 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:29.285+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:29.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:29 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:29.285+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:29.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:29 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:29.285+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:29.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:29 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:29.285+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:29.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:29 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:29.285+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:29.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:29 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:29.355+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:29.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:29 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:29.355+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:29.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:29 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:29.355+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:29.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:29 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:29.355+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:29.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:29 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:29.355+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:29.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:29 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:29.355+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.726331+0000 front 2023-09-20T22:06:59.726477+0000 (oldest deadline 2023-09-20T22:07:25.026286+0000) 2023-09-20T22:07:30.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:30 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:30.325+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:30.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:30 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:30.325+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:30.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:30 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:30.325+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:30.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:30 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:30.325+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:30.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:30 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:30.325+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:30.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:30 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:30.325+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:30.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:30 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:30.325+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:30.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:30.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:30.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:30.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:30.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:30.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:30.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:30.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:30.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:30.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:30.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:30 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:30.309+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.726331+0000 front 2023-09-20T22:06:59.726477+0000 (oldest deadline 2023-09-20T22:07:25.026286+0000) 2023-09-20T22:07:30.845 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:30.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:30.845 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:30.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:30.846 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:30.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:30.846 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:30.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:30.846 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:30.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:30.846 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:30.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:31.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:31 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:31.346+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:31.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:31 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:31.346+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:31.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:31 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:31.346+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:31.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:31 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:31.346+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:31.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:31 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:31.346+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:31.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:31 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:31.346+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:31.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:31 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:31.346+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:31.747 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:31 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:31.288+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:31.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:31 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:31.288+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:31.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:31 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:31.288+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:31.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:31 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:31.288+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:31.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:31 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:31.288+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:31.748 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:31 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:31.288+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.726331+0000 front 2023-09-20T22:06:59.726477+0000 (oldest deadline 2023-09-20T22:07:25.026286+0000) 2023-09-20T22:07:31.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:31 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:31.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:31.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:31 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:31.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:31.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:31 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:31.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:31.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:31 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:31.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:31.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:31 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:31.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:31.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:31 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:31.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:32.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:32 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:32.244+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:17.013622+0000 front 2023-09-20T22:06:17.013689+0000 (oldest deadline 2023-09-20T22:06:41.614728+0000) 2023-09-20T22:07:32.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:32 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:32.244+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.217954+0000 front 2023-09-20T22:06:27.217882+0000 (oldest deadline 2023-09-20T22:06:52.517956+0000) 2023-09-20T22:07:32.497 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:32 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:32.244+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:32.518729+0000 front 2023-09-20T22:06:32.518625+0000 (oldest deadline 2023-09-20T22:07:00.119565+0000) 2023-09-20T22:07:32.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:32 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:32.244+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:41.220566+0000 front 2023-09-20T22:06:41.220522+0000 (oldest deadline 2023-09-20T22:07:08.821404+0000) 2023-09-20T22:07:32.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:32 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:32.244+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:48.822291+0000 front 2023-09-20T22:06:48.822145+0000 (oldest deadline 2023-09-20T22:07:14.121967+0000) 2023-09-20T22:07:32.498 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:32 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9[140484]: 2023-09-20T22:07:32.244+0000 7f641b15c700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.726331+0000 front 2023-09-20T22:06:59.726477+0000 (oldest deadline 2023-09-20T22:07:25.026286+0000) 2023-09-20T22:07:32.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:32 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:32.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:32.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:32 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:32.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:32.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:32 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:32.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:32.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:32 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:32.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:32.845 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:32 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:32.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:32.845 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:32 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:32.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:32.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:32 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:32.377+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:32.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:32 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:32.377+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:32.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:32 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:32.377+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:32.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:32 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:32.377+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:32.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:32 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:32.377+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:32.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:32 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:32.377+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:32.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:32 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:32.377+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:33.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:33 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:33.329+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:33.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:33 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:33.329+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:33.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:33 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:33.329+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:33.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:33 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:33.329+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:33.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:33 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:33.329+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:33.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:33 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:33.329+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:33.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:33 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:33.329+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:33.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:33 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:33.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:33.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:33 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:33.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:33.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:33 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:33.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:33.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:33 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:33.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:33.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:33 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:33.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:33.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:33 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:33.383+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:33.997 INFO:journalctl@ceph.osd.9.smithi022.stdout:Sep 20 22:07:33 smithi022 bash[175947]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-9 2023-09-20T22:07:34.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:34 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:34.385+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:34.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:34 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:34.385+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:34.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:34 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:34.385+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:34.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:34 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:34.385+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:34.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:34 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:34.385+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:34.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:34 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:34.385+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:34.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:34 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:34.378+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:34.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:34 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:34.378+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:34.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:34 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:34.378+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:34.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:34 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:34.378+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:34.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:34 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:34.378+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:34.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:34 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:34.378+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:34.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:34 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:34.378+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:35.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:35 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:35.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:35.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:35 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:35.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:35.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:35 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:35.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:35.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:35 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:35.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:35.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:35 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:35.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:35.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:35 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:35.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:35.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:35 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:35.357+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:35.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:35 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:35.357+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:35.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:35 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:35.357+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:35.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:35 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:35.357+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:35.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:35 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:35.357+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:35.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:35 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:35.357+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:35.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:35 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:35.357+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:35.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.9.service' 2023-09-20T22:07:35.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:07:35.768 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-09-20T22:07:35.768 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-09-20T22:07:35.768 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.10 2023-09-20T22:07:36.110 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:35 smithi089 systemd[1]: Stopping Ceph osd.10 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:07:36.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:36 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:36.137+0000 7f32cc5f3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:07:36.610 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:36 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:36.137+0000 7f32cc5f3700 -1 osd.10 97 *** Got signal Terminated *** 2023-09-20T22:07:36.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:36 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:36.137+0000 7f32cc5f3700 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:07:36.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:36 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:36.332+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:36.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:36 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:36.332+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:36.611 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:36 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:36.332+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:36.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:36 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:36.332+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:36.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:36 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:36.332+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:36.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:36 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:36.332+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:36.612 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:36 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:36.332+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:36.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:36 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:36.360+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:36.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:36 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:36.360+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:36.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:36 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:36.360+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:36.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:36 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:36.360+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:36.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:36 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:36.360+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:36.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:36 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:36.360+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:37.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:37 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:37.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:37.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:37 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:37.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:37.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:37 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:37.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:37.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:37 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:37.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:37.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:37 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:37.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:37.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:37 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:37.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:37.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:37 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:37.368+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:37.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:37 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:37.368+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:37.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:37 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:37.368+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:37.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:37 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:37.368+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:37.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:37 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:37.368+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:37.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:37 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:37.368+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:37.862 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:37 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:37.368+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:38.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:38 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:38.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:38.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:38 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:38.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:38.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:38 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:38.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:38.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:38 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:38.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:38.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:38 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:38.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:38.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:38 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:38.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:38.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:38 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:38.388+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:38.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:38 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:38.388+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:38.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:38 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:38.388+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:38.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:38 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:38.388+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:38.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:38 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:38.388+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:38.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:38 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:38.388+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:38.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:38 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:38.388+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:39.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:39 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:39.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:39.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:39 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:39.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:39.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:39 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:39.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:39.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:39 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:39.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:39.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:39 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:39.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:39.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:39 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:39.339+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:39.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:39 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:39.431+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:39.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:39 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:39.431+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:39.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:39 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:39.431+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:39.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:39 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:39.431+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:39.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:39 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:39.431+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:39.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:39 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:39.431+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:39.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:39 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:39.431+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:40.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:40 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:40.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:40.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:40 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:40.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:40.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:40 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:40.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:40.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:40 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:40.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:40.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:40 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:40.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:40.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:40 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:40.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:40.595 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:40 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:40.340+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2023-09-20T22:07:14.358335+0000 front 2023-09-20T22:07:14.358366+0000 (oldest deadline 2023-09-20T22:07:39.658285+0000) 2023-09-20T22:07:40.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:40 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:40.461+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:18.526769+0000 front 2023-09-20T22:06:18.526693+0000 (oldest deadline 2023-09-20T22:06:40.226758+0000) 2023-09-20T22:07:40.860 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:40 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:40.461+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2023-09-20T22:06:27.228547+0000 front 2023-09-20T22:06:27.228570+0000 (oldest deadline 2023-09-20T22:06:52.528754+0000) 2023-09-20T22:07:40.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:40 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:40.461+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:35.429892+0000 front 2023-09-20T22:06:35.429802+0000 (oldest deadline 2023-09-20T22:07:00.729945+0000) 2023-09-20T22:07:40.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:40 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:40.461+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:40.730446+0000 front 2023-09-20T22:06:40.730481+0000 (oldest deadline 2023-09-20T22:07:04.830488+0000) 2023-09-20T22:07:40.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:40 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:40.461+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:49.531418+0000 front 2023-09-20T22:06:49.531444+0000 (oldest deadline 2023-09-20T22:07:14.831593+0000) 2023-09-20T22:07:40.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:40 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:40.461+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.433228+0000 front 2023-09-20T22:06:59.433189+0000 (oldest deadline 2023-09-20T22:07:24.733062+0000) 2023-09-20T22:07:40.861 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:40 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10[164289]: 2023-09-20T22:07:40.461+0000 7f32c83e8700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:04.733958+0000 front 2023-09-20T22:07:04.733819+0000 (oldest deadline 2023-09-20T22:07:30.033622+0000) 2023-09-20T22:07:41.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:41 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:41.357+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:41.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:41 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:41.357+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:41.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:41 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:41.357+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:41.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:41 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:41.357+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:41.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:41 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:41.357+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:41.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:41 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:41.357+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:41.845 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:41 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:41.357+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2023-09-20T22:07:14.358335+0000 front 2023-09-20T22:07:14.358366+0000 (oldest deadline 2023-09-20T22:07:39.658285+0000) 2023-09-20T22:07:41.904 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 20 22:07:41 smithi089 bash[178557]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-10 2023-09-20T22:07:42.845 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:42 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:42.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:42.846 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:42 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:42.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:42.846 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:42 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:42.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:42.846 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:42 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:42.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:42.846 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:42 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:42.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:42.846 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:42 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:42.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:42.847 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:42 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:42.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2023-09-20T22:07:14.358335+0000 front 2023-09-20T22:07:14.358366+0000 (oldest deadline 2023-09-20T22:07:39.658285+0000) 2023-09-20T22:07:43.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.10.service' 2023-09-20T22:07:43.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:07:43.307 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-09-20T22:07:43.307 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-09-20T22:07:43.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.11 2023-09-20T22:07:43.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 systemd[1]: Stopping Ceph osd.11 for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:07:43.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:43.440+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:43.593 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:43.440+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:43.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:43.440+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:43.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:43.440+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:43.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:43.440+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:43.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:43.440+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:43.594 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:43.440+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2023-09-20T22:07:14.358335+0000 front 2023-09-20T22:07:14.358366+0000 (oldest deadline 2023-09-20T22:07:39.658285+0000) 2023-09-20T22:07:44.093 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:43.667+0000 7f484fc02700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:07:44.093 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:43.667+0000 7f484fc02700 -1 osd.11 97 *** Got signal Terminated *** 2023-09-20T22:07:44.093 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:43 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:43.667+0000 7f484fc02700 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-09-20T22:07:44.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:44 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:44.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:44.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:44 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:44.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:44.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:44 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:44.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:44.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:44 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:44.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:44.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:44 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:44.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:44.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:44 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:44.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:44.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:44 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:44.395+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2023-09-20T22:07:14.358335+0000 front 2023-09-20T22:07:14.358366+0000 (oldest deadline 2023-09-20T22:07:39.658285+0000) 2023-09-20T22:07:45.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:45 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:45.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:45.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:45 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:45.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:45.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:45 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:45.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:45.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:45 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:45.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:45.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:45 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:45.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:45.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:45 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:45.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:45.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:45 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:45.396+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2023-09-20T22:07:14.358335+0000 front 2023-09-20T22:07:14.358366+0000 (oldest deadline 2023-09-20T22:07:39.658285+0000) 2023-09-20T22:07:46.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:46 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:46.378+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:46.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:46 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:46.378+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:46.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:46 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:46.378+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:46.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:46 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:46.378+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:46.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:46 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:46.378+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:46.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:46 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:46.378+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:46.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:46 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:46.378+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2023-09-20T22:07:14.358335+0000 front 2023-09-20T22:07:14.358366+0000 (oldest deadline 2023-09-20T22:07:39.658285+0000) 2023-09-20T22:07:47.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:47 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:47.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:47.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:47 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:47.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:47.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:47 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:47.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:47.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:47 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:47.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:47.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:47 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:47.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:47.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:47 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:47.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:47.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:47 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:47.368+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2023-09-20T22:07:14.358335+0000 front 2023-09-20T22:07:14.358366+0000 (oldest deadline 2023-09-20T22:07:39.658285+0000) 2023-09-20T22:07:48.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:48 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:48.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6806 osd.0 since back 2023-09-20T22:06:15.746482+0000 front 2023-09-20T22:06:15.746568+0000 (oldest deadline 2023-09-20T22:06:40.446449+0000) 2023-09-20T22:07:48.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:48 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:48.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2023-09-20T22:06:33.750081+0000 front 2023-09-20T22:06:33.750044+0000 (oldest deadline 2023-09-20T22:06:57.250241+0000) 2023-09-20T22:07:48.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:48 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:48.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6814 osd.3 since back 2023-09-20T22:06:42.452141+0000 front 2023-09-20T22:06:42.452196+0000 (oldest deadline 2023-09-20T22:07:08.852707+0000) 2023-09-20T22:07:48.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:48 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:48.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2023-09-20T22:06:51.754024+0000 front 2023-09-20T22:06:51.753975+0000 (oldest deadline 2023-09-20T22:07:15.854042+0000) 2023-09-20T22:07:48.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:48 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:48.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2023-09-20T22:06:59.355145+0000 front 2023-09-20T22:06:59.355187+0000 (oldest deadline 2023-09-20T22:07:22.255234+0000) 2023-09-20T22:07:48.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:48 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:48.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.22:6822 osd.6 since back 2023-09-20T22:07:06.956383+0000 front 2023-09-20T22:07:06.956424+0000 (oldest deadline 2023-09-20T22:07:29.856150+0000) 2023-09-20T22:07:48.844 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:48 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11[165582]: 2023-09-20T22:07:48.361+0000 7f484a9f5700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2023-09-20T22:07:14.358335+0000 front 2023-09-20T22:07:14.358366+0000 (oldest deadline 2023-09-20T22:07:39.658285+0000) 2023-09-20T22:07:49.843 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 20 22:07:49 smithi123 bash[179807]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-osd-11 2023-09-20T22:07:50.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@osd.11.service' 2023-09-20T22:07:50.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:07:50.972 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-09-20T22:07:50.972 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-09-20T22:07:50.972 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.a 2023-09-20T22:07:51.497 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 22:07:51 smithi022 systemd[1]: Stopping Ceph mds.a for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:07:51.997 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 22:07:51 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-a[148978]: 2023-09-20T22:07:51.520+0000 7fdea6fbd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:07:51.997 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 22:07:51 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-a[148978]: 2023-09-20T22:07:51.520+0000 7fdea6fbd700 -1 mds.a *** got signal Terminated *** 2023-09-20T22:07:53.747 INFO:journalctl@ceph.mds.a.smithi022.stdout:Sep 20 22:07:53 smithi022 bash[176218]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-a 2023-09-20T22:07:54.078 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.a.service' 2023-09-20T22:07:54.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:07:54.164 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-09-20T22:07:54.164 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-09-20T22:07:54.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.d 2023-09-20T22:07:54.497 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 22:07:54 smithi022 systemd[1]: Stopping Ceph mds.d for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:07:54.997 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 22:07:54 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-d[147971]: 2023-09-20T22:07:54.661+0000 7f4bd772a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:07:54.997 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 22:07:54 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-d[147971]: 2023-09-20T22:07:54.661+0000 7f4bd772a700 -1 mds.d *** got signal Terminated *** 2023-09-20T22:07:57.747 INFO:journalctl@ceph.mds.d.smithi022.stdout:Sep 20 22:07:57 smithi022 bash[176363]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-d 2023-09-20T22:07:57.951 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.d.service' 2023-09-20T22:07:58.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:07:58.035 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-09-20T22:07:58.036 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-09-20T22:07:58.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.g 2023-09-20T22:07:58.247 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 20 22:07:58 smithi022 systemd[1]: Stopping Ceph mds.g for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:07:58.997 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 20 22:07:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-g[148589]: 2023-09-20T22:07:58.551+0000 7f9ea4806700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:07:58.998 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 20 22:07:58 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-g[148589]: 2023-09-20T22:07:58.551+0000 7f9ea4806700 -1 mds.g *** got signal Terminated *** 2023-09-20T22:08:02.747 INFO:journalctl@ceph.mds.g.smithi022.stdout:Sep 20 22:08:02 smithi022 bash[176507]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-g 2023-09-20T22:08:02.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.g.service' 2023-09-20T22:08:02.916 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:08:02.916 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-09-20T22:08:02.917 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-09-20T22:08:02.917 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.j 2023-09-20T22:08:03.247 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 22:08:03 smithi022 systemd[1]: Stopping Ceph mds.j for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:08:03.997 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 22:08:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-j[147584]: 2023-09-20T22:08:03.501+0000 7f5fb922d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:08:03.998 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 22:08:03 smithi022 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-j[147584]: 2023-09-20T22:08:03.501+0000 7f5fb922d700 -1 mds.j *** got signal Terminated *** 2023-09-20T22:08:05.978 INFO:journalctl@ceph.mds.j.smithi022.stdout:Sep 20 22:08:05 smithi022 bash[176651]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-j 2023-09-20T22:08:06.008 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.j.service' 2023-09-20T22:08:06.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:08:06.091 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-09-20T22:08:06.091 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-09-20T22:08:06.092 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.b 2023-09-20T22:08:06.450 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 22:08:06 smithi089 systemd[1]: Stopping Ceph mds.b for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:08:06.860 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 22:08:06 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b[171416]: 2023-09-20T22:08:06.449+0000 7f4e8699c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:08:06.860 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 22:08:06 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b[171416]: 2023-09-20T22:08:06.449+0000 7f4e8699c700 -1 mds.b *** got signal Terminated *** 2023-09-20T22:08:09.516 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 20 22:08:09 smithi089 bash[178826]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-b 2023-09-20T22:08:09.704 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.b.service' 2023-09-20T22:08:09.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:08:09.741 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-09-20T22:08:09.741 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-09-20T22:08:09.742 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.e 2023-09-20T22:08:10.110 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 22:08:09 smithi089 systemd[1]: Stopping Ceph mds.e for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:08:10.610 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 22:08:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-e[170732]: 2023-09-20T22:08:10.124+0000 7f5fb2c7a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:08:10.610 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 22:08:10 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-e[170732]: 2023-09-20T22:08:10.124+0000 7f5fb2c7a700 -1 mds.e *** got signal Terminated *** 2023-09-20T22:08:13.802 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 20 22:08:13 smithi089 bash[178965]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-e 2023-09-20T22:08:13.807 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.e.service' 2023-09-20T22:08:13.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:08:13.842 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-09-20T22:08:13.842 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-09-20T22:08:13.842 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.h 2023-09-20T22:08:14.110 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 22:08:13 smithi089 systemd[1]: Stopping Ceph mds.h for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:08:14.610 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 22:08:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-h[171074]: 2023-09-20T22:08:14.213+0000 7f7966eed700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:08:14.610 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 22:08:14 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-h[171074]: 2023-09-20T22:08:14.213+0000 7f7966eed700 -1 mds.h *** got signal Terminated *** 2023-09-20T22:08:17.784 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 20 22:08:17 smithi089 bash[179113]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-h 2023-09-20T22:08:17.789 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.h.service' 2023-09-20T22:08:17.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:08:17.825 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-09-20T22:08:17.825 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-09-20T22:08:17.825 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.k 2023-09-20T22:08:18.110 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 22:08:17 smithi089 systemd[1]: Stopping Ceph mds.k for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:08:18.610 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 22:08:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-k[170392]: 2023-09-20T22:08:18.205+0000 7ff564ff8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:08:18.610 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 22:08:18 smithi089 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-k[170392]: 2023-09-20T22:08:18.205+0000 7ff564ff8700 -1 mds.k *** got signal Terminated *** 2023-09-20T22:08:21.763 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 20 22:08:21 smithi089 bash[179259]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-k 2023-09-20T22:08:21.769 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.k.service' 2023-09-20T22:08:21.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:08:21.803 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-09-20T22:08:21.803 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-09-20T22:08:21.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.c 2023-09-20T22:08:22.093 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 22:08:21 smithi123 systemd[1]: Stopping Ceph mds.c for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:08:22.593 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 22:08:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-c[172252]: 2023-09-20T22:08:22.177+0000 7fcf107e0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:08:22.593 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 22:08:22 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-c[172252]: 2023-09-20T22:08:22.177+0000 7fcf107e0700 -1 mds.c *** got signal Terminated *** 2023-09-20T22:08:25.593 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 20 22:08:25 smithi123 bash[180081]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-c 2023-09-20T22:08:25.611 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.c.service' 2023-09-20T22:08:25.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:08:25.649 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-09-20T22:08:25.649 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-09-20T22:08:25.649 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.f 2023-09-20T22:08:26.035 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 22:08:25 smithi123 systemd[1]: Stopping Ceph mds.f for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:08:26.343 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 22:08:26 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-f[171914]: 2023-09-20T22:08:26.035+0000 7f4f01a65700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:08:26.343 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 22:08:26 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-f[171914]: 2023-09-20T22:08:26.035+0000 7f4f01a65700 -1 mds.f *** got signal Terminated *** 2023-09-20T22:08:29.496 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 20 22:08:29 smithi123 bash[180228]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-f 2023-09-20T22:08:29.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.f.service' 2023-09-20T22:08:29.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:08:29.720 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-09-20T22:08:29.721 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-09-20T22:08:29.721 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.i 2023-09-20T22:08:30.093 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 22:08:29 smithi123 systemd[1]: Stopping Ceph mds.i for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:08:30.593 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 22:08:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-i[172592]: 2023-09-20T22:08:30.100+0000 7fc9ab955700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:08:30.593 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 22:08:30 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-i[172592]: 2023-09-20T22:08:30.100+0000 7fc9ab955700 -1 mds.i *** got signal Terminated *** 2023-09-20T22:08:33.728 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 20 22:08:33 smithi123 bash[180367]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-i 2023-09-20T22:08:33.733 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.i.service' 2023-09-20T22:08:33.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:08:33.768 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-09-20T22:08:33.768 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-09-20T22:08:33.769 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.l 2023-09-20T22:08:34.093 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 20 22:08:33 smithi123 systemd[1]: Stopping Ceph mds.l for f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da... 2023-09-20T22:08:34.593 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 20 22:08:34 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-l[171577]: 2023-09-20T22:08:34.158+0000 7f3925b4b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-09-20T22:08:34.593 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 20 22:08:34 smithi123 ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-l[171577]: 2023-09-20T22:08:34.158+0000 7f3925b4b700 -1 mds.l *** got signal Terminated *** 2023-09-20T22:08:38.516 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 20 22:08:38 smithi123 bash[180511]: ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da-mds-l 2023-09-20T22:08:38.718 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da@mds.l.service' 2023-09-20T22:08:38.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-09-20T22:08:38.753 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-09-20T22:08:38.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da --force --keep-logs 2023-09-20T22:08:39.158 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T22:08:42.576 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da --force --keep-logs 2023-09-20T22:08:42.882 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T22:08:44.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da --force --keep-logs 2023-09-20T22:08:44.769 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T22:08:46.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-20T22:08:46.470 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-20T22:08:46.507 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-09-20T22:08:46.537 INFO:tasks.cephadm:Archiving crash dumps... 2023-09-20T22:08:46.539 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/crash to /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182/remote/smithi022/crash 2023-09-20T22:08:46.541 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/crash -- . 2023-09-20T22:08:46.623 INFO:teuthology.orchestra.run.smithi022.stderr:tar: /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-20T22:08:46.623 INFO:teuthology.orchestra.run.smithi022.stderr:tar: Error is not recoverable: exiting now 2023-09-20T22:08:46.626 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/crash to /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182/remote/smithi089/crash 2023-09-20T22:08:46.627 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/crash -- . 2023-09-20T22:08:46.729 INFO:teuthology.orchestra.run.smithi089.stderr:tar: /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-20T22:08:46.729 INFO:teuthology.orchestra.run.smithi089.stderr:tar: Error is not recoverable: exiting now 2023-09-20T22:08:46.731 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/crash to /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182/remote/smithi123/crash 2023-09-20T22:08:46.732 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/crash -- . 2023-09-20T22:08:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/crash: Cannot open: No such file or directory 2023-09-20T22:08:46.775 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2023-09-20T22:08:46.776 INFO:tasks.cephadm:Checking cluster log for badness... 2023-09-20T22:08:46.777 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-09-20T22:08:46.859 INFO:teuthology.orchestra.run.smithi022.stderr:grep: /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph.log: No such file or directory 2023-09-20T22:08:46.864 INFO:tasks.cephadm:Compressing logs... 2023-09-20T22:08:46.864 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-20T22:08:46.869 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-20T22:08:46.872 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-20T22:08:46.903 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-09-20T22:08:46.904 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-20T22:08:46.909 INFO:teuthology.orchestra.run.smithi089.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-20T22:08:46.909 INFO:teuthology.orchestra.run.smithi089.stderr:: No such file or directory 2023-09-20T22:08:46.910 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.5.log 2023-09-20T22:08:46.914 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.2.log 2023-09-20T22:08:46.914 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.11.log 2023-09-20T22:08:46.914 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mgr.z.log 2023-09-20T22:08:46.914 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.8.log 2023-09-20T22:08:46.915 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.f.log 2023-09-20T22:08:46.915 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.c.log 2023-09-20T22:08:46.915 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.i.log 2023-09-20T22:08:46.915 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mon.c.log 2023-09-20T22:08:46.916 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.c.log: /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-volume.log 2023-09-20T22:08:46.916 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.l.log 2023-09-20T22:08:46.917 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.10.log 2023-09-20T22:08:46.917 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.h.log 2023-09-20T22:08:46.918 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.e.log 2023-09-20T22:08:46.918 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.10.log: /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.4.log 2023-09-20T22:08:46.919 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-volume.log: /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.l.log: 93.2% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mgr.z.log.gz 2023-09-20T22:08:46.919 INFO:teuthology.orchestra.run.smithi123.stderr: 94.4% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.l.log.gz 2023-09-20T22:08:46.921 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.1.log 2023-09-20T22:08:46.921 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.k.log 2023-09-20T22:08:46.922 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-volume.log 2023-09-20T22:08:46.922 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.7.log 2023-09-20T22:08:46.922 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mon.b.log 2023-09-20T22:08:46.922 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.b.log 2023-09-20T22:08:46.923 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mgr.y.log 2023-09-20T22:08:46.923 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.b.log: /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mgr.y.log: 93.0% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mgr.y.log.gz 2023-09-20T22:08:46.947 INFO:teuthology.orchestra.run.smithi022.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-09-20T22:08:46.947 INFO:teuthology.orchestra.run.smithi022.stderr:: No such file or directory 2023-09-20T22:08:46.953 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-09-20T22:08:46.954 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.0.log 2023-09-20T22:08:46.955 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.g.log 2023-09-20T22:08:46.956 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.a.log 2023-09-20T22:08:46.956 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.g.log: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-09-20T22:08:46.957 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.6.log 2023-09-20T22:08:46.958 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.3.log 2023-09-20T22:08:46.960 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.d.log 2023-09-20T22:08:46.961 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mgr.x.log 2023-09-20T22:08:46.961 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.d.log: 94.3% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.g.log.gz 2023-09-20T22:08:46.962 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.9.log 2023-09-20T22:08:46.963 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-volume.log 2023-09-20T22:08:46.964 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.j.log 2023-09-20T22:08:46.965 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mon.a.log 2023-09-20T22:08:46.991 INFO:teuthology.orchestra.run.smithi089.stderr: 91.0% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-volume.log.gz 2023-09-20T22:08:47.022 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.j.log: /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mon.a.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-20T22:08:47.121 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-volume.log.gz 2023-09-20T22:08:47.211 INFO:teuthology.orchestra.run.smithi123.stderr: 94.6% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.c.log.gz 2023-09-20T22:08:47.435 INFO:teuthology.orchestra.run.smithi123.stderr: 91.0% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-volume.log.gz 2023-09-20T22:08:47.474 INFO:teuthology.orchestra.run.smithi022.stderr: 93.8% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.d.log.gz 2023-09-20T22:08:47.560 INFO:teuthology.orchestra.run.smithi123.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-20T22:08:47.649 INFO:teuthology.orchestra.run.smithi089.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2023-09-20T22:08:47.661 INFO:teuthology.orchestra.run.smithi022.stderr: 90.2% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mgr.x.log.gz 2023-09-20T22:08:47.893 INFO:teuthology.orchestra.run.smithi089.stderr: 93.8% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.k.log.gz 2023-09-20T22:08:47.910 INFO:teuthology.orchestra.run.smithi123.stderr: 94.5% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.i.log.gz 2023-09-20T22:08:48.230 INFO:teuthology.orchestra.run.smithi123.stderr: 91.9% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mon.c.log.gz 2023-09-20T22:08:49.370 INFO:teuthology.orchestra.run.smithi089.stderr: 93.5% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.e.log.gz 2023-09-20T22:08:49.442 INFO:teuthology.orchestra.run.smithi089.stderr: 91.9% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mon.b.log.gz 2023-09-20T22:08:49.554 INFO:teuthology.orchestra.run.smithi022.stderr: 90.5% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mon.a.log.gz 2023-09-20T22:08:51.238 INFO:teuthology.orchestra.run.smithi089.stderr: 93.3% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.h.log.gz 2023-09-20T22:08:53.371 INFO:teuthology.orchestra.run.smithi022.stderr: 94.2% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.a.log.gz 2023-09-20T22:09:18.793 INFO:teuthology.orchestra.run.smithi022.stderr: 97.6% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.j.log.gz 2023-09-20T22:09:25.589 INFO:teuthology.orchestra.run.smithi123.stderr: 96.8% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.f.log.gz 2023-09-20T22:10:45.401 INFO:teuthology.orchestra.run.smithi123.stderr: 90.6% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.8.log.gz 2023-09-20T22:10:53.956 INFO:teuthology.orchestra.run.smithi123.stderr: 90.9% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.11.log.gz 2023-09-20T22:11:07.803 INFO:teuthology.orchestra.run.smithi123.stderr: 90.7% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.2.log.gz 2023-09-20T22:11:54.299 INFO:teuthology.orchestra.run.smithi089.stderr: 90.7% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.10.log.gz 2023-09-20T22:11:59.510 INFO:teuthology.orchestra.run.smithi123.stderr: 90.7% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.5.log.gz 2023-09-20T22:11:59.520 INFO:teuthology.orchestra.run.smithi123.stderr: 2023-09-20T22:11:59.521 INFO:teuthology.orchestra.run.smithi123.stderr:real 3m12.632s 2023-09-20T22:11:59.521 INFO:teuthology.orchestra.run.smithi123.stderr:user 5m37.438s 2023-09-20T22:11:59.521 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m11.923s 2023-09-20T22:12:15.443 INFO:teuthology.orchestra.run.smithi022.stderr: 90.8% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.6.log.gz 2023-09-20T22:12:15.968 INFO:teuthology.orchestra.run.smithi022.stderr: 90.8% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.0.log.gz 2023-09-20T22:12:38.898 INFO:teuthology.orchestra.run.smithi089.stderr: 90.7% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.7.log.gz 2023-09-20T22:12:42.663 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.3.log.gz 2023-09-20T22:12:47.994 INFO:teuthology.orchestra.run.smithi089.stderr: 90.7% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.1.log.gz 2023-09-20T22:12:56.979 INFO:teuthology.orchestra.run.smithi089.stderr: 95.6% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-mds.b.log.gz 2023-09-20T22:13:19.617 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.9.log.gz 2023-09-20T22:13:19.621 INFO:teuthology.orchestra.run.smithi022.stderr: 2023-09-20T22:13:19.621 INFO:teuthology.orchestra.run.smithi022.stderr:real 4m32.715s 2023-09-20T22:13:19.622 INFO:teuthology.orchestra.run.smithi022.stderr:user 7m10.296s 2023-09-20T22:13:19.622 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m50.524s 2023-09-20T22:14:05.574 INFO:teuthology.orchestra.run.smithi089.stderr: 91.2% -- replaced with /var/log/ceph/f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da/ceph-osd.4.log.gz 2023-09-20T22:14:05.576 INFO:teuthology.orchestra.run.smithi089.stderr: 2023-09-20T22:14:05.576 INFO:teuthology.orchestra.run.smithi089.stderr:real 5m18.688s 2023-09-20T22:14:05.577 INFO:teuthology.orchestra.run.smithi089.stderr:user 7m17.381s 2023-09-20T22:14:05.577 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m16.783s 2023-09-20T22:14:05.578 INFO:tasks.cephadm:Archiving logs... 2023-09-20T22:14:05.579 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182/remote/smithi022/log 2023-09-20T22:14:05.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-20T22:15:16.033 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182/remote/smithi089/log 2023-09-20T22:15:16.035 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-20T22:16:31.966 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182/remote/smithi123/log 2023-09-20T22:16:31.967 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-20T22:17:32.566 INFO:tasks.cephadm:Removing cluster... 2023-09-20T22:17:32.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da --force 2023-09-20T22:17:33.193 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T22:17:35.336 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da --force 2023-09-20T22:17:36.140 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T22:17:37.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da --force 2023-09-20T22:17:37.994 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: f59fb6f0-57fd-11ee-9ab8-7b867c8bd7da 2023-09-20T22:17:38.972 INFO:tasks.cephadm:Removing cephadm ... 2023-09-20T22:17:38.972 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-20T22:17:39.022 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-20T22:17:39.057 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2023-09-20T22:17:39.085 INFO:tasks.cephadm:Teardown complete 2023-09-20T22:17:39.086 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-20T22:17:39.097 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-20T22:17:39.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-20T22:17:39.103 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-20T22:17:39.105 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-20T22:17:39.260 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-20T22:17:39.271 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-20T22:17:39.272 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-20T22:17:39.277 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-20T22:17:39.279 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-20T22:17:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-09-20T22:17:39.296 INFO:teuthology.orchestra.run.smithi089.stderr:bash: ntpq: command not found 2023-09-20T22:17:39.314 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-20T22:17:39.315 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-09-20T22:17:39.315 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 -1508us[-1508us] +/- 86ms 2023-09-20T22:17:39.315 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -1814us[-1464us] +/- 70ms 2023-09-20T22:17:39.315 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +2874us[+2874us] +/- 62ms 2023-09-20T22:17:39.316 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T22:17:39.316 INFO:teuthology.orchestra.run.smithi022.stderr:bash: ntpq: command not found 2023-09-20T22:17:39.319 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-20T22:17:39.319 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2023-09-20T22:17:39.319 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 -1501us[-1484us] +/- 86ms 2023-09-20T22:17:39.319 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -1786us[-1770us] +/- 70ms 2023-09-20T22:17:39.320 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +2880us[+2897us] +/- 62ms 2023-09-20T22:17:39.320 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T22:17:39.353 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-20T22:17:39.354 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2023-09-20T22:17:39.354 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 43 -1493us[-1493us] +/- 85ms 2023-09-20T22:17:39.354 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv02.front.sepia.ceph.com 3 7 377 104 -1736us[-1241us] +/- 69ms 2023-09-20T22:17:39.354 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +2864us[+2864us] +/- 62ms 2023-09-20T22:17:39.354 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-09-20T22:17:39.356 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-20T22:17:39.366 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-20T22:17:39.367 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-20T22:17:39.378 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-20T22:17:39.672 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-20T22:17:39.850 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-20T22:17:39.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-20T22:17:40.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T22:17:40.071 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-20T22:17:40.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T22:17:40.107 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-20T22:17:40.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T22:17:40.141 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-20T22:17:40.154 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-20T22:17:40.165 INFO:teuthology.task.internal:Duration was 3311.307025 seconds 2023-09-20T22:17:40.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-20T22:17:40.180 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-20T22:17:40.180 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-20T22:17:40.185 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-20T22:17:40.188 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-20T22:17:40.270 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-20T22:17:40.300 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-20T22:17:40.350 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-20T22:17:40.625 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-20T22:17:40.625 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2023-09-20T22:17:40.626 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-20T22:17:40.691 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2023-09-20T22:17:40.691 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-20T22:17:40.718 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2023-09-20T22:17:40.718 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-20T22:17:40.758 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-20T22:17:40.759 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-20T22:17:40.763 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-20T22:17:40.766 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-20T22:17:40.881 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-20T22:17:40.881 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-20T22:17:40.886 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-20T22:17:40.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-20T22:17:41.054 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-20T22:17:41.071 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-20T22:17:41.072 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-20T22:17:41.161 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-20T22:17:41.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-20T22:17:41.221 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-20T22:17:41.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-20T22:17:41.239 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-20T22:17:41.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-20T22:17:41.289 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2023-09-20T22:17:41.317 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2023-09-20T22:17:41.375 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2023-09-20T22:17:41.390 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-20T22:17:41.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T22:17:41.432 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-20T22:17:41.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T22:17:41.449 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-20T22:17:41.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-20T22:17:41.466 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-20T22:17:41.479 INFO:teuthology.task.internal:Transferring archived files... 2023-09-20T22:17:41.480 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182/remote/smithi022 2023-09-20T22:17:41.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-20T22:17:41.565 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182/remote/smithi089 2023-09-20T22:17:41.566 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-20T22:17:41.605 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182/remote/smithi123 2023-09-20T22:17:41.605 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-20T22:17:41.646 INFO:teuthology.task.internal:Removing archive directory... 2023-09-20T22:17:41.646 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-20T22:17:41.651 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-20T22:17:41.654 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-20T22:17:41.698 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-20T22:17:41.712 INFO:teuthology.task.internal:Not uploading archives. 2023-09-20T22:17:41.712 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-20T22:17:41.723 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-20T22:17:41.724 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-20T22:17:41.735 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-20T22:17:41.737 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-20T22:17:41.751 INFO:teuthology.orchestra.run.smithi089.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 20 22:17 /home/ubuntu/cephtest 2023-09-20T22:17:41.755 INFO:teuthology.orchestra.run.smithi123.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 20 22:17 /home/ubuntu/cephtest 2023-09-20T22:17:41.774 INFO:teuthology.orchestra.run.smithi022.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 20 22:17 /home/ubuntu/cephtest 2023-09-20T22:17:41.777 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-20T22:17:41.789 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-20T22:17:41.819 INFO:teuthology.nuke:Checking targets against current locks 2023-09-20T22:17:41.849 DEBUG:teuthology.nuke:shortname: smithi022 2023-09-20T22:17:41.849 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-20T22:17:41.880 DEBUG:teuthology.nuke:shortname: smithi089 2023-09-20T22:17:41.880 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-20T22:17:41.891 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-20 21:13:35.765830', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-20T22:17:41.916 DEBUG:teuthology.nuke:shortname: smithi123 2023-09-20T22:17:41.917 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-20T22:17:41.922 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-20 21:13:35.767733', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-20T22:17:41.967 INFO:teuthology.orchestra.console.smithi022:Power off 2023-09-20T22:17:41.968 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-20T22:17:41.980 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-20_10:42:39-fs-wip-vshankar-testing-20230920.072635-testing-default-smithi/7399182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-20 21:13:35.769330', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-20T22:17:41.999 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2023-09-20T22:17:41.999 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:42.028 INFO:teuthology.orchestra.console.smithi089:Power off 2023-09-20T22:17:42.029 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-20T22:17:42.040 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2023-09-20T22:17:42.053 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off 2023-09-20T22:17:42.053 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:42.069 INFO:teuthology.orchestra.console.smithi123:Power off 2023-09-20T22:17:42.070 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-20T22:17:42.080 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-09-20T22:17:42.114 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2023-09-20T22:17:42.115 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:42.144 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-09-20T22:17:46.042 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:46.081 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:46.146 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:46.156 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2023-09-20T22:17:46.191 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-09-20T22:17:46.256 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-09-20T22:17:50.158 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:50.192 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:50.257 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:50.268 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2023-09-20T22:17:50.302 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-09-20T22:17:50.367 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-09-20T22:17:50.369 INFO:teuthology.orchestra.console.smithi022:Power off completed 2023-09-20T22:17:54.306 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:54.368 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-20T22:17:54.415 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2023-09-20T22:17:54.480 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-09-20T22:17:54.517 INFO:teuthology.orchestra.console.smithi089:Power off completed 2023-09-20T22:17:54.581 INFO:teuthology.orchestra.console.smithi123:Power off completed 2023-09-20T22:17:54.687 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} duration: 3311.3070254325867 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_vshankar@teuthology status: fail success: false 2023-09-20T22:17:54.687 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-20T22:17:54.778 INFO:teuthology.run:FAIL